Benchmarks

The benchmarks can be found in the bench/ folder. To run them, first instantiate the environment

$ julia --project=bench -e "using Pkg; Pkg.instantiate()"

then execute the bench/bench.jl file

$ julia --project=bench bench/bench.jl

System Information

Julia Version 1.10.4
Commit 48d4fd48430 (2024-06-04 10:41 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 16 × 13th Gen Intel(R) Core(TM) i5-1340P
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, goldmont)
Threads: 1 default, 0 interactive, 1 GC (on 16 virtual cores)

Evaluation benchmark

This benchmark tests how long it takes to evaluate a single point in the PSF model. This may seem contrived, but we expect performance to scale directly from this measure: if it takes 1 microsecond to evaluate a single point, it should take ~1 second to evaluate a 1000×1000 image, with speedups potentially from multithreading or SIMD loop evaluation.

table = CSV.read(benchdir("evaluation_results.csv"), DataFrame)

3×3 DataFrame
 Row │ name      psfmodels   astropy
     │ String15  Float64     Float64
─────┼─────────────────────────────────
   1 │ Gaussian  2.33112e-8  7.1579e-5
   2 │ AiryDisk  1.9672e-8   5.7172e-5
   3 │ Moffat    2.33253e-8  5.3287e-5

@df table groupedbar(
    :name, [:psfmodels :astropy];
    ylabel="time (s)", yscale=:log10, legend=:outertopright,
    label=["PSFModels.jl" "Astropy"], size=(500, 300),
)

Fitting benchmark

This benchmark tests how long it takes to fit a PSF Model to a stamp with size (39, 39). In all cases, we use equivalently complex models, the default fitters for PSFModels.jl, and the default LevMarLSQFit fitter for astropy.

table = CSV.read(benchdir("fitting_results.csv"), DataFrame)

3×3 DataFrame
 Row │ name      psfmodels   astropy
     │ String15  Float64     Float64
─────┼──────────────────────────────────
   1 │ Gaussian  0.00349649  0.00731791
   2 │ AiryDisk  0.00849787  0.00952454
   3 │ Moffat    0.00393968  0.00754826

@df table groupedbar(
    :name, [:psfmodels :astropy];
    ylabel="time (s)", yscale=:log10, legend=:outertopright,
    label=["PSFModels.jl" "Astropy"], size=(500, 300),
)