GitHub

Roman

Adapted from STScI Roman Notebooks.

In this example, we show how to use ASDF.jl to load and view some simulated astronomical data created in preparation for the future (Nancy Grace Roman Space Telescope) mission.

Data availability

Simulated data products are currently provided by STScI via AWS S3 buckets. Note: The data product used for this example is a moderately large file (~300 MB).

Load

data_dir = joinpath("..", "..", "data")
mkpath(data_dir)
fpath = joinpath(data_dir, "roman.asdf")

if !isfile(fpath)
    using AWSS3, AWS
    aws_config = AWS.AWSConfig(; creds = nothing, region = "us-east-1")
    AWSConfig(nothing, "us-east-1", "json", 3)

    # This is a large file, will take some time to download
    s3_get_file(
        aws_config,
        "stpubdata",
        "roman/nexus/soc_simulations/tutorial_data/r0003201001001001004_0001_wfi01_f106_cal.asdf",
        fpath
    )
end
using ASDF

af = ASDF.load_file(fpath; extensions = true, validate_checksum = false)

af.metadata["roman"]
Dict{Any, Any} with 16 entries:
  "dq_border_ref_pix_top"    => NDArray(LazyBlockHeaders(BlockHeader[BlockHeade…
  "data"                     => NDArray(LazyBlockHeaders(BlockHeader[BlockHeade…
  "dq_border_ref_pix_bottom" => NDArray(LazyBlockHeaders(BlockHeader[BlockHeade…
  "amp33"                    => NDArray(LazyBlockHeaders(BlockHeader[BlockHeade…
  "border_ref_pix_top"       => NDArray(LazyBlockHeaders(BlockHeader[BlockHeade…
  "dq"                       => NDArray(LazyBlockHeaders(BlockHeader[BlockHeade…
  "dq_border_ref_pix_left"   => NDArray(LazyBlockHeaders(BlockHeader[BlockHeade…
  "var_rnoise"               => NDArray(LazyBlockHeaders(BlockHeader[BlockHeade…
  "border_ref_pix_bottom"    => NDArray(LazyBlockHeaders(BlockHeader[BlockHeade…
  "border_ref_pix_right"     => NDArray(LazyBlockHeaders(BlockHeader[BlockHeade…
  "var_poisson"              => NDArray(LazyBlockHeaders(BlockHeader[BlockHeade…
  "err"                      => NDArray(LazyBlockHeaders(BlockHeader[BlockHeade…
  "border_ref_pix_left"      => NDArray(LazyBlockHeaders(BlockHeader[BlockHeade…
  "meta"                     => Dict{Any, Any}("file_date"=>"2020-01-01T00:00:0…
  "dq_border_ref_pix_right"  => NDArray(LazyBlockHeaders(BlockHeader[BlockHeade…
  "var_flat"                 => NDArray(LazyBlockHeaders(BlockHeader[BlockHeade…

Plot

using CairoMakie

img = af.metadata["roman"]["data"][]

let
    fig, ax, hm = heatmap(img[begin:1000, begin:1000]; colorscale = asinh, colorrange = (0.5, 4))
    Colorbar(fig[1, 2], hm)
    fig
end
Example block output
Note

Some ASDF files produced by the Python implementation of ASDF may save a checksum in its header block computed from the original decompressed file. This will cause ASDF.jl to fail because in constrast, it computes the checksum based on the compressed (i.e., "used data"), as per the current specification for ASDF. To handle this potenial failure mode, we pass validate_checksum = false to avoid running the default checksum.