API

block_magic_token

The 4-byte sentinel UInt8[0xd3, 0x42, 0x4c, 0x4b].

host_byteorder

Native byte order of the running machine, detected at load time. Defined by ASDF.Byteorder.

ASDFFile

The in-memory representation of a loaded ASDF file, combining the parsed YAML metadata tree with the binary block infrastructure needed to lazily materialize any array data it references.

Fields

ASDFLibrary

Software provenance metadata, serialized as a !core/software-1.0.0 YAML tag. [ASDF.write_file] inserts an entry automatically under the key "asdf/library" if one is not already present, using the package's own name, author, homepage, and version.

BlockHeader

Parsed representation of a single ASDF binary block header.

Every binary block in an ASDF file begins with a fixed-layout header that describes the block's compression, size, and integrity checksum. BlockHeader captures all decoded fields from that header, together with the IO handle and file position needed to subsequently read the block's data payload.

Fields

File layout

The block occupies the following byte range within io, starting at position:

The extended header (always 48 bytes in the current implementation) contains, in order: flags (4 B), compression (4 B), allocated_size (8 B), used_size (8 B), data_size (8 B), and checksum (16 B).

All multi-byte integers in the header are stored in big-endian byte order.

Blocks

Module-level accumulator that collects ASDF.NDArrayWrapper values and their corresponding file positions during a single call to ASDF.write_file.

Blocks acts as a two-phase write buffer. In the first phase, as the YAML tree is serialized, each non-inline ASDF.NDArrayWrapper appends itself to arrays and reserves a sequential source index. In the second phase, ASDF.write_file iterates over arrays, compresses and writes each block to disk, and records the resulting file offsets in positions. The finalized positions vector is then written as the ASDF block index at the end of the file.

Fields

ASDF.Byteorder

Represents the byte order of array data stored in a block. Available variants:

• Byteorder_little : Little-endian
• Byteorder_big: Big-endian

ASDF.Byteorder(str::AbstractString)::Byteorder

Convenience conversion for ASDF.Byteorder. Inverse of Base.string(byteorder::Byteorder).

Examples

julia> ASDF.Byteorder("little")
Byteorder_little::Byteorder = 0

ChunkedNDArray

A logical N-dimensional array assembled from a collection of arbitrarily-positioned ASDF.NDArrayChunk tiles, each backed by its own binary block or inline data.

ChunkedNDArray is the in-memory representation of a !core/chunked-ndarray-1.X.Y YAML node. It defines the shape and element type of the full logical array, but defers all data access to the individual chunks. The full array is only allocated and populated when Base.getindex(ndarray::NDArray) is called.

Fields

Identifies the compression algorithm used for a data block. Available variants:

Maps ASDF datatype strings to Julia types. Note this is unrelated to Base.DataType. Defined mappings:

LazyBlockHeaders

A mutable container holding the complete list of ASDF.BlockHeader values scanned from an ASDF file, shared by reference with every ASDF.NDArray and ASDF.ChunkedNDArray constructed during parsing.

Reference sharing

Every ASDF.NDArray created during parsing of a given file holds a reference to the same LazyBlockHeaders instance. When ndarray[] is called, it indexes into lazy_block_headers.block_headers using the array's zero-based source field:

header = ndarray.lazy_block_headers.block_headers[ndarray.source + 1]
data   = ASDF.read_block(header)

Because block_headers is populated after all NDArray objects are constructed, no array needs to be updated individually when block scanning completes. The shared mutable reference propagates the result automatically.

Mutability

LazyBlockHeaders is a mutable struct solely to allow block_headers to be assigned after construction. The field is written exactly once per file load, immediately after YAML.load within ASDF.load_file returns. It is never modified again during normal use. Treat it as effectively immutable after load_file returns.

NDArray

A lazily-materialized N-dimensional array stored in an ASDF file, either as a binary block or inline within the ASDF file.

NDArray is the in-memory representation of an !core/ndarray-1.0.0 YAML node. It holds the array's shape, type, and layout metadata, but defers reading and decompressing block data until the array is explicitly materialized by calling Base.getindex(ndarray::NDArray).

Fields

NDArrayChunk

A positioned tile within an ASDF.ChunkedNDArray, pairing an ASDF.NDArray with a zero-based origin that locates the tile in the parent array's coordinate space.

Fields

NDArrayWrapper

A write-side wrapper around a Julia array that carries compression and layout options. Used as the value type when building a document dict for ASDF.write_file.

find_all_blocks(io::IO, pos::Int64=Int64(0); validate_checksum::Bool)

Scans an IO stream and returns all ASDF.BlockHeader values found.

load_file(filename::AbstractString; extensions = false, validate_checksum = true)

Reads an ASDF file from disk.

Block data is located lazily. Block headers are scanned after the YAML is parsed, and array data (ndarray) is read only when Base.getindex(ndarray::NDArray) is called, i.e., ndarray[].

read_block(header::BlockHeader)

Uses an ASDF.BlockHeader to read, verify, and decompress the data payload.

read_block_header(io::IO, position::Int64; validate_checksum::Bool)

Constructs an ASDF.BlockHeader by parsing the file at a given offset.

write_file(filename::AbstractString, document::Dict{Any,Any})

Writes an ASDF file to disk. document is a plain Dict whose values may include NDArrayWrapper instances. These are serialized as binary blocks with appropriate compression.

Layout of the output file:

1. ASDF/YAML header (#ASDF 1.0.0, #ASDF_STANDARD 1.2.0, %YAML 1.1)
2. YAML tree (!core/asdf-1.1.0)
3. Binary blocks — one per NDArrayWrapper that has inline == false
4. Block index (#ASDF BLOCK INDEX)

Base.getindex(chunked_ndarray::ChunkedNDArray)

Allocates a dense array of shape reverse(shape) and fills it by calling chunk.ndarray[] for each chunk, placing the result at the correct offset.

Base.getindex(ndarray::NDArray)

Returns the fully materialized array. See ASDF.NDArray for definitions. When block-backed (source !== nothing), this reads and decompresses the block, applies offset and strides via a StridedViews.StridedView, reinterprets bytes to Type(datatype), and byte-swaps if byteorder != host_byteorder. The returned array satisfies:

size(result) == Tuple(reverse(ndarray.shape))
eltype(result) == Type(ndarray.datatype)
sizeof(eltype) .* strides(result) == Tuple(reverse(ndarray.strides))

string(byteorder::Byteorder)::AbstractString

Convenience conversion for ASDF.Byteorder. Inverse of ASDF.Byteorder(str::AbstractString).

Examples

julia> string(ASDF.Byteorder_little)
"little"