API/Reference

abstract type AbstractCCDData{T} <: AbstractMatrix{T}

Supertype for CCDData based on AbstractMatrix interface.

Every subtype of this type should have methods data and hdr defined.

CCDData <: AbstractCCDData
CCDData(data::AbstractMatrix, [hdr::FITSHeader])

Struct to store ImageHDU, derived from AbstractCCDData.

CCDData acts like a matrix with a header associated.

ccd = CCDData(zeros(4, 4))

ccd[1]

This accesses the 1st element in matrix associated with ccd.

ccd["SIMPLE"]

One can also access the header directly from ccd, the key can be Symbol as well.

ccd[:SIMPLE] = false

Header values can be directly modified from ccd.

One can perform arithmetic operations on it as well:

ccd1 = CCDData(zeros(4, 4))
ccd2 = CCDData(ones(4, 4))
sum_ccd1 = ccd1 + ccd2
sum_ccd2 = ccd2 + ccd1

sum_ccd1 has the header of ccd1 whereas sum_ccd2 has the header ccd2.

If header is not provided in the CCDData constructor, default_header is used to generate the header.

CCDData(path::AbstractString; hdu = 1)

Loads HDU from hdu index in FITS file at path as CCDData.

CCDData(hdu::ImageHDU)

Loads ImageHDU as CCDData.

arrays(collection)

Generator for arrays of images of entries in data frame.

Iterates over collection using each path and hdu to load data into an Array.

Examples

collection = fitscollection("~/data/tekdata")
data = arrays(collection) |> collect

This returns all image arrays present in collection. This can also be used via a for-loop

collection = fitscollection("~/data/tekdata")
for arr in arrays(collection)
    @assert arr isa Array
    println(size(arr))
end

# output
(1048, 1068)
(1048, 1068)
...

arrays(f,
       collection;
       path = nothing,
       save_prefix = nothing,
       save_suffix = nothing,
       save = any(!isnothing, (save_prefix, path, save_suffix)),
       save_delim = "_",
       ext = r"fits(\.tar\.gz)?"i,
       kwargs...)

Iterates over the image arrays of the collection applying function f at each step.

The output from f can be saved using the appropriate keyword arguments. The save_prefix argument will add a prefix to each filename delimited by save_delim. save_suffix will add a suffix prior to the extension, which can be manually provided via ext, similar to fitscollection. Files will be saved in the directory they are stored unless path is given. Finally, save will default to true if any of the previous arguments are set, but can be manually overridden (useful for testing). Files will be saved using CCDReduction.writefits.

Examples

collection = fitscollection("~/data/tekdata")
processed_images = map(arrays(collection)) do arr
    trim(arr, (:, 1040:1059))
end

The above generates processed_images which consists of trimmed versions of image arrays present in collection. For saving the processed_images simultaneously with the operations performed

processed_images = map(arrays(collection; path = "~/data/tekdata", save_prefix = "trimmed")) do img
    trim(img, (:, 1040:1059))
end

The trimmed image arrays are saved as trimmed_(original_name) (FITS files) at path = "~/data/tekdata" as specified by the user.

ccds(collection)

Generator for CCDDatas of entries in data frame.

Iterates over collection using each path and hdu to load data into a CCDData.

Examples

collection = fitscollection("~/data/tekdata")
for hdu in ccds(collection)
    @assert hdu isa CCDData
end

ccds(f,
     collection;
     path = nothing,
     save_prefix = nothing,
     save_suffix = nothing,
     save = any(!isnothing, (save_prefix, path, save_suffix)),
     save_delim = "_",
     ext = r"fits(\.tar\.gz)?"i,
     kwargs...)

Iterates over the CCDDatas of the collection applying function f at each step.

The output from f can be saved using the appropriate keyword arguments. The save_prefix argument will add a prefix to each filename delimited by save_delim. save_suffix will add a suffix prior to the extension, which can be manually provided via ext, similar to fitscollection. Files will be saved in the directory they are stored unless path is given. Finally, save will default to true if any of the previous arguments are set, but can be manually overridden (useful for testing). Files will be saved using CCDReduction.writefits.

Example

collection = fitscollection("~/data/tekdata")
processed_images = map(ccds(collection)) do img
    trim(img, (:, 1040:1059))
end

The above generates processed_images which consists of trimmed versions of images present in collection.

For saving the processed_images simultaneously with the operations performed

processed_images = map(ccds(collection; path = "~/data/tekdata", save_prefix = "trimmed")) do img
    trim(img, (:, 1040:1059))
end

The trimmed images are saved as trimmed_(original_name) (FITS files) at path = "~/data/tekdata" as specified by the user.

combine(frames...; method = median, [hdu = 1], [header_hdu = 1])
combine(frames; method = median, [hdu = 1], [header_hdu = 1])

Combine multiple frames using method. Multiple frames can also be passed in a vector or as generators for combining.

To pass a custom method, it must have a signature like method(::AbstractArray; dims).

If frames are strings, they will be loaded into CCDDatas first. The HDU indices can be specified with hdu as either an integer or a tuple corresponding to each file.

Header of output file (if applicable) is specified by header_hdu which by default is 1.

Examples

julia> frame = [reshape(1.0:4.0, (2, 2)) for i = 1:4];

julia> combine(frame)
2×2 Matrix{Float64}:
 1.0  3.0
 2.0  4.0

julia> combine(frame, method = sum)
2×2 Matrix{Float64}:
 4.0  12.0
 8.0  16.0

crop(frame, shape; force_equal = true, [hdu = 1])

Crops frame to the size specified by shape anchored by the frame center.

This will remove rows/cols of the frame equally on each side. When there is an uneven difference in sizes (e.g. size 9 -> 6 can't be removed equally) the default is to increase the output size (e.g. 6 -> 7) so there is equal removal on each side. To disable this, set force_equal=false, which will remove the extra slice from the end of the axis.

If frame is a string, it will be loaded into CCDData first. The HDU loaded can be specified by hdu which by default is 1.

Examples

julia> frame = reshape(1:25, (5, 5));

julia> crop(frame, (3, 3))
3×3 Matrix{Int64}:
 7  12  17
 8  13  18
 9  14  19

julia> crop(frame, (4, 3), force_equal = false)
4×3 Matrix{Int64}:
 6  11  16
 7  12  17
 8  13  18
 9  14  19

See Also

cropview

cropview(frame, shape; force_equal = true)

Crops frame to the size specified by shape anchored by the frame center.

This function is same as the crop function but returns a view of the frame.

See Also

crop

filenames(collection)

Generator for filenames of entries in data frame.

Iterates over collection using each path.

Examples

collection = fitscollection("~/data/tekdata")
for path in filenames(collection)
    @assert path isa String
    println(path)
end

# output
"~/data/tekdata/tek001.fits"
"~/data/tekdata/tek002.fits"
...

filenames(f,
          collection;
          path = nothing,
          save_prefix = nothing,
          save_suffix = nothing,
          save = any(!isnothing, (save_prefix, path, save_suffix)),
          save_delim = "_",
          ext = r"fits(\.tar\.gz)?"i,
          kwargs...)

Iterates over the file paths of the collection applying function f at each step.

The output from f can be saved using the appropriate keyword arguments. The save_prefix argument will add a prefix to each filename delimited by save_delim. save_suffix will add a suffix prior to the extension, which can be manually provided via ext, similar to fitscollection. Files will be saved in the directory they are stored unless path is given. Finally, save will default to true if any of the previous arguments are set, but can be manually overridden (useful for testing). Files will be saved using CCDReduction.writefits.

Examples

collection = fitscollection("~/data/tekdata")
data = map(filenames(collection)) do path
    fh = FITS(path)
    data = getdata(fh[1]) # assuming all 1-hdu are ImageHDUs
    close(fh)
    data
end

The above generates data which consists of image arrays corresponding to 1st hdu of FITS file paths present in collection. For saving the data simultaneously with the operations performed

data = map(filenames(collection; path = "~/data/tekdata", save_prefix = "retrieved_from_filename")) do img
    fh = FITS(path)
    data = getdata(fh[1]) # assuming all 1-hdu are ImageHDUs
    close(fh)
    data
end

The retrieved data is saved as retrieved_from_filename_(original_name) (FITS files) at path = "~/data/tekdata" as specified by the user.

find_ccd(As)

Return the first CCDData among the arguments.

fitscollection(dir;
               recursive=true,
               abspath=true,
               keepext=true,
               ext=r"fits(\.tar\.gz)?",
               exclude=nothing,
               exclude_dir=nothing,
               exclude_key=("", "HISTORY"))

Walk through dir collecting FITS files, scanning their headers, and culminating into a DataFrame that can be used with the generators for iterating over many files and processing them. If recursive is false, no subdirectories will be walked through.

The table returned will contain the path to the file, the name of the file, and index of the corresponding HDU, and each FITS header column and value. If two FITS files have distinct columns, they will both appear in the table with missing in the appropriate rows.

If abspath is true, the path in the table will be absolute. If keepext is true, the name in the table will include the file extension, given by ext. ext will be used with endswith to filter for fits files compatible with FITSIO.FITS. exclude is a pattern that can be used with occursin to exclude certain filenames. For example, to exclude any files containing "sky",

fitscollection(...; exclude="sky")

to exclude exact filenames, regex strings will prove powerful

fitscollection(...; exclude=r"^tek001\d")

finally, using external tools like Glob.jl allows further customization

using Glob
fitscollection(...; exclude=fn"tek001*.fits") # same as regex match above

Similarly, exclude_dir allows excluding entire folders using pattern matching (e.g. skipping a backup folder exclude_dir="backup"). exclude_key allows excluding certain entries in the header unit of ImageHDU in FITS files (e.g. skipping "HISTORY" and "" exclude_key = ("HISTORY", "")).

For more information about the file matching and path deconstruction, see the extended help (??fitscollection)

Extended Help

Parts of a path

Let's look at some file paths starting from "/data". Here are examples of how they would be parsed

 root  dir   base   ext
[----][---][------][---]
/data/test/tek0001.fits

 root    dir     base   ext
[----][-------][------][---]
/data/test/sci/tek0001.fits

If keepext is true, name=base * ext, otherwise it is just base. If abspath is true, the path will be root * dir * base * ext, otherwise it will be dir * base * ext. These options allow flexility in creating a table that can be easily saved and loaded to avoid having to manually filter files. Especially consider how abspath can allow keeping tables that will transfer easily between computers or between data sources with common structures.

flat_correct!(frame, flat_frame; norm_value = mean(flat_frame), [hdu = 1])

In-place version of flat_correct

See Also

flat_correct

flat_correct(frame, flat_frame; norm_value = mean(flat_frame), [hdu = 1])

Correct frame for non-uniformity using the calibrated flat_frame.

By default, the flat_frame is normalized by its mean, but this can be changed by providing a custom norm_value.

If either are strings, they will be loaded into CCDData first. The HDU loaded can be specified by hdu as either an integer or a tuple corresponding to each file.

Examples

julia> frame = ones(3, 3);

julia> flat = fill(2.0, (3, 3));

julia> flat_correct(frame, flat, norm_value = 1.0)
3×3 Matrix{Float64}:
 0.5  0.5  0.5
 0.5  0.5  0.5
 0.5  0.5  0.5

julia> flat_correct(frame, flat)
3×3 Matrix{Float64}:
 1.0  1.0  1.0
 1.0  1.0  1.0
 1.0  1.0  1.0

See Also

flat_correct!

CCDReduction.getdata(::FITSIO.ImageHDU)

Loads the given HDU as an Array, permuting the dimensions appropriately.

FITSIO.jl takes over memory read in by cfitsio, which reads in row-major form, whereas when Julia takes that memory, it is assumed as column major. Therefore all data read by FITSIO.read is transposed. This function allows the user to read data in a consistent way to Array by transposing after reading.

subtract_bias!(frame, bias_frame; [hdu = 1])

In-place version of subtract_bias

See Also

subtract_bias

subtract_bias(frame, bias_frame; [hdu = 1])

Subtract the bias_frame from frame.

If either are strings, they will be loaded into CCDData first. The HDU loaded can be specified by hdu as either an integer or a tuple corresponding to each file.

Examples

julia> frame = [1.0 2.2 3.3 4.5];

julia> bias = [0.0 0.2 0.3 0.5];

julia> subtract_bias(frame, bias)
1×4 Matrix{Float64}:
 1.0  2.0  3.0  4.0

See Also

subtract_bias!

subtract_dark!(frame, dark_frame; data_exposure = 1, dark_exposure = 1, [hdu = 1])

In-place version of subtract_dark

See Also

subtract_dark

subtract_dark(frame, dark_frame; data_exposure = 1, dark_exposure = 1, [hdu = 1])

Subtract the dark_frame from frame.

If either are strings, they will be loaded into CCDData first. The HDU loaded can be specified by hdu as either an integer or a tuple corresponding to each file.

Examples

julia> frame = ones(3, 3);

julia> dark_frame = ones(3, 3);

julia> subtract_dark(frame, dark_frame)
3×3 Matrix{Float64}:
 0.0  0.0  0.0
 0.0  0.0  0.0
 0.0  0.0  0.0

julia> subtract_dark(frame, dark_frame, data_exposure = 1, dark_exposure = 4)
3×3 Matrix{Float64}:
 0.75  0.75  0.75
 0.75  0.75  0.75
 0.75  0.75  0.75

See Also

subtract_dark!

subtract_overscan!(frame, idxs; dims = axes_min_length(idxs))

In-place version of subtract_overscan

See Also

subtract_overscan

subtract_overscan(frame, idxs; dims = axes_min_length(idxs), [hdu = 1])

Subtract the overscan frame from image.

dims is the dimension along which overscan_frame is combined. The default value of dims is the axis with smaller length in overscan region. If idxs is a string it will be parsed as FITS-style indices.

If frame is a string, it will be loaded into CCDData first. The HDU loaded can be specified by hdu which by default is 1.

Examples

julia> frame = [4.0 2.0 3.0 1.0 1.0];

julia> subtract_overscan(frame, (:, 4:5), dims = 2)
1×5 Matrix{Float64}:
 3.0  1.0  2.0  0.0  0.0

julia> subtract_overscan(frame, "[4:5, 1:1]", dims = 2)
1×5 Matrix{Float64}:
 3.0  1.0  2.0  0.0  0.0

See Also

subtract_overscan!

trim(frame, idxs; [hdu = 1])

Trims the frame to remove the region specified by idxs.

This function trims the array in a manner such that final array should be rectangular. The indices follow standard Julia convention, so (:, 45:60) trims all columns from 45 to 60 and (1:20, :) trims all the rows from 1 to 20. The function also supports FITS-style indices.

If frame is a string, it will be loaded into CCDData first. The HDU loaded can be specified by hdu which by default is 1.

Examples

julia> frame = ones(5, 5);

julia> trim(frame, (:, 2:5))
5×1 Matrix{Float64}:
 1.0
 1.0
 1.0
 1.0
 1.0

julia> trim(frame, "[2:5, 1:5]")
5×1 Matrix{Float64}:
 1.0
 1.0
 1.0
 1.0
 1.0

See Also

trimview

trimview(frame, idxs)

Trims the frame to remove the region specified by idxs.

This function is same as the trim function but returns a view of the frame.

See Also

trim

CCDReduction.writefits(file_path, data; header = nothing)
CCDReduction.writefits(file_path, ccd::CCDData)

Writes data/ccd in FITS format at file_path.

FITSIO takes over memory write in by cfitsio, which writes in row-major form, whereas when Julia gives that memory, it is assumed as column major. Therefore all data written by FITSIO.write is transposed. This function allows the user to write the data in a consistent way to FITS file by transposing before writing.