API/Reference

For now, here is a list of all documented functions.

API/Reference

CCDReduction.AbstractCCDDataType
AbstractCCDData{T}

Supertype for CCDData based on AbstractMatrix interface.

Every subtype of this type should have methods data and hdr defined.

source
CCDReduction.CCDDataType
CCDData(data::AbstractMatrix, [hdr::FITSHeader])

Struct to store ImageHDU, derived from AbstractCCDData.

CCDData acts like a matrix with a header associated.

ccd = CCDData(zeros(4, 4))

ccd[1]

This accesses the 1st element in matrix associated with ccd.

ccd["SIMPLE"]

One can also access the header directly from ccd, the key can be Symbol as well.

ccd[:SIMPLE] = false

Header values can be directly modified from ccd.

One can perform arithmetic operations on it as well:

ccd1 = CCDData(zeros(4, 4))

ccd2 = CCDData(ones(4, 4))

sum_ccd1 = ccd1 + ccd2

sum_ccd2 = ccd2 + ccd1

sum_ccd1 has the header of ccd1 whereas sum_ccd2 has the header ccd2.

If header is not provided in the CCDData constructor, default_header is used to generate the header.

source
CCDReduction.CCDDataMethod
CCDData(path::AbstractString; hdu = 1)

Loads HDU from hdu index in FITS file at path as CCDData.

source
CCDReduction.arraysFunction
arrays(collection)

Generator for arrays of images of entries in data frame.

Iterates over collection using each path and hdu to load data into an Array.

Examples

collection = fitscollection("~/data/tekdata")
data = arrays(collection) |> collect

This returns all image arrays present in collection. This can also be used via a for-loop

collection = fitscollection("~/data/tekdata")
for arr in arrays(collection)
@assert arr isa Array
println(size(arr))
end

# output
(1048, 1068)
(1048, 1068)
...
source
CCDReduction.arraysMethod
arrays(f,
collection;
path = nothing,
save_prefix = nothing,
save_suffix = nothing,
save = any(!isnothing, (save_prefix, path, save_suffix)),
save_delim = "_",
ext = r"fits(\.tar\.gz)?"i,
kwargs...)

Iterates over the image arrays of the collection applying function f at each step.

The output from f can be saved using the appropriate keyword arguments. The save_prefix argument will add a prefix to each filename delimited by save_delim. save_suffix will add a suffix prior to the extension, which can be manually provided via ext, similar to fitscollection. Files will be saved in the directory they are stored unless path is given. Finally, save will default to true if any of the previous arguments are set, but can be manually overridden (useful for testing). Files will be saved using CCDReduction.writefits.

Examples

collection = fitscollection("~/data/tekdata")
processed_images = map(arrays(collection)) do arr
trim(arr, (:, 1040:1059))
end

The above generates processed_images which consists of trimmed versions of image arrays present in collection. For saving the processed_images simultaneously with the operations performed

processed_images = map(arrays(collection; path = "~/data/tekdata", save_prefix = "trimmed")) do img
trim(img, (:, 1040:1059))
end

The trimmed image arrays are saved as trimmed_(original_name) (FITS files) at path = "~/data/tekdata" as specified by the user.

source
CCDReduction.ccdsFunction
ccds(collection)

Generator for CCDDatas of entries in data frame.

Iterates over collection using each path and hdu to load data into a CCDData.

Examples

collection = fitscollection("~/data/tekdata")
for hdu in ccds(collection)
@assert hdu isa CCDData
end
source
CCDReduction.ccdsMethod
ccds(f,
collection;
path = nothing,
save_prefix = nothing,
save_suffix = nothing,
save = any(!isnothing, (save_prefix, path, save_suffix)),
save_delim = "_",
ext = r"fits(\.tar\.gz)?"i,
kwargs...)

Iterates over the CCDDatas of the collection applying function f at each step.

The output from f can be saved using the appropriate keyword arguments. The save_prefix argument will add a prefix to each filename delimited by save_delim. save_suffix will add a suffix prior to the extension, which can be manually provided via ext, similar to fitscollection. Files will be saved in the directory they are stored unless path is given. Finally, save will default to true if any of the previous arguments are set, but can be manually overridden (useful for testing). Files will be saved using CCDReduction.writefits.

Example

collection = fitscollection("~/data/tekdata")
processed_images = map(ccds(collection)) do img
trim(img, (:, 1040:1059))
end

The above generates processed_images which consists of trimmed versions of images present in collection.

For saving the processed_images simultaneously with the operations performed

processed_images = map(ccds(collection; path = "~/data/tekdata", save_prefix = "trimmed")) do img
trim(img, (:, 1040:1059))
end

The trimmed images are saved as trimmed_(original_name) (FITS files) at path = "~/data/tekdata" as specified by the user.

source
CCDReduction.combineMethod
combine(frames...; method = median, [hdu = 1], [header_hdu = 1])
combine(frames; method = median, [hdu = 1], [header_hdu = 1])

Combine multiple frames using method. Multiple frames can also be passed in a vector or as generators for combining.

To pass a custom method, it must have a signature like method(::AbstractArray; dims).

If frames are strings, they will be loaded into CCDDatas first. The HDU indices can be specified with hdu as either an integer or a tuple corresponding to each file.

Header of output file (if applicable) is specified by header_hdu which by default is 1.

Examples

julia> frame = [reshape(1.0:4.0, (2, 2)) for i = 1:4];

julia> combine(frame)
2×2 Matrix{Float64}:
1.0  3.0
2.0  4.0

julia> combine(frame, method = sum)
2×2 Matrix{Float64}:
4.0  12.0
8.0  16.0

source
CCDReduction.cropMethod
crop(frame, shape; force_equal = true, [hdu = 1])

Crops frame to the size specified by shape anchored by the frame center.

This will remove rows/cols of the frame equally on each side. When there is an uneven difference in sizes (e.g. size 9 -> 6 can't be removed equally) the default is to increase the output size (e.g. 6 -> 7) so there is equal removal on each side. To disable this, set force_equal=false, which will remove the extra slice from the end of the axis.

If frame is a string, it will be loaded into CCDData first. The HDU loaded can be specified by hdu which by default is 1.

Examples

julia> frame = reshape(1:25, (5, 5));

julia> crop(frame, (3, 3))
3×3 Matrix{Int64}:
7  12  17
8  13  18
9  14  19

julia> crop(frame, (4, 3), force_equal = false)
4×3 Matrix{Int64}:
6  11  16
7  12  17
8  13  18
9  14  19


See Also

cropview

source
CCDReduction.cropviewMethod
cropview(frame, shape; force_equal = true)

Crops frame to the size specified by shape anchored by the frame center.

This function is same as the crop function but returns a view of the frame.

Note

This function returns a view of the frame, so any modification to output array will result in modification of frame.

See Also

crop

source
CCDReduction.filenamesFunction
filenames(collection)

Generator for filenames of entries in data frame.

Iterates over collection using each path.

Examples

collection = fitscollection("~/data/tekdata")
for path in filenames(collection)
@assert path isa String
println(path)
end

# output
"~/data/tekdata/tek001.fits"
"~/data/tekdata/tek002.fits"
...
source
CCDReduction.filenamesMethod
filenames(f,
collection;
path = nothing,
save_prefix = nothing,
save_suffix = nothing,
save = any(!isnothing, (save_prefix, path, save_suffix)),
save_delim = "_",
ext = r"fits(\.tar\.gz)?"i,
kwargs...)

Iterates over the file paths of the collection applying function f at each step.

The output from f can be saved using the appropriate keyword arguments. The save_prefix argument will add a prefix to each filename delimited by save_delim. save_suffix will add a suffix prior to the extension, which can be manually provided via ext, similar to fitscollection. Files will be saved in the directory they are stored unless path is given. Finally, save will default to true if any of the previous arguments are set, but can be manually overridden (useful for testing). Files will be saved using CCDReduction.writefits.

Examples

collection = fitscollection("~/data/tekdata")
data = map(filenames(collection)) do path
fh = FITS(path)
data = getdata(fh[1]) # assuming all 1-hdu are ImageHDUs
close(fh)
data
end

The above generates data which consists of image arrays corresponding to 1st hdu of FITS file paths present in collection. For saving the data simultaneously with the operations performed

data = map(filenames(collection; path = "~/data/tekdata", save_prefix = "retrieved_from_filename")) do img
fh = FITS(path)
data = getdata(fh[1]) # assuming all 1-hdu are ImageHDUs
close(fh)
data
end

The retrieved data is saved as retrieved_from_filename_(original_name) (FITS files) at path = "~/data/tekdata" as specified by the user.

source
CCDReduction.fitscollectionMethod
fitscollection(dir;
recursive=true,
abspath=true,
keepext=true,
ext=r"fits(\.tar\.gz)?",
exclude=nothing,
exclude_dir=nothing,
exclude_key=("", "HISTORY"))

Walk through dir collecting FITS files, scanning their headers, and culminating into a DataFrame that can be used with the generators for iterating over many files and processing them. If recursive is false, no subdirectories will be walked through.

The table returned will contain the path to the file, the name of the file, and index of the corresponding HDU, and each FITS header column and value. If two FITS files have distinct columns, they will both appear in the table with missing in the appropriate rows.

Duplicate Keys

In certain cases, there are multiple FITS headers with the same key, e.g., COMMENT. In these cases, only the first instance of the key-value pair will be stored.

If abspath is true, the path in the table will be absolute. If keepext is true, the name in the table will include the file extension, given by ext. ext will be used with endswith to filter for fits files compatible with FITSIO.FITS. exclude is a pattern that can be used with occursin to exclude certain filenames. For example, to exclude any files containing "sky",

fitscollection(...; exclude="sky")

to exclude exact filenames, regex strings will prove powerful

fitscollection(...; exclude=r"^tek001\d")

finally, using external tools like Glob.jl allows further customization

using Glob
fitscollection(...; exclude=fn"tek001*.fits") # same as regex match above

Similarly, exclude_dir allows excluding entire folders using pattern matching (e.g. skipping a backup folder exclude_dir="backup"). exclude_key allows excluding certain entries in the header unit of ImageHDU in FITS files (e.g. skipping "HISTORY" and "" exclude_key = ("HISTORY", "")).

For more information about the file matching and path deconstruction, see the extended help (??fitscollection)

Extended Help

Parts of a path

Let's look at some file paths starting from "/data". Here are examples of how they would be parsed

 root  dir   base   ext
[----][---][------][---]
/data/test/tek0001.fits

root    dir     base   ext
[----][-------][------][---]
/data/test/sci/tek0001.fits

If keepext is true, name=base * ext, otherwise it is just base. If abspath is true, the path will be root * dir * base * ext, otherwise it will be dir * base * ext. These options allow flexility in creating a table that can be easily saved and loaded to avoid having to manually filter files. Especially consider how abspath can allow keeping tables that will transfer easily between computers or between data sources with common structures.

source
CCDReduction.flat_correctMethod
flat_correct(frame, flat_frame; norm_value = mean(flat_frame), [hdu = 1])

Correct frame for non-uniformity using the calibrated flat_frame.

By default, the flat_frame is normalized by its mean, but this can be changed by providing a custom norm_value.

If either are strings, they will be loaded into CCDData first. The HDU loaded can be specified by hdu as either an integer or a tuple corresponding to each file.

Note

This function may introduce non-finite values if flat_frame contains values very close to 0 due to dividing by zero. The default behavior will return Inf if the frame value is non-zero, and Nan if the frame value is 0.

Examples

julia> frame = ones(3, 3);

julia> flat = fill(2.0, (3, 3));

julia> flat_correct(frame, flat, norm_value = 1.0)
3×3 Matrix{Float64}:
0.5  0.5  0.5
0.5  0.5  0.5
0.5  0.5  0.5

julia> flat_correct(frame, flat)
3×3 Matrix{Float64}:
1.0  1.0  1.0
1.0  1.0  1.0
1.0  1.0  1.0


See Also

flat_correct!

source
CCDReduction.getdataMethod
CCDReduction.getdata(::FITSIO.ImageHDU)

Loads the given HDU as an Array, permuting the dimensions appropriately.

FITSIO.jl takes over memory read in by cfitsio, which reads in row-major form, whereas when Julia takes that memory, it is assumed as column major. Therefore all data read by FITSIO.read is transposed. This function allows the user to read data in a consistent way to Array by transposing after reading.

source
CCDReduction.subtract_biasMethod
subtract_bias(frame, bias_frame; [hdu = 1])

Subtract the bias_frame from frame.

If either are strings, they will be loaded into CCDData first. The HDU loaded can be specified by hdu as either an integer or a tuple corresponding to each file.

Examples

julia> frame = [1.0 2.2 3.3 4.5];

julia> bias = [0.0 0.2 0.3 0.5];

julia> subtract_bias(frame, bias)
1×4 Matrix{Float64}:
1.0  2.0  3.0  4.0


See Also

subtract_bias!

source
CCDReduction.subtract_darkMethod
subtract_dark(frame, dark_frame; data_exposure = 1, dark_exposure = 1, [hdu = 1])

Subtract the dark_frame from frame.

If either are strings, they will be loaded into CCDData first. The HDU loaded can be specified by hdu as either an integer or a tuple corresponding to each file.

Examples

julia> frame = ones(3, 3);

julia> dark_frame = ones(3, 3);

julia> subtract_dark(frame, dark_frame)
3×3 Matrix{Float64}:
0.0  0.0  0.0
0.0  0.0  0.0
0.0  0.0  0.0

julia> subtract_dark(frame, dark_frame, data_exposure = 1, dark_exposure = 4)
3×3 Matrix{Float64}:
0.75  0.75  0.75
0.75  0.75  0.75
0.75  0.75  0.75


See Also

subtract_dark!

source
CCDReduction.subtract_overscanMethod
subtract_overscan(frame, idxs; dims = axes_min_length(idxs), [hdu = 1])

Subtract the overscan frame from image.

dims is the dimension along which overscan_frame is combined. The default value of dims is the axis with smaller length in overscan region. If idxs is a string it will be parsed as FITS-style indices.

If frame is a string, it will be loaded into CCDData first. The HDU loaded can be specified by hdu which by default is 1.

Examples

julia> frame = [4.0 2.0 3.0 1.0 1.0];

julia> subtract_overscan(frame, (:, 4:5), dims = 2)
1×5 Matrix{Float64}:
3.0  1.0  2.0  0.0  0.0

julia> subtract_overscan(frame, "[4:5, 1:1]", dims = 2)
1×5 Matrix{Float64}:
3.0  1.0  2.0  0.0  0.0


See Also

subtract_overscan!

source
CCDReduction.trimMethod
trim(frame, idxs; [hdu = 1])

Trims the frame to remove the region specified by idxs.

This function trims the array in a manner such that final array should be rectangular. The indices follow standard Julia convention, so (:, 45:60) trims all columns from 45 to 60 and (1:20, :) trims all the rows from 1 to 20. The function also supports FITS-style indices.

If frame is a string, it will be loaded into CCDData first. The HDU loaded can be specified by hdu which by default is 1.

Examples

julia> frame = ones(5, 5);

julia> trim(frame, (:, 2:5))
5×1 Matrix{Float64}:
1.0
1.0
1.0
1.0
1.0

julia> trim(frame, "[2:5, 1:5]")
5×1 Matrix{Float64}:
1.0
1.0
1.0
1.0
1.0


See Also

trimview

source
CCDReduction.trimviewMethod
trimview(frame, idxs)

Trims the frame to remove the region specified by idxs.

This function is same as the trim function but returns a view of the frame.

Note

This function returns a view of the frame, so any modification to output array will result in modification of frame.

See Also

trim

source
CCDReduction.writefitsMethod
CCDReduction.writefits(file_path, data; header = nothing)
CCDReduction.writefits(file_path, ccd::CCDData)

Writes data/ccd in FITS format at file_path.

FITSIO takes over memory write in by cfitsio, which writes in row-major form, whereas when Julia gives that memory, it is assumed as column major. Therefore all data written by FITSIO.write is transposed. This function allows the user to write the data in a consistent way to FITS file by transposing before writing.

source