API/Reference
For now, here is a list of all documented functions.
Index
CCDReduction.AbstractCCDData
CCDReduction.CCDData
CCDReduction.CCDData
CCDReduction.CCDData
CCDReduction.arrays
CCDReduction.arrays
CCDReduction.ccds
CCDReduction.ccds
CCDReduction.combine
CCDReduction.crop
CCDReduction.cropview
CCDReduction.filenames
CCDReduction.filenames
CCDReduction.find_ccd
CCDReduction.fitscollection
CCDReduction.flat_correct
CCDReduction.flat_correct!
CCDReduction.getdata
CCDReduction.subtract_bias
CCDReduction.subtract_bias!
CCDReduction.subtract_dark
CCDReduction.subtract_dark!
CCDReduction.subtract_overscan
CCDReduction.subtract_overscan!
CCDReduction.trim
CCDReduction.trimview
CCDReduction.writefits
API/Reference
CCDReduction.AbstractCCDData
— Typeabstract type AbstractCCDData{T} <: AbstractMatrix{T}
Supertype for CCDData
based on AbstractMatrix
interface.
Every subtype of this type should have methods data
and hdr
defined.
CCDReduction.CCDData
— TypeCCDData <: AbstractCCDData
CCDData(data::AbstractMatrix, [hdr::FITSHeader])
Struct to store ImageHDU
, derived from AbstractCCDData
.
CCDData
acts like a matrix with a header associated.
ccd = CCDData(zeros(4, 4))
ccd[1]
This accesses the 1st element in matrix associated with ccd
.
ccd["SIMPLE"]
One can also access the header directly from ccd
, the key can be Symbol
as well.
ccd[:SIMPLE] = false
Header values can be directly modified from ccd
.
One can perform arithmetic operations on it as well:
ccd1 = CCDData(zeros(4, 4))
ccd2 = CCDData(ones(4, 4))
sum_ccd1 = ccd1 + ccd2
sum_ccd2 = ccd2 + ccd1
sum_ccd1
has the header of ccd1
whereas sum_ccd2
has the header ccd2
.
If header is not provided in the CCDData
constructor, default_header
is used to generate the header.
CCDReduction.CCDData
— MethodCCDData(path::AbstractString; hdu = 1)
Loads HDU from hdu
index in FITS
file at path
as CCDData
.
CCDReduction.CCDData
— MethodCCDData(hdu::ImageHDU)
Loads ImageHDU
as CCDData
.
CCDReduction.arrays
— Functionarrays(collection)
Generator for arrays of images of entries in data frame.
Iterates over collection
using each path
and hdu
to load data into an Array
.
Examples
collection = fitscollection("~/data/tekdata")
data = arrays(collection) |> collect
This returns all image arrays present in collection
. This can also be used via a for-loop
collection = fitscollection("~/data/tekdata")
for arr in arrays(collection)
@assert arr isa Array
println(size(arr))
end
# output
(1048, 1068)
(1048, 1068)
...
CCDReduction.arrays
— Methodarrays(f,
collection;
path = nothing,
save_prefix = nothing,
save_suffix = nothing,
save = any(!isnothing, (save_prefix, path, save_suffix)),
save_delim = "_",
ext = r"fits(\.tar\.gz)?"i,
kwargs...)
Iterates over the image arrays of the collection applying function f
at each step.
The output from f
can be saved using the appropriate keyword arguments. The save_prefix
argument will add a prefix to each filename delimited by save_delim
. save_suffix
will add a suffix prior to the extension, which can be manually provided via ext
, similar to fitscollection
. Files will be saved in the directory they are stored unless path
is given. Finally, save
will default to true
if any of the previous arguments are set, but can be manually overridden (useful for testing). Files will be saved using CCDReduction.writefits
.
Examples
collection = fitscollection("~/data/tekdata")
processed_images = map(arrays(collection)) do arr
trim(arr, (:, 1040:1059))
end
The above generates processed_images
which consists of trimmed versions of image arrays present in collection
. For saving the processed_images
simultaneously with the operations performed
processed_images = map(arrays(collection; path = "~/data/tekdata", save_prefix = "trimmed")) do img
trim(img, (:, 1040:1059))
end
The trimmed image arrays are saved as trimmed_(original_name)
(FITS files) at path = "~/data/tekdata"
as specified by the user.
CCDReduction.ccds
— Functionccds(collection)
Generator for CCDData
s of entries in data frame.
Iterates over collection
using each path
and hdu
to load data into a CCDData
.
Examples
collection = fitscollection("~/data/tekdata")
for hdu in ccds(collection)
@assert hdu isa CCDData
end
CCDReduction.ccds
— Methodccds(f,
collection;
path = nothing,
save_prefix = nothing,
save_suffix = nothing,
save = any(!isnothing, (save_prefix, path, save_suffix)),
save_delim = "_",
ext = r"fits(\.tar\.gz)?"i,
kwargs...)
Iterates over the CCDData
s of the collection applying function f
at each step.
The output from f
can be saved using the appropriate keyword arguments. The save_prefix
argument will add a prefix to each filename delimited by save_delim
. save_suffix
will add a suffix prior to the extension, which can be manually provided via ext
, similar to fitscollection
. Files will be saved in the directory they are stored unless path
is given. Finally, save
will default to true
if any of the previous arguments are set, but can be manually overridden (useful for testing). Files will be saved using CCDReduction.writefits
.
Example
collection = fitscollection("~/data/tekdata")
processed_images = map(ccds(collection)) do img
trim(img, (:, 1040:1059))
end
The above generates processed_images
which consists of trimmed versions of images present in collection
.
For saving the processed_images
simultaneously with the operations performed
processed_images = map(ccds(collection; path = "~/data/tekdata", save_prefix = "trimmed")) do img
trim(img, (:, 1040:1059))
end
The trimmed images are saved as trimmed_(original_name)
(FITS files) at path = "~/data/tekdata"
as specified by the user.
CCDReduction.combine
— Methodcombine(frames...; method = median, [hdu = 1], [header_hdu = 1])
combine(frames; method = median, [hdu = 1], [header_hdu = 1])
Combine multiple frames using method
. Multiple frames can also be passed in a vector or as generators for combining.
To pass a custom method, it must have a signature like method(::AbstractArray; dims)
.
If frames
are strings, they will be loaded into CCDData
s first. The HDU indices can be specified with hdu
as either an integer or a tuple corresponding to each file.
Header of output file (if applicable) is specified by header_hdu
which by default is 1.
Examples
julia> frame = [reshape(1.0:4.0, (2, 2)) for i = 1:4];
julia> combine(frame)
2×2 Matrix{Float64}:
1.0 3.0
2.0 4.0
julia> combine(frame, method = sum)
2×2 Matrix{Float64}:
4.0 12.0
8.0 16.0
CCDReduction.crop
— Methodcrop(frame, shape; force_equal = true, [hdu = 1])
Crops frame
to the size specified by shape
anchored by the frame center.
This will remove rows/cols of the frame
equally on each side. When there is an uneven difference in sizes (e.g. size 9 -> 6 can't be removed equally) the default is to increase the output size (e.g. 6 -> 7) so there is equal removal on each side. To disable this, set force_equal=false
, which will remove the extra slice from the end of the axis.
If frame
is a string, it will be loaded into CCDData
first. The HDU loaded can be specified by hdu
which by default is 1.
Examples
julia> frame = reshape(1:25, (5, 5));
julia> crop(frame, (3, 3))
3×3 Matrix{Int64}:
7 12 17
8 13 18
9 14 19
julia> crop(frame, (4, 3), force_equal = false)
4×3 Matrix{Int64}:
6 11 16
7 12 17
8 13 18
9 14 19
See Also
CCDReduction.cropview
— Methodcropview(frame, shape; force_equal = true)
Crops frame
to the size specified by shape
anchored by the frame center.
This function is same as the crop
function but returns a view of the frame.
This function returns a view of the frame, so any modification to output array will result in modification of frame.
See Also
CCDReduction.filenames
— Functionfilenames(collection)
Generator for filenames of entries in data frame.
Iterates over collection
using each path
.
Examples
collection = fitscollection("~/data/tekdata")
for path in filenames(collection)
@assert path isa String
println(path)
end
# output
"~/data/tekdata/tek001.fits"
"~/data/tekdata/tek002.fits"
...
CCDReduction.filenames
— Methodfilenames(f,
collection;
path = nothing,
save_prefix = nothing,
save_suffix = nothing,
save = any(!isnothing, (save_prefix, path, save_suffix)),
save_delim = "_",
ext = r"fits(\.tar\.gz)?"i,
kwargs...)
Iterates over the file paths of the collection applying function f
at each step.
The output from f
can be saved using the appropriate keyword arguments. The save_prefix
argument will add a prefix to each filename delimited by save_delim
. save_suffix
will add a suffix prior to the extension, which can be manually provided via ext
, similar to fitscollection
. Files will be saved in the directory they are stored unless path
is given. Finally, save
will default to true
if any of the previous arguments are set, but can be manually overridden (useful for testing). Files will be saved using CCDReduction.writefits
.
Examples
collection = fitscollection("~/data/tekdata")
data = map(filenames(collection)) do path
fh = FITS(path)
data = getdata(fh[1]) # assuming all 1-hdu are ImageHDUs
close(fh)
data
end
The above generates data
which consists of image arrays corresponding to 1st hdu of FITS file paths present in collection
. For saving the data
simultaneously with the operations performed
data = map(filenames(collection; path = "~/data/tekdata", save_prefix = "retrieved_from_filename")) do img
fh = FITS(path)
data = getdata(fh[1]) # assuming all 1-hdu are ImageHDUs
close(fh)
data
end
The retrieved data is saved as retrieved_from_filename_(original_name)
(FITS files) at path = "~/data/tekdata"
as specified by the user.
CCDReduction.find_ccd
— Methodfind_ccd(As)
Return the first CCDData
among the arguments.
CCDReduction.fitscollection
— Methodfitscollection(dir;
recursive=true,
abspath=true,
keepext=true,
ext=r"fits(\.tar\.gz)?",
exclude=nothing,
exclude_dir=nothing,
exclude_key=("", "HISTORY"))
Walk through dir
collecting FITS files, scanning their headers, and culminating into a DataFrame
that can be used with the generators for iterating over many files and processing them. If recursive
is false, no subdirectories will be walked through.
The table returned will contain the path to the file, the name of the file, and index of the corresponding HDU, and each FITS header column and value. If two FITS files have distinct columns, they will both appear in the table with missing
in the appropriate rows.
In certain cases, there are multiple FITS headers with the same key, e.g., COMMENT
. In these cases, only the first instance of the key-value pair will be stored.
If abspath
is true, the path in the table will be absolute. If keepext
is true, the name in the table will include the file extension, given by ext
. ext
will be used with endswith
to filter for fits files compatible with FITSIO.FITS
. exclude
is a pattern that can be used with occursin
to exclude certain filenames. For example, to exclude any files containing "sky",
fitscollection(...; exclude="sky")
to exclude exact filenames, regex strings will prove powerful
fitscollection(...; exclude=r"^tek001\d")
finally, using external tools like Glob.jl allows further customization
using Glob
fitscollection(...; exclude=fn"tek001*.fits") # same as regex match above
Similarly, exclude_dir
allows excluding entire folders using pattern matching (e.g. skipping a backup folder exclude_dir="backup"
). exclude_key
allows excluding certain entries in the header unit of ImageHDU
in FITS files (e.g. skipping "HISTORY"
and ""
exclude_key = ("HISTORY", "")
).
For more information about the file matching and path deconstruction, see the extended help (??fitscollection
)
Extended Help
Parts of a path
Let's look at some file paths starting from "/data"
. Here are examples of how they would be parsed
root dir base ext
[----][---][------][---]
/data/test/tek0001.fits
root dir base ext
[----][-------][------][---]
/data/test/sci/tek0001.fits
If keepext
is true
, name=base * ext
, otherwise it is just base
. If abspath
is true
, the path will be root * dir * base * ext
, otherwise it will be dir * base * ext
. These options allow flexility in creating a table that can be easily saved and loaded to avoid having to manually filter files. Especially consider how abspath
can allow keeping tables that will transfer easily between computers or between data sources with common structures.
CCDReduction.flat_correct!
— Methodflat_correct!(frame, flat_frame; norm_value = mean(flat_frame), [hdu = 1])
In-place version of flat_correct
See Also
CCDReduction.flat_correct
— Methodflat_correct(frame, flat_frame; norm_value = mean(flat_frame), [hdu = 1])
Correct frame
for non-uniformity using the calibrated flat_frame
.
By default, the flat_frame
is normalized by its mean, but this can be changed by providing a custom norm_value
.
If either are strings, they will be loaded into CCDData
first. The HDU loaded can be specified by hdu
as either an integer or a tuple corresponding to each file.
This function may introduce non-finite values if flat_frame
contains values very close to 0
due to dividing by zero. The default behavior will return Inf
if the frame value is non-zero, and Nan
if the frame value is 0
.
Examples
julia> frame = ones(3, 3);
julia> flat = fill(2.0, (3, 3));
julia> flat_correct(frame, flat, norm_value = 1.0)
3×3 Matrix{Float64}:
0.5 0.5 0.5
0.5 0.5 0.5
0.5 0.5 0.5
julia> flat_correct(frame, flat)
3×3 Matrix{Float64}:
1.0 1.0 1.0
1.0 1.0 1.0
1.0 1.0 1.0
See Also
CCDReduction.getdata
— MethodCCDReduction.getdata(::FITSIO.ImageHDU)
Loads the given HDU as an Array
, permuting the dimensions appropriately.
FITSIO.jl takes over memory read in by cfitsio, which reads in row-major form, whereas when Julia takes that memory, it is assumed as column major. Therefore all data read by FITSIO.read
is transposed. This function allows the user to read data in a consistent way to Array
by transposing after reading.
CCDReduction.subtract_bias!
— MethodCCDReduction.subtract_bias
— Methodsubtract_bias(frame, bias_frame; [hdu = 1])
Subtract the bias_frame
from frame
.
If either are strings, they will be loaded into CCDData
first. The HDU loaded can be specified by hdu
as either an integer or a tuple corresponding to each file.
Examples
julia> frame = [1.0 2.2 3.3 4.5];
julia> bias = [0.0 0.2 0.3 0.5];
julia> subtract_bias(frame, bias)
1×4 Matrix{Float64}:
1.0 2.0 3.0 4.0
See Also
CCDReduction.subtract_dark!
— Methodsubtract_dark!(frame, dark_frame; data_exposure = 1, dark_exposure = 1, [hdu = 1])
In-place version of subtract_dark
See Also
CCDReduction.subtract_dark
— Methodsubtract_dark(frame, dark_frame; data_exposure = 1, dark_exposure = 1, [hdu = 1])
Subtract the dark_frame
from frame
.
If either are strings, they will be loaded into CCDData
first. The HDU loaded can be specified by hdu
as either an integer or a tuple corresponding to each file.
Examples
julia> frame = ones(3, 3);
julia> dark_frame = ones(3, 3);
julia> subtract_dark(frame, dark_frame)
3×3 Matrix{Float64}:
0.0 0.0 0.0
0.0 0.0 0.0
0.0 0.0 0.0
julia> subtract_dark(frame, dark_frame, data_exposure = 1, dark_exposure = 4)
3×3 Matrix{Float64}:
0.75 0.75 0.75
0.75 0.75 0.75
0.75 0.75 0.75
See Also
CCDReduction.subtract_overscan!
— Methodsubtract_overscan!(frame, idxs; dims = axes_min_length(idxs))
In-place version of subtract_overscan
See Also
CCDReduction.subtract_overscan
— Methodsubtract_overscan(frame, idxs; dims = axes_min_length(idxs), [hdu = 1])
Subtract the overscan frame from image.
dims
is the dimension along which overscan_frame
is combined. The default value of dims
is the axis with smaller length in overscan region. If idxs
is a string it will be parsed as FITS-style indices.
If frame
is a string, it will be loaded into CCDData
first. The HDU loaded can be specified by hdu
which by default is 1.
Examples
julia> frame = [4.0 2.0 3.0 1.0 1.0];
julia> subtract_overscan(frame, (:, 4:5), dims = 2)
1×5 Matrix{Float64}:
3.0 1.0 2.0 0.0 0.0
julia> subtract_overscan(frame, "[4:5, 1:1]", dims = 2)
1×5 Matrix{Float64}:
3.0 1.0 2.0 0.0 0.0
See Also
CCDReduction.trim
— Methodtrim(frame, idxs; [hdu = 1])
Trims the frame
to remove the region specified by idxs.
This function trims the array in a manner such that final array should be rectangular. The indices follow standard Julia convention, so (:, 45:60)
trims all columns from 45 to 60 and (1:20, :)
trims all the rows from 1 to 20. The function also supports FITS-style indices.
If frame
is a string, it will be loaded into CCDData
first. The HDU loaded can be specified by hdu
which by default is 1.
Examples
julia> frame = ones(5, 5);
julia> trim(frame, (:, 2:5))
5×1 Matrix{Float64}:
1.0
1.0
1.0
1.0
1.0
julia> trim(frame, "[2:5, 1:5]")
5×1 Matrix{Float64}:
1.0
1.0
1.0
1.0
1.0
See Also
CCDReduction.trimview
— Methodtrimview(frame, idxs)
Trims the frame
to remove the region specified by idxs.
This function is same as the trim
function but returns a view of the frame.
This function returns a view of the frame, so any modification to output array will result in modification of frame.
See Also
CCDReduction.writefits
— MethodCCDReduction.writefits(file_path, data; header = nothing)
CCDReduction.writefits(file_path, ccd::CCDData)
Writes data
/ccd
in FITS format at file_path
.
FITSIO
takes over memory write in by cfitsio
, which writes in row-major form, whereas when Julia gives that memory, it is assumed as column major. Therefore all data written by FITSIO.write
is transposed. This function allows the user to write the data in a consistent way to FITS file by transposing before writing.