Data Processing

Information about the publicly available FIFI-LS, FLITECAM, FORCAST, and HAWC+ data pipelines is available here.

SOFIA FITS Keywords Dictionary (pdf)

• Pipeline User Manuals and Data Handbooks
• Levels
• Data Products Timeline
• Current Pipeline Versions
• Quality Assurance and Known Issues
• FAQ

Pipeline User Manuals and Data Handbooks

The pipeline User Manuals and Guest Observer (GO) Data Handbooks describe data products, processing steps, calibration procedures, and known issues.

EXES [pdf]
FIFI-LS [pdf]
FLITECAM [pdf]
FORCAST [pdf]
HAWC+ [pdf]

back to the top

Levels

The SOFIA Science Center provides raw and calibrated data for all instruments via the SOFIA Science Archive at IRSA.

Currently, there are five levels of SOFIA data products:

Level 0: raw SI data in standardized format (HAWC+ only)
Level 1: raw SI data in standardized format (FITS);  for HAWC+, the demodulated (chop and nod subtracted) data comprise Level 1
Level 2: data corrected for instrument artifacts (e.g. flat-fielded, dark-subtracted, bad pixels removed)
Level 3: flux calibrated and telluric corrected data (using FITS keywords; Jy/pix)
Level 4: Level 4 data products comprise everything beyond flux calibrated (Level 3) products. Any data product generated from the combination of Level 3 files (e.g., a map or a mosaic) is considered a Level 4 product. For most instruments, Level 4 data products are usually generated from observations obtained across multiple flights or even multiple flight series and observing cycles. For HAWC+ nod-pol observations, the data product that includes the polarization vectors is Level 4.

Details on the processing steps can be found in the data handbooks for each instrument.

The instrument, and desired processing level can be specified in the archive search parameters.

The SOFIA Science Center (SSC) provides the processed, flux calibrated Level 2, Level 3, and (when appropriate) Level 4 data products for the commissioned Facility Class science instruments, which currently include FORCAST (both imaging and grism modes), HAWC+, FIFI-LS, and (now-retired) FLITECAM (both imaging and grism modes). The instruments teams for the PI Class science instruments EXES and GREAT provide the data products for those instruments.

back to the top

Data Products Timeline

The data products for each instrument, and at each level of processing, are expected to be available on the timescales listed below. In all cases the times are measured from the end of the flight series in which the observation was performed. This timeline is for nominal processing only which does not include additional manual processing to address any problems encountered in the data.

Level 4 data is available for FIFI-LS, FORCAST, GREAT, and HAWC+ instruments. It is typically delivered at the same time as Level 3 data, except for data compiled from cross-mission observations, which is situationally dependent.

Tables of synthetic fluxes for SOFIA/FORCAST standard stars

All Level 3 imaging data have been flux calibrated using all standards available from a single flight series, as described in the GO Data Handbooks. Tables of filter wavelengths and color corrections for each observing cycle are given below:

FORCAST Cycle 1: FORCAST_ColorCorrecns_OC1.pdf
FORCAST Cycle 2 onward: FORCAST_ColorCorrecns_OC4.pdf
HAWC+ Cycle 5: HAWC_ColorCorrecns_OC5.pdf

back to the top

Current Pipeline Versions

Here are the current versions of the Data Reduction Pipelines:

When searching for and retrieving reduced data from the SOFIA Science Archive at IRSA, please check the version of the pipeline used to generate the data products. If the pipeline version does not agree with the entry given above, please contact the Help Desk (sofia_help@sofia.usra.edu) to inquire if the data require re-processing or are in the process of being re-reduced. Not all revisions to the pipelines result in the need to re-process data. If re-processing is underway, new versions of the data products will be available shortly. The status of the data products for any specific set of observations can be provided upon request.

back to the top

Quality Assurance and Known Issues

Summary of QA Process and Keywords

Calibrated FORCAST imaging data processed before Cycle 3 (2015) do not include the on-source integration time listed in their headers. The document FORCAST Imaging Exposure Time outlines the procedure for calculating the on-source integration time in Level 2 and 3 merged and co-added data files for these observations.

Catalog of Known Issues for each Observing Series.

back to the top

FAQ

FAQ coming soon.

back to the top