CALACS Design: Lessons Learned from CALSTIS - STScI

Instrument Science Report ACS-98-01

CALACS Design: Lessons Learned from CALSTIS Warren J. Hack 26 May 1998

ABSTRACT This report documents the basic design of those parts of CALSTIS relevant for CALACS. A listing of the basic calibration steps required for ACS and a comparison of ACS with STIS provide the basis for determining what functionality needs to be retained or used from CALSTIS. Although CALSTIS provides a solid foundation for CALACS design, ACS data requires changes in how memory is managed along with new functionality to interpret the complex ACS associations. These changes are highlighted in the description of the overall design of CALACS, with its 5 tasks: CALACS, ACSCCD, ACS2D, ACSREJ, and ACSSUM. The HSTIO changes necessary to support this new pipeline and the implementation plan with testing schedule complete the description of how CALACS will be created from CALSTIS.

1. Introduction ACS uses detectors designed for STIS, therefore, it is natural to determine if software designed for STIS data can be adapted to reduce ACS observations. At the very least, lessons learned from the STIS calibration pipeline software CALSTIS should be applied in creating the ACS calibration pipeline software, CALACS. This paper describes differences in data format and processing requirements for ACS in the next section. The following section outlines the current CALSTIS software, point out what functions will be needed for CALACS, and provides details of the file I/O currently used in CALSTIS. These elements are put together in an outline of the CALACS software in Section 4. Keywords used for CALACS are given in Section 6, while the required changes in HSTIO are described in Section 7. The remainder of this paper provides an outline of the development schedule for CALACS (in Section 8).

1

2. ACS Data and Software Requirements There are a number of difference between ACS and STIS data that preclude a direct use of most of the CALSTIS code. The type of rewriting that needs to be done relies, of course, on the exact nature of the new data, so by understanding the new ACS data, a clear picture of the required changes becomes evident. Once the changes are identified, then the CALSTIS code will be copied and modified as needed to create a separate pipeline for the ACS data. Differences between ACS and STIS data can be classified into two areas: data format and reduction requirements. The ACS data format itself derives from the STIS FITS file format using FITS extensions for science data, data quality arrays and error arrays, resulting in a three extension IMSET within the FITS file. However, there are these differences: •

2 chip readout for the wide field camera (WFC)

•

maximum 2048x4096 science data arrays in each of 2 chips

•

strictly imaging data (aside from grism exposures)

The decision was made to store the WFC data using 2 IMSETs in one FITS file, with one IMSET for each chip. Each IMSET will consist of a science (SCI) array, a data quality (DQ) array and an error (ERR) array. The size of the image arrays, though, results in a maximum size of 160Mb for a single calibrated WFC ACS observation, under the assumption that the ERR array is stored in floating point format. The following are the design decisions made for the baseline ACS pipeline: •

CR-SPLIT and REPEATOBS options are mutually exclusive

•

geometric correction will be handled ‘off-line’ as a stand-alone task

•

no spectroscopic reductions (even for GRISM data) in baseline version

•

use of NICMOS styled associations for CR-SPLIT and REPEATOBS data, but no other associations

• no combining of dithered observations • no special reductions for polarized observations In addition to the changes to the calibration steps which will be performed by CALACS, one major change in design is required in order to allow it to run at all on most systems: line-by-line image I/O. This will require the greatest amount of work. The effect of this new file I/O model will be discussed in detail in the following section along with the summary of the functions that will still be needed from CALSTIS. CALACS will retain as much of the same code from CALSTIS as possible in an effort to allow the two pipelines to eventually share libraries of functions in later builds for tasks which are camera independent, such as cosmic ray rejection.

2

3. Description of Relevant CALSTIS Functions Modularity defines the basic design used for CALSTIS, a modularity of functionality which was used to build separate tasks run under a controlling task, CALSTIS0. In all, CALSTIS contains 12 separate tasks each of which can be run individually from the command line when they are not called from CALSTIS0. Figure 1 illustrates the overall structure of CALSTIS0, with the spectroscopic steps being combined into one ‘function’ call since they will not be used in CALACS. Figure 1: Flow Diagram of CALSTIS0. CRCORR

Yes (CCD)

No (MAMA)

ATOD,DQI, or BLEV

Yes CALSTIS 1 (AtoD, DQI, BLEV)

CRCORR

Yes CALSTIS 2 Cosmic Ray Reject

CALSTIS 1

Yes

FLAT

CALSTIS 1 FLTCORR

SPECTROSCOPY

Spec OBSTYPE

GEO CORR

(CALSTIS1,11,7,4,12,7,6)

Yes CALSTIS 7

Yes RPT CORR CALSTIS 8

3

CALSTIS relies heavily on modularity, something which will work to our advantage in adapting STIS code for CALACS. File management is one aspect of CALSTIS which is shown in Figure 1. Since each step may or may not be used for the calibration of an image being processed, the responsibility for accessing the image lies with the individual functions which perform the calibration steps. This results in CALSTIS0 only passing the image name from one step to the next. This makes it easy to customize the file I/O to match the needs of the data and calibration step. CALSTIS Handling of Headers and Image Data Overall, CALSTIS attempts to perform as many operations in memory and in place on the images as possible. This involves having the header information in memory during processing as well as the image data. Several structures are defined within CALSTIS to package up the relevant keywords and switch values and pass them along for processing. The structure definitions for CALSTIS0 and CALSTIS1 can be found in Appendix A. Data arrays for the images are read into and stored in an HSTIO structure SingleGroup. This structure packages the primary header information, the extension specific header information, and the data arrays for the SCI, DQ and ERR arrays (for STIS). All three arrays are read in and written out using HSTIO routines that handle the reading or writing with one function call. The data and header I/O can be seen in the following summary of routines relevant to ACS data: [CALSTIS0] • Read in primary image header and store key values in several structures, including StisInfo and specific structures for controlling CALSTIS1. • Use values in StisInfo structure to control what processing steps are performed. None of these values are modified by underlying functions. • Pass information from CALSTIS1 specific structures to processing steps in CALSTIS1. This information is not directly modified at any point. [CALSTIS1] •Open image. Read primary header into local header structure. •Update StisInfo1 structure with switch information input from CALSTIS0 and from header structure. •Close input image and free structure header structure. •Pass StisInfo1 structure to function Do2D for processing of each IMSET in the input image. [Do2D]

4

Read data arrays into memory as SingleGroup structure. Extract IMSET specific header information into StisInfo1 structure. Pass SingleGroup structure to following functions for operation on data arrays in memory: doDQI StisInfo1[Input] SingleGroup[Input/Output] Operates on DQ array in-place doAtoD StisInfo1[Input] SingleGroup[Input/Output] Applies AtoD correction in-place. doBlev StisInfo1[Input] SingleGroup[Input] extver SingleGroup(&z)[Output] &meanblev &done &driftcorr Subtract bias from SingleGroup and output to SingleGroup(&z). [CALSTIS0] • Check values in StisInfo structure to determine if cosmicray rejection needs to be performed. If so, pass name of files from StisInfo to CALSTIS2. [CALSTIS2] •Open input file template pointing to list of images to be processed [CRREJ_DO] Open Input files and temp files, make sure images have same characteristics (done in CRREJ_CHECK) Extract parameters from image headers [CRREJ_SKY] Calculate sky for each image, read in each image into memory for the calculation. Read first image in list into memory as SingleGroup [CRREJ_INIT] Use line-by-line I/O to read in individual lines from all images in list at once and compute comparison image [CRREJ_LOOP] Read in one image at a time to perform final rejection against the comparison image. Resultant cleaned image SingleGroup computed in place.

5

[CRREJ_DO] Cleaned image scaled by exposure time and sky value with the results kept in SingleGroup. Header values updated in SingleGroup. Combined SingleGroup written to output file in final processed form. SingleGroup finally closed and temp arrays freed. [CALSTIS2] •Input file list closed and return to calling routine [CALSTIS0] • Complete basic 2D calibration on all IMSETS (including combined cosmic-ray rejected products) as required: [CALSTIS1] •Process each IMSET based on the calibration switches set in the primary header. [Do2D] doNoise StisInfo1[Input] SingleGroup[Input/Output] Evaluate ERR array in-place doLoRes StisInfo1[Input] SingleGroup[Input] SingleGroup(&y)[Output] &done Bins SingleGroup to create SingleGroup(&y) doNonLin StisInfo1[Input] SingleGroup[Input/Output] &gsat &lsat Corrects or flags non-linearity in-place. doBias StisInfo1[Input] SingleGroup[Input/Output] Subtract bias image from SingleGroup. doDark StisInfo1[Input] SingleGroup[Input/Output] &meandark Subtract dark image from SingleGroup. doFlat StisInfo1[Input] SingleGroup[Input/Output] Divide SingleGroup by flat-fields. doShad StisInfo1[Input] SingleGroup[Input/Output] Applies shutter shading correction to SingleGroup. doPhot StisInfo1[Input] SingleGroup[Input/Output]

6

Updates SingleGroup header in-place with calculated photometry information. doStat SingleGroup[Input/Output] sts.sdqflags Computes good min/max/means for data and error and updates SingleGroup header in-place. [Do2D] •Update SingleGroup structure header in memory after each processing step is performed. •Write out SingleGroup structure after all processing steps are performed, complete with updated header information. •Close image and free memory used by SingleGroup. [CALSTIS0] • Check StisInfo structure to see if images need to be summed together. If so, pass some information from StisInfo to CALSTIS8 for processing. [CALSTIS8] •Initialize StisInfo8 structure, and header structure. •Open input image and read primary header into header structure. •Copy pertinent keywords into StisInfo8 structure and delete primary header structure. [SUMGRPS] initialize 2 SingleGroups x and y Read in input image (SingleGroup x) Get IMSET specific header info into local variables For each successive image, read SingleGroup into memory as y and add to SingleGroup x in memory. For each image, update keyword values in SingleGroup x’s header in memory Free each SingleGroup y once added to x Update header info in x, then write SingleGroup to output file. Free SingleGroup x [CALSTIS0] • Free reference file structures and quit. In the above summary, function and task names are given in bold and structure variables are listed in italics. 7

The handling of the header information and data arrays all are completed in memory until the processing is completed, which works well for datasets small enough to fit in memory but becomes untenable for very large data arrays. In general, the data array I/O is confined to the lowest level routines possible, reserving that activity until absolutely necessary and when it needs to be done the least. The header information, on the other hand, is read in early and controls the operations of the program. This information is then passed down to the processing steps but not modified until the data arrays are modified, and even then it is done in memory and only written out upon completion of the reduction step. This safeguards the input file’s headers from being modified out of step with the reductions and localizes any errors in writing the headers to the lowest level routines possible, making for a more secure program overall.

4. CALACS Design In comparison to CALSTIS, the overall design for the CALACS package will be much less complex, with fewer separate tasks to maintain. The same basic procedures used in CALSTIS will be applied to the ACS data, including the separate calibrations for MAMA and CCD data, as shown in Figure 2. This design will form the basis for CALACS, the task responsible for managing the operation of the remainder of the pipeline. The task CALACS will serve primarily as a wrapper for the other pipeline tasks, retaining some aspects of CALSTIS0, revising the STIS-specific logic to be applicable to ACS data, and incorporating new code to interpret the NICMOSstyle ACS association file. As a result, the task CALACS will be new code which shares basic design features with CALSTIS0. The ACS pipeline, shown in Figure 2, will be derived from CALSTIS tasks as follows: CALACS

CALSTIS0

Control operation of all tasks

ACSCCD

CALSTIS1

AtoD, DQI, BLEV functions only

ACS2D

CALSTIS1

DQI plus remaining functions, including MAMA specific steps

ACSREJ

CALSTIS2

complete task rewritten for ACS

ACSSUM

CALSTIS8

complete task rewritten for ACS

These new tasks will be revised to accommodate the larger image sizes for ACS, the use of NICMOS-style associations, and the replacement of STIS-specific logic and data parameters. The management of the associations will take place in CALACS as it replaces looping over the IMSETs in a file with looping through the files listed in the ACS association file. The routines listed in Table 2 will be revised as they contain logic specific to STIS data which can not be applied to ACS observations. Finally, a new convention for naming the CALACS tasks breaks from the CALSTIS model where tasks are named CALSTIS1, CALSTIS2, and so on, however, it provides a name which better describes how each task was designed to be applied.

8

In an extension of the design of CALSTIS1, the functions called prior to cosmic-ray rejection and those called afterwards will be separated into individual tasks for ACS, as shown in Figure 2. This will not only isolate calls specific to the CCDs from those used for all the images, but will allow the creation of the intermediate overscan trimmed image on disk for use in further processing steps. Therefore, the task ACSCCD will perform the initial processing on the input images resulting in the overscan trimmed image being written to disk, while the remaining CALSTIS1 tasks will be merged into the ACS2D task. The ACS2D task will also include the tasks which initialize the data quality and error arrays so that it can be used for both CCD and MAMA observations. The cosmic ray rejection task CALSTIS2 will require significant revision to use the memory model required by CALACS, but will still remain a single task as ACSREJ. Finally, the task for summing repeated observations will also need to be modified for line-by-line I/O and to conform to the CALACS memory model. The following sections describe what changes in design will be needed for the CALACS package to accommodate ACS data. CALACS Association Table A major change in the design of CALACS will be in the need to interpret association tables and process the appropriate files. Association files are necessary due to the complex set of relationships between the exposures. The current baseline version of CALACS will recognize and correctly process CR-SPLIT or REPEAT-OBS exposures resulting in either combined cosmic-ray rejected images or a combined image of all the repeated exposures. As shown in Figure 2, the CALACS task would process related CR-SPLIT exposures through ACSCCD then combine them into a single CR-combined product using ACSREJ. This CR-combined product would then be processed through ACS2D to produce the calibrated product designated in the association table. This process would then be repeated for the remaining sets of CR-SPLIT exposures in the association table. In future enhancements, each CR-combined product could represent a position in a dither pattern, which would need to be combined into a dither-combined exposure. To support the creation of the CR-combined products and future dither combining, the NICMOS-style association table was adopted containing MEMNAME, MEMTYPE and MEMPRSNT columns. Unlike the NICMOS tables, ACS tables will involve multiple output products. The MEMNAME column lists all the exposure filenames which comprise the entire association, complete with output product names. The MEMTYPE columns specifies the type of role the file has in the association. A different set of MEMTYPEs specific to ACS were adopted in order to provide the support for the

9

multiple products. CALACS will recognize the following MEMTYPE values in the baseline version: MEMTYPE EXP_CRn

Description Input CR-SPLIT exposure for CR-combined image n

PROD_CRn

CR-combined exposure n

EXP_RPTn

Input REPEAT-OBS exposure #n

PROD_RPT

REPEAT-OBS combined output product

(PROD_DTH)

(Dither-combined output product, not in baseline version)

An example association table for a 2 position dither set with CR-SPLIT=2 is given in Table 1. The MEMTYPE for each component of the first CR-SPLIT exposure, j1xx52ecm and j1xx52egm, are given as EXP_CR1. The output product is not only given a name based on the association table’s name, j1xx5201, but it is designated in the table with a MEMTYPE of PROD_CR1. This example not only shows how the MEMTYPE can be used to associate input and output products, but also demonstrates how the MEMNAME will be used as well. The last digit of each product’s filename corresponds to the output product number in the MEMTYPE, with the designation of 0 being reserved for the final dither-combined output product (which will not be produced in the baseline version). Table 1. Sample Association Table J1XX52010_ASN MEMNAME

MEMTYPE

MEMPRSNT

j1xx52ecm

EXP_CR1

yes

j1xx52egm

EXP_CR1

yes

j1xx52011

PROD_CR1

yes

j1xx52emm

EXP_CR2

yes

j1xx52eom

EXP_CR2

yes

j1xx52012

PROD_CR2

yes

j1xx52010

PROD_DTH

yes

CALACS will then use the number in the MEMTYPE to associate the input exposures, designated as EXP_CR or EXP_RPT, and process all of them before combining them into the output product. This output product then gets calibrated by the rest of the pipeline. Code from the NICMOS pipeline software CALNICB will be copied and revised to handle the interpretation of ACS association tables.

10

CALACS Memory Model Accommodating the larger ACS images will require a significant change in the memory management of the IMSETS. The working memory design model will assume a maximum size of approximately 80Mb per observational IMSET and that one IMSET could be held in memory as the output image, which would be built up during processing. A limitation on writing out an entire IMSET at once is imposed by the FITS kernel which only allows one extension of an IMSET to be open for line-by-line I/O at the same time. Without keeping the entire output in memory, a temporary file for each extension would have to be created as needed, then copied into the final output file one extension at a time. This would dramatically increase the amount of disk I/O involved in working with the image. As a result, we decided on using a hybrid I/O model for CALACS. By keeping an output IMSET in memory during processing, we can operate on the entire IMSET in place. Once processing is complete in the task, the output file will be written out using the currently available HSTIO functions without the need for temporary files. Line-by-line I/O will then be used to read in the potentially large input images, especially when more than one image is operated on at the same time, such as in bias image subtraction or in summing images. In particular, the output image would be created in memory as a SingleGroup from one of the input images, then the other image(s) would be read in line-by-line and applied to the output image in-place. The functions which need to be revised for line-by-line I/O are listed in Table 2, along with some comments on the form those revisions will take. Table 2. Functions Which Require ACS Motivated Revisions Function

Revisions Needed

doBlev

Operate on X in-place, but specify data section within overscan sections for use by PutSingleGroup to write out smaller image to file.

doBias

Remove the dependency on binning and convert to line-by-line I/O

doDark


doFlat


doShad


doNonLin

Remove the dependency on binning.

doDQI

Remove dependency on binning and dispaxis.

11

Function

Revisions Needed

crrej_loop

Convert to operate on image sections

sumGrps

Convert to use line-by-line I/O

sub2d,div2d,mult2d


RptSum


FindBin

Convert to work with only one line at a time. Also, remove dependency on binning and dispaxis as they impose STIS specific logic.

CALACS Data Products Output from this calibration pipeline will consist of files containing the calibrated observations stored as IMSETs in each file. Each IMSET consists of a SCI (science data) extension, a DQ (data quality) extension, and an ERR (error) extension for a maximum size of 80Mb plus the header information. The file extensions which will be used for the ACS observations are listed in Table 3, which closely mimics the extensions used by STIS. Table 3. Filename extensions used by CALACS: IPPPSSOOT_RAW Raw data _ASN Association file for observation set _FLT Calibrated individual exposure (not produced if CR-split) _BLV_TMP Overscan-trimmed individual exposure (not kept unless requested by the user) _CRJ_TMP Uncalibrated, CR-split-combined image (renamed to _CRJ if ACS2D is not run, otherwise deleted) _CRJ Calibrated, CR-split-combined image. _SFL Calibrated, Repeat-Obs-combined image. _DTH Calibrated, Dither-combined image (not in baseline) The initial inputs to the task CALACS will be the _RAW files from generic conversions and the _ASN (association) table for the complete observation set. Output files for a REPEAT_OBS association will be individually calibrated _FLT exposures and a calibrated summed _SFL image, while CR-SPLIT observations will produce _CRJ images. The _BLV_TMP file will be created for CCD images as output from ACSCCD, will serve as input to either ACSREJ or ACS2D and will be deleted once it is no longer needed for processing. Similarly, for CR-SPLIT exposures, _CRJ_TMP will be created as output from ACSREJ and will either be copied to an _CRJ file is ACS2D is not run, or simply deleted upon completion of ACS2D. Finally, if later versions CALACS ever perform dither-combining of _CRJ images, they will produce a _DTH image.

12

Optional ERR Array Not everyone wants or needs the ERR extension. Since the ERR extension is identical in size to the SCI extension (in uncompressed form), the ERR array comprises 32Mb of an output IMSET, or up to 64Mb for the entire FITS file for a 2 chip WFC observation. Therefore, an option will be built into CALACS to output a NULL ERR extension with a keyword WRTERR set to ‘NO’. This would cause CALACS to process the observation as usual, including the ERR arrays, and then simply write out a NULL ERR extension upon completion with the keyword signifying that the errors were not written out at all, rather than assuming that they are zero because of their absence. It should be stressed that the presence of a NULL ERR array will not adversely affect pipeline should this observation be run through again, and that HSTIO already knows how to work with these NULL arrays. Figure 2: Flow Diagram for CALACS. Detector

CCD ACSCCD

Another Image in CR-SPLIT set?

(AtoD, DQI, BLEV, Bias)

MAMA

Another Image in association?

CRCORR ACS2D

Yes ACSREJ

DQI,Flat,Dark,...

Cosmic Ray Rejection

ACS2D Flat, Dark,... Another Set of CR-SPLIT Images?

RPT CORR

Yes ACSSUM

5. CALACS Tasks This section describes how the design changes described in the previous section will be applied to the CALSTIS code to create the baseline tasks that will make up CALACS.

13

ACSCCD Summary This routine contains the initial processing tasks that need to be performed on all ACS CCD data and will be comprised of four main functions from CALSTIS1, performed in the following order: •

error array initialization in ‘doNoise’,

•

data quality initialization in ‘doDQI’,

•

AtoD conversion in ‘doAtoD’,

•

bias level subtraction and trimming in ‘doBlev’, and

•

bias image subtraction in ‘doBias’.

The upper level task, ACSCCD, will use header keyword values to determine what steps should be performed, and only perform them if needed, rather than blindly applying the correction more than once on the data. The bulk of this task will be based on the code in CALSTIS1, but then be simplified to control only these four tasks using logic applicable to ACS rather than using the current STIS-specific logic. Input to this task will be an image list or single image, which could either be provided by the task CALACS or the user directly. It will process each image in the list or the single image and create an output image with the _BLV_TMP file extension. Two of the functions in this task do not need any revision at all to work with ACS data: ‘doNoise’ and ‘doAtoD’. These functions simply operate in-place on the output image stored in memory. The function ‘doDQI’, on the other hand, needs major revision to function properly on ACS observations. First, it relies on keywords which are unnecessary for ACS, such as dispaxis for designating the dispersion axis in the spectroscopic observation. It also defines variables specific to STIS observations; namely, high_res which designates whether the observation was subsampled or not, specifically for MAMA high-resolution observations and associated reference files. Finally, the routine determines how to process the data based on logic specific to STIS observations which can not be generalized to ACS observations without ‘falsifying’ the values for the STIS specific parameters to treat ACS data appropriately. For STIS, MAMA data are designated as high resolution with a binning of 1, when ACS data would be considered low resolution for the same binning factor. In short, the entire function needs to be re-written specifically for ACS. Since binning will not be possible for the MAMA, all code related to binning of MAMA data will be removed, along with all code related to the keyword dispaxis. Binning, however, will still be a possibility for CCD observations, so that code will remain in place, but likely will be untested until needed to support flight-software changes to allow CCD binning. The function ‘doBlev’ requires conversion to line-by-line I/O when fitting the bias level from the overscan region to the image data. In a change from the CALSTIS design, bias image subtraction will now be performed prior to cosmic-ray rejection using ‘doBias’. This move was made to

14

allow proper accounting of the bias contribution from each pixel in the CR-combined image. When performed after ACSREJ, the bias image is scaled by the number of images which were combined. However, for cosmic-ray rejected pixels, the flux in that pixel would be scaled by the exposure time (not number of total images), scaling the bias contribution as well, and if the cosmic-ray pixel came from an image which had a different exposure time, the bias contribution would be incorrectly scaled. Therefore, in order to correctly account for the bias contribution from each pixel, this correction will be performed prior to combining the images. This function will need to be re-written to apply the correction to an image which still contains the overscan regions, and to read the bias reference image using line-by-line I/O, in order to operate with ACS images. Finally, ACSCCD will require a new HSTIO function to write out to disk the section of the observation that does not contain the overscan data. This trimmed output image would then be the final product from this task, and would be used as input into the remaining of the processing steps in CALACS. ACS2D Summary Every observation will eventually be processed by this task, whether it is MAMA data or CCD data, as it contains the basic functions necessary for calibration. It will be a direct derivative of CALSTIS1, with the exclusion of the AtoD conversion and bias level subtraction and the STIS specific function ‘doLoRes’ for rebinning sub-sampled MAMA data. It will, however, contain the data quality initialization function used in ACSCCD and the error array initialization function ‘doNoise’ for the benefit of those observations which would never be run through ACSCCD. A check will be made to insure that the array initialization is not performed on CCD data twice. Calibration switches in the image header control the performance of the remaining calibration functions as they do in CALSTIS1, with MAMA specific functions being initiated only when the MAMA specific calibration switch was set, for example. The primary tasks contained in ACS2D will be (in operational order): doNoise

Applies a simple noise model to blank error array, if not done in ACSCCD

doDQI

Initialize data quality flags, if not done in ACSCCD

doNonLin

Apply and flag non-linear data

doDark

Perform dark image subtraction

doFlat

Combine and subtract flat field images

doShad

Perform shutter shading correction for CCD, if not CR-combined

doPhot

Compute photometric values for header

doStat

Computes min, mean and max of good SCI data

In addition to these primary functions, other functions will be used to ensure that the appropriate comments are saved to the history section of the header. Of these functions, ‘doFlat’,

15

‘doDark’, and ‘doShad’ rely on binning information specific to STIS to determine the resolution of the data compared to the reference data. These functions can not be re-written for use with both STIS data and ACS data as a result of this use of STIS specific information and logic. As with ACSCCD, the code will be copied from CALSTIS and revised specifically for use with ACS data, retaining CCD binning and discarding MAMA binning. Revising the STIS specific tasks for ACS will actually result in much simpler code. The revisions will eliminate the need for the switch doppcorr and related keyword value for dispaxis. Both of these keywords and related code will have to be removed from all ACS code because of the conflict that would arise in trying to apply them to ACS data in ‘doDQI’ and elsewhere. Therefore, of all the routines derived from CALSTIS1, the following will need to be rewritten specifically for ACS: doDQI, doDark, doFlat, doNonLin, and doShad, along with the function ‘GetGrpInfo1’ which sets parameters based on header keyword values and switch settings. In essence, this results in a completely new, ACS specific task which relies solely on logic applicable to ACS observations. Along with these revisions, changes to support line-by-line I/O for those functions working with more than one image at a time will be performed. These revisions will be based on code copied from CALSTIS but this code will evolve separately without trying to apply it to STIS data. ACSREJ Summary Unlike CALSTIS2, this task will not be able to keep multiple images in memory due to the potentially very large size of the images arrays. Instead, line-by-line I/O will be used while still keeping the overall design and basic algorithm for finding and rejecting the cosmic rays applied in CALSTIS2. Slight modifications, however, may be necessary to insure compatibility with the list of images extracted from the association file. CALSTIS2 relies on more direct access of the science and data quality arrays through the use of lower-level CVOS routines, allowing this routine to be revised without relying on changes to HSTIO like the rest of CALACS. A revised version of the cosmic ray rejection task could then be developed and tested separately from the CALACS, using STIS data if necessary. This new task will be nearly identical to CALSTIS2 with the majority of the revisions taking place in the function ‘crrej_loop’. The processing will start by computing the initial comparison image from the list of images using line-by-line I/O, and will not need any modifications from CALSTIS2 at this point. This comparison image will have to be written out and the filename passed onto the function ‘crrej_loop’ for final rejection and creation of the summed output image. This routine will need to be re-written to work with only 1 output image and read in sections of the input image and the comparison image. Once cosmic-rays are detected in the input image section, they will be marked and added to output image in memory. This can be seen in Figure 3, where those elements kept in memory are drawn in bold lines and file I/O is indicated by arrows.

16

One additional change in the way ACSREJ will work involves the application of the shuttershading correction, previously done in ‘doShad’. This correction, while small for long exposure images, can be significant for very short exposures. CALSTIS2 does not apply this correction, since it is applied after CALSTIS2 completes processing the CR-SPLIT observations. The problem with that approach is that it improperly applies the shading correction to pixels that have had the values from one or more input images rejected for cosmic rays. It assumes that there have been N exposures and bases the correction on that number. To properly apply it after cosmic ray rejection would require keeping track of the number of input exposures contributing to each pixel, which is not available from the input data sets. The way that the shading correction will be applied as part of cosmic ray rejection in ACSREJ is described as follows: 1. the initial guess image is computed in the same manner as CALSTIS2. 2. each input image pixel is compared to the initial guess image by scaling the initial guess image to match the effective exposure times of the single image and the initial guess image. The effective exposure times include the shutter shading term correction, i.e., for a single exposure, effective exposure time = nominal exposure time (as expressed by the header keyword) + shutter shading correction for that pixel. The effective exposure time for the initial guess image is the total of the individual nominal exposure times + N times the shutter shading correction for that pixel where N is the number of exposures in the CRsplit set. Depending on whether or not the individual pixels value is within the acceptable range of the value derived from the initial guess image, it is marked as accepted or rejected. 3. If the pixel for that image is accepted, it is summed into the output array unscaled. The output array consists of a simple sum of input array values with no adjustments other than those applied before the CR rejection step. If the pixel is rejected, it is not summed into the output array. The mask for the input image is updated to indicate a value of 1 for accepted pixels and 0 for rejected pixels. (The format of the mask array will be described in more detail below) 4. when all input images have been processed, every pixel in the summed output array is scaled to reflect the proper number value corresponding to the total nominal exposure time (the same for all pixels). This is done by scaling the pixel by the factor

( Σ i ( T i + ∆t i ( x ,y ) )m i ( x ,y ) ) ⁄ T

where T i is the nominal exposure time for image i, ∆t i ( x ,y ) is the shading correction for the pixel x, y, and m i ( x ,y ) is the mask value for the i’th image at pixel x, y, and T is the total nominal exposure time for all input images (regardless of whether they were all used for that particular pixel). In short, the cosmic ray rejection step sums all the accepted values, computes the true exposure time for that pixel and scales the sum to correspond to the total nominal exposure time. A mask array will be used instead of an accumulated exposure time array throughout this task. This array will use 1 bit for every pixel requiring only 1MB per input image. In contrast, an accu-

17

mulated time array would require keeping another 32MB array in memory during processing, a requirement which was deemed excessive in addition to the 80Mb of memory already being used for the output image. The accumulated exposure time will be calculated from the mask arrays using the above equation after all input arrays are processed and will require much less memory if the shading correction can be represented as a highly subsampled image or in a functional form (e.g., a 2nd order polynomial). Figure 3: ACS Cosmic Ray Rejection Input Images

Comparison: Median/Min of Input Images

Identify Cosmic Rays

Written to Disk

Input Images

Summed Output

Written to Disk

In short, the calculations will be done on each image one section at a time, and the summed cosmic-ray corrected image will be built up in memory one section at a time as well, along with a bit-mask reflecting what images contributed to each pixel. This will minimize the amount of memory necessary for the computations, but the line-by-line I/O should not have a dramatic impact on the speed of the processing when done. Testing will be conducted to not only insure that this section-by-section approach identifies the same cosmic rays identified by CALSTIS2, but also to see what impact the line-by-line I/O has on the processing speed. ACSSUM Summary This routine stands as the probably the simplest task in CALACS, simply summing a list of images together one line at a time and updating 4 keywords to reflect this operation. In general, CALSTIS8 can be used with very little modification, primarily to allow the use of file list read in from the association file and to use line-by-line I/O for the input images, while deleting all code related to spectral orders. The function ‘SumGrps’ will need to be revised to loop over image files rather than extensions of one image, since STIS associations were always contained in the same image file. The first image on the list will be read into memory, just as it is done now in CALSTIS8. However, only enough memory for 1 line of data will be allocated for use by the remainder of the input images, then each image will be read in line-by-line using the new HSTIO functions. The function will then loop over each line in the input image and add it to the output image in memory. Once all the images are summed together, the output image will be written out as done in CALSTIS8 now and the task will be complete. Unlike CALSTIS8, however, we do not have images of different

18

sizes which will be summed together, like the different spectral orders. Therefore, all code related to spectral orders will be removed for ACSSUM.

6. Keyword Usage Processing of the images in CALACS will be controlled by the population of keywords and calibration switches in the input image headers, as it is in CALSTIS. Those keywords essential for CALACS remain a subset of those used by CALSTIS, with keywords related to spectroscopy and to binning of the images being absent. Table 4 provides a summary of those keywords which will be derived from CALSTIS for use by ACS, along with any that are unique to ACS. Table 4. ACS Keywords Used in CALACS Keyword

STIS Function(s)

I/O

Header

Sample of Possible Values

APERTURE

calstis1,calstis8

Input

Primary

WFC1,WFC2,WFC1-FIX, WFC2-FIX,WFC,HRC,SBC,...

ASN_MTYP

calstis1

Input

SCI

PRODUCT,REPEATOBS,CRSPLIT,NONE COMPLETE,OMIT,PERFORM,SKIPPED

ATODCORR

atodHistory

Input/Output

Primary

BADINPDQ

calstis2

Output

Primary

BIASCORR

do2D

Input/Output

Primary

BIAS_REJ

calstis1

Input

Primary

BINAXIS1,BINAXIS2

calstis0,do2d

Input

Primary

BLEVCORR

do2D

Input/Output

Primary

COMPLETE,OMIT,PERFORM,SKIPPED

CCDAMP

calstis1

Input

Primary,SCI

ABCD,ABC,ABD,AC,AD,ACD, BC,BD,BCD

CCDGAIN

calstis1

Input

Primary,SCI

1,2,4,8


CD1_1, CD2_2

do2D

Input

SCI,ERR,DQ

CENWAVE

photmode

Input

Primary

CRCORR

do2D,calstis2

Input/Output

Primary

CRPIX1,CRPIX2

calstis1

Input

SCI,ERR,DQ

CRRADIUS

calstis2

Output

Primary

CRSIGMAS

calstis2

Output

Primary

CRTHRESH

calstis2

Output

Primary

DARKCORR

do2D

Input/Output

Primary


DETECTOR

calstis1,calstis8

Input

Primary

WF1,WF2,HRC,SBC

DQICORR

do2D

Input/Output

Primary


EXPSCORR

calstis0

Input

Primary


EXPEND

calstis8

Input/Output

SCI

EXPSTART

do2D

Input

SCI,ERR,DQ

EXPTIME

do2D,calstis8

Input/Output

SCI,ERR,DQ


FILENAME

calstis8

Output

Primary

FLATCORR

calstis1

Input/Output

Primary


FLUXCORR

calstis8

Input

Primary


GLINCORR

calstis1

Input/Output

Primary


GLOBLIM

doNonLin

Output

SCI

EXCEEDED, NOT-EXCEEDED, NOTAPPLICABLE, UNDETERMINED

GLOBRATE

calstis1

Input

SCI

GOODMAX

do2D,calstis8

Output

SCI,ERR

GOODMIN

do2D,calstis8

Output

SCI,ERR

19

Keyword

STIS Function(s)

I/O

Header

GOODMEAN

do2D,calstis8

Output

SCI,ERR

INITGUES

calstis2

Output

Primary

LFLGCORR

calstis1

Input/Output

Primary

LTM1_1,LTM2_2

calstsi1

Input/Output

SCI,ERR,DQ

LTV1,LTV2

calstis1

Input/Output

SCI,ERR,DQ



MEANBLEV

do2D

Output

SCI

MEANDARK

do2D

Output

SCI

MEANEXP

calstis2

Output

Primary

NCOMBINE

calstis1

Input

SCI

NEXTEND

calstis1,calstis8

Input

Primary

NGOODPIX

do2D,calstis8

Output

SCI, ERR

NRPTEXP

calstis8

Input

Primary

OBSMODE

calstis1,calstis8

Input

Primary

ACCUM,ACQ,ACQ/PEAK, ALIGN,ANNEAL

PHOTCORR

do2D

Input/Output

Primary


PHOTFLAM

doPhot

Output

SCI

PHOTMODE

doPhot

Output

SCI

PHOTZPT

doPhot

Output

SCI

PHOTPLAM

doPhot

Output

SCI

PHOTBW

doPhot

Output

SCI

REJ_RATE

calstis2

Output

Primary

ROOTNAME

calstis1,calstis8

Input

Primary

RPTCORR

calstis8

Output

Primary

SCALENSE

calstis2

Output

Primary


SDQFLAGS

do2D

Input

Primary,SCI

SHADCORR

calstis1

Input/Output

Primary

SKYSUB

calstis2

Output

Primary

SKYSUM

calstis2

Output

Primary

SNRMAX

do2D,calstis8

Output

SCI

SNRMEAN

do2D,calstis8

Output

SCI

SNRMIN

do2D,calstis8

Output

SCI

SUBARRAY

calstis1

Input

Primary

TRUE,FALSE

STATFLAG

calstis0

Input

Primary

T,F

TEXPTIME

calstis2

Output

Primary


Unique to ACS ATODGN1, ATODGN2

do2D

Output

SCI

CCDOFST1, CCDOFST2

calstis1

Input

SCI

CDRKFILE

doDark

Input

Primary

Input

SCI,DQ,ERR

1,2

calstis1,calstis8

Input

Primary

CLEAR1L,F555W,F775W,F625W,F550M,F850 LP,CLEAR1S,POL0UV,POL60UV,...

N/A

Primary

doDark

Input

Primary

Input

Primary

Output

SCI

Input

Primary,ERR

CHIP FILTER1, FILTER2 GEOFILE MDRKFILE OSCNTAB READNSE1, READNSE2

do2D

WRTERR Reference Tables APERTAB

do2D

Input

Primary

ATODTAB

doAtoD

Input

Primary

20

Keyword

STIS Function(s)

I/O

Header

BIASFILE

doBias

Input

Primary

BPIXTAB

doDQI

Input

Primary

CCDTAB

do2D

Input

Primary

CRREJTAB

calstis2

Input

Primary

PFLTFILE

doFlat

Input

Primary

DFLTFILE

doFlat

Input

Primary

LFLTFILE

doFlat

Input

Primary

MLINTAB

do2D

Input

Primary

PHOTTAB

do2D

Input

Primary

SHADFILE

doShad

Input

Primary


7. HSTIO Development The line-by-line I/O and sectional output required by CALACS will be based on additional functions to HSTIO which have been implemented by Michele de la Peña. This section describes the changes necessary for these new capabilities. Since HSTIO is an existing entity, it was important in the design of these new functions to maintain consistency with the current implementation. In particular, this refers to the nomenclature, and modularity of the functions. For line-by-line I/O, the acquisition of a line closely (but not exactly!) parallels the acquisition of an entire image. Although in this specific implementation an IMSET consists of three components (SCI, ERR, and DQ), the design can be extended in the same manner as HSTIO currently accommodates both the STIS and NICMOS IMSETs. The new functions for line-by-line I/O will have the following properties: 1. The primary header (globalhdr) and the individual extension headers will only be loaded once. 2. As typical, the getSingleGroupLine structure should always be initialized before it is used. 3. One level of indirection has been removed as compared to getting an entire image. 4. A corresponding “put” line high-level routine will be written to output a single line to a single image. Until there seems to be a real need to output single lines to groups of images, a suite of “put” routines will be considered low priority and will only be implemented as time permits. Three new data structures are needed: SingleGroupLine, FloatHdrLine, ShortHdrLine. These new structures will be nearly identical to the current SingleGroup, FloatHdr and ShortHdr structures within HSTIO, with the addition of a flag to tell whether the header has already been loaded or not, and the line number. These new structures will be used by the following new functions for implementing the actual I/O: High-Level (public) void initSingleGroupLine (SingleGroupLine *x);

21

int openSingleGroupLine (char *filename, int extver, SingleGroupLine *x); int getSingleGroupLine (char *filename, int extver, int line, SingleGroupLine *x); void closeSingleGroupLine (SingleGroupLine *x); void freeSingleGroupLine (SingleGroupLine *x);

In addition, many new low-level functions were developed to support these cuntions.All other routines (open, close, acquiring header information, getFloatLine/getShortLine, etc.) are already available in HSTIO. The new routines could be used as shown in the following semi-pseudocode. This pseudocode example is to be taken as an illustration of how a line could be obtained from an IMSET to fill an awaiting buffer. Once the routines are implemented, tested examples will be provided as illustrations of how the routines should be used. initSingleGroupLine() { Initializes data structure SingleGroupLine members initFloatHdrLine() Initializes the data structure SciHdrLine members initFloatHdrLine() Initializes the data structure ErrHdrLine members initShortHdrLine() Initializes the data structure DQHdrLine members } openSingleGroupLine() { This routine incorporates much functionality in setting up to acquire image lines. After this function is called, the programmer has access to the length of the data lines so a buffer can now be allocated dynamically to accommodate the data lines. Internally, the length has been determined by NAXIS1 (for images with true size) or by NPIX1 (for constant images). openInputImage() Opens the primary and fills the IODesc structure allocSingleGroupLine() allocate space for the global (primary) header allocFloatHdrLine (SCI) allocFloatLine() allocHdr() allocFloatHdrLine() (ERR) allocFloatLine() allocHdr() allocFloatHdrLine() (DQ) allocShortLine() allocHdr() getHeader() Obtain the primary header information closeImage() Closes the primary getSciHdr() == getFloatHdr()

22

openInputImage() Opens the SCI extension getHeader() Obtains the SCI header getErrHdr() == getFloatHdr() openInputImage() Opens the ERR extension getHeader() Obtains the ERR header getDQHdr() == getShortHdr() openInputImage() Opens the DQ extension getHeader() Obtains the DQ header allocSciLine() == allocFloatLine() Allocates the proper amount of memory for SCI line allocErrLine() == allocFloatLine() Allocates the proper amount of memory for ERR line allocDQLine() == allocShortLine() Allocates the proper amount of memory for DQ line } At this stage, the SCI, ERR, and DQ extensions have not been closed. getSingleGroupLine() { Obtain the requested data line from the IMSET getSciLine() == getFloatLine() Gets a line from the SCI extension, length based upon NAXIS1 getErrLine() == getFloatLine() Gets a line from the ERR extension, length based upon NPIX1 getDQLine() == getShortLine() Gets a line from the DQ extension, length based upon NPIX1 } closeSingleGroupLine() Closes the SCI, ERR, and DQ image extensions. freeSingleGroupLine() Frees all allocated memory for the data structure

A new public routine has been added to the HSTIO library which allows one to output to a new file a subsection of an image in memory. This subsection from memory becomes the full size of the output image. The new function for supporting this output is: int putSectSingleGroup (char *, int, SingleGroup *, int, int, int, int, int);

In addition, there are several low-level routines which support this function. This one function can be called by itself in order to handle the output of an IMSET section.

23

8. Development Schedule Although based heavily on a fully tested software package, this new pipeline will require considerable effort to implement the design changes described in this report. A tentative time line for this development, with time set aside for adequate testing, can be seen in Table 5. This schedule breaks down the development of the software into components which can be finished independently and then tested before adding more complexity to the product. This has the benefit of finding problems early in the development stage when they are easier to identify and to fix, so that by the time all the software is complete, the majority of problems will already have been addressed. The final testing phase should consist little more than verification that all the parts work together as expected, since all the individual parts should already have been debugged. Finally, time has been implicitly provided for any changes in design that may become necessary as we learn more about how to share code with STIS and about the data from the instrument itself. Table 5. Proposed Development Time Line Develop Time Develop HSTIO Line-by-Line I/O Functions

Complete Date 1 May 1998

Write CALACS and ACSCCD tasks

4 weeks

1 June 1998

Test CALACS and ACSCCD tasks

1 week

8 June 1998

Write ACS2D

4 weeks

6 July 1998

Test ACS2D with CALACS

2 weeks

20 July 1998

Revise CALSTIS2 into ACSREJ

4 weeks

24 Aug 1998

Test ACSREJ

2 weeks

11 Sept 1998

Revise CALSTIS8 into ACSSUM

2 weeks

25 Sept 1998

Test Full CALACS Package

2 weeks

9 Oct 1998

Revise Full CALACS package based on testing

2 weeks

23 Oct 1998

Support File Requirements for Development The testing schedule provides the basis for specifying what files need to be delivered at what time. The first phase of CALACS development focuses on building the CALACS task itself for controlling the operation of the package and on revising CALSTIS1 code to create ACSCCD. This will only require the following reference files by 1 June 1998: Tables similar to STIS files: ATODTAB BPIXTAB

24

CCDTAB New for ACS: OSCNTAB

provides details on overscan regions for Bias-level subtraction and trimming

Reference File: BIASFILE The second phase for CALACS involves building the ACS2D task, containing the majority of the basic reduction steps for both CCD and MAMAs. This will require the following reference files by 6 July 1998 for testing: MLINTAB - MAMA linearity tables Darks (MDRKFILE, CDRKFILE) Flat-fields (LFLTFILE, PFLTFILE, DFLTFILE) SHADFILE PHOTTAB APERTAB In addition, some test MAMA images would be necessary for testing of the MAMA specific routines.The next phase will be the building of the cosmic-ray rejection task, ACSREJ. The following reference file will be needed by 24 Aug 1998 for testing: CRREJTAB Some CR-SPLIT CCD images would be necessary for this testing as well. The format of the reference tables, rather than the specific values in them, will be critical to insure that CALACS can properly read and interpret the contents of each reference table. Those tables which have STIS equivalents could simply be STIS tables with headers changed to reflect ACS specific information. The actual values in the tables are expected to be volatile, and are not critical to the initial testing. The ACS CCD images in FITS format needed to conduct the basic testing will most likely need to be hand-tailored by to include all the necessary keywords and proper values for CALACS in their headers, both the primary and extension headers.

9. Summary The development of CALACS can be described as a simplified version of CALSTIS, with different memory usage and image I/O. Initial work on CALACS would start with a complete copy of the necessary CALSTIS tasks; namely, CALSTIS0, CALSTIS1, CALSTIS2, and CALSTIS8. Unnecessary code related to spectroscopy would then be removed, and variable definitions would be trimmed to fit the needs of ACS. At this point, the line-by-line I/O and association handling

25

would be added to produce the final product. This development will take place on a snapshot of the CALSTIS software and evolve separately to support ACS without any concern for sharing functions or tasks with STIS. The overall design of CALSTIS makes this development efficient and easy to implement. CALACS will not just build on this modularity, but also provide a basis for efficiently implementing the desired enhancements in the future. In addition, only minimal enhancements to HSTIO have been found to be required to support this calibration software for ACS. Therefore, CALACS should be easier to develop than previous calibration software, and no more difficult to maintain or enhance in the future.

26

Appendix A: CALSTIS Structures and Variables This section describes the primary structures and variables used within CALSTIS. These structures are used primarily to determine what processing needs to be performed, and to keep track of the results of the processing so that the image headers can be updated properly. Throughout CALSTIS, similar structure names are used for the structure containing the relevant header keywords: StisInfo in CALSTIS0, StisInfo1 in CALSTIS1, and StisInfo8 in CALSTIS8. CALSTIS0 Structure CALSTIS0 acts as a primary switching station for the observations, sending the data to the tasks for processing appropriate for the observation type. The structure StisInfo contains all the necessary information for making these decisions, as shown in Table 6. Table 6. Definition of Structure StisInfo. Variable

Description of variable input, outroot, and temporary file names

char rawfile[STIS_LINE];

uncalibrated science data

char outroot[STIS_LINE];

root name for output

char wavfile[STIS_LINE];

input wavecal file

char crjfile[STIS_LINE];

CR rejected, flat fielded science

char fltfile[STIS_LINE];

flat fielded science

char x1dfile[STIS_LINE];

extracted 1-D spectrum

char x2dfile[STIS_LINE];

rectified 2-D spectrum

char sx2file[STIS_LINE];

summed, rectified 2-D

char sx1file[STIS_LINE];

summed, extracted 1-D

char sflfile[STIS_LINE];

summed, not rectified 2-D

char blv_tmp[STIS_LINE];

blevcorr, then CR flagged

char crj_tmp[STIS_LINE];

CR rejected, summed

char fwv_tmp[STIS_LINE];

flat fielded wavecal

char cwv_tmp[STIS_LINE];

FF, source-subtracted wavecal

char w2d_tmp[STIS_LINE];

2-D extracted wavecal

char rootname[STIS_CBUF];

root name for set of obs

info about input science or wavecal file int detector;

integer code for detector

27

Variable

Description of variable

int obstype;

spectroscopic or imaging

int nimages;

number of “groups” in file Info for comparing the binning and gain of the science file and wavecal. If they’re the same (only relevant for the CCD), the samebin flag should be set to one.

int scibin[2], wavbin[2];

binning factors

int scigain, wavgain;

ccdgain values

int samebin; calibration switches int sci_basic_2d_a;

do calstis1a? (dqicorr or blevcorr)

int sci_basic_2d;

do calstis1 for science file?

int sci_expscorr;

do calstis1 on CR flagged but non-summed?

int sci_crcorr;

do cosmic-ray rejection for science file?

int sci_rptcorr;

combine repeatobs science data?

int sci_wavecorr;

do calstis11, calstis4, and calstis12?

int sci_2d_rect;

do calstis7 for science file?

int sci_1d_extract;

do calstis6?

int sci_geocorr;

do 2-D image rectification?

int wav_basic_2d;

do calstis1 for wavecal?

int wav_subscicorr;

do calstis11? (not a header switch)

CALSTIS1 Structure CALSTIS1 controls a large number of processing steps and requires a great deal of information from the image header to manage that calibration steps. This results in a structure which contains more keywords and header information than the similar structure in CALSTIS0. The definition of this structure is given in Table 7. In addition to the keyword values, this structure also contains specific information about the format of the observation, information such as binning amount, whether the observation is a subarray, and so on. Since most functions require a large amount of input information, this structure provides a simple way to pass all this information around.

28

Table 7. Definition of CALSTIS1 Structure StisInfo1. Variable

Description of Variable input and output image names

char input[STIS_LINE];

input image to be calibrated

char output[STIS_LINE];

output calibrated image

char outblev[STIS_LINE];

output text file for bias levels

char rootname[STIS_CBUF];

root name for set of obs

char assoc_typ[STIS_CBUF];

ACCUM, or WAVE for a wavecal command-line flags

int printtime;

print time after each step?

int verbose;

print additional info? keywords and file names for reference files

RefFileInfo *refnames; info about input image char det[STIS_CBUF];

name of detector

char aperture[STIS_CBUF];

aperture name

char opt_elem[STIS_CBUF];

name of grating or mirror

char obsmode[STIS_CBUF];

e.g. ACCUM or TIMETAG

int detector;

integer code for detector

int ncombine;

number previously summed together

int nimages;

number of “groups” in file

double exptime;

exposure time

short sdqflags;

serious data quality values coordinate info

int subarray;

is current image a subarray?

int bin[2];

size of pixel in detector coordinates

int dispaxis;

dispersion axis, 1 ==> X, 2 ==> Y

int dispsign;

sign of CD1_1 (dispersion), +1 or -1 MAMA-specific info (regarding linearity)

double global_limit;

count rate for 10% global nonlinearity

double tau;

time constant for global nonlinearity

29

Variable

Description of Variable

double local_limit;

count rate for 10% local nonlinearity

float expand;

radius for flagging local nonlinearity MAMA-specific info (regarding Doppler shift)

double globrate;

global count rate

double expstart;

exposure start time (MJD)

double doppzero;

Doppler shift zero phase time (MJD)

double doppmag;

Doppler shift magnitude (high-res pixels)

double orbitper;

Assumed HST orbital period (seconds) CCD-specific info

float bias_rej;

number of sigma for rejecting outliers

char ccdamp[STIS_CBUF];

CCD amplifier read out (A,B,C,D)

int ccdgain;

commanded gain of CCD

int ccdoffset;

commanded offset

int binaxis[2];

BINAXIS1, BINAXIS2 from header

float atodgain;

actual gain of CCD

float ccdbias;

CCD bias offset number

float readnoise;

readout noise

float saturate;

CCD saturation level calibration flags (switches)

int doppcorr;

Doppler convolution needed (flats, etc)?

int lorscorr;

convert MAMA data to low-res

int dqicorr;

data quality initialization

int atodcorr;

analog to digital correction

int blevcorr;

subtract bias from overscan

int biascorr;

subtract bias image

int glincorr;

global nonlinearity correction

int lflgcorr;

flag local nonlinearity

int darkcorr;

subtract dark image

int flatcorr;

apply flat field

int pfltcorr;

apply pixel-to-pixel flat

30

Variable

Description of Variable

int dfltcorr;

apply delta flat

int lfltcorr;

apply low-order flat

int shadcorr;

correct short exposures for shutter time

int photcorr;

add photometry header keyword values

int filtcorr;

include filter throughput filtcorr is not a switch; it’s associated with photcorr

int noisecorr;

initialize error array? (yes)

int statcorr;

compute statistics? (yes) calibration images and tables

RefImage bias;

bias image

RefImage dark;

dark image

RefImage pflt;

pixel-to-pixel flat field

RefImage dflt;

delta flat

RefImage lflt;

low-order flat

RefImage shad;

shutter shading correction image

RefTab bpix;

bad pixel table

RefTab ccdpar;

CCD parameters table

RefTab mlin;

MAMA nonlinearity info table

RefTab atod;

analog to digital correction table

RefTab phot;

photometric throughput table

RefTab apertab;

aperture (filter) throughput table

31