JWST Pipeline/Analysis Tools Perry Greenfield Science Software Branch

Running the JWST pipeline Uses a python framework (stpipe) to connect processing steps into a pipeline. Framework handles: – Command line interface – Configuration files – Logging and error handling – Input and output One can run a pipeline – From the shell command line Directly or by invoking a configuration file – Interactive Python prompt Multiple ways to customize: – Command line options to stop after a specified step – And restart at a step – Insert one’s own shell level command inside a pipeline (i.e., non-Python) – Define one’s own Python step and insert it, or modify existing ones

A (very) Simple Step from jwst_lib.stpipe import Step, Cmdline from jwst_lib import models class FlatFieldStep(Step): reference_file_types = [‘flat’] def process(self, input): with models.ImageModel(input) as im: self.flat = self.get_reference_file(im, ‘flat’).name flat_model = models.FlatModel(self.flat) result = flat_field.correct(im, flat_model) return result

Executing a Step > strun jwst_pipeline.FlatFieldStep myimage.fits From the command line: > strun flat_field.cfg myimage.fits or From Python: >>> from jwst_pipeline import FlatFieldStep >>> result = FlatFieldStep.call(‘myimage.fits’, config_file=‘flat_field.cfg’) Configuration (.cfg) files use ini-file format and stpipe uses the ConfigObj library to parse them

A (very) Simple Pipeline from jwst_lib.stpipe import Pipeline from jwst_pipeline.bias import bias_step from jwst_pipeline.dark import dark_step from jwst_pipeline.reset import reset_step class SimplePipeline(Pipeline): step_defs = {“bias”: bias_step.SuperBiasStep, “dark”: dark_step.DarkCurrentStep, “reset”: reset_step.ResetStep} def process(self, input): input = self.bias(input) input = self.dark(input) if input.meta.instrument.name == “MIRI”: input = self.reset(input) return input

Pipeline Config file name = "SloperPipeline" class = "jwst_pipeline.pipeline.SloperPipeline" save_calibrated_ramp = True [steps] [[dq_init]] config_file = dq_init.cfg [[saturation]] config_file = saturation.cfg [[ipc]] config_file = ipc.cfg [[superbias]] config_file = superbias.cfg [[bias_drift]] config_file = bias_drift.cfg [[reset]] config_file = reset.cfg [[lastframe]] config_file = lastframe.cfg [[linearity]] config_file = linearity.cfg [[dark_current]] config_file = dark_current.cfg [[jump]] config_file = jump.cfg [[ramp_fit]] config_file = ramp_fit.cfg

Step Config File name = "jump" class = "jwst_pipeline.jump.JumpStep" rejection_threshold = 5.0 do_yintercept = False yint_threshold = 1.0

Calibration Reference Data System Figures out what reference files to use for data set JWST pipelines automatically will download rules and any needed reference files to local computer when run remotely CRDS allows specifying different rules configuration than current operation system is using Rules can be locally customized to use user- provided reference files

Tools being developed for JWST Priorities driven by JWST Data Analysis Development Forum – Organized by Harry Ferguson and Susan Kassin Currently time series tools not high on the list of priorities – If you want to change that, you should get involved (even if outside STScI) Pipeline needs may impact data analysis resources.

Sprint-oriented Framework 1-d interactive spectral analysis Image utilities (imarith, imstat, etc.) Non-interactive fitting of 1-d spectra Signal processing: filtering and interpolation Tools for cubes I Tools for cubes II MOS visualization, data-quality inspection 2-d visualization Release preparations for imutils & imexam Parameter files PSF and PSF-kernel tools I JWST data file registry, I/O, and associations 1D spectral tools II (and initial release preparations) Photometry on non-resampled data Background estimation toolbox & tutorials (Spectroscopic and imaging) Image & spectral geometry, registration & resampling I Synthetic photometry MOS tools II (and initial release preparations) JWST Ramp inspection and analysis tools PSF/LSF and PSF/LSF-kernel tools II Visualization of resampled and non-resampled data Spectral fitting on non-resampled data Cube tools III (and initial release preparations) Astrometry on non-resampled data Improved source extraction tools (remaining SExtractor-like functionality, particularly overlapping source segmentation) Image & spectral geometry, registration & resampling II 2D/3D spectral data combination Tools to go from quick-look to publication-quality plots Grism analysis and visualization tools Optimal spectral extraction tools (optimal S/N extraction) Coronagraphy tools to augment pipeline tools NRM analysis tools Time-series tools (e.g., transit analysis)

Astropy Community based astronomical core library for Python Started in 2011 >140 code contributors to date

Astropy Core Works with both Python 2 and 3 Regression tests Documentation (sphinx) Hosted on github Goal is to foster better interoperability of software tools written in Python

Astropy contents Units, quantities, physical constants Tables I/O (FITS, text tables, VOTables) Time and Dates Astronomical Coordinate Systems Generalized World Coordinate Systems Much more flexible than FITS WCS Models and Fitting Analytic functions Convolution and filtering Cosmological calculations Astrostatistics VO cone search

Affiliated packages Use astropy core libraries Same python coverage, tests, documentation Selected examples: – APLpy (image + plotting) – Ginga (image viewer) – Astroquery – Photutils – Sncosmo – Spectral-cube – WCSAxes