1. Fiorentino & Rosa Wavelength calibration in physical model based calibration pipelines. Astronomical Data Analysis III S. Agata sui due Golfi, Naples, April 2004

2. Fiorentino & Rosa04/30/04ADA III - Napoli Overview IPMG at ST-ECF - Who we are. HST Spectrographs & traditional pipelines. Predictive calibration based on: 1.physical model of the instrument 2.simulated annealing technique for optimization 1.Show how we implement this into the science data pipeline.

3. Fiorentino & Rosa04/30/04ADA III - Napoli IPMG at ST-ECF Comprehensive empirical calibration pipeline already exists for the HST STIS Spectrograph We aim to improve those components which benefit from physically motivated corrections Current work includes 1.Wavelength Calibration 2.Calibration lamp line list - measurements at NIST 3.Detector Model repairing the Charge Transfer (CTE)

4. Fiorentino & Rosa04/30/04ADA III - Napoli What is STIS ? STIS is the HST imaging spectrograph. – spatially resolved spectroscopy from 1150 Å to 10,300 Å at low to medium spectral resolution – echelle spectroscopy (high resolution) in the ultraviolet. – time tagging of photons in the ultraviolet (high time resolution). Since 1997 on board HST Unlikely to be replaced during the remaining HST lifetime

5. Fiorentino & Rosa04/30/04ADA III - Napoli STIS optical layout

6. Fiorentino & Rosa04/30/04ADA III - Napoli STIS Pipeline ‘calstis’ calstis for spectra - series of modules that 1.Control the data flow through the pipeline 2.Basic 2-D image reduction (e.g. bias subtraction) 3.Reject cosmic rays from CCD data 4.Process the contemporaneously obtained wavecal data to ascertain zero point shifts in the spectral and spatial directions 5.Extract 1 dimensional spectra – need to know geometry 6.Perform spectroscopic wavelength and flux calibration 7.Sum any CR-SPLIT and REPEATOBS exposures.

7. Fiorentino & Rosa04/30/04ADA III - Napoli Pipeline Flow for Spectroscopic Data

8. Fiorentino & Rosa04/30/04ADA III - Napoli Where the empirical wavelength calibration is currently used. Determine MSM offset from wavecal. Its purpose is to find the offset of the spectrum from the expected location, owing to non repeatability of the MSM. Spectroscopic Calibration and Extraction. 1-D spectral extraction. A spectrum is extracted along a narrow band, summing over the cross-dispersion direction and subtracting background values to produce a 1-D array of fluxes for each spectral order. In order to calculate the offsets and to assign wavelengths the empirical pipeline uses bi-dimensional polynomial dispersion solutions. Therefore it can only apply linear translations (offsets), but not rotations.

9. Fiorentino & Rosa04/30/04ADA III - Napoli STIS Auto Wavecals A standard wavecal is usually only a few seconds long. X and Y displacements based on a few lines. X and Y are not the same on the whole detector because, the differential rotation (“splaying‘”) of individual echelle orders resulting from the combined effects of the echelle and cross-dispersing elements, cause different orders to be differentially rotated (“splayed”).

10. Fiorentino & Rosa04/30/04ADA III - Napoli Short and Long Wavecal

11. Fiorentino & Rosa04/30/04ADA III - Napoli Short and long wavecal (detail)

12. Fiorentino & Rosa04/30/04ADA III - Napoli Traditional Pipeline’s accuracy vs. Enhanced calibration. 1)Image shift (-3,+3) pixels due to the MSM. 2)Thermal effects cause the spectrum to drift by about 0.1 pixels up to 0.35 pixels per orbit. 3)Shift not always precisely determined due to, for instance, a short wavecal. 1)The Absolute Wavelength zero points shifts are not predicted with the traditional calibration (errors in E140H up to 1.3km/s 0.5-1.0 Pixel). We aim to reach 0.1 pixel precision. 2)We will have an homogenous calibration for each mode and overall the lifetime of STIS.

13. Fiorentino & Rosa04/30/04ADA III - Napoli The alternative: predictive calibration “The calibration of astronomical data can be significantly improved by constructing instrument models which incorporate as fully as possible a knowledge of optical and detector physics” A typical example is the wavelength calibration – empirical dispersion relations should be replaced by a physical model (simple ray trace) of the spectrograph This usually yields better than 0.1 % accuracy (1 pix in 1000) straight away Distortions may be added to go to sub-pixel accuracy

14. Fiorentino & Rosa04/30/04ADA III - Napoli Predictive Calibration: Echelle model & Simulated Annealing. Mathematical model with about 35 parameters which need to be optimized. Derivatives cannot be easily formulated and analytical inversion is impossible. Simulated Annealing (SA) is one of the technique which cope with such a problem. Although easy in principle, its implementation may not be trivial.

15. Fiorentino & Rosa04/30/04ADA III - Napoli Simulated Annealing. SA exploits an analogy between the way in which a metal cools and freezes into a minimum energy crystalline structure and the search for a minimum in a more general system. SA don’t get trapped at local minima. The algorithm accepts also changes that increase objective function f with a probability following the Boltzmann probability distribution. Not all sets of parameters which minimize the cost function are physically acceptable therefore our SA algorithm will make those configurations extremely costly.

16. ADA III - Napoli Randomize according to the T Better than the current solution ? Start Store it. Yes Accept or reject based on Boltzmann Probability Distribution. NO Random tries > Max_Iteration ? No Decrease Temperature Min Temperature reached ? Yes Exit No SA Data Flow

17. Fiorentino & Rosa04/30/04ADA III - Napoli Fitlines Mode,Slit,Central wavelength, Catalog lines Wavecal (FITS Image) Calculates the predicted X,Y lines positions on the detector. Output: observed X,Y lines positions on the detector. Center a box around the predicted lines positions and estimate the exact observed positions by fitting the line shape with a Gaussoid.

18. Fiorentino & Rosa04/30/04ADA III - Napoli STIS Anneal Annealing Algorithm to optimize the set of parameters for each configuration. repeat the anneling process. If config file is good store it If not good One or more sets of lines positions from Fitlines. One or more configuration file each with 35 parameters

19. ADA III - Napoli Master Catalog Mode, CenWave, SlitPos, Config File What else? ExtractSubCatalog Fitlines SubCatalog Wavecal exposure STISAnneal (X,Y)measured Store it or not Store it ? New Cfg File If good store it Not Yet good ? New Master ? If not good Reject Bad lines, Change Weights, Intensity thresholds Not possible to anneal all parameters at the same time therefore needs to identify set of them to be annealed. Learning curve for a new instrument. Reference Files Data Flow

20. Fiorentino & Rosa04/30/04ADA III - Napoli SA into the Science pipeline Once all the reference files have been determined we will be able to predict, for a given configuration and for each order and lambda, the position on of the corresponding line on the detector. However, in order to cope with the non repeatibility of the MSM, another SA need to be run each time a science exposure is taken.

21. Fiorentino & Rosa04/30/04ADA III - Napoli SA into the Science Pipeline Final set of config files for each mode, Central wavelength, slit, Epoch. User selects a mode, Central wavelength … Config File extracted but MSM positions may no longer be accurate. Run Fitlines + Fast-Anneal in order to calculate the the actual MSM position Run Calstis

22. Fiorentino & Rosa04/30/04ADA III - Napoli Discover Dependencies Run Fitlines +Anneal Select a mode and fetch all the wavecals. N Wavecals extracted. N Config files Analyze config files against environmental conditions. Relation T, Focal Length ? Enhance the model Number of config files reduced.

23. Fiorentino & Rosa04/30/04ADA III - Napoli Modeling Echelle Spectrographs At the ST-ECF we are currently implementing a STIS model based on first optical principles. It incorporates off-plane grating equations and 3D rotations in order to account for line tilt and order curvature. Similar formalism had already been partially implemented and applied for FOS(HST), UVES, CASPEC pipelines with significant science improvement. See Ballester and Rosa A&AS 126, 563-571 (1997). www.stecf.org/poa/pcrel/scicase.html www.eso.org/observing/dfo/quality/Messenger/UVES_Messenger_101.html

24. Fiorentino & Rosa04/26/04ADA III - Napoli Good only for HST spectrographs ? Predictive calibration can be applied to any spectrograph. We aim to implement the STIS pipeline such that can be easily re-used for other spectrograph (i.e. Object oriented code). Although this is just a part of a pipeline…

25. Fiorentino & Rosa04/30/04ADA III - Napoli Status of the STIS implementation Prototype implementation finished (C++). Wavelength calibration translated into C in order to import into the existing IRAF/C STIS pipeline. Reference files production is in C++ and does not need to be translated since it is an offline tools. Future items: Analyze science cases in order to test the CE_CALSTIS. Enhance the model (MSM model).

26. Fiorentino & Rosa04/30/04ADA III - Napoli Referenced articles & URL links Ballester and Rosa Astron. & Astrophysic. Suppl.Ser 126, 563-571 (1997). Ballester & Rosa ADASS XIII, Instrument Modeling in Observational Astronomy. Kirkpatrick, S., C. D. Gelatt Jr., M. P. Vecchi, "Optimization by Simulated Annealing",Science, 220, 4598, 671-680, 1983. Metropolis,N., A. Rosenbluth, M. Rosenbluth, A. Teller, E. Teller, "Equation of State Calculations by Fast Computing Machines", J. Chem. Phys.,21, 6, 1087-1092, 1953. URL links: www.stecf.org/poa/pcrel/scicase.html www.stecf.org/poa/index2.html www.eso.org/observing/dfo/quality/Messenger/UVES_Messenger_101.html

27. Fiorentino & Rosa04/30/04ADA III - Napoli Science Improved: FOS case. Effect of the improved dispersion relation. We looked at the interstellar absorption lines imprinted on the spectrum of a low red-shift quasar (PG 1115+407, PI B. Wills). There were two separate FOS observations red and black dots. All measurements have been reduced to barycentric velocities. The solid line is the weighted average of HI 21 cm line observations with the dashed lines indicating the range of velocities found in the line of sight.

28. Fiorentino & Rosa04/30/04ADA III - Napoli Standard Calfos dispersion solution

29. Fiorentino & Rosa04/30/04ADA III - Napoli Improved dispersion solution.

30. Fiorentino & Rosa04/30/04ADA III - Napoli STIS Spectroscopic Capabilities

31. Fiorentino & Rosa04/30/04ADA III - Napoli Traditional Pipeline’s accuracy vs. enhanced calibration. Image shift (-3,+3) pixels due to the MSM. Thermal effects cause the spectrum to drift of about 0.1 pixels up to 0.35 pixels per orbit. Shift not always precisely determined due to, for instance, a short wavecal.