1. NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 PACS page 1 SPIRE Imaging Fourier Transform Spectrometer (FTS) Pipeline Data Processing Nanyao Lu (NHSC/IPAC)

2. PACS page 2 Nanyao Lu NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 List of Topics Overview of SPIRE (FTS) Spectrometer Overview of the FTS Pipeline

3. PACS page 3 Nanyao Lu NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 SPIRE Spectrometer Fourier Transform Spectrometer (FTS): The entire spectral coverage of 194-671 micron is observed in one go! (SMEC) (194-313 um) (303-671 um)

4. PACS page 4 Nanyao Lu NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 Two Bolometer Detector Arrays 194 – 313 microns 303 – 671 microns Beam = 17”- 21” Beam = 29”- 42”

5. PACS page 5 Nanyao Lu NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 Observing Modes Telescope Pointing Single Pointing Raster Spatial Sampling Sparse (2 beam spacing) Intermediate (1 beam spacing) Full (1/2 beam spacing; Nyquist) Spectral Resolution High: 0.04 cm -1 (1.2 GHz), R=1290 – 370, e.g., line fluxes. Intermediate: 0.24 cm -1 (7.2 GHz), R = 210– 60. Low: 0.83 cm -1 (25 GHz), R = 62 – 18, e.g., dust continuum. High + Low: Both High and Low scans. Note: Data sampling at 25μm in OPD; Nyquist wave num. = 200 cm -1 Spectral resolution depends on the scan length

6. PACS page 6 Nanyao Lu NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 From Interferogram to Spectrum Interferogram Optical path difference (cm) Signal (volts) Fourier Transform Source Spectrum

7. PACS page 7 Nanyao Lu NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 List of Topics Overview of SPIRE FTS Spectrometer Overview of the FTS Pipeline

8. PACS page 8 Nanyao Lu NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 Level 1 products: unmodified interfergrams average spectrum (apodized) average spectrum (unapodized) Interferograms (stored in Level 1) Level 0.5 products: detector time lines scan mirror time line house keeping time lines Spectrometer Pipeline Data Flow SPIRE Common Pipeline 1. Modify Detector Timelines 2. Create Interferogram 3. Modify Interferogram 4. Fourier Transform 5. Modify Spectra ( V → Jy) 6. Spectral Mapping Level 2 product: Spectral Cubes (still under development)

9. PACS page 9 Nanyao Lu NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 1 st Level Deglitching Remove Electrical Crosstalk Clipping Correction Time-domain Phase Correction Bath Temperature Correction Cross talk matrix V(t) Step 1: Modify Timelines V(t) Level 0.5 Timelines Modified Level 0.5 Timelines Non-linearity Correction V(t) Bolometer Nonlinearity Table Bath temp. corr. product Time constants

10. PACS page 10 Nanyao Lu NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 1 st Level Deglitching Remove Electrical Crosstalk Clipping Correction Time-domain Phase Correction Bath Temperature Correction Cross talk matrix V(t) Step 1: Modify Timelines V(t) Modified Level 0.5 Timelines Non-linearity Correction V(t) Bolometer Nonlinearity Table Bath temp. corr. product Time constants Clipping Correction

11. PACS page 11 Nanyao Lu NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 1 st Level Deglitching Remove Electrical Crosstalk Clipping Correction Time-domain Phase Correction Bath Temperature Correction Cross talk matrix V(t) Step 1: Modify Timelines V(t) Level 0.5 Timelines Modified Level 0.5 Timelines Non-linearity Correction V(t) Bolometer nonlinearity table Bath temp. correction table Time constants

12. PACS page 12 Nanyao Lu NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 Create Interferograms Once time domain processing is complete, the detector signals and SMEC positions can be merged to create interferograms. The created “unmodified” interferograms are also stored in Level 1 in case users want to do their own interferogram-to- spectrum process. V(t) Step 2: Create Interferograms Level 0.5 Timelines V(t) Unmodified Interferograms V(x) (Stored in Level 1) SMEC Positions x(t') Pointing P(t'')

13. PACS page 13 Nanyao Lu NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 Create Interferograms Once time domain processing is complete, the detector signals and SMEC positions can be merged to create interferograms. The created “unmodified” interferograms are also stored in Level 1 in case users want to do their own interferogram-to- spectrum process. V(t) Step 2: Create Interferograms Level 0.5 Timelines V(t) Unmodified Interferograms V(x) (Stored in Level 1) SMEC Positions x(t') Pointing P(t'')

14. PACS page 14 Nanyao Lu NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 Telescope/SCAL/Beamsplitter Correction Baseline Removal 2 nd Level Deglitching Phase Correction (Default apodization) V(x) Step 3: Modify Interferograms V(x) (Level 1) Interferograms Modified Interferogram Products (both unapodized and apodized) Reference background interferogram Nonlinear phase calibration table Norton Beer Order-1.5 function

15. PACS page 15 Nanyao Lu NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 Telescope/SCAL/Beamsplitter Correction Baseline Correction 2 nd Level Deglitching Phase Correction (Default) Apodization V(x) Step 3: Modify Interferograms V(x) (Level 1) Interferograms Modified Interferogram Products Reference background interferogram Nonlinear phase Norton Beer Order-1.5 function

16. PACS page 16 Nanyao Lu NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 Fourier Transform Apply the Fourier Transform to each interferogram to create a set of spectra for each spectrometer detector. Step 4: Transform Interferograms Spectra V(σ) Modified Interferograms V(x)

17. PACS page 17 Nanyao Lu NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 Fourier Transform Apply the Fourier Transform to each interferogram to create a set of spectra for each spectrometer detector. Step 4: Transform Interferograms Spectra V(σ) Modified Interferograms V(x)

18. PACS page 18 Nanyao Lu NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 Spectral Averaging Flux Conversion: V->Jy Remove Optical Crosstalk V(σ) Step 5: Modify Spectra I(σ) Spectra Level 1 Spectrum Products Extended-source case volt-to-Jy factors (both unapodized and apodized) Detector optical crosstalk matrix Spectra are all in extended-source calibration at Level 1.

19. PACS page 19 Nanyao Lu NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 Spectral Averaging Flux Conversion: V->Jy Remove Optical Crosstalk V(σ) Galaxy IC 342: SLW Channel Spectra I(σ) Spectra Level 1 Spectrum Products Point-source case volt-to-Jy factors Detector optical crosstalk matrix

20. PACS page 20 Nanyao Lu NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 Spectral Averaging Flux Conversion: V->Jy Remove Optical Crosstalk V(σ) Galaxy IC 342: SSW Channel Spectra I(σ) Spectra Level 1 Spectrum Products Point-source case volt-to-Jy factors Detector optical crosstalk matrix

21. PACS page 21 Nanyao Lu NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 Spatial Regridding V(t) Step 6: Spatial Regridding (Level 1 to 2) Level 1 Spectra I(σ) Level 2 Spectral Cube I(σ) (Under development) Level 1 Spectra I(σ) Level 1 Spectra I(σ) For all observing modes but the sparse spatial sampling mode, for which only a point-source spectrum is given at Level 2.

22. PACS page 22 Nanyao Lu NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 Caveats and Remarks Noise doesn’t average down as 1/sqrt(n) after about n = 25 repeats as a result of some systematic fringes. Flux calibration is accurate to 10-20% for SSW, ~30% for SLW. The background subtraction still uncertain below 25 cm -1 in SLW. So the continuum level could be off significantly there. However, line calibration Is not affected. Extended-source flux calibration provided for all detector channels in all observing modes. Additional point-source calibration is provided only for the central detectors (SSWD4 & SLWC3) in the sparse observing mode. Lines are usually unresolved, but have side lobes following a SINC function. A SINC function fit is required for total flux.

23. PACS page 23 Nanyao Lu NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 Residual Background in SLW

24. PACS page 24 Nanyao Lu NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 Reprocess your FTS Observations You probably want to use data processed with the latest calibration files. A modified HIPE 4 pipeline script is available at ~/scripts_readonly/SPIRE/spec/SPIRE_spec_SOF1_pipeline_hipe4_modified.py, which you can use to reprocess your data with any of the following options: Use the latest calibration files (i.e., spire_cal_4_0). Only process the central detectors (to speed up data processing & to avoid overloading your computer memory). Using a user-supplied interferogram for telescope/sky background subtraction.

25. PACS page 25 Nanyao Lu NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 Final Spectra are great! SPIRE FTS SOF1 Pipeline and Calibration Files Trevor Fulton 25

26. PACS page 26 Nanyao Lu NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 You can play with FTS spectrum of Mrk 231 SPIRE FTS SOF1 Pipeline and Calibration Files Trevor Fulton 26 (Van der Werf etal 2010)

27. PACS page 27 Nanyao Lu NHSC SPIRE Data School – Pasadena 28 th - 30 th June 2010 You can play with FTS spectrum of Mrk 231 SPIRE FTS SOF1 Pipeline and Calibration Files Trevor Fulton 27 There are 3 files in ~/scripts_readonly/spire/spec: >>> Copy the following 2 files fro there to your home directory: SPIRE_spec_SOF1_pipeline_hipe4_modified.py SCalSpecInterRef_CR_nominal_20050222_50002972_average_fourier_ALL_DETS.fits >>> Copy the following data to your ~/.hcss/lstore/ and then untar it there: 50002975.tar