1. PACS page 1 NHSC SPIRE Data Processing Webinars 8 th Feb 2012 PACS page 1 Overview of SPIRE Photometer Pipeline C. Kevin Xu (NHSC/IPAC)

2. PACS page 2 NHSC SPIRE Data Processing Webinars 8 th Feb 2012 Goals: Show how SPIRE Photometer pipeline works (functionalities of major modules). Explain what is new in HIPE 8. Brief summary of remaining issues. Will concentrate on scan map “user pipelines” (covering small map, large map, SPIRE/PACS parallel mode).

3. PACS page 3 NHSC SPIRE Data Processing Webinars 8 th Feb 2012 User Pipelines You can find these “user pipelines” in HIPE: User pipelines (Jython scripts): Simplified version of Standard Product Generation (SPG) pipelines.

4. PACS page 4 NHSC SPIRE Data Processing Webinars 8 th Feb 2012 5’ Our focus (“user pipelines”) SPIRE Pipeline & Data Products (in HIPE)

5. PACS page 5 NHSC SPIRE Data Processing Webinars 8 th Feb 2012 Scan Map Pipeline Flow Chart Baseline removal + Mapper A “reversed sequence” relative to the chain of data acquisition place-holder Default: concurrent deglitcher + wavelet deglitcher Alternative: σ-κ deglitcher FFT based (run after deglitch & repair to avoid ringing) Include: (1) Non-linearity correction (2) Volt to Jy/beam conversion Remove 1/f noise due to T-drift FFT based Default: naïve mapper Alternative: madmapper Telescope pointing & orientation SPIRE detectors positions Choices: Default: scan median baseline removal Advanced : polynomial baseline removal Most advanced: an iterative destriper

6. PACS page 6 NHSC SPIRE Data Processing Webinars 8 th Feb 2012 Scan-By-Scan Processing The pipeline processes timelines scan by scan (to ease the demand on RAM). Problem: ringing at the two ends of each scan due to FFT based modules. Solution: (1) Before the process, attaching “turn-around” data blocks to ends of the scan. (2) During the process, the ringing is confined to the “turn-around” data. (3) After the process, cut-off the “turn-around” data blocks from the scan. Uniform scan speed distance Turn- around

7. PACS page 7 NHSC SPIRE Data Processing Webinars 8 th Feb 2012 Highlights of a User Pipeline (Jython Script) New in HIPE 8.0 (for extended source, optional)

8. PACS page 8 NHSC SPIRE Data Processing Webinars 8 th Feb 2012 Signal “Jump” Sudden (spontaneous?) jump in the timeline of a single channel. Mostly seen in thermistors (see below). Very rarely seen in detectors. The average frequency is ~ 1/day ( a few hundred instances since launch). The cause is still unknown. Effect: The pipeline uses thermistor timelines in the correction for detector signal drift due to bath temperature drift (major source of 1/f noise for SPIRE). A thermistor “jump” affects this correction, introducing artificial stripes in the final map. Jump Stripe caused by the jump

9. PACS page 9 NHSC SPIRE Data Processing Webinars 8 th Feb 2012 Thermistor Jump Solution (in HIPE 8.0) a “reversed sequence” relative to the chain of data acquisition scan by scan processing Jump Detector (automatically identify & mask any thermistor affected by a jump) Exclude masked thermistors New module Destriper + Mapper

10. PACS page 10 NHSC SPIRE Data Processing Webinars 8 th Feb 2012 Status of SPIRE Scan Map Pipeline in HIPE 8.0. General assessment: In many cases, data from HSA are already science quality! The official calibration accuracy is ±7% (5% from model, 2% RMS). An example (on the right): The image from HSA looks good. Caveat: detailed inspections show thin stripes (can be improved using a more advanced baseline remover such as the iterative destriper). A major issue yet to resolve: stripes due to bias voltage drift (the “burp”) (affecting a few observations) SPIRE 3-color map of NGC 5315 (a planetary nebula) (Public data taken from HSA)

11. PACS page 11 NHSC SPIRE Data Processing Webinars 8 th Feb 2012 Bias Drift in “Burp” Period Every time when SPIRE is switched on after a cooler recycle, the first ~6 h sees a rapid rise of bias voltage (the “burp”). This effect has not been corrected in pipeline. It causes stripes in maps observed during the “burp” period (see example below). A pipeline module is now being developed to correct this effect (available soon). For point source surveys, the stripes might be removed by polynomial baseline removal. After cooler recycle 1 (~40 h) After cooler recycle 2 (~40 h) “burp” ~ 6h bias drift ~10σ  Resistor voltage Bias voltage An example of stripes caused by bias drift (Standard pipeline product)

12. PACS page 12 NHSC SPIRE Data Processing Webinars 8 th Feb 2012 Summary Scan-Map pipeline covers nearly all SPIRE PHOT AOTs (small map, large map, map in SPIRE/PACS parallel mode). It follows a “reversed sequence” relative to the chain of data acquisition. For a general user, Level0_5 should be the best starting point. From Level0_5 to Level 1, the pipeline processes any observation data set scan by scan (to ease the demand on the RAM). The current pipeline (HIPE 8.0.1) does a good job (“science ready”) in general. However, the pipeline still has some open issues: that (1) Stripes due to the “burp effect” (affecting ~10% observations). (2) Residual stripes (can be corrected by the destriper module in HIPE). (3) Residual glitches in maps (2nd level deglitcher being developed inside the destriper module).

13. PACS page 13 NHSC SPIRE Data Processing Webinars 8 th Feb 2012 Home work: Run Photometer Large Map user pipeline (with your own data or NGC 5315 data). Next: A demo of SPIA (SPIRE Photometer Interactive Analysis), a GUI based data reduction tool (including all pipeline modules and more).