1. XMM data reduction with SAS Simon Vaughan (original notes by Tim Roberts)

2. XMM data reduction 2 Overview Context & basics Obtaining XMM-Newton data and identifying useful files Setting up the analysis environment Reducing and cleaning EPIC data Producing images, light curves and spectra using XMMSELECT

3. XMM data reduction 3 What you are aiming for… Cas A supernova remnant Silicon Continuum Iron The nearest star in X-raysJupiter’s X-ray aurora Relativistic Fe line emission from close to a black hole X-rays from the Cen A radio jet Hot gas in the Coma cluster of galaxies

4. XMM data reduction 4 Obtaining X-ray data Astrophysical source of X-rays Intervening absorption (e.g. Galactic neutral gas) X-ray optics (e.g. grazing incidence mirrors) X-ray detectors (e.g. CCDs)

5. XMM data reduction 5 Peculiarities of X-ray data X-ray detectors are photon counting (as opposed to measuring incoming flux)  X-ray data composed of lists of events and their attributes (time, energy etc…) X-ray data is usually photon limited – products have few or no counts in many bins  Requires specific data analysis techniques and statistical approaches

6. XMM data reduction 6 X-ray data products Event list – time-tagged events, with a position (detector/sky space) and energy  Detector attributes e.g. CCD pixel pattern for event – allows rejection of “poor” events Filter event list then project in 1-, 2-D to give data products  Images, energy spectra, light curves Calibration essential in interpreting data  e.g. exposure maps + PSF, response matrices

7. XMM data reduction 7 XMM-Newton X-ray telescopes Optical monitor XMM-Newton instruments: EPIC – pn, MOS × 2 RGS × 2 OM X-ray Detectors

8. XMM data reduction 8 Obtaining XMM-Newton data Write your own proposal  Unfortunately, a high risk process where success may not be rewarded for up to 1.5 years Use the archive  XSA provides access to all datasets beyond the 1- year proprietary period  Accessed via: http://xmm.vilspa.esa.es – java interface best run via netscapehttp://xmm.vilspa.esa.es

9. XMM data reduction 9 EPIC set-up

10. XMM data reduction 10 Select object of interest… …then execute query

11. XMM data reduction 11 Select data…..then move it to your basket (will need to log in)… … before going to check out

12. XMM data reduction 12 Request multiple then highlight both ODF and PPS… …then submit request, and follow instructions

13. XMM data reduction 13 Data receipt E-mailed ftp instructions – follow, unpack tar files Two types of data  ODF – observation data files – telemetry data reformatted to FITS files  PPS – pipeline processing system – top-level science products including event lists, images, source lists, catalogue cross-correlations TIP: load INDEX.HTM into browser – summary info

14. XMM data reduction 14

15. XMM data reduction 15 PPS files Good for “first look” at data Specific naming convention: Where: PiiiiiijjkkaablllCCCCCCnmmm.zzz aa – detector: pn, m1, m2, r1, r2, om CCCCCC – file ID e.g. P(M)IEVLI – pn (MOS) imaging event list, IMAGE_n – image (n gives band ID) zzz – file type: ASC, PDF, PNG, HTM, TAR, FTZ

16. XMM data reduction 16 Should I reprocess? i.e. are the PPS files sufficiently well formatted and calibrated? New (proprietary) data  Should be OK – just gone through most recent version of pipeline Archival data  Whilst reprocessing of archival datasets does occur, perhaps best to adopt “better safe than sorry” approach and reprocess

17. XMM data reduction 17 A manageable directory structure /data/05/sav2/xmm/ tons180/ odf/processed/ pn/ …processed files… mos/ …processed files… rgs/ …processed files… om/ …processed files… pps/ mrk766/source_name/

18. XMM data reduction 18 Setting up the user environment To run on XROA system: > sas-setup-new initialises latest version > setenv SAS_ODF (path_to_ODF_directory) e.g. /data/05/sav2/xmm/tons180/odf > setenv SAS_CCFPATH /usr/local/ccf > cifbuild >& cifbuild.log builds ccf.cif (= Calibration Index File) > setenv SAS_CCF (path_to_ccf.cif_file) > odfingest >& odfingest.log builds ***SUM.SAS file in ODF directory – ODF summary file necessary for reprocessing

19. XMM data reduction 19 Pipeline processing of EPIC data Each pipeline (pn, MOS) needs one command > emchain (or emproc ) > epchain (or epproc ) NB – may need to set ftools up first > lhea-setup-new Output is calibrated event lists ( *EVLI* ) Caveats – multiple event lists may be formed if more than one exposure in dataset, can take some time to run!

20. XMM data reduction 20 What is the result? One ‘event’ list file [*EVLI*] per exposure  An ‘event’ is a detection (usually an X-ray)  Each event on a CCD is tagged with: which detector (camera/CCD) time (CCDs are ‘read out’ periodically) position (X,Y) on detector ‘pattern’ indicating how many pixels are involved energy (~amount of charge deposited in the pixels) quality ‘flag’ indicating known good/bad pixels/events To get a ‘science product’ you need to filter this list all the unwanted times, patterns, image areas, etc…

21. XMM data reduction 21 Event patterns (grades)

22. XMM data reduction 22 The SAS GUI Run by > sas & Simply scroll, double-click on utility e.g. emproc PRO: transparency CON: only handles one dataset at a time

23. XMM data reduction 23 What next? Clean data, and produce science products – XMMSELECT GUI Can run this from SAS GUI… …but quicker to start from command line > xmmselect table=P0106860101PNU002PIEVLI0000.FIT%events NB – can take some time to load large datasets esp. pn data containing considerable background flaring

24. XMM data reduction 24 Logical expression used to filter data Filtering criteria and ranges Product selection Process: Edit ranges Click parameter Repeat… Select param(s). for product Select product

25. XMM data reduction 25 Filtering options Main choices are:  PI – output energy range for product in eV  Time – portion(s) of the observation to include  Pattern – charge signature on one or more pixels pn: 0 – 4 MOS: 0 – 12  Flag – quality control for events pn: FLAG==0 MOS: #xmmea_em

26. XMM data reduction 26 Cleaning data: flare exclusion Perhaps largest problem with XMM-Newton data: space weather Enhances background – dilutes source signals Energy-dependent – reduces effectiveness of spectroscopy Can remove by identifying periods when flaring at worst and excluding them from products

27. XMM data reduction 27 Do I need to flare-filter? YES! Examine PPS MOS images – look for enhancement in background in region visible to sky

28. XMM data reduction 28 Filtering Do for pn: select events above 10 keV, flag & pattern Select “time” Extract “OGIP Rate Curve” Set output file, bin size (10 sec normally OK)

29. XMM data reduction 29

30. XMM data reduction 30 Excluding time intervals For one large flare, or a small number of flares work out time intervals when you want to accumulate data (zoom in grace )  xmmselect : (TIME in (xxx:yyy))&& For noisy data, create Good Time Intervals (GTI) file > tabgtigen table=pn_rates.fits:RATE expression=‘RATE<1.5’ gtiset=pn_gti.fits timecolumn=TIME

31. XMM data reduction 31 Total 21.9 ks data

32. XMM data reduction 32 Caveats Energy spectra of flares can vary  Conservative approach is to also check light curve from 0.3 – 10 keV data using GTI filter MOS & pn may have different start/stop times  If need both instruments on (light curves, some spectra) add extra filter with Tstart, Tstop  OK to use pn GTI file on MOS!  If treated separately – do similar filtering for MOS Very heavy flaring – create new filtered event file before extracting products

33. XMM data reduction 33 Images Select energy, flag, pattern, time Indicate X, Y & extract image Select image tab – set output name, binning size  pn: x/ybinsize=80  MOS: binsize=20

34. XMM data reduction 34

35. XMM data reduction 35 Individual source products Next step: extract spectra, light curves for individual sources Selection of correct source, background regions very important! Rules of thumb:  Avoid: other sources(!), chip gaps, out-of-time events, diffuse sources (if possible)  Consider: distance from read-out, detector structure, same quadrant (pn)

36. XMM data reduction 36 Al-K Si-K

37. XMM data reduction 37 Regions 30-arcsec around NGC 1313 X-1 45-arcsec background region Save both regions separately! Read-out direction Set to background using “info” then “property”

38. XMM data reduction 38 Then extract…. Light curves:  Create for source, background separately  Use “2-D region” button to automatically transfer region to xmmselect selected expression  Select “time”, then “OGIP rate curve”  Choose “withtimeranges=yes”  Set “timemin” and “timemax” same for source, background  But note: using flare filter means light curves broken up (i.e. data gaps present) Spectra:  Highlight source, background regions in ds9, then…

39. XMM data reduction 39 Especget Select PI, time filter Choose “OGIP spectral products” choice to optimise region Change “stem” in “filenames” Run (may be slow!) Spectrum appears in grace window RMFs, ARFs produced

40. XMM data reduction 40 Resources This talk (and others):  http://www.star.le.ac.uk/~sav2/stats/ XMM-Newton SAS web pages  Via http://xmm.vilspa.esa.eshttp://xmm.vilspa.esa.es  Particularly useful documentation: HEASARC ABC guide SAS user’s guide Talk to other experienced users! Once you’ve mastered the GUI, try the command line (more power!)