1. STATUS OF APOGEE DATA PRODUCTS, REDUCTION, AND ANALYSIS Jon Holtzman (NMSU) David Nidever (UVa) Ana Garcia-Perez (UVa) Carlos Allende-Prieto (IAC) Szabolcs Mezaros (IAC) The APOGEE team

2. 2 Data Overview Target selection : input photometry, selection flags, extinction, etc. Observations: up-the-ramp readouts + telescope/weather information  raw APOGEE data files. Exposure sets are taken in visits: multiple frames at different dither positions Data reduction: Visit analysis: basic calibration, extraction, sky correction, initial RV Visit combination: wavelength resampling, refined RVs Chemical analysis (ASPCAP) Stellar parameter determination Individual element abundances

3.  Most major results summarized in two data files:  allVisit : merges target information, individual visit RVs, individual visit stellar parameters. One record per visit per star  allStar : merges target information, average RVs (soon, combined frame RV), average individual visit stellar parameters (soon, stellar parameters from combined frames)  Available in both FITS table, and (wide!) ASCII file  Files are sorted by RA, with accompanying RA “index” table (FITS table version) for faster searches  allStar file contains ID records (FITS table version) of individual visits in allVisit file Summary data files

4. objid : TMASS-style object name file: apVisit file name fiber: fiber number plate: plate number mjd: mjd observation date location : field location ID field: field name ra : RA dec : DEC glon : galactic longitude glat : galactic latitude jmag : 2MASS J hmag : 2MASS H kmag : 2MASS K primtarg : primary target flag sectarg : secondary target flag jd: JD of observations bc: Barycentric correction vtype: Radial velocity method type vrel: derived radial velocity vrelerr: error in radial velocity vhelio: derived barycentric RV vlsr: derived VLSR vgsr: derived VGSR allVisit contents rv_teff: effective temperature used for RV template rv_logg: log g used for RV template rv_feh: [Fe/H] used for RV template rv_chi2: chi2 of RV fit snr: median S/N ratio of spectrum synthfile: synthetic spectrum file name psfile: plot file teff : ASPCAP Teff from visit logg : ASPCAP logg from visit vmicro : ASPCAP vmicro from visit metals : ASPCAP metals from visit c : ASPCAP stepar carbon from visit n : ASPCAP stepar nitrogen from visit alpha : ASPCAP alpha from visit aspcap_chi2: ASPCAP chi2 from visit fit aspcap_vers : ASPCAP version ak : extinction used in target selection akmethod : extinction method used ak_wise_allsky : extinction using WISE ALLSKY release wash_m : Washington M wash_m_err wash_t2 : Washington T2 wash_t2_err ddo51 : DDO 51 ddo51_err irac_3_6 : IRAC 3.6micron mag irac_3_6_err mag_4_5 : 4.5micron mag (IRAC or WISE) mag_4_5_err irac_5_8 : IRAC 5.8micron mag irac_5_8_err irac_8_0 : IRAC 8.0micron mag

5. obj : TMASS-style object name locid : field location ID field : field name jmag : 2MASS J hmag : 2MASS H kmag : 2MASS K ra : RA dec : DEC glon : galactic longitude glat : galactic latitude ak : extinction used in target selection akmethod : extinction method used ak_wise_allsky : extinction using WISE ALLSKY release primtarg : primary target flag sectarg : secondary target flag nvisits : number of visits pre : =1 for pre-2012 shutdown, otherwise 0 vavg: average RV vscatter : scatter in RV snr : S/N of combined frame teff_avg : Average ASPCAP Teff from multiple visits logg_avg : Average ASPCAP logg from multiple visits vmicro_avg : Average ASPCAP vmicro from multiple visits metals_avg : Average ASPCAP metals from multiple visits c_avg : Average ASPCAP stepar carbon from multiple visits n_avg : Average ASPCAP stepar nitrogen from multiple visits alpha_avg : Average ASPCAP alpha from multiple visits : allStar contents teff_sig : Sigma of ASPCAP Teff from multiple visits logg_sig : Sigma of ASPCAP logg from multiple visits vmicro_sig : Sigma of ASPCAP vmicro from multiple visits metals_sig : Sigma of ASPCAP metals from multiple visits c_sig : Sigma of ASPCAP stepar carbon from multiple visits n_sig : Sigma of ASPCAP stepar nitrogen from multiple visits alpha_sig : Sigma of ASPCAP alpha from multiple visits teff : ASPCAP Teff from combined frame logg : ASPCAP logg from combined frame vmicro : ASPCAP vmicro from combined frame metals : ASPCAP metals from combined frame c : ASPCAP stepar carbon from combined frame n : ASPCAP stepar nitrogren from combined frame alpha : ASPCAP alpha from combined frame aspcap_chi2 :ASPCAP chi2 aspcap_vers : ASPCAP version wash_m : Washington M wash_m_err wash_ t2 : Washington T2 wash_t2_err ddo51 : DDO 51 ddo51_err irac_3_6 : IRAC 3.6micron mag irac_3_6_err mag_4_5 : 4.5micron mag (IRAC or WISE) mag_4_5_err irac_5_8 : IRAC 5.8micron mag irac_5_8_err irac_8_0 : IRAC 8.0micrton mag, irac_8_0_err

6. 6 Other data products Data model available at http://data.sdss3.org/datamodel, data at http:/data.sdss3.org/sas/bosswork/groups/apogee (to be renamed…)http://data.sdss3.org/datamodel Raw data: apR-[abc]-ID8.fits (in APOGEE_DATA/) Reduced data (in APOGEE_REDUX/vers/) Basic calibrations: red/MJD5/ap2D-[abc]-ID8.fits Extracted 1D: red/MJD5/ap1D-[abc]-ID8.fits Sky corrected: plates/PLATE4/MJD5/apCframe-[abc]-ID8.fits Dither combined: plates/PLATE4/MJD5/apPlate-[abc]-ID8.fits, or separated apVisit-PLATE4-MJD5-FIBER3.fits (with initial RVs in headers), native wavelength scale (with wavelength array) Visit combined: stars/LOC4/apStar-NAME.fits: rebinned to fixed log lambda dispersion Chemical analysis (in APOGEE_ASPCAP/vers/) Best fit stellar parameters and model spectra from individual visit spectra in results/aspcap-PLATE4-MJD5.fits

7. 7 Sample spectra Spectra plots available on embedded web pages, + RV plots, links to FITS files

8. 8 DR10 DR10 contents still TBD, but will likely include Table of all possible targets with target selection data Design, field, plate data allVisit table allStar table Separate ASPCAP tables for different versions? To include …?

9. 9 Data quantity status Sky coverage S/N and number of visits Reduction processing: visit processing routine and current, happens within 2-3 days of data taking; combination processing happening ~monthly ASPCAP processing: uniform (but non-optimal) processing through ~March 2012 S/N vs H…

10. Mostly relatively standard reduction steps Basic calibration (flats, darks, linearity): flats seem stable, minor improvements have been made over current pipeline version Darks need some analysis Linearity corrections not currently implemented Extraction Currently done with empirical PSF, considering 3 fibers at a time Sky correction: telluric and sky subtraction Dither combination Visit RV Visit combination / refined RVs Data quality: reduction

11. Data quality: sky and telluric

12. Velocity Uncertainty Scatter from multiple measurements Some real variability Peak RV scatter ~100 m/s Velocity zeropoint Comparison to GC RVs RV offset average -0.26 +/- 0.2 km/s Data quality: RVs

13. ASPCAP tuning/development still active Tuning of linelists Fit method (PCA vs flux space) being changed Parameter space (7 param vs 6 param plus fixed/coupled microturbulence) still under consideration Fitting of combined spectra as well as individual visit spectra Some tests of ASPCAP stellar parameters available Repeatability: internal precision Clusters Kepler asteroseismic targets Data quality: ASPCAP

14. ASPCAP precision Sample: Combined S/N>220 N(ASPCAP>=5 )

15. ASPCAP accuracy: metallicities ASPCAP metallicities seem fairly good near solar, but are systematically high for metal- poor clusters TBD: understanding and decision on whether to implement calibration relations

16. ASPCAP accuracy: gravities ASPCAP gravities systematically high: ~0.4 dex near solar ~1 dex for metal-poor clusters Significant issue for distances Again, calibratable or need to solve? Comparison with asteroseismic gravities

17. Catalog RJCE extinctions derived assuming fixed color for all stars: expected metallicity dependence could give systematic errors of 0.1-0.2 mag in A(K) Simple to implement metallicity dependent A(K) Still some uncertainties at metal-poor end: extinctions derived from spectroscopic temperature and color-temp relation don’t quite match RJCE extinctions Data quality: Extinctions

18. APOGEE data is flowing through reduction pipeline Much processed through ASPCAP, soon to be routine Data products available in SAS Spectra summary tables ASPCAP fits Important to be aware of known issues: publication quality TBD? We’re interested to hear about unknown issues! Conclusions

20. Hope to implement late summer/fall Design work by Nick MacDonald + spare fibers: target + sky fibers Main goals: Calibration star targets RV monitoring Bright star survey NMSU 1m – APOGEE Fiber feed Challenge: robotic spectroscopy – acquisition and guiding

21. Determine mean M_H of PDF given Te, log g, [Fe/H] and adopted errors, with prior on IMF and SFH; use H and WISE reddenings to get dist (similar to Santiago et al SEGUE distances) Coming: priors on spatial distribution, selection function, better error treatment (e.g., Burnett and Binney) Preliminary distances and abundance gradients with APOGEE : Michael Hayden, Jon Holtzman (NMSU) Inner disk coverage with ASPCAP still limited but should be available soon Initial gradient results comparable with SEGUE (Cheng et al.)

22. Dec 8, 2011 22 Line Widths/ Resolution (1) ThAr UNe Sky Solid: fiber 150 Open: fibers 10/290 Each point is median of results from lots of frames (few per night for lamps, one per night for skies) Simple Gaussian fits Some lines appear to be resolved (blends) Wavelength range/dispersion as expected Resolution close to or achieves SRD specification of R~22500 (significant improvement from pre-shutdown in red)

23. Dec 8, 2011 23 Line Widths/ Resolution (1) ThAr UNe Sky Solid: fiber 150 Open: fibers 10/290 Each point is median of results from lots of frames (few per night for lamps, one per night for skies) Simple Gaussian fits Some lines appear to be resolved (blends) Wavelength range/dispersion as expected Resolution close to or achieves SRD specification of R~22500 (significant improvement from pre-shutdown in red)

24. Dec 8, 2011 24 Line Widths/ Resolution (2) Stability and monitoring: line widths appear to be constant in time (mostly?) Significant variation across chip not unexpected

25. Dec 8, 2011 25 Line Widths/ Resolution (3) Issues Moderate undersampling in blue Resolution varies across chip (not always best in center!) Beware sky lines for LSF characterization? Caveat: Measurements from uniformly illuminated fibers Possible LSF measurements from fits to telluric lines, or perhaps asteroid (or solar analog)

26. Dec 8, 2011 26 S/N and throughput SRD calls for S/N=100 at H=12.2 in 3 hr exposure per pixel at 1.6 microns, airmass=2, seeing=1.5”, clear conditions. Pixel is taken to mean a “Nyquist sampled pixel”; since dithering provides spectra at ~2X Nyquist sampling, this means S/N=70.7 per dithered pixel. S/N calculated for all 500s exposures taken to date, using median S/N for 12<H<12.2, where noise comes from noise model Some S/N verification from looking at “featureless” region of tellurics; ratio of observed/expected scatter generally between 1.0 and 2.0 Dotted lines are SRD requirement assuming signal-limited exposures Note that S/N in excess of SRD doesn’t necessarily imply we are overexposing because we could reduce number of exposures

27. Dec 8, 2011 27 Persistence APOGEE detectors have noticeable persistence issues Blue detector has “superpersistence” over top ~1/3 rd Some characterization done in spring, but little or no effort since then Attempting to minimize as much as possible: Cold shutter installed and in routine use Dome flats are only nighttime frames taken that aren’t science frames; taken AFTER science exposures of a plate, followed by two short dark Evening cals minimized, and taken early, with more complete calibration set taken in morning; kept cal exposures to a “minimum”

28. Dec 8, 2011 28 Dither pair accuracy Modifications made to mechanism algorithm between runs 2 and 3 made significant improvements! SRD spec: 0.5 micron accuracy or 0.028 pixels (dotted lines) Algorithm for combining dither frames should handle arbitrary shift size so long as it is well measured, although deviations from 0.5 lead to correlated noise issues Detailed tests on understanding whether 2 samples are sufficient for full LSF reconstruction have not yet been done (dither combination of monochromatic sources complicated by intensity variations): test on single line looks good

29. Dec 8, 2011 29 Observing software Observing software functional and relatively stable: ICS software STABLE : some modifications have been made for dithering and for safety interlock. Possible desire to implement additional exposure type keywords Instrument control MOSTLY through SOP: few items still done with STUI scripts: cartridge change, evening cals, morning cals. Appears to be going smoothly, but possible desire to incorporate everything into SOP (priority? manpower?) Python quicklook actor MOSTLY STABLE: handles header annotation of frames, checksum vs raw frames, initiates quicklook, quickred, bundling. Some issues with long term stability, so restarted every day automatically (TBD priority for modification? Manpower?) Quicklook/webapp for observing support to be discussed later Morning cron job checks for missing bundled frames, creates MD5 sums for SAS transfer SAS transfer of raw data running stably during subsequent day

30. Dec 8, 2011 30 Observing support Observers have been given primary contacts: Frinchaboy: scheduling issues Holtzman : actor / observing issues Shetrone: observing issues Alerts set up for instrument parameters, disk space Things seem to be running fairly smoothly (from team perspective, but it’s really the observer perspective that is important here!)

31. Dec 8, 2011 31 Calibration data Calibration data is taken on a regular basis: Daily internal cals: ThArNe and UNe lamps at 2 dither positions Quartz exposure 3 long darks Internal flats Monthly internal cals, plan not yet routine – needs improved procedure/responsible party 10 long darks 10 internal flats Persistence monitor Sparse pack quartz On-telescope calibration data: Mirror petal flats taken after every cartridge to 1) provide PSFs, 2) provide fiber to fiber throughput Sky frames taken once per night (4 150s exposures) for LSF monitoring (but note possible issues with these) Every plate has sky and telluric star fibers for sky subtraction and telluric correction Overall approach has been relatively conservative, however, little or no impact on efficiency from internal cals, some from on-telescope cals

32. Dec 8, 2011 32 Calibration data (2) Calibration data suggests instruments appears to be relatively stable: Wavelength cals Flat fields Darks LSFs Appears that current internal calibration data is sufficient (and taking more that might possible be relevant would probably imply taking an efficiency and persistence hit) Reduction pipeline currently not doing great job at sky subtraction and telluric correction Relatively little effort dedicated to date Not yet clear if calibration data is sufficient Have been using 35/35 sky/telluric as opposed to original plan of 25/25

33. Dec 8, 2011 33 Data inspection / QA Data is now being routinely processed (although almost certainly not optimally!) quick manual construction of pre-processing file (check on plate/exposure assocation, cal frames, etc.) done morning after observing (~15 minutes) based on observing logs Raw data cubes downloaded from SAS, generally by night following observing Reduction started second day after observing, processed through 3D->2D, 2D->1D, sky correction, dither combination, RV measurement in 6-8 hours per plate, all plates reduced simultaneously on multiple processors Daily calibration data processed in parallel (not yet quite routinely implemented)

34. Dec 8, 2011 34 Data inspection / QA (2) Data reduction pipeline produces web pages for quick inspection / QA Summary MJD QA page: Highlights any frames for which no reduced frame exists Frames with bad checksums Missing sequence numbers plot of zeropoints for all exposures, continuum sky levels for all exposures Table of all science exposures, with derived S/N, zeropoints, attempts at identifying missing and faint fibers Summary web page for each plate: Zeropoint, continuum sky, and S/N plots vs mag for every exposure Maps of zeropoints, sky as f (zeta, eta) Web page of all dither-combined spectra and error spectrum for each plate Links to RV page showing best matching spectrum Some of these are getting looked at now Need to develop specific checklist of what should be looked at (may motivate some reorganization of information on pages), but I think we are very close This checks mostly issues with conditions, data issues, and data reduction issues, thus indirectly instrument health

35. Dec 8, 2011 35 Data inspection / QA (3) Instrument health montoring through daily calibration monitor output: not yet fully implemented, but many pieces in place (few days work) Wavelength calibration: start/end wavelengths, wavelength solution vs. fiducial solution LSF: line width monitoring Dither pair separation (should be included in plate pages) Daily flat vs fiducial flat Daily dark vs fiducial dark Cartridge flats vs fiducial (perhaps should be included in plate pages) Need to develop specific checklist of things to look at and simple web interface (high priority, getting close to implementation) Increment of survey reduced data statistics RV histogram for plate Sky map of observed plates S/N histogram Delta mag histogram

36. Dec 8, 2011 36 Possible areas for improvement EFFICIENCY Shortening default exposure time Implement variable exposure time based, e.g., on quicklook Use of twilight time Additional cartridges or nighttime replugging of a cartridge Reduction of number of sky and/or telluric fibers Reducing/eliminating daily sky exposures Developing refined S/N criteria, e.g., based on actual ASPCAP results ANALYSIS Develop methods for addressing persistence Improve extraction methods Improve telluric correction, and perhaps, sky subtraction Improve construction of dither-combined frames Implement combination of multiple visit spectra