1. 1 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo Overview of SDSS: I. Data Product Yun-Young Choi Korean Scientist Group KIAS

2. 2 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo Contents Data Distribution Sky Coverage Data Products Imaging Data Products Spectroscopic Data Products Other data products Database Access Catalog Archive Server: SkyServer Catalog Archive Server SkyServer Data Archive Server User Interfaces SDSS Query Tools Access to Files Imaging Files Spectroscopy Files Hardware 2.5m Telescope and Instruments Imaging System Filters Great Circle Drift Scanning Spectroscopic System Photometric Telescope Data Acquisition SoftwareSoftware: (See also Algorithms on the DR4 web!)Algorithms Data Processing Factory Astrometric Pipeline The Postage Stamp Pipeline The Frames Pipeline Measurements of Flux Measurements of Shape and Morphology Photometric Calibration Target Selection Galaxies Luminous Red Galaxies Quasars Other Science Targets Plate Definition Spectroscopic Pipelines Extraction and Calibration of Spectra Measuring Spectra Ref. EDR paper by Stoughton et al. 2002 AJ 123, 485Stoughton et al. 2002 AJ 123, 485

3. 3 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo Learning about the SDSS and the Data PLEASE make use of SDSS documentation on the web!! Survey summary: York et al. (2000 AJ 120, 1579) - technical summary paper of SDSS York et al. (2000 AJ 120, 1579) SDSS Project Book (1996-1997) - to get a general idea for the scope of SDSS SDSS Project Book Data Processing and Formats of the outputs: EDR paper by Stoughton et al. (2002 AJ 123, 485) – the best description available of the photometric and spectroscopic pipelines and target selection algorithms.Stoughton et al. (2002 AJ 123, 485 Technical Papers: Technical Papers Many of the pipelines and much of the SDSS hardware Data Release SDSS DR4 web site (http://www.sdss.org/dr4/ ):http://www.sdss.org/dr4/ o Data Products, Sky Coverage, Data Access, Algorithms Data ProductsSky CoverageData AccessAlgorithms http://www.astro.princeton.edu/~strauss/welcome/welcome.html

4. 4 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo Accessing the data CAS and DAS Data Archive Server (DAS) Flat files generated by the pipelines: flat-field images, the catalogs of objects in each field, the spectra, and so on. Collaborator's Resource of the DAS at FNAL Catalog Archive Server (CAS) SQL database with fast search capabilities   See Schema Browser of SkyserverSchema BrowserSkyserver Auxiliary Data Value-added catalogs (both public and private versions) Value-added catalogs NYU Value-Added Galaxy Catalog SDSS quasar catalog Princeton reductions of the imaging and spectroscopic dataimagingspectroscopic CMU-PITT spectroscopic galaxy and cluster catalogsspectroscopic galaxycluster etc.

5. 5 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo SDSS Data Flow 1. 1.Imaging Data Acquisition at APO 2. 2.DLTs mailed to Fermilab 3. 3.Pipelines run in successive order: Serial Stamp Collecting (SSC) Pipeline (bookkeeping) Postage Stamp Pipeline (PSP) (flatfields, skybackground, PSF, etc) frames (aka Photo) astrom (astrometric calibration: assign precise coordinates to each objects) nfcalib (final calibration: refinement in the positional and photometric calibration 4. 4.Stuffing operational database, resolving 5. 5.Spectroscopic target pipelines  See DR4 web page, Target selectionTarget selection 6. 6.Plate drilling at UW 7. 7.Spectroscopic Data Acquisition at APO 8. 8.DLTs mailed to Fermilab 9. 9.spectro2d and spectro1d runs 10. 10.Stuffing database 11. 11.All data eventually loaded into CAS http://www.sdss.org/dr4/dataflow/index.html

6. 6 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo SDSS Terms ●Run: a scan made by the imaging camera ●Runs make up a strip, 2 interleaved N and S strips make up each 2.5 deg wide survey stripe ●Rerun: number assigned to each (re)processing of the same run ●Camcol: 1 of the 6 camera columns comprising each run ●Field: 2048 pixel x 1361 pixel areas dividing each run ●Target: reduction used for spectroscopic target selection ●Best: best available reduction ●Primary: ``main'' observation of an object. Objects in the SDSS can be duplicated (runs, stripes, fields, etc.) scan direction

7. 7 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo Survey coordinate system (,  )

8. 8 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo DR4 Sky Coverage Imaging Area 6670 sq. deg.Spectroscopic Area 5320 sq. deg. Use “Foot” (footprint) server to convert RA/Dec to run/rerun/camcol/fieldFoot See sky coverage page for the lists of target/best stripes, runs, fields, …sky coverage

9. 9 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo

10. 10 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo SDSS Imaging Corrected frames ●fpC-$run-$filter$camcol-$field.fit ( fpC-002126-g6-0428.fit ) ●bias subtracted, flat-fielded, and purged of bright stars. 0.396”/pix 2048 x 1489 pixels (note adjacent fpC images overlap; e.g., along field direction primary area is only 1361pixels long) Background includes SOFTBIAS (=1000) + SKY (in the header keyword SKY)  Retrieve from Data Archive Server (DAS)Data Archive Server (DAS)

11. 11 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo Image-Related Products 1.Atlas imagesAtlas images ●fpAtlas*.fit: “postage-stamp” images of individual objects in a single field 2.PSF filesPSF files ●psField*.fit: Photometric Calibration By Field - PSF information etc. ●SDSS PSF ~ a sum of 2 Gaussians and a power-law wing 3.Image masksImage masks ●fpM*.fit: binary masks indicating saturation, interpolation, detected objects, etc. 4.Bad region masksBad region masks ●mask*.csv: ascii files indicating regions affected by saturation/bright stars, satellite trails, bad seeing, and survey holes  Retrieve from Data Archive Server (DAS)Data Archive Server (DAS)  Codes to read: See DR4 web page, Getting and using imagesGetting and using images

12. 12 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo ● tsField (Target Summary for a field) tsField (Target Summary for a field ●Field table in CAS (ref. schema browser)CAS (ref. schema browser) ●tsField-$run-$camcol-$rerun-$field.fit binary fits tables ●Field quality (good, acceptable, bad, …) ●Photometric calibration zeropoints and extinction coefficients ●PSF and seeing information ●Sky (atmospheric extinction corrected) ●Astrometric transformation ●Plus other stuff … Field Information

13. 13 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo ● tsObj (Calibrated Photometric Catalog) tsObj (Calibrated Photometric Catalog) ●PhotoObj and similar tables/views in CAS ●tsObj-$run-$camcol-$rerun-$field.fit binary fits tables ●Field quality, seeing, sky are also in fits header ●Fits tables contain Run, rerun, camcol, field, id Position: RA/Dec, etc. Object classifications and photometric processing flags Target selection flags Various calibrated SDSS magnitudes Surface brightness, concentration index, radial profiles Adaptive moments Lots of other quantities … see data modeldata model Calibrated Object Lists

14. 14 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo SDSS Photometry ● Count rate f/f 0 = counts/exptime * 10 0.4*(aa + kk * airmass) exptime=53.907456, aa=zeropoint, kk=extinction coefficient tsField file or Field table ● Conventional magnitude = -2.5 * log10(f/f 0 ) ● asinh magnitude = -(2.5/ln(10)) * [asinh((f/f 0 )/2b)+ln(b)] asinh magnitude Lupton, Gunn, & Szalay 1999, AJ, 118 b is a softening parameter, set to be about 1 sigma of sky noise in each filter (see DR4 web site) Difference between asinh and conventional mag < 1% for objects brighter than asinh mag m(f/f 0 = 10b) = 22.12, 22.60, 22.29, 21.85, 20.32 for ugriz Corrections to convert SDSS magnitude system to AB magnitudes: u(AB) = 22.5 - 2.5*log10(f u ) - 0.042 g(AB) = 22.5 - 2.5*log10(f g ) + 0.036 r(AB) = 22.5 - 2.5*log10(f r ) + 0.015 i(AB) = 22.5 - 2.5*log10(f i ) + 0.013 z(AB) = 22.5 - 2.5*log10(f z ) - 0.002

15. 15 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo SDSS Magnitudes 1.PSF: Fit of a PSF model to the object 2.Fiber: Magnitude inside the 3” aperture to estimated the flux seen by spectroscopic fiber aperture (images convolved to 2” seeing first) 3.Petrosian: Magnitude including the same fraction of flux, independent of galaxy’s angular size. Very noise for faint galaxy 4.Model: The better of the fits to a PSF-convolved deVaucouleurs (deV) or exponential (exp) galaxy profile (aperture set by r band) 5.cmodel: The best linear combination of the deVaucouleurs and exponential fits in each band; the flux is F composite = fracDeV F deV + (1 - fracDeV) F exp  See DR4 web page, Photometry in AlgorithmsPhotometry in Algorithms

16. 16 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo SDSS Magnitudes (cont’d) Galactic Extinction Correction Schlegel, Finkbeiner & Davis (1998) Fluxes are not Galactic extinction corrected, and are in nanomaggies. Given as reddening in the tsObj files or extinction in the CAS To compute magnitudes in arbitrary circular aperture, can use the radial surface brightness profiles Given as the average surface brightness in a series of annuli Units: maggies/sq. arcsec, where 1 maggie of flux has an AB magnitude of 0 See profMean in the tsObj files and the PhotoProfile table in CAS

17. 17 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo Example Magnitude Usage Photometry of bright galaxies : e.g. for main sample galaxies, use Petrosian magnitudes (model independent, S/N remains good to r=20 or so) Photometry of galaxies : cmodel magnitude (smaller noise for faint galaxies) Colors of galaxies : model magnitudes (aperture from r band applied to all filters, unlike for cmodel magnitudes) Photometry of distant quasars : PSF magnitude (unresolved objects) Colors of stars : PSF magnitude

18. 18 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo Target Selection Flags ● Flags are used extensively to indicate type, status, quality, etc., etc. primTarget: 32-bit primary target selection flag Main galaxies, quasars, high-z quasars, LRGs.. secTarget: secondary target selection flag Used to indicate calibration stars, skies, guide stars, etc.  See DR4 web page, Target selectionTarget selection

19. 19 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo primTarget flags QSO_HIZ = 0x1 = bit # 0 = 1 QSO_CAP = 0x2 = bit # 1 = 2 QSO_SKIRT = 0x4 = bit # 2 = 4 QSO_FIRST_CAP = 0x8 = bit # 3 = 8 QSO_FIRST_SKIRT = 0x10 = bit # 4 = 16 QSO_MAG_OUTLIER = 0x2000000 = bit #25 =33554432 QSO_REJECT = 0x20000000 = bit #29 =536870912 GALAXY_RED = 0x20 = bit # 5 = 32 GALAXY_RED_II = 0x4000000 = bit #26 = 67108864 GALAXY = 0x40 = bit # 6 = 64 GALAXY_BIG = 0x80 = bit # 7 = 128 GALAXY_BRIGHT_CORE= 0x100 = bit # 8 = 256 ROSAT_A = 0x200 = bit # 9 = 512 ROSAT_B = 0x400 = bit #10 = 1024 ROSAT_C = 0x800 = bit #11 = 2048 ROSAT_D = 0x1000 = bit #12 = 4096 ROSAT_E = 0x8000000 = bit #27 =134217728 STAR_BHB = 0x2000 = bit #13 = 8192 STAR_CARBON = 0x4000 = bit #14 = 16384 STAR_BROWN_DWARF = 0x8000 = bit #15 =32768 STAR_SUB_DWARF = 0x10000 = bit #16 = 65536 STAR_CATY_VAR = 0x20000 = bit #17 = 131072 STAR_RED_DWARF = 0x40000 = bit #18 = 262144 STAR_WHITE_DWARF = 0x80000 = bit #19 = 524288 STAR_PN = 0x10000000 = bit #28 =268435456 SERENDIP_BLUE = 0x100000 = bit #20 = 1048576 SERENDIP_FIRST = 0x200000 = bit #21 = 2097152 SERENDIP_RED = 0x400000 = bit #22 = 4194304 SERENDIP_DISTANT = 0x800000 = bit #23 = 8388608 SERENDIP_MANUAL = 0x100000 = bit #24 = 16777216 e.g., to select main sample galaxies: (primTarget & 64) > 0 0.24

20. 20 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo Photometric Processing Flags ● Flags objc_flags, objc_flags2 in tsObj files, flags in CAS (also flags for each individual filter) Important to check flags to eliminate problem objects or junk Examples given to derive clean samples of stars and galaxies Primary objects to avoid duplicates Reject saturated objects Reject objects with deblending problems Reject objects with interpolation problems Other cuts …  See DR4 web page, Object FlagsObject Flags  See sdssMaskbits.par in IDLUTILsdssMaskbits.par

21. 21 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo Tiling Tile – spectroscopic fiber-plug plate A circular FoV with r=1.49°, 640 fibers (48 for observations of blank sky and spectrophotometric standards) approximately 2000 plates by the end of its 5 year survey Minimum separation of fiber center, 55”: 100 kpc at about z = 0.2 Problem: the dynamics of binary galaxies, the study of subclustering, and velocity dispersions in rich clusters, etc. Minimizing the # of tiles necessary to observe all the desired targets and maximizing efficiency when placing these tiles and assigning targets to each tile. Allocating fibers to the set of decollided targets (about 90% of all targets) → using unallocated fibers to resolve fiber collisions in the overlaps. a higher priority target is guaranteed never to be eliminated from the sample due to a collision with a lower priority object Priorities: brown dwarfs and hot standards > QSOs > galaxies and LRGs Priorities: brown dwarfs and hot standards > QSOs > galaxies and LRGs Tiling Spectroscopic Plates reference: Blanton et al. (2003) AJ 125,2276Blanton et al. (2003) AJ 125,2276 55" approximately 2000 plates by the end of its 5 year survey

22. 22 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo Tile Placement Distributing tiles uniformly across the sky and then using a cost minimization scheme to perturb the tiles to a more efficient solution. SDSS requirements: assign fibers to 99% of the decollided targets.

23. 23 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo Spectra 1.2D spPlate-$plate-$mjd.fitspPlate- 640 spectra together in single fits file for each plate Flux-calibrated spectra, inverse variance, quality masks, and other supporting information No measured parameters (e.g., redshifts) 2.1D spSpec-$plate-$mjd-$fiber.fitspSpec- 1 fits file for each fiber of each plate Calibrated spectrum, continuum-subtracted spectrum, noise, quality mask Spectral classification, redshift, redshift error and confidence Binary tables with line measurements, line indices, emission-line and cross-correlation redshift information, … 3.spDiag-$mjd-$plate.par Summary information for all 640 spectra on each plate Object id, target flags Spectral classification, redshift, redshift error and confidence  In CAS, use the SpecObj, SpecLine, and SpecLineIndex tables/views to get spectroscopic parameters

24. 24 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo Spectroscopic Information ● Generate vacuum wavelengths using = 10 (COEFF0 + COEFF1*i), where i denotes the (zero indexed) pixel number, and COEFF1 and COEFF2 are from file headers ● Check redshift confidence z_Conf, and also status z_Status and warning z_Warnin flags (eg. z_Conf > 0.7 based on eye checks of galaxy spectra) ● The spectrophotometry is good (e.g., overall residual offsets vs. gri fiber magnitudes/colors of < a few percent); see documentation on DR4 web site for details

25. 25 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo ● Mostly on Stripe 82, including u-selected galaxies, low-z galaxies, deep LRGs, faint quasars, spectra of everything, stellar programs, … Retrieve from DAS or using PlateX table in CAS see sdss-archive/2511, and instructions are forthcoming on DR4 web site See the Southern Equatorial Survey plates page at http://www-sdss.fnal.gov/targetlink/southernEqSurvey/ http://www-sdss.fnal.gov/targetlink/southernEqSurvey See Ivan Baldry’s page and catalogs at http://mrhanky.pha.jhu.edu/~baldry/sdss-southern / http://mrhanky.pha.jhu.edu/~baldry/sdss-southern / Southern Survey and Special Spectroscopic Programs