1 Aug SDSS-KSG Summer Overview of SDSS: I. Data Product Yun-Young Choi Korean Scientist Group KIAS

2 Aug SDSS-KSG Summer Contents Data Distribution Sky Coverage Data Products Imaging Data Products Spectroscopic Data Products Other data products Database Access Catalog Archive Server: SkyServer Catalog Archive Server SkyServer Data Archive Server User Interfaces SDSS Query Tools Access to Files Imaging Files Spectroscopy Files Hardware 2.5m Telescope and Instruments Imaging System Filters Great Circle Drift Scanning Spectroscopic System Photometric Telescope Data Acquisition SoftwareSoftware: (See also Algorithms on the DR4 web!)Algorithms Data Processing Factory Astrometric Pipeline The Postage Stamp Pipeline The Frames Pipeline Measurements of Flux Measurements of Shape and Morphology Photometric Calibration Target Selection Galaxies Luminous Red Galaxies Quasars Other Science Targets Plate Definition Spectroscopic Pipelines Extraction and Calibration of Spectra Measuring Spectra Ref. EDR paper by Stoughton et al AJ 123, 485Stoughton et al AJ 123, 485

3 Aug SDSS-KSG Summer Learning about the SDSS and the Data PLEASE make use of SDSS documentation on the web!! Survey summary: York et al. (2000 AJ 120, 1579) - technical summary paper of SDSS York et al. (2000 AJ 120, 1579) SDSS Project Book ( ) - to get a general idea for the scope of SDSS SDSS Project Book Data Processing and Formats of the outputs: EDR paper by Stoughton et al. (2002 AJ 123, 485) – the best description available of the photometric and spectroscopic pipelines and target selection algorithms.Stoughton et al. (2002 AJ 123, 485 Technical Papers: Technical Papers Many of the pipelines and much of the SDSS hardware Data Release SDSS DR4 web site ( ): o Data Products, Sky Coverage, Data Access, Algorithms Data ProductsSky CoverageData AccessAlgorithms

4 Aug SDSS-KSG Summer Accessing the data CAS and DAS Data Archive Server (DAS) Flat files generated by the pipelines: flat-field images, the catalogs of objects in each field, the spectra, and so on. Collaborator's Resource of the DAS at FNAL Catalog Archive Server (CAS) SQL database with fast search capabilities   See Schema Browser of SkyserverSchema BrowserSkyserver Auxiliary Data Value-added catalogs (both public and private versions) Value-added catalogs NYU Value-Added Galaxy Catalog SDSS quasar catalog Princeton reductions of the imaging and spectroscopic dataimagingspectroscopic CMU-PITT spectroscopic galaxy and cluster catalogsspectroscopic galaxycluster etc.

5 Aug SDSS-KSG Summer SDSS Data Flow 1. 1.Imaging Data Acquisition at APO 2. 2.DLTs mailed to Fermilab 3. 3.Pipelines run in successive order: Serial Stamp Collecting (SSC) Pipeline (bookkeeping) Postage Stamp Pipeline (PSP) (flatfields, skybackground, PSF, etc) frames (aka Photo) astrom (astrometric calibration: assign precise coordinates to each objects) nfcalib (final calibration: refinement in the positional and photometric calibration 4. 4.Stuffing operational database, resolving 5. 5.Spectroscopic target pipelines  See DR4 web page, Target selectionTarget selection 6. 6.Plate drilling at UW 7. 7.Spectroscopic Data Acquisition at APO 8. 8.DLTs mailed to Fermilab 9. 9.spectro2d and spectro1d runs Stuffing database All data eventually loaded into CAS

6 Aug SDSS-KSG Summer SDSS Terms ●Run: a scan made by the imaging camera ●Runs make up a strip, 2 interleaved N and S strips make up each 2.5 deg wide survey stripe ●Rerun: number assigned to each (re)processing of the same run ●Camcol: 1 of the 6 camera columns comprising each run ●Field: 2048 pixel x 1361 pixel areas dividing each run ●Target: reduction used for spectroscopic target selection ●Best: best available reduction ●Primary: ``main'' observation of an object. Objects in the SDSS can be duplicated (runs, stripes, fields, etc.) scan direction

7 Aug SDSS-KSG Summer Survey coordinate system (,  )

8 Aug SDSS-KSG Summer DR4 Sky Coverage Imaging Area 6670 sq. deg.Spectroscopic Area 5320 sq. deg. Use “Foot” (footprint) server to convert RA/Dec to run/rerun/camcol/fieldFoot See sky coverage page for the lists of target/best stripes, runs, fields, …sky coverage

9 Aug SDSS-KSG Summer

10 Aug SDSS-KSG Summer SDSS Imaging Corrected frames ●fpC-$run-$filter$camcol-$field.fit ( fpC g fit ) ●bias subtracted, flat-fielded, and purged of bright stars ”/pix 2048 x 1489 pixels (note adjacent fpC images overlap; e.g., along field direction primary area is only 1361pixels long) Background includes SOFTBIAS (=1000) + SKY (in the header keyword SKY)  Retrieve from Data Archive Server (DAS)Data Archive Server (DAS)

11 Aug SDSS-KSG Summer Image-Related Products 1.Atlas imagesAtlas images ●fpAtlas*.fit: “postage-stamp” images of individual objects in a single field 2.PSF filesPSF files ●psField*.fit: Photometric Calibration By Field - PSF information etc. ●SDSS PSF ~ a sum of 2 Gaussians and a power-law wing 3.Image masksImage masks ●fpM*.fit: binary masks indicating saturation, interpolation, detected objects, etc. 4.Bad region masksBad region masks ●mask*.csv: ascii files indicating regions affected by saturation/bright stars, satellite trails, bad seeing, and survey holes  Retrieve from Data Archive Server (DAS)Data Archive Server (DAS)  Codes to read: See DR4 web page, Getting and using imagesGetting and using images

12 Aug SDSS-KSG Summer ● tsField (Target Summary for a field) tsField (Target Summary for a field ●Field table in CAS (ref. schema browser)CAS (ref. schema browser) ●tsField-$run-$camcol-$rerun-$field.fit binary fits tables ●Field quality (good, acceptable, bad, …) ●Photometric calibration zeropoints and extinction coefficients ●PSF and seeing information ●Sky (atmospheric extinction corrected) ●Astrometric transformation ●Plus other stuff … Field Information

13 Aug SDSS-KSG Summer ● tsObj (Calibrated Photometric Catalog) tsObj (Calibrated Photometric Catalog) ●PhotoObj and similar tables/views in CAS ●tsObj-$run-$camcol-$rerun-$field.fit binary fits tables ●Field quality, seeing, sky are also in fits header ●Fits tables contain Run, rerun, camcol, field, id Position: RA/Dec, etc. Object classifications and photometric processing flags Target selection flags Various calibrated SDSS magnitudes Surface brightness, concentration index, radial profiles Adaptive moments Lots of other quantities … see data modeldata model Calibrated Object Lists

14 Aug SDSS-KSG Summer SDSS Photometry ● Count rate f/f 0 = counts/exptime * *(aa + kk * airmass) exptime= , aa=zeropoint, kk=extinction coefficient tsField file or Field table ● Conventional magnitude = -2.5 * log10(f/f 0 ) ● asinh magnitude = -(2.5/ln(10)) * [asinh((f/f 0 )/2b)+ln(b)] asinh magnitude Lupton, Gunn, & Szalay 1999, AJ, 118 b is a softening parameter, set to be about 1 sigma of sky noise in each filter (see DR4 web site) Difference between asinh and conventional mag < 1% for objects brighter than asinh mag m(f/f 0 = 10b) = 22.12, 22.60, 22.29, 21.85, for ugriz Corrections to convert SDSS magnitude system to AB magnitudes: u(AB) = *log10(f u ) g(AB) = *log10(f g ) r(AB) = *log10(f r ) i(AB) = *log10(f i ) z(AB) = *log10(f z )

15 Aug SDSS-KSG Summer SDSS Magnitudes 1.PSF: Fit of a PSF model to the object 2.Fiber: Magnitude inside the 3” aperture to estimated the flux seen by spectroscopic fiber aperture (images convolved to 2” seeing first) 3.Petrosian: Magnitude including the same fraction of flux, independent of galaxy’s angular size. Very noise for faint galaxy 4.Model: The better of the fits to a PSF-convolved deVaucouleurs (deV) or exponential (exp) galaxy profile (aperture set by r band) 5.cmodel: The best linear combination of the deVaucouleurs and exponential fits in each band; the flux is F composite = fracDeV F deV + (1 - fracDeV) F exp  See DR4 web page, Photometry in AlgorithmsPhotometry in Algorithms

16 Aug SDSS-KSG Summer SDSS Magnitudes (cont’d) Galactic Extinction Correction Schlegel, Finkbeiner & Davis (1998) Fluxes are not Galactic extinction corrected, and are in nanomaggies. Given as reddening in the tsObj files or extinction in the CAS To compute magnitudes in arbitrary circular aperture, can use the radial surface brightness profiles Given as the average surface brightness in a series of annuli Units: maggies/sq. arcsec, where 1 maggie of flux has an AB magnitude of 0 See profMean in the tsObj files and the PhotoProfile table in CAS

17 Aug SDSS-KSG Summer Example Magnitude Usage Photometry of bright galaxies : e.g. for main sample galaxies, use Petrosian magnitudes (model independent, S/N remains good to r=20 or so) Photometry of galaxies : cmodel magnitude (smaller noise for faint galaxies) Colors of galaxies : model magnitudes (aperture from r band applied to all filters, unlike for cmodel magnitudes) Photometry of distant quasars : PSF magnitude (unresolved objects) Colors of stars : PSF magnitude

18 Aug SDSS-KSG Summer Target Selection Flags ● Flags are used extensively to indicate type, status, quality, etc., etc. primTarget: 32-bit primary target selection flag Main galaxies, quasars, high-z quasars, LRGs.. secTarget: secondary target selection flag Used to indicate calibration stars, skies, guide stars, etc.  See DR4 web page, Target selectionTarget selection

19 Aug SDSS-KSG Summer primTarget flags QSO_HIZ = 0x1 = bit # 0 = 1 QSO_CAP = 0x2 = bit # 1 = 2 QSO_SKIRT = 0x4 = bit # 2 = 4 QSO_FIRST_CAP = 0x8 = bit # 3 = 8 QSO_FIRST_SKIRT = 0x10 = bit # 4 = 16 QSO_MAG_OUTLIER = 0x = bit #25 = QSO_REJECT = 0x = bit #29 = GALAXY_RED = 0x20 = bit # 5 = 32 GALAXY_RED_II = 0x = bit #26 = GALAXY = 0x40 = bit # 6 = 64 GALAXY_BIG = 0x80 = bit # 7 = 128 GALAXY_BRIGHT_CORE= 0x100 = bit # 8 = 256 ROSAT_A = 0x200 = bit # 9 = 512 ROSAT_B = 0x400 = bit #10 = 1024 ROSAT_C = 0x800 = bit #11 = 2048 ROSAT_D = 0x1000 = bit #12 = 4096 ROSAT_E = 0x = bit #27 = STAR_BHB = 0x2000 = bit #13 = 8192 STAR_CARBON = 0x4000 = bit #14 = STAR_BROWN_DWARF = 0x8000 = bit #15 =32768 STAR_SUB_DWARF = 0x10000 = bit #16 = STAR_CATY_VAR = 0x20000 = bit #17 = STAR_RED_DWARF = 0x40000 = bit #18 = STAR_WHITE_DWARF = 0x80000 = bit #19 = STAR_PN = 0x = bit #28 = SERENDIP_BLUE = 0x = bit #20 = SERENDIP_FIRST = 0x = bit #21 = SERENDIP_RED = 0x = bit #22 = SERENDIP_DISTANT = 0x = bit #23 = SERENDIP_MANUAL = 0x = bit #24 = e.g., to select main sample galaxies: (primTarget & 64) >

20 Aug SDSS-KSG Summer Photometric Processing Flags ● Flags objc_flags, objc_flags2 in tsObj files, flags in CAS (also flags for each individual filter) Important to check flags to eliminate problem objects or junk Examples given to derive clean samples of stars and galaxies Primary objects to avoid duplicates Reject saturated objects Reject objects with deblending problems Reject objects with interpolation problems Other cuts …  See DR4 web page, Object FlagsObject Flags  See sdssMaskbits.par in IDLUTILsdssMaskbits.par

21 Aug SDSS-KSG Summer Tiling Tile – spectroscopic fiber-plug plate A circular FoV with r=1.49°, 640 fibers (48 for observations of blank sky and spectrophotometric standards) approximately 2000 plates by the end of its 5 year survey Minimum separation of fiber center, 55”: 100 kpc at about z = 0.2 Problem: the dynamics of binary galaxies, the study of subclustering, and velocity dispersions in rich clusters, etc. Minimizing the # of tiles necessary to observe all the desired targets and maximizing efficiency when placing these tiles and assigning targets to each tile. Allocating fibers to the set of decollided targets (about 90% of all targets) → using unallocated fibers to resolve fiber collisions in the overlaps. a higher priority target is guaranteed never to be eliminated from the sample due to a collision with a lower priority object Priorities: brown dwarfs and hot standards > QSOs > galaxies and LRGs Priorities: brown dwarfs and hot standards > QSOs > galaxies and LRGs Tiling Spectroscopic Plates reference: Blanton et al. (2003) AJ 125,2276Blanton et al. (2003) AJ 125, " approximately 2000 plates by the end of its 5 year survey

22 Aug SDSS-KSG Summer Tile Placement Distributing tiles uniformly across the sky and then using a cost minimization scheme to perturb the tiles to a more efficient solution. SDSS requirements: assign fibers to 99% of the decollided targets.

23 Aug SDSS-KSG Summer Spectra 1.2D spPlate-$plate-$mjd.fitspPlate- 640 spectra together in single fits file for each plate Flux-calibrated spectra, inverse variance, quality masks, and other supporting information No measured parameters (e.g., redshifts) 2.1D spSpec-$plate-$mjd-$fiber.fitspSpec- 1 fits file for each fiber of each plate Calibrated spectrum, continuum-subtracted spectrum, noise, quality mask Spectral classification, redshift, redshift error and confidence Binary tables with line measurements, line indices, emission-line and cross-correlation redshift information, … 3.spDiag-$mjd-$plate.par Summary information for all 640 spectra on each plate Object id, target flags Spectral classification, redshift, redshift error and confidence  In CAS, use the SpecObj, SpecLine, and SpecLineIndex tables/views to get spectroscopic parameters

24 Aug SDSS-KSG Summer Spectroscopic Information ● Generate vacuum wavelengths using = 10 (COEFF0 + COEFF1*i), where i denotes the (zero indexed) pixel number, and COEFF1 and COEFF2 are from file headers ● Check redshift confidence z_Conf, and also status z_Status and warning z_Warnin flags (eg. z_Conf > 0.7 based on eye checks of galaxy spectra) ● The spectrophotometry is good (e.g., overall residual offsets vs. gri fiber magnitudes/colors of < a few percent); see documentation on DR4 web site for details

25 Aug SDSS-KSG Summer ● Mostly on Stripe 82, including u-selected galaxies, low-z galaxies, deep LRGs, faint quasars, spectra of everything, stellar programs, … Retrieve from DAS or using PlateX table in CAS see sdss-archive/2511, and instructions are forthcoming on DR4 web site See the Southern Equatorial Survey plates page at See Ivan Baldry’s page and catalogs at / / Southern Survey and Special Spectroscopic Programs