1. Cooking with Sloan: Special DR6 Recipes Jordan Raddick, Ani Thakar Alex Szalay, Maria Nieto-Santisteban, Nolan Li, Wil O’Mullane, Adrian Pope, Tamas Budavari, George Fekete, Sam Carliles JHU Brian Yanny, Svetlana Lebedeva FermiLab Jim Gray Microsoft Research

2. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 2 Outline SDSS-II and DR6 CAS Data Access –SkyServer, ImgCutout, CasJobs –Help resources and sample queries –Restricted collab access –SDSS and other datasets VO services EPO content

3. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 3 SDSS-II Legacy –Continuation of SDSS-I (fill out 10k sq.deg.) –Completeness is same as for SDSS-I –Flux limits are the same –Target all galaxies with r_petro < 17.77, plus LRGs SEGUE –Detailed 3-d map of the Galaxy Spectra of 240,000 stars in the disk and spheroid Age, composition and phase space distribution –CAS component cataloged in SegueDRx DB Supernova Survey –Repeated scans of SDSS Southern Stripe over 3 mths/yr –Data not available in CAS yet, will be on DAS soon

4. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 4 SDSS Data Overview Data Archive Server (DAS) FITS files (raw data) Images, spectra, corrected frames, atlas images, binned images, masks Online form-based access Rsync and wget file retrieval Data Archive Server (DAS) FITS files (raw data) Images, spectra, corrected frames, atlas images, binned images, masks Online form-based access Rsync and wget file retrieval Catalog Archive Server (CAS) Science parameters extracted to catalogs Stuffed into relational DBMS (SQL Server) Heavily indexed, optimized Online access via SkyServer Several levels of access, query tools Catalog Archive Server (CAS) Science parameters extracted to catalogs Stuffed into relational DBMS (SQL Server) Heavily indexed, optimized Online access via SkyServer Several levels of access, query tools SDSS Data Release www.sdss.org das.sdss.org/DRx-cgi-bin/DAS www.sdss.org das.sdss.org/DRx-cgi-bin/DAS cas.sdss.org/collab/ skyserver.sdss.org cas.sdss.org/collab/ skyserver.sdss.org

5. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 5 CAS Datasets (databases) BestDRx –Latest, greatest calibration of the data –Photometric and spectroscopic objects –The default and most accessed (by far) dataset TargDRx –The calibration from which spectroscopic targets were chosen RUNS –All the runs (processings) other than Best, Target SegueDRx –New with DR6 (SDSS-II)

6. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 6 CAS Data Access SkyServer: http://cas.sdss.org/collabdr6 –Also pwd access at http://cas.sdss.org/collabdr6pw for non-collab IPshttp://cas.sdss.org/collabdr6pw –See message sdss-archive/2935 for uname/pwd CasJobs –DR6 and DR6QA targets point to BestDR6 DB –TARGDR6, SEGUEDR6 targets –Need to have “collab” privilege set in user profile

7. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 7 CAS Data Access SkyServer –Synchronous access – need only web browser –Meant to support several levels of users –Visual tools to browse image and catalog data –API access, e.g. emacs interface, sqlcl (command- line) –Strict limits on execution time and output size (but higher for collaboration)

8. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 8 Navigating SkyServer Major sections linked from main page

9. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 9 Finding your favorite object Get position from SIMBAD

10. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 10 Finding your favorite object Advanced Tools -> Navigate Enter RA/Dec in DMS or HH:MM:SS

11. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 11 Finding your favorite object Click to get position, magnitude Notebook “shopping cart”

12. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 12 Finding your favorite object Explore – full SDSS data for object Chart – create finding chart

13. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 13 Searching the data Search for data that meets certain criteria Return matching data, and only matching data Example: search for galaxies with spectra in Abell 168 –RA = 18.84, Dec = 0.26, radius = 3 arcmin –Limit to galaxies g < 18 –Return RA, Dec, magnitudes, redshift

14. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 14 Searching the Data Tools -> Imaging Query Get data as HTML or CSV Constraints by position or imaging

15. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 15 SQL SQL – computer language for searching databases Advantages –You get longer queries and more results returned –You can write more complex queries –You can use command-line tools

16. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 16 SQL Concepts Data are stored in a database Similar data types are stored in tables –photoObj (photometry), specObj (spectroscopy), etc. A small part of the photoObj table:

17. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 17 SQL Concepts Within a table: –Horizontal rows are individual data points, or records –Vertical columns are types of data, or columns A request to a database to return data is called a query Queries usually request data that meets certain constraints

18. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 18 SQL as a foreign language Languages have grammar and vocabulary Dutch grammar –In sub-clause, verb goes at the end English: –I think that astronomy is fun. Dutch: –Ik denk dat het sterrenkunde leuk is.

19. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 19 SQL Grammar Select – choose which columns of data you want to see From – choose the table(s) from which you want to retrieve data Where – set constraints on the search

20. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 20 Dutch vocabulary

21. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 21 SQL vocabulary Go to Help -> Schema Browser

22. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 22 Learning SQL Go to Help -> Sample SQL Queries Go to Tools -> Search -> SQL Copy, paste, modify

23. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 23 Visual search results Go to Advanced Tools -> Image List Use query to fill form Query must have: select x, ra, dec

24. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 24 CasJobs Go to Search Tools -> CasJobs Best method for long or complex queries

25. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 25 CasJobs Register and log in Quick mode: 1 minute Submit: 24 hours Results stored in MyDB

26. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 26 CasJobs: other features MyDB database to save results of your queries –Define your own functions, procedures too –Share your tables with collaborators (groups) History of Output data as FITS/CSV/VOTable Plot data in browser Table Import (upload) for your own data Groups to share your results with collaborators Command-line access Java tool also downloadable

27. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 27 Collab-only access collabdrx SkyServer sites –http://cas.sdss.org/collabdrx –http://cas.sdss.org/collabdrxpw collab privilege in CasJobs –Restricted data, longer queues (DR6QA 24-hr queue) –If you have collab set, you will see these queues –If you don’t have it, email sdss-helpdesk@fnal.govsdss-helpdesk@fnal.gov

28. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 28 SkyServer help Schema Browser FAQ Glossary – of SDSS terms Algorithms Helpdesk

29. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 29 CasJobs help SQL IntroSchema BrowserShares SQL Intro, Schema Browser with SkyServer FAQHas its own FAQ page –Lists differences between CasJobs and SkyServer due to distributed query execution Advanced CasJobs QueriesAdvanced CasJobs Queries page –Neighbor searches with fixed and variable search radii –Cursors –Compound queries

30. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 30 Find the right tool for the job When you want to browse images, or when you know a position… ToolsVisual ToolsTools  Visual Tools  Navigate –Find images and data: Navigate Explore –Browse objects one at a time: Explore Finding Chart –Find objects on a frame, compare your own observations: Finding Chart Image List –View multiple objects or thumbnail search results: Image List

31. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 31 Finding the right tool (contd.) When you don’t know specific objects… ToolsSearchTools  Search  –Find objects in a patch of sky: Radial or Rectangular Imaging Query Form –Simple search for imaging data: Imaging Query Form Spectro Query –Simple search for spectral data: Spectro Query form SQL –More sophisticated searches: SQL –Long, complex queries: CasJobs

32. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 32 Finding the right tool (contd.) When you have a list of objects, and you want to see what SDSS knows about them… ToolsObject CrossidTools  Object Crossid  Upload –Imaging crossid: Upload SpecList –Spectro crossid: SpecList

33. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 33 Data access: general tips Use collab or astro sites –Less “frills”, more direct access to tools –Some extra features –e.g. Imaging/Spectro Form Query not on public page Each release has separate sites –http://cas.sdss.org/collab/ (the public release)http://cas.sdss.org/collab/ –http://cas.sdss.org/collabdr6/ (not yet public)http://cas.sdss.org/collabdr6/ ContactUse Contact link to contact help-desk (not direct E-mail)

34. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 34 CAS Dos and Don’ts Do notDo not submit a query unless you have some idea how long it will take! –It could tie up the server for hours (sometimes days)! –Use “select count(*)” or “select top 10” instead Do use predefined views for commonly-used objects and parameters –PhotoObjPhotoPrimary –PhotoObj, PhotoPrimary for photometry –PhotoTag –PhotoTag if you only need popular fields –SpecObj –SpecObj for spectra

35. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 35 Educational Resources Extensive education content in SkyServer Projects –Use Projects link on the menu at the top –K-12 and college level student exercises and teacher resources

36. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 36 Educational projects Go to public site – http://cas.sdss.orghttp://cas.sdss.org Go to Projects Great as labs for intro classes

37. Jordan Raddick and Ani Thakar SDSS catalog data demo, March 30, 2007 37 Focused practice Get into groups of two Share a computer Consider some task you do often in research or teaching –Some ideas in your handout Use SkyServer or CasJobs to accomplish the task We will answer questions, offer advice You have 45 minutes… go!