1. Astronomy of the Next Decade: From Photons to Petabytes R. Chris Smith AURA Observatory in Chile CTIO/Gemini/SOAR/LSST

2. “Classical” Optical Astronomy 1-4 investigators propose for telescope time  Obtain 1 to 5 nights Travel to distant telescope site Observe  or not: clouds (OUCH!) Take 5 to 50 GB of data home (on tapes) Reduce & Analyze “by hand”  Extract every detail from those bits  Often takes months per night of data 2Photons to Petabytes 2014

3. The “Modern Classic” 3Photons to Petabytes 2014

4. Optical Windows 4Photons to Petabytes 2014

5. Classical Astronomy still dominates new facilities Even new large facilities (VLT, Gemini, ALMA, GMT, E-ELT) are and will be scheduled for “individual projects”  In units of nights, sometimes hours! But methods are changing…  Sloan Digital Sky Survey led the way Statistical analyses -> new discoveries  Surveys and science with massive datasets are growing, filling an important need Photons to Petabytes 20145

6. Today’s BIG Questions: Dark Energy & Dark Matter Dark Energy is the dominant constituent of the Universe. Dark Matter is next. 95% of the Universe is in Dark Energy and Dark Matter, for which we have little or no detailed understanding. 1998 and 2003 Science breakthroughs of the year, 2011 Nobel Prize 6Photons to Petabytes 2014

7. Attacking the Question of Dark Energy & Others “Classical” approach won’t work  Not enough telescope time in 2-5 night “chunks” LARGE SURVEYS  Goal: Provide large, uniform, well calibrated, controlled, and documented datasets to allow for advanced statistical analyses  Larger and broader collaborations provide both manpower and diverse expertise NEED…  NEW INSTRUMENTS  NEW TELESCOPES  NEW METHODS 7Photons to Petabytes 2014

8. Sociology of Dark Energy Dark Energy may be pushing the universe APART But it is pulling the Astronomy, Physics, Mathematics, and CompSci communities TOGETHER  New physics, new algorithms, new processing capabilities (h/w & s/w), new access methods (DBs, fast networks) 8Photons to Petabytes 2014

9. Selected Examples: Coming soon to nearby mountaintops… New Instruments (DECam) New Telescopes (LSST) Photons to Petabytes 20149

10. Dark Energy Survey (DES) 5 year project to improve our understanding of Dark Energy  Key DOE/NSF collaboration: Fermilab/NOAO/NCSA  International collaboration: Brazil, UK, Spain, Germany Characterize Dark Energy with four methods  Supernovae  Weak Lensing (also measure Dark Matter)  Galaxy clustering  Baryon Acoustic Oscillations All depend on careful statistical analyses of large datasets 10Photons to Petabytes 2014

11. Dark Energy Camera Optical Lenses CAMERA: 62 2048 x 4096 pixel CCDs 570 Megapixel camera The largest focal plane for astronomy in S. Hemisphere 11Photons to Petabytes 2014

12. DECam is here TODAY First light images: September 12, 2012 Fornax galaxy cluster Photons to Petabytes 201412

13. Photons to Petabytes 201413

14. Small Magellanic Cloud ~5 billion measurable stars in this single image! Photons to Petabytes 201414

15. A “modest” data challenge Each image 1GB; 300–700 GB of raw data/night  Data must be moved from Chile to NCSA before next night begins (<18 hours ), preferably in real time  YEAR 1: Each image transferred in <120 sec!  Data must be processed within <24 hours to inform next night’s observing: using NCSA resources  YEAR 1: Real-time pipeline processing on Tololo with LIneA QuickReduce pipeline: robust and reliable  Initial processing completed at NCSA in <24 hours, still with only limited data quality specifications TOTAL Dataset will be ~5 PB Photons to Petabytes 201415

16. The next step… ca. 1950 POSS (Photographic) ca. 2000 SDSS (Digital) ca. 2012 DES (Digital + Depth) ca. 2020 LSST (Digital Sky +Time Domain) 16Photons to Petabytes 2014

17. LSST is designed to image the whole sky every few nights for 10 years, giving us a movie-like window into our dynamic Universe. 8.4 M Telescope –3.5 Degree Field Of View –Telescope Located in Chile on Cerro Pachón 3.2 Billion Pixel Camera ~40 Second Cadence –Two 15 second exposures –Full sky coverage every few nights Advanced Data Management Systems Public Data –Alerts of new events –Catalogs of object –Archives of images Next Step = LSST: Creating a “Digital Universe” Next Step = LSST: Creating a “Digital Universe” 17Photons to Petabytes 2014

18. The Large Synoptic Survey Telescope – Massively Parallel Astrophysics Survey the entire sky every 3-4 nights, to simultaneously detect and study:  Dark Matter via Weak gravitational lensing  Dark Energy via thousands of SNe per year  Potentially hazardous near earth asteroids  Tracers of the formation of the solar system  Fireworks in the heavens – GRBs, quasars…  Periodic and transient phenomena ...…the unknown 18Photons to Petabytes 2014

19. Why is the LSST so unique? Primary Mirror Diameter Gemini South Telescope 0.2 degrees 8 m Field of View 3.5 degrees (Full moon is 0.5 degrees) LSST 8.4 m 19Photons to Petabytes 2014

21. 1,380 m 2 service and maintenance facility 30 m diameter dome Control room and heat producing equipment (lower level) 1.2 m diameter atmospheric telescope 350 ton telescope Base Facility Includes the facilities, and hardware to collect the light, control the survey, calibrate conditions, and support all LSST summit and base operations. Telescope and Site 21Photons to Petabytes 2014

22. Camera 3.2 Gigapixel science array – 10 square degree FOV! Wavefront and guide sensors 2 second readout 5 filters in camera Filter L1 Lens Utility Trunk—houses support electronics and utilities Cryostat—contains focal plane & its electronics Focal plane L2 Lens L3 Lens Camera ¾ Section 1.65 m (5’-5”) 22Photons to Petabytes 2014

23. Each image roughly 6.5GB Cadence: ~1 image every 15s 15 to 18 TB per night, 30TB “reduced”!  ALL must be transferred to NCSA archive center within image timescale (17s), >>10 Gbps REAL TIME reduction, analysis, & alerts  Send out alerts of transient sources within 60s ~2 million events per night every night for 10 years  Provide automatic data quality evaluation, alert to problems  Change survey observing strategy on the fly based on conditions, last field visited, etc. Petascale Data Management 23Photons to Petabytes 2014

24. LSST: “Data Science” in real time TRANSIENT SCIENCE (Data Stream)  >3 Terabytes per hour (reduced) that must be mined in real time for alerts.  20 billion objects will be monitored for important variations in real time.  ~2 million events per night every night for 10 years New approaches must be developed for knowledge extraction in real time NON-TRANSIENT SCIENCE  >10 10 objects in a 20 PB final database catalog, backed by a 100 PB final image archive New approaches to data mining needed to sift through data to identify samples, or individual objects, of interest Photons to Petabytes 201424

25. Summit Site Summit Facility Telescope and Camera Data Acquisition Crosstalk Correction Data Management Sites and Centers Base Site Base Facility Long-term storage (copy 1) Data Access Center Data Access and User Services HQ Site HQ Facility Observatory Management Science Operations Education and Public Outreach French Site Processing Center Data Release Production Archive Site Archive Center Alert Production Data Release Production Calibration Products Production EPO Infrastructure Long-term Storage (copy 2) Data Access Center Data Access and User Services Photons to Petabytes 201425

26. LSST Data Management: Baseline Solutions High-speed connectivity  Mountain to Base: >100 Gbps  Base to Archive: >10 Gbps (hopefully 100Gbps)  Archive to User: variable, UI challenge Supercomputer processing & storage  Base in La Serena, NCSA, Others? (France?, Brazil?)  100 PB final image archive  Distributed (Grid) analysis facilities Petascale DB (~20 PB final catalog)  Based on open source RDBMS Photons to Petabytes 201426

27. LSST: Strategic Partnerships Distributed Computing Systems  Supercomputer center(s) to provide bulk storage, large scale processing (e.g., NCSA, NLHPC in Chile)  Grid processing, storage, advanced DB  Data Access for member countries/institutions Connectivity  High-speed Chilean bandwidth (REUNA)  International bandwidth (AmLIGHT, RedCLARA) Scientific Analysis Challenges: Data Mining & Astro-Informatics or Astro-Statistics  Separating small signals from systematic effects  Automatically finding unique objects: one in billions Photons to Petabytes 201427

28. LSST Outreach Data will be used in classrooms, science museums, and online Classroom Emphasis on: Data-enabled research experiences Citizen Science College classes Collaboration through Social Networking 28Photons to Petabytes 2014

29. Integrated Project Schedule with Key Milestones FUNDING STARTS NOW!

30. The Science of Big Data Data growing exponentially, in all sciences Changes the nature of science from hypothesis-driven to data-driven discovery Cuts across all sciences Industry and government face the same challenges Convergence of physical and life sciences through Big Data (statistics and computing) A new scientific revolution Photons to Petabytes 201430

31. Photons to Petabytes 201431

32. 32

33. DOE/NSF Joint Interface and Management Review Tucson, Arizona May 30-June 1, 2012 Construction NOW First light in 2019 Operations in 2022