National Center for Supercomputing Applications Observational Astronomy NCSA projects radio astronomy: CARMA & SKA optical astronomy: DES & LSST access: NVO

The big questions: –The formation and evolution of the universe from the earliest cosmic times to the present –The nature of the “dark energy” driving the current acceleration of the universe These fundamental questions require a new era of ultra-sensitive, high-resolution imaging survey telescopes at multiple wavelengths, requiring: Surveys over large solid angle and redshift cosmic volumes Fine synoptic time-sampling and/or high astronomical completeness. These survey telescopes will have: High receptor count and data acquisition rates Software/hardware boundary far closer to receptors than at present Efficient, high-throughput survey operations modes Key questions in current astrophysics

Processing requirements: –High sensitivity, wide-field imaging –Demanding time and frequency non-imaging analysis and transient detection Data management and processing implications: –Large O(10 9 ) survey catalogs –High associated data rates (TBps) –Compute processing rates (PF) –Petabyte and Exabyte archives with sophisticated community access mechanisms The new era of great survey telescopes LSST SKA

Data rates in observational Astronomy (Szalay & Gray) Data rates are driven by: Contemporary astrophysics questions require surveys of large cosmic volumes Moore’s Law advances in detector counts and data output Increasingly sophisticated data processing needed. Data rates are exponential and require fundamentally new approaches to data management and processing. Optical data rates Telescope area CCD pixel count Radio data rates

Deep processing and data product production at Archive Center at NCSA Archive Capacities: 12 PB – 83 PB over 10 year lifetime Compute Requirements: 80 TF – 270 TF over 10 year lifetime A new optical survey telescope to be located in Chile Map entire visible sky every 3-4 days Rapid remapping for detecting moving objects in solar system and more distant transient phenomena Increasing sensitivity with each remapping will result in deep images of distant universe A tool for constraining dark energy A Data Challenge across two continents 3 Gpixel camera will produce 13 TB raw data per night Real-time processing at Base Site to detect and report new transients

Just a few LSST challenges To meet demands, part of data center must be optimized for high I/O bandwidth Drives up “spindle count” (the number of drives) over their capacity Current models suggest that meeting performance requires 10x excess DB storage Investigating new hardware and database technologies to address issues Allowing Community to leverage LSST processing cyberinfrastructure for custom processing and analysis All sky analysis not possible on desktop Scientists need to insert new processing into existing pipelines VM-based clusters and cloud technologies can provide better access for user code Sharing a massive database with the community Single DB releases will grow to over 10 TB with trillions of rows Community access will be a mix of simple and complex queries Simple queries need to return quickly (50 simultaneous queries in < 2 sec) A few complex queries may require full table scans

New-era radio interferometer in the decade One square kilometer of collecting area Science: 1)cradle of life 2)probing the dark ages 3)cosmic magnetism 4)strong field tests of gravity 5)galaxy evolution, cosmology, & dark energy A peta- or exa-scale processing and data management challenge for this decade.

National Center for Supercomputing Applications Observational astronomy faces a coming tsunami of data promising enormous scientific advances but threatening to overwhelm computational and data management resources offering great opportunities for NCSA