Final Data Archiving of the Sloan Digital Sky Survey-an Example Denver Sci Com Don York Elisabeth introduces presentation and emphasizes desire for discussion and ideas to inform scientists as to what might be possible with what is regarded as critical library involvement 9/13/16 Denver Sci Com
The science world is changing with the advent of petabyte (and more) datasets that are available. In the case of astronomy, the telescopes that produce the data sets are of an ever-changing sky. Facilities go out of date, but the data do not (see summary, Hipparchus 140 B. C. and Halley 1718.) A major issue is archiving the data so it remains publicly available and safe after observations cease. The same issues do or will confront scientists and agencies in geology, climate studies, high energy physics, … 9/13/16 Denver Sci Com
16 years of digital data released publicly from SDSS. The Sloan Digital Sky Survey is an exemplary case for considering the solution to the problem (and many more such projects are coming). In retrospect, SDSS was the start of “BIG DATA” 16 years of digital data released publicly from SDSS. 13 Data Releases science-ready and freely accessible (including advanced products) Papers from first eight years of data (2000-2007) still heavily cited (600 papers per year). Comparable to HST at <1/10 the cost. Cumulative total: over 7200 papers, 330,000 citations. Project was planned as a focused survey, but the data are now used by astronomers in most branches of astronomy, as well as by STEM groups. Survey observations will continue for 5-10 years. Archive is expected to have a very long life (>50 years) 9/13/16 Denver Sci Com
How are we proceeding? Scope (over 10 years) Strong belief that libraries are the ultimate best respository Centuries of curation experience Exist at the center of academia Already experienced with research collections Moving to modern digital collections Working models of distributed systems within expansive networks Scope (over 10 years) Collect paper archives, drawings, “history” into federated catalogue (project involves dozens of institutions and hundreds of astronomers) Collect and archive all versions so research results can be verified later. Scientists (in close coordination with libraries) upgrade reduction software to accompany archival data (and allow modifications) 9/13/16 Denver Sci Com
Three major stakeholders: Who pays for the archive? Three major stakeholders: The international community of scientists who use the data. The governments that fund the research. The libraries that distribute and preserve the data for their campus constituents and enable new approaches to education. Discussions with long lead times and with all at the table will be necessary to divide up the costs in the way most natural to these stakeholders. Serving of data will be needed by many Federally funded projects, should be done nationally (NOT by specialized hardware, library by library). Inexpensive compared to costs of the projects that generate the data, if done this way. Prototypes underway. 9/13/16 Denver Sci Com
FIN 9/13/16 Denver Sci Com
Slides for questions http://skyserver.sdss.org/dr7/en/ http://www.sdss.org/dr12/ DR7 homepageFamous PlacesNGC450SDSSnavigate, play with image, go to blue pea, spectrum. DR12 homepageSDSS Sky ServerSearchFamous Places 9/13/16 Denver Sci Com
SDSS IV Participating Institutions (34) Brazilian Participation Group, Carnegie Institution for Science, Carnegie Mellon University, Chilean Participation Group, French Participation Group, Harvard-Smithsonian Center for Astrophysics, Instituto de Astrofísica de Canarias, The Johns Hopkins University, Kavli Institute for the Physics and Mathematics of the Universe (IPMU) / University of Tokyo, Lawrence Berkeley National Laboratory, Leibniz Institut für Astrophysik Potsdam (AIP), Max-Planck-Institut für Astronomie (MPIA Heidelberg), Max-Planck-Institut für Astrophysik (MPA Garching), Max-Planck-Institut für Extraterrestrische Physik (MPE), 9/13/16 Denver Sci Com
National Astronomical Observatory of China, New Mexico State University, New York University, University of Notre Dame, Observatório Nacional / MCTI, The Ohio State University, Pennsylvania State University, Shanghai Astronomical Observatory, United Kingdom Participation Group, Universidad Nacional Autónoma de México, University of Arizona, University of Colorado Boulder, University of Oxford, University of Portsmouth, University of Utah, University of Virginia, University of Washington, University of Wisconsin, Vanderbilt University, Yale University. 9/13/16 Denver Sci Com
What would be the motivations for libraries to be interested? Questions What would be the motivations for libraries to be interested? -Preserve usabililty of Data for their own institutional researchers. -Gain experience with a widely used system and prepare for use of additional and bigger astronomy projects (LSST, DESI, DES,….). -Maintain in a more complex media the traditional spirit of openness and careful curation that is the hallmark of the print function of libraries. 9/13/16 Denver Sci Com