1. GES 2007, 3.5.2007 The German Astrophysical Virtual Observatory (GAVO) Knowledge Networking for Astronomy in Germany and abroad Gerard Lemson 1,2, Wolfgang Voges 1, Joachim Wambsganss 2 GAVO team 1 Max-Planck-Institut für extraterrestrische Physik, Garching 2 Astronomisches Rechen Institut/Zentrum für Astronomy der Universität Heidelberg

2. GES 2007, 3.5.2007 Overview Knowledge networking for astronomy: the Virtual Observatory Standardisation: the International Virtual Observatory Alliance (IVOA) The Virtual Observatory in Germany: GAVO Theory in the VObs

3. GES 2007, 3.5.2007 The status quo Astronomy produce a huge range of valuable data products, –Of sometimes astronomical sizes (soon Petabytes/yr). –From ground-based observatories and satellites –From all-sky surveys, and from large variety of individual targeted observations. –In all wavelengths. –Don’t forget computer simulations Requires large variety of disciplines/specialisations to produce and analyse. Data products are relatively quickly made public (~1 yr). –Many already available as online archives, more or less standardised, homogeneous. In combination promises interesting new science.

4. GES 2007, 3.5.2007 Some astronomical data products John Hibbard http://www.cv.nrao.edu/~jhibbard/n4038/n4038.html NASA/CXC/SAO/G. Fabbiano et al. Di Matteo, Springel and Hernquist, 2005 Images Simulations Spectra Catalogues

5. GES 2007, 3.5.2007

6. But Where is the data? What do these archives contain? How can they be accessed? How do we analyse these (very) large datasets? How can we combine them?... The Virtual Observatory (VObs) is the proposed answer to these questions.

7. GES 2007, 3.5.2007 Requires archiving curation online availability description access tools filtering tools analysis tools standardisation Promises improved communication reuse of results validation comparison combination federation collaboration new science The VObs... → O(16) national VObs projects, organized in the IVOA

8. GES 2007, 3.5.2007 IVOA

9. GES 2007, 3.5.2007 Mission statement Facilitate the international coordination and collaboration necessary for the development and deployment of the tools, systems and organizational structures necessary to enable the international utilization of astronomical archives as an integrated and interoperating virtual observatory.

10. GES 2007, 3.5.2007 IVOA organisation Since 2002 Activities divided in working - and interest groups, “tiger teams” Meet twice a year in interoperability meetings (two weeks from now, Beijing) Active mailing lists, wiki pages Standardised standardisation process

11. GES 2007, 3.5.2007 IVOA activities Working groups create standards for –publication and discovery (Resource Registry) –(meta-)data description (DM, Semantics) –selection and remote filtering (DAL, VOQL) –formats for transmitted data (VOTable) –(web) services, distributed workflows (GWS) –application interoperability (Applications) –event notification (VOEvent) interest groups represent special interests –Data curation –Grid –Theory

12. GES 2007, 3.5.2007 Some results Resource registry data models + various implementations VOTable format (XML schema) + many support tools Data models for –space-time coordinate systems –characterisation of observations –spectra Simple data access protocols for –source catalogues –2D images –1D spectra In development: –Astronomical data query language (ADQL) –Astronomical web service standards (UWS) –Application messaging –Simulation data models + access protocols

13. GES 2007, 3.5.2007 The German Astrophysical Virtual Observatory http://www.g-vo.org

14. GES 2007, 3.5.2007 GAVO I: 2002-2005 BMBF funded Partners –Astrophysikalisches Institut Potsdam (AIP) –Astronomisches Rechen Institut/Zentrum für Astronomy der Universität Heidelberg (ARI) –Max-Planck-Institut für extraterrestrische Physik (MPE), Garching –Hamburger Sternwarte –Asociated partner: Max-Planck-Institut fur Astrophysik (MPA), Garching Activities –R&D –Prototyping –Special attention: Archive publication: ROSAT, RAVE Data mining: cross-matching, classification Grid computing: simulations, distributed cluster finder Theory: virtual telescopes, archiving simulations, IVOA

15. GES 2007, 3.5.2007 GAVO II: 2006-2008 BMBF funded Partners –AIP, MPE, MPA, –Technische Universität Munchen-Informatik –Universität Tübingen Focus: move to scientifically useful services Projects Millennium database (see later) IVOA representation (theory, VOQL) Standard services (SIA, SSA,SCS) Custom services VObs expertise center at ARI...

16. GES 2007, 3.5.2007 VObs expertise center @ ARI IVOA compatible metadata repository for community Implementation data access query protocols Storage (smaller) data sets, especially science ready data Tools Outreach/PR Help-desk

17. GES 2007, 3.5.2007 Theory in the VObs: some observations Simulations not as simple as observations –less homogeneous –complex observables –no standardisation on data formats –archiving ad hoc, for local use Current IVOA standards somewhat irrelevant –no common sky –no common objects –requires data models for content, physics, code Moore’s law for N-body) simulations –Very large simulations possible –NB: also makes useful lifetime relatively short

18. GES 2007, 3.5.2007 “Moore’s law” for N-body simulations Courtesy Simon White

19. GES 2007, 3.5.2007 Virgo collaboration’s Millennium database Largest cosmological simulation to date –10 billion particles evolving under gravity –500 Mpc (~2Gly) box –64 snapshots –350000 CPU hours –O(30Tb) raw data Derived data –density fields –clusters, merger trees –galaxies, merger trees –realistic, “observed” galaxy catalogues Courtesy Volker Springel

20. GES 2007, 3.5.2007 Time evolution: merger trees

21. GES 2007, 3.5.2007 Real and Mock catalogues Courtesy Volker Springel

22. GES 2007, 3.5.2007 Database + web server Derived data products only SQLServer database Apache web server –portal: http://www.mpa-garching.mpg.de/millennium/http://www.mpa-garching.mpg.de/millennium/ –public DB access: http://www.g-vo.org/Millenniumhttp://www.g-vo.org/Millennium –private access: http://www.g-vo.org/MyMillenniumhttp://www.g-vo.org/MyMillennium Access methods –browser producing various formats, plotting capabilities –stream based wget + IDL, R, etc allows –finite query time (30sec-7min) Features –efficient tree storage+access –spatial indexing –MyDB

23. GES 2007, 3.5.2007

25. Usage statistics Up since Aug 2006 Community notified via preprint server http://xxx.lanl.gov/abs/astro-ph/0608019 http://xxx.lanl.gov/abs/astro-ph/0608019 130 registered scientific users >1.4 million individual SQL queries > 4 billion returned rows (since March 8 2007)

26. GES 2007, 3.5.2007 Summary VObs is natural extension of astronomy’s history of data archiving, standardisation, online and open access. IVOA active, questions are complex technically, but politics are sometimes even harder. GAVO relatively small, but has found some niches, particularly theory. To be successful requires use by non-VO scientists, requires proper PR.

27. GES 2007, 3.5.2007 Thank you. Further thanks to: Volker Springel, Simon White, Gabriella DeLucia, Jeremy Blaizot, Manfred Kitzbichler (MPA, Garching), Carlos Frenk, John Helly, Richard Bower (ICC, Durham, UK), Alex Szalay (JHU, Baltimore) Opening picture courtesy of NASA Goddard Space Flight Center.