, TriestEuro-VO DCA WP5 kick off Euro-VO DCA WP4 and GAVO and (a little) AstroGrid-D Gerard Lemson Euro-VO DCA WP5 kick-off meeting Trieste,

, TriestEuro-VO DCA WP5 kick off Overview Euro-VO DCA WP4 –Goal and motivation –Theory in VO –Theory in IVOA –Theory in Euro-VO DCA: WP4 –WP4 and WP5: theory WP5 and/or grid in WP4 ? GAVO ( & AstroGrid-D ) –GAVO I –GAVO II –Theory in GAVO –Grid in GAVO –AstroGrid-D –(GA)VO and AstroGrid-D

, TriestEuro-VO DCA WP5 kick off Euro-VO DCA WP4 Objectives –This task aims at assessing the inclusion of theory data and services in the Virtual Observatory Description of work –The task is tackled by the Theory Expert Group, which has yearly meetings and discussions to establish a framework for the inclusion of Theoretical Astronomy Services in the VObs. The TEG organizes a Workshop. Deliverables –WP4-D10: Euro-VO Theory Workshop –WP4-D11: TEG Report: Framework for inclusion of Theoretical Astronomy Services in the VObs Expected results –The final report will describe a framework for inclusion of theoretical data and services in the VObs –Motivation of the European Theoretical astronomy community to participate in the VObs endeavour.

, TriestEuro-VO DCA WP5 kick off Reminder Broad goal of VObs –Make results of astronomical research, data and applications, more readily available to larger community, and create value-adding services. Facilitate results- –communication –checking –(re)use –comparison –combination Organised in IVOA

, TriestEuro-VO DCA WP5 kick off The observational VObs Most VO efforts concentrate on observational data sets –simple observables: photons detected at a certain time from a certain area on the sky –long history of archiving –pre-existing standards (FITS) –valuable over long time (digitising 80 yr old plates) –long history of comparing different observations: interoperability Standards observationally biased –common sky: cone search, SIAP, region –common objects: XMatch –data models: characterisation of sky/time/energy (/no polarisation yet)

, TriestEuro-VO DCA WP5 kick off The same object John Hibbard Radio NASA/CXC/SAO/G. Fabbiano et al. X-Ray Optical

, TriestEuro-VO DCA WP5 kick off Theory in VO Apply same ideas to theoretical resources (datasets and services). Plenty of resources available online. Not standardised: –2004, whitepaper detailing issues related to standardisation of theory

, TriestEuro-VO DCA WP5 kick off Theory in the VO: issues Simulations not so simple –complex observables –no standardisation (not even HDF5) –archiving ad hoc, for local use Moore’s law makes useful lifetime relatively short: few years later can do better

, TriestEuro-VO DCA WP5 kick off “Moore’s law” for N-body simulations Courtesy Simon White

, TriestEuro-VO DCA WP5 kick off Standardisation issues Current IVOA standards somewhat irrelevant –assume common sky –common objects –comparison direct In theory –no common sky –no common objects –comparison statistical requires new data models –content, physics, code

, TriestEuro-VO DCA WP5 kick off Not the same object Toomre & Toomre, 1972 Di Matteo, Springel and Hernquist, 2005 Courtesy Volker Springel

, TriestEuro-VO DCA WP5 kick off So why bother ? Simulations are interesting: –For many cases only way to see processes in action –Complex observations require sophisticated models for interpretation Bridging gap in specialisations: not everyone has required expertise to create simulations, though they can analyse them. Many use cases do not require the latest/greatest –exposure time calculator –survey design

, TriestEuro-VO DCA WP5 kick off Theory in the IVOA IVOA Interest group on theory –founded May 2004 –“horizontal” –use cases Current projects: –Theory UCDs/Semantics (Laurie Shaw) –SNAP (Claudio Gheller) –Simulation (meta-) data model (GL) Meeting on SNAP in Garching, early 2007

, TriestEuro-VO DCA WP5 kick off Euro-VO DCA WP4 (repeat) Objectives –This task aims at assessing the inclusion of theory data and services in the Virtual Observatory Description of work –The task is tackled by the Theory Expert Group, which has yearly meetings and discussions to establish a framework for the inclusion of Theoretical Astronomy Services in the VObs. The TEG organizes a Workshop. Deliverables –WP4-D10: Euro-VO Theory Workshop –WP4-D11: TEG Report: Framework for inclusion of Theoretical Astronomy Services in the VObs Expected results –The final report will describe a framework for inclusion of theoretical data and services in the VObs –Motivation of the European Theoretical astronomy community to participate in the VObs endeavour.

, TriestEuro-VO DCA WP5 kick off WP4: tasks Procurement WP4 personnel: Draft whitepaper “Framework for inclusion of Theory data and services in the VObs” (End cycle 1) EURO-VO Theory Workshop (Deliverable 10, Month 17) Whitepaper ready (Deliverable 11, month 26) Participation IVOA interop meetings with theory

, TriestEuro-VO DCA WP5 kick off WP4 first steps Formation Theoretical astronomy Experts Group (TEG) CNRSHervé WOZNIAK ESAPedro OSUNA ESONone INAFSanti CASSISI INTAMiguel CERVIÑO MPGGerard LEMSON (Chair) NOVATBD (Joop SCHAYE) LUTBD (?) Census theory data centers (with WP2) Cooperation in IVOA theory interest group activities Coordination with WP 5 (which is why I’m here)

, TriestEuro-VO DCA WP5 kick off WP4 and WP5: some first ideas Grid support for WP4 –Grid must be addressed in WP4 whitepaper –Grid can facilitate/is required for implementations of prototype for standards (see below) Theory use cases/requirements for WP5. –Asynchronous, queued querying of large simulation database OGSA-DAI? (GAVO+AstroGrid-D) –SNAP on Millennium raw data grid services wrapping custom data access modules (GAVO+AstroGrid-D) –Scientifically useful virtual telescopes –Parameter studies using “small” simulations –Large simulations ??

, TriestEuro-VO DCA WP5 kick off Observed Simulated Thanks to Alexis Finoguenov, Ulrich Briel, Peter Schuecker, MPE) Thanks to Volker Springel Example: Virtual Telescope

, TriestEuro-VO DCA WP5 kick off Virtual telescope configuration

, TriestEuro-VO DCA WP5 kick off Virtual telescope configuration Components: –simulation of source, wrapped with “photon producer” service –spec and implementation for “pre-observation image” (POI) –“foreground” accepting, reemitting POI ISM/IGM lenses –“software telescope”, simulating realistic observations –comparison software Required (VO/Grid) infrastructure supporting this workflow –registry for publishing/discovering components –computational resources –scheduler for executing job –streaming support for flowing POI between components –storage for results –workflow engine?

, TriestEuro-VO DCA WP5 kick off Discussion Suggestions very welcome

, TriestEuro-VO DCA WP5 kick off Intermezzo

, TriestEuro-VO DCA WP5 kick off GAVO and AstroGrid-D

, TriestEuro-VO DCA WP5 kick off GAVO and AstroGrid-D GAVO I –Funded by German Ministry of –Dec 2002 – March 2005 –3 months extension –3 FTEs AstroGrid-D –1 of 5 funded “communities” taking part in D-Grid project –O(13) FTEs –Autumn 2005 – 2007/8 GAVO II –Delayed funding, due to need to distinguish from AstroGrid-D –June 2006-July 2008 –5 FTEs (so far 3, ARI (>~1.5), MPE (1.5), TUM (.5), AIP (<~.5), Tübingen (1))

, TriestEuro-VO DCA WP5 kick off GAVO (I + II) projects Archives: –ROSAT All Sky Survey cone search: photons, sources SIAP: fields –ROSAT pointed observations: SIAP standardised approach to publish FITS images under SIAP uses FITS metadata repository (SQLServer) SQL based mapping and SIAP implementation easily generalised to other datasets and protocols (SSA) –RAVE –CDFS optical follow up spectroscopy: SSA –Heidelberg catalogues and GAVO archive: SCS/SkyNode Data mining –Cross-matcher (also on Grid, TUM) –SED classifier –Cluster finder

, TriestEuro-VO DCA WP5 kick off GAVO (I + II) projects ctd. Grid –evaluation Globus –Gadget 2 on Globus MPI (AIP + MPE) –cluster finder on the grid VOQL/SkyNode –VOQL tiger team (Benjamin Gufler) –HiSbase peer-to-peer approach to multi catalogue cross- matching –Mock catalogues for testing Theory –IVOA: Initialised Theory Interest Group (chair) SNAP Data models Semantics –Millennium databasedatabase –(proto-) Virtual telescopes (planck, MoMaF2, Dolag et al)planckMoMaF2Dolag et al

, TriestEuro-VO DCA WP5 kick off AstroGrid-D Part of D-Grid Community project Provide –use cases –middleware –adaptation of existing applications/data to the (D-)Grid

, TriestEuro-VO DCA WP5 kick off AstroGrid-D partner projects (incomplete) Astrophysikalisches Institut Potsdam –project management –robotic telescopes on the grid –galaxy merger simulations MPE: –cluster finder –catalogue cross-matcher (with TUM) –adapt simulation visualisation to the Grid –access to persistent data (in database) MPA –Planck Process Coordinator on the Grid Technischer Universitat München-Informatik –StarGlobe: streaming data querying/combination

, TriestEuro-VO DCA WP5 kick off AstroGrid-D partner projects, ctd. Zuse Institut Berlin –Grid Application Toolkit –GridSphere web portal –Information Services (“registry”, RDF+SparQL) Albert Einstein Institute, Berlin –Cactus tool kit –Grid based simulation monitoring Astronomisches Rechen Zentrum, Heidelberg –Stellar dynamics: Grape on the Grid –GaiaGrid –Planck ProC collaboration

, TriestEuro-VO DCA WP5 kick off GAVO + AstroGrid-D, synergies Groups involved in both projects: AIP, ARI, MPE, TUM VO offers different kind of use cases: –VObs aims to enable external users to use resources –Heterogeneous world: Interoperate with resources outside of AstroGrid-D (but within IVOA standards) Grid can help implement VO services

, TriestEuro-VO DCA WP5 kick off GAVO + AstroGrid-D ctd. Planned (from GAVO point of view) –Cluster finder (MPE) –SNAP implementation (MPA) –OGSA-DAI (GSM?, DSA?, CAS Jobs) access to Millennium DB (MPA) –Real virtual telescopes (MPA+...) –Cross matching (MPE+TUM) Possibly –VOEvent and Robotic Telesopes ? –AstroGrid-D Information Services as IVOA Registry

, TriestEuro-VO DCA WP5 kick off SNAP using grid services Simple Numerical Access Protocol –Project of IVOA theory interest group –Based on simulation data model (input welcome !) –Query interesting simulations –Select and retrieve subset Implementation: SNAP on Millennium snapshots –Custom access code deployed near large datasets –Authenticated access –Standardised publication for use in Grid workflows (ProC? Streaming?) –Data staging and movement

, TriestEuro-VO DCA WP5 kick off Millennium snaphots getBox() C-module Grid Service Globus container Information Service SNAP web service IS aware Workflow Engine

, TriestEuro-VO DCA WP5 kick off Millennium website and OGSA-DAI Dataset – postprocessing products of Millennium simulation ( garching.pg.de/millennum) –halo/galaxy catalogues –density fields (environment queries) Functions –support generic SQL queries on large (O(5Tb)), complex database –support mydb –some graphics script based access

, TriestEuro-VO DCA WP5 kick off Implementation Microsoft SQLServer database (SkyServer inspired) –data model for efficient retrieval merger tree histories –spatial indexing TOMCAT web server – –authentication using RDB –wget based access, IDL+R modules Graphics –VOPlot applet (small results, browser) –TOPCAT (plugin) Synchronous queries –JDBC –7 min time limit Issues –I/O bottle necks when > 4 users query same table –short maximum query time –no asynchronous access –simplistic user management (compared say to CAS Jobs)

, TriestEuro-VO DCA WP5 kick off

, TriestEuro-VO DCA WP5 kick off Why Grid interesting? Need –Asynchronous access Queuing Improve result management, data delivery –Bulk uploading into MyDB –More generic authentication (X.509) Will try CAS jobs (SDSS/JHU) Investigate OGSA-DAI –Map portal to OGSA-DAI functionality –Monitoring ? Logging ? Performance ? Will try AstroGrid-UK’s DSA Other suggestions ?

, TriestEuro-VO DCA WP5 kick off

, TriestEuro-VO DCA WP5 kick off Thank you !