Virtual Observatory Interfaces reused in the Virtual Atomic and Molecular Data Centre Guy Rixon Institute of Astronomy University of Cambridge September 2009

Rixon: VObs interfaces reused in VAMDC 2 VAMDC reusing VObs IT Virtual Observatory VAMDC Paradigms,Standards,Implementations Too complicated? Start from scratch? Too simplistic? WHY?

Rixon: VObs interfaces reused in VAMDC 3 Past, present and future Time Usefulness Era of Separate web-sites VAMDC project ??? IT support for use of A & M data does not start and end with the VAMDC project.

Rixon: VObs interfaces reused in VAMDC 4 Goals Open Easy to add users (and user communities) Extensible Easy to add data and applications Composable Easy to combine with other software Sustainable Easy (cheap) to maintain Finished in < 3 years

Rixon: VObs interfaces reused in VAMDC 5 Goal conflicts Extensible, composable, cheap: pick any two... “Good Software Takes Ten Years. Get Used To it” – Joel Spolsky

Rixon: VObs interfaces reused in VAMDC 6 How does the Virtual Observatory do? Tries to be open, extensible, composable, etc. Web services Standard protocols Resource discovery via registry Workflow support via data grids

Rixon: VObs interfaces reused in VAMDC 7 Open and Composable GUI ScriptWorkflowEtc. Vobs service HTTP calls across WAN Web (HTTP) services

Rixon: VObs interfaces reused in VAMDC 8 Extensible: adding data VObs clients Site ASite BSite C PublicPublic Private IVOA standard interfaces

Rixon: VObs interfaces reused in VAMDC 9 My community's apps My community's apps The IVOA “API library” Open & extensible: adding applications PublicPublic Private IVOA standard interfaces “The Virtual Observatory” My personal apps My personal apps My personal apps My community's apps Standard Apps & utilities Standard Apps & utilities Generic apps & utilities

Rixon: VObs interfaces reused in VAMDC 10 Extensible and sustainable: registry Publish, describe Discover, choose Resources registered in on-line directory Scientists can discover meaning of resources Software can discover interface details No need to update software to use new resources

Rixon: VObs interfaces reused in VAMDC 11 What can be registered? Hypothetical extension by VAMDC

Rixon: VObs interfaces reused in VAMDC 12 Structure of registration document Resource identity Curation Content synopsis Capability (TAP) Capability (VAMDC) Tableset “Dublin core”: meaningful to science users Used to select services. Each Capability describes a major interface Lists access URL(s) and support for optional parts of interfaces. Machine readable detail; drives UIs. Describes the tables available for query. Machine-readable detail; drives query building UIs.

Rixon: VObs interfaces reused in VAMDC 13 Relational vs. XML storage Relational: Robust, mature Choice of RDBMS SQL query Hard to add new schemata Native XML DB New(ish) technology Few implementations XQuery Trivial to add new schemata Registries have been built on both relational and native-XML databases. XQuery + schema flexibility wins every time.

Rixon: VObs interfaces reused in VAMDC 14 Publishing, harvesting, searching “Full” registry NVO (STSci) Publishing registry Vizier Publishing registry CASU Publishing registry VAMDC registry Etc. All registrations “harvested” to full registry VAMDC providers register Services here Virtual Observatory VAMDC VAMDC users search here

Rixon: VObs interfaces reused in VAMDC 15 Extensible and composable: data grid Service AService BService C VOSpace Fast network My app. Slower network

Rixon: VObs interfaces reused in VAMDC 16 UIs: VOSpec Multi-wavelength spectral analysis tool developed at ESAC by ESA-VO Create a SED with spectra from: a wide range of different providers registered in the VO Registry local data (FITS, ASCII, VOT) Many fitting/ plotting functions ISO LWS HST ISO PHT EUVE HUT HST FOS IUE INES

Rixon: VObs interfaces reused in VAMDC 17 VOSpec cont. VOSpec provides access to atomic and molecular line Databases (SLAP): NIST Atomic Spectra CHIANTI (X-rays) CIELO (XMM-Newton) ‏ IASD (ISO) ‏ LERMA (molecular) ‏ STSCI (UV) on-the-fly identification of transitions associated to a given observed feature.

Rixon: VObs interfaces reused in VAMDC 18 UIs: Taverna

Rixon: VObs interfaces reused in VAMDC 19 UIs: VODesktop

Rixon: VObs interfaces reused in VAMDC 20 VODesktop: actions on resources

Rixon: VObs interfaces reused in VAMDC 21 VODesktop: query builder

Rixon: VObs interfaces reused in VAMDC 22 VODesktop/SAMP: UI integration

Rixon: VObs interfaces reused in VAMDC 23 VODesktop: data-grid access

Rixon: VObs interfaces reused in VAMDC 24 Looter's checklist Web-service paradigm Registry paradigm Data-grid paradigm VODesktop Other UIs/desktop applications (as applicable) Service standards (as applicable) Service implementations (as applicable)

Rixon: VObs interfaces reused in VAMDC 25 IVOA standard services StandardPurposeData modelRelevance? RegistryDiscovery of servicesExtensibleHigh Cone searchData access: sky searchAstronomicalNone SIAPData access: imagesAstronomicalNone SSAPData access: spectraAstronomicalLow/medium TAPData access: tablesGenericHigh SimDB/SimDAPData access: simulationsGeneric??? SLAPData access: line listsAstronomicalMedium/high VOSpaceGrid-accessible storageGenericMedium UWS-PA/CEAGrid executionGenericMedium

Rixon: VObs interfaces reused in VAMDC 26 The data-model issue Web-service shell Data-model Resource adaptor Requests Results What the client needs Elaborate data-model makes it hard to repurpose the service, especially if domain- specific For reuse, prefer services with little or no data model

Rixon: VObs interfaces reused in VAMDC 27 TAP: Table Access Protocol Web-service shell Data-model Resource adaptor: async. queries on JDBC Requests: SQL Results: VOTable/HTML/CSV/??? Relational database TAP data-model is very generic: “relational data” Works with any DB schema

Rixon: VObs interfaces reused in VAMDC 28 TAP: possible use in VAMDC year 1 TAP service TAP DB VAMDC App (desktop) Describe DB schemata in registry Reuse same implementation on each VAMDC site Sites provide these Deal with schemata Variations here One per VAMDC site

Rixon: VObs interfaces reused in VAMDC 29 TAP: possible evolution Web-service shell Data-model Resource adaptor: async. queries on JDBC Requests: XQuery Results: XSAMS Relational database Replace the TAP data- model with something VAMDC-specific Works with any DB schema Extend this for VAMDC

Rixon: VObs interfaces reused in VAMDC 30 VOSpace Storage accessible to users and services Useful in workflows, grid jobs Can use simple storage or data grid Site ASite B iRODS Site C iRODS Site C iRODS VOSpace

Rixon: VObs interfaces reused in VAMDC 31 SLAP: Simple Line Access Model Web-service shell Data-model Resource adaptor Requests Results: VOTable Relational database SLAP data-model is very simple: “just enough for astronomy” Needs a particular schema

Rixon: VObs interfaces reused in VAMDC 32 CEA & grid CEA = Common Execution Architecture Describe, register, discover, run server-side apps: grid “A standard for everything else” Powerful: CEA UI runs any registered app Allows long-running jobs (asynchronous execution) Can be connected to the Grids, e.g. EGEE Integrates with VOSpace, Taverna workflow

Rixon: VObs interfaces reused in VAMDC 33 If all else fails......design a new service type! Need to specify: Service semantics Web-service protocol (e.g. REST) Query language (e.g. XQuery) Representation of results (e.g. XSAMS) XML schema for registration Consider following IVOA “DAL” architecture

Rixon: VObs interfaces reused in VAMDC 34 VAMDC strategy Adapting to new service types VODesktop UI Registry parser AstroRuntime SLAP strategy SSAP strategy TAP strategy CEA strategy VAMDC strategy

Rixon: VObs interfaces reused in VAMDC 35 Summary VAMDC needs adaptable, future-proof software on the cheap We think the VObs model will work We will copy the VObs ideas We will use their service protocols to start We will use the ready-made software We will exploit the flexibility of the system progressively to fit it to VAMDC.