A PPARC funded project AstroGrid approach to the Virtual Observatory Architecture ADASS XIV Pasadena Oct-2004 Tony Linde Andrew Lawrence Keith Noddle

2 26-Oct-2004AstroGrid approach to Virtual Observatory architecture Goal Develop software infrastructure to enable Virtual Observatory Deploy that infrastructure in the UK Virtual Observatory “The VO vision can be summed up as the desire to make all archives speak the same language − all searchable and analysable by the same tools, all data sources accessible through a common interface, all data held in distributed databases that appear as one.” Andy Lawrence, 09/2003

3 26-Oct-2004AstroGrid approach to Virtual Observatory architecture Overview of AstroGrid/-2 Duration:Sept 2001 – Dec 2007 Funding:£7.7M (PPARC) Personnel:~26 (23.4 FTE) Goal:Develop VObs Infrastructure Deploy UK VObs Scope:Astrophysics, Solar, STP, … Optical, X-Ray, Radio, … AG1 Phase A: Sept 2001 – Dec 2002 Analysis, R&D, Architecture AG1 Phase B: Jan 2003 – Dec 2004 Build, test & deliver AstroGrid-2: Jan 2005 – Dec 2007 Structure tbd

4 26-Oct-2004AstroGrid approach to Virtual Observatory architecture AstroGrid consortium Edinburgh Leicester Cambridge MSSL JBO RAL QUB From 2005, AstroGrid-2 + Bristol Exeter Leeds Portsmouth

5 26-Oct-2004AstroGrid approach to Virtual Observatory architecture Lead Personnel Project Lead Andy Lawrence (Edinburgh)  Project Scientist Nic Walton (Cambridge)  Project Manager Tony Linde (Leicester)  Technical Lead Keith Noddle (Leicester) 

6 26-Oct-2004AstroGrid approach to Virtual Observatory architecture Links to other ventures EU FP5: AVO, FP6: Euro-VO (UK rep) Stop Press: €3.3M for VOTech US-NVO Interoperable registries (done) Interoperable applications (goal – Jan’05) IVOA Standards workgroups GGF, Globus, OGSA-DAI Other e-Science projects in UK GridPP, EGSO, myGrid, …

7 26-Oct-2004AstroGrid approach to Virtual Observatory architecture Links (URLs) Virtual observatory ventures: AVO/Euro-VO: VOTech: US-VO: IVOA:  Mailing lists, Wiki EGSO: e-Science/Grid ventures NeSC: GGF: Globus:

8 26-Oct-2004AstroGrid approach to Virtual Observatory architecture Project Approach Agile: iterative & incremental development Open project: Wiki Forum News Jabber via jabber.astrogrid.org Open Source code Academic Free License AstroGrid releases

9 26-Oct-2004AstroGrid approach to Virtual Observatory architecture Development Approach Iterative full cycle each iteration 3 month(-ish) iterations Incremental add functionality each iteration refactor wherever necessary Use-case driven from ‘key 10’ science cases Architecture led

10 26-Oct-2004AstroGrid approach to Virtual Observatory architecture AstroGrid-2 Aspects Extension  Infrastructure  Grid (GT4 + ) integration  Intelligent resource discovery (semantic grid)  Full security: AA(A) Shibboleth integration? Additional components  Data exploration & Visualization  Client-based access: VO Workbench  Technical Scope

11 26-Oct-2004AstroGrid approach to Virtual Observatory architecture AstroGrid-2 Layer Diagram

12 26-Oct-2004AstroGrid approach to Virtual Observatory architecture More info: News

13 26-Oct-2004AstroGrid approach to Virtual Observatory architecture More info: Forum

14 26-Oct-2004AstroGrid approach to Virtual Observatory architecture More info: Wiki

15 26-Oct-2004AstroGrid approach to Virtual Observatory architecture AstroGrid Posters at ADASS Portal and Workflow AstroGrid and the Registry: Enabling Resource Discovery Access Control in AstroGrid software The AstroGrid Common Execution Architecture The Publisher's AstroGrid Library

16 26-Oct-2004AstroGrid approach to Virtual Observatory architecture Portal and Workflow Supports workflow paradigm (from the simple to the complex) Queries and workflows are designed using the portal Work is run remotely and asynchronously Archives searched and results manipulated Results are stored in a virtual file system Queries and workflows can be re-used and shared

17 26-Oct-2004AstroGrid approach to Virtual Observatory architecture Enabling Resource Discovery Types of Registries:– Full, Publish, Special Registry is the main focal point for all Astrogrid components Agreed Standards with IVOA Search and harvest interface OAI standard for harvest interface Adheres to the Astrogrid CEA standard Types of resource Generic services, web services, applications, … Data collections AstroGrid-specific resources (e.g. MySpace servers) Use of XQuery language with eXist XML database. Harvesting US NVO VizieR

18 26-Oct-2004AstroGrid approach to Virtual Observatory architecture Access Control AstroGrid has taken the lead in AAA because our system is based on writable storage Implement single-sign-on (SSO) authentication Using a community service (so, account can be used with a number of portals or other UIs) Programme-to-programme authentication based on message security with SOAP headers and WS-Security Facade API to hide details from implementations Underneath, use JAX-RPC/Axis handlers to secure messages. Little authorization support in 2004, more in 2005 groups, quotas, community authorization, file sharing Currently specific to AstroGrid Working with IVOA to design and adopt a common standard

19 26-Oct-2004AstroGrid approach to Virtual Observatory architecture Common Execution Architecture Uniform interface for running applications in VObs Set of WSDL and schema Define the interactions in language independent way Adapters (ExecutionControllers) implemented Using Java Web services To run legacy command-line applications To run legacy HTTP GET/POST apps within the framework ExecutionControllers Allow asynchronous operation of the applications Callback and polling mechanisms for delivery of results Tightly integrated with JES (Job Execution System) Allows programmable workflows

20 26-Oct-2004AstroGrid approach to Virtual Observatory architecture Publisher's AstroGrid Library Task: What's involved in publishing data to the VO? Library of components Make a data owners life a (relatively) straight-forward transition to a data owner/publisher Variety of ‘plugins’ Cope with different collections of data (modes of storage)  To make life easier for installers  and test how they might be presented to the VO Ready-made web UI Suitable for astronomers to use, Includes useful configuration & monitoring features VO-compatible interfaces for tools to access the data Features to communicate with other VO services E.g. Registries