who are we ? what is a VO ? what is a Grid ? how do we get there ? Andy Lawrence S.P.I.E. Hawaii Aug 2002 AstroGrid

Belfast Cambridge Edinburgh Jodrell Leicester MSSL RAL Andy Lawrence S.P.I.E. Hawaii Aug 2002

AstroGrid Optical Infrared X-ray Radio Solar Space Plasma Andy Lawrence S.P.I.E. Hawaii Aug 2002

people PL = Andy Lawrence PS = Nic Walton PM = Tony Linde

slogans the archive is the sky (stolen from US-NVO) everybody can be a power user shift the results not the data a supercomputer on your desk

the project PPARC funded project Sept –approx $6M Grid technology development programme –links to GridPP, MyGrid (Bio), OGSA (Globus) Stepping stone to Virtual Observatory –partner in AVO –working links with US-VO using Unified Process open collaborative web sites

status elaboration phase two-thirds finished –science problems / use cases / architecture –technology assessment reports –sw demos : portal, auth/auth, ontology; AVO science demos –collaborative web pages set up –Phase A study report to be published November construction phase begins ~end 2002

collectivisation and democratisation thirty year trend towards communal organisation –facility class (common-user) instruments –facility class data reduction s/w –calibrated archives with simple tools –information services (Vizier, ADS, NED) –large consortium projects (MACHO, 2dF, SDSS, UKIDSS, VISTA...) next steps –inter-operable archives (joint queries) –automated resource discovery (registry) –facility-class exploration and analysis tools (data mining)

the Virtual Observatory concept Aim to make all archives speak the same language –all searchable and analysable by the same tools –all data sources accessible through a uniform interface –all data held in distributed databases that appear as one –archives form the Digital Sky –eventual interface to real observatories the archive is the sky

shared managed distributed resources –documents + data + software + storage + cycles + expertise network : ability to pass messages web : transparent document system computational grid : transparent CPU datagrid: transparent data access and services information grid, knowledge grid... ? Virtual Organisations ? the Grid concept a supercomputer on your desktop everybody can be a power user

what do we need ? sociology –need agreed formats for data, metadata, provenance –need standardised semantics ("ontology") internet technology –need protocols for publishing and exchanging data –need registry for publishing service availability and semantics –need method of transmitting authentication/authorisation –need methods for managing distributed resources

bottlenecks i/o bottleneck –need database supercomputers –need innovative search and analysis algorithms network bottleneck –data centres must provide analysis service –facility class analysis code needed shift the results not the data

grid geometry needed not a centralised superarchive not P2P ( like Napster) not a hierarchy ( like LHC grid) service providers + users ( like most commerce) –some unplanned open use –some registered use –variety of access rights

two rivers academic / Globus –remote log on –identity/authentication/authorisation –resource management commercial / W3C –exchange of data (B2B) –service description and publication –"Web services" = XML + SOAP + WSDL (cf GLU)

Globus problems only half works data transfer primitive –flat files only not a services-user structure

Web service problems one-to-one bulky no general auth/auth solution no accepted service registry solution no ontology solution

Grid Services ? web services + auth/auth + lifetime management to be developed by OGSA project –OGSA-DAI integrates database access –AstroGrid and MyGrid "early adopters"

who does what the VO projects will : –evolve agreed data standards –build infrastructural toolkit for data exchange –build registry of available data services Data Centres will : –maintain data in whatever format they like –use toolkit to write standard data services –publish services to registry AstroGrid also will : –provide first example data services for key UK databases –provide example "point of entry" user interface –provide some resource : warehouse, CPU, "MySpace".

observatories / instrument builders need to : –output data to new standards –link observing request to VO research –decide whether to provide data services or farm out to data centres who does what part(II)

FIN

Web DB access today browser front end user CGI request html web page DB engine SQL data

Web service application web service user XML request XML data DB engine SQL native data anything