AstroGrid status SOFT the VO the future AstroGrid presentation to GSC Andy Lawrence July 2003

status six months into Phase B team of 26 people (23.4 FTEs) spent 1.5M (40% of budget) 10,000 lines of code working s/w released working on infrastructure, standards –not tools

Delivered in Phase-A requirements analysis technology evaluation comprehensive Phase A report demonstration s/w (with AVO) initial VO standards (with AVO and NVO) respected position in UK e-sci programme world ranking position in VO work

Delivered in Itn-01,02 basic engine room components –registry schema, implementation, and query service –data access and job control methods –MySpace manager, explorer, and registry preliminary interface and contents –simple portal, no tools –populated with a handful of DCs and datasets further international/esci visibility –use of OGSI, OGSA-DAI, formed GGF-RG –defined IVOA standards process OGSI experiments –intercontinental visualisation with grid services

Planned for Itn03,04 engine room continued –consolidate Registry, Data Access, MySpace –add Auth/Auth, workflow tool (poss Triana) –Registry harvesting interface and contents –astronomer useable interface –meaningful contents –data federation services international programme –standards for Registry, VOQL

demonstrations Jan 2003 : with AVO –modest technical advance –new tools, well integrated, excellent PR July 2003 : AGOC and Sydney –large technical advance –no new tools, minimal interface next steps –feedback from AGSAG beta testers crucial –wrap tools for next demo

Target

Current

successes well motivated hardworking team rigorous s/w eng processes code cut and released on schedule basic engine room components in place status in international VO status in astronomical community

opportunities our infrastructure could become standard additional tool access via AVO collaboration

failures portal work failed : GUI hacked fixed process take-up slow fixed

threats US diverging on registry standard OGSA-DAI vs standard Grid too little Data Centres effort –incorporating datasets –writing services

Progress on AstroGrid goals (1) develop standards for data, metadata, data exchange and provenance GOOD (2) develop a software infrastructure for data services GOOD (3) establish a physical grid of resources GOOD but DC effort worrying (4) construct and maintain a Service and Resource Registry GOOD (5) implement a working VO system of real scientific use to astronomers TBD NEXT SIX MONTHS (6) provide a user interface to that VO system BEHIND (7) provide or adapt a set of science user tools to work with the VO LACK OF FUNDING (8) establish a leading position for the UK in VO work GOOD

The VO is a system that will.. allows users to interrogate multiple data centres in a seamless and transparent way provide new powerful analysis and visualisation tools within that system give data centres a standard framework for publishing and delivering services using their data. AstroGrid Phase A Report October 2002

multi- views of a Supernova Remnant Shocks seen in the X- ray Heavy elements seen in the optical Dust seen in the IR Relativistic electrons seen in the radio

What happens to the Earth's magnetosphere during a coronal mass ejection ? Event imaged by space-based solar observatory Effect detected later by satellites and ground radar

needles in a haystack Hambly et al faint moving object is a cool white dwarf - may be solution to the dark matter problem - but hard to find : one in a million - even harder across multiple archives

The VO is a system that will.. allows users to interrogate multiple data centres in a seamless and transparent way provide new powerful analysis and visualisation tools within that system, give data centres a standard framework for publishing and delivering services using their data. VO-1 VO-2 VO-3

The VO is a system that will.. allows users to interrogate multiple data centres in a seamless and transparent way provide new powerful analysis and visualisation tools within that system, give data centres a standard framework for publishing and delivering services using their data. VO-1 VO-2 VO-3 looking good need effort for DCs to "publish and deliver" almost untouched

the future : data services VO : geometry –not a warehouse; not a hierarchy; not P2P –small set of service centres + large popn of end-users VO : content –not software monolith : framework + standards –needs content : data services and user tools implies work for Data Centres –writing services; framework uptake; resource management –upcoming opportunities internationally competitive –establish a few centres as professionally competitive

yesterday browser front end CGI request html web page DB engine SQL data

today application web service SOAP/XML request SOAP/XML data DB engine SQL native data anything standard formats

tomorrow application web service job results anything web service web service web service web service web service Registry Workflow GLUE AstroPass MySpace standard semantics publish WSDL

day after tomorrow application grid service job results anything grid service grid service grid service grid service grid service Registry Workflow GLUE AstroPass MySpace pooled resource standard semantics agents ontology

work needed application grid service job results anything grid service grid service grid service grid service grid service Registry Workflow GLUE AstroPass MySpace pooled resource standard semantics agents TOOLS STANDARDS INFRASTRUCTURE TECHNOLOGY RESEARCH DATA SERVICES (access and analysis) GRID UPTAKE DATA PIPELINES ontology

expected status 2004 tools : ok but minimal standards : good but evolving glue : good but could be better data access services : ok but under-resourced data analysis services : almost untouched grid uptake : ready to deploy but resource worry technology research : treading water

AstroGrid-2 priority areas –core : evolve and improve, build tools –establish content : data services, resource grid –analysis services : visualisation, datamining : research and deploy –next technology wave : ontology, agents expertise needed –existing AstroGrid developer team –alliance of Data Centres –add Computer Science expertise

CS expertise Leeds : visualisation experts Portsmouth : kd-tree algorithms Exeter : Intelligent Agents for astronomy commercial experience : Linde, Noddle local links : Bristol, Edinburgh, Cambridge external links : OGSA-DAI, GGF, MS, IBM

Infrastructure framework largely in place but.. needs to be robust and reliable needs continuing work on standards –IVOA and Grid needs suite of tools key idea is personalisation –tool requests from competitive calls delivers VO-1

R&D really mostly D –assess, customise, integrate keep on the next wave –ontology, agents keep promise that everybody can be a power user –large volume and multi-D visualisation and datamining services delivers VO-2 and re-newed VO-1

Data Centres Data Centres are the heart of the VO there is no UK VO without well resourced DCs top priority is support for VO/uptake second priority is core support for DCs –not mission specific resources –not pipeline or archive development –basic physical resource management and data curation delivers VO-1 delivers VO-3

Costed Model Options M1 : full programme £9.25M M2 : full research £6.68M –lose DCA core support –world beating VO infrastructure but UK content compromised M3 : partial research £5.28M –lose some lines and trim others; minimise new algorithms –good working VO but loses opportunity for UK lead in new technology –approximately flat continuation of AG1-PhaseB M4 : minimum viable £3.75M –no research; unlikely to lead any third wave of VO development –working VO but relies on external components

FIN