1. OGSA-DAI Users’ Meeting Introduction Malcolm Atkinson Director www.nesc.ac.uk 7 th April 2004

2. Overview Today’s Meeting Users meet each other We hear Users’ Complaints, Requirements & Priorities Users understand better the limits of one team’s work We jointly identify the way ahead OGSA-DAI Vision What we aspire to achieve Short, medium and long term Relationship with GGF DAIS WG Standardisation effort OGSA-DAI Today Current and Imminent Releases The Road Map Which you — the Users — should influence

3. OGSA-DAI History Lesson DB & BiGUM meetings August 2001 — Paul Watson paper — prime mover Database Task Force 01Q3 – 02Q3  Norman Paton, Dave Pearson & Paul Watson  http://www.cs.man.ac.uk/grid-db/dbtf.html Architecture Task Force 01Q3 – 02Q2  Urged UK engagement in OGSA  OGSI Sep 01 to Sep 03 http://www.nesc.ac.uk/teams/atf/01-03/ OGSA-DAI project Start Feb 02 – end Sept 03 Produced releases R1 to R3.1 Produced Alpha Release of DQP http://www.ogsadai.org.uk/ DAIT project Start Nov 03 – end ~ March 06 Continues OGSA-DAI development GGF4 Toronto March 02 DAIS BoF OGSI announced GGF5 Edinburgh July 02 DAIS WG launched GGF10 Berlin March 04 WS-Resource Framework

4. Oxford Glasgow Cardiff Southampton London Belfast Daresbury Lab RAL OGSA-DAI Partners EPCC & NeSC Newcastle IBM USA IBM Hursley Oracle Manchester Cambridge Hinxton $5 million, 20 months, started February 2002 Additional 24 months, starts October 2003

5. OGSA-DAI Vision 1 There are many sources of Data Structure is the essence of using data Structure supported by DBMS: Relational & XML Structure also recognised in files: BinX & GGF DFDL Sources of Data are Autonomously Managed Vary in model, structure, schema Vary in content Vary in Access & Use policies Vary in Platform Discovery & Design enabled by Combining Data From multiple sources Recorded in multiple data resources

6. OGSA-DAI Vision 2 Developing applications using heterogeneous data sources is difficult: Dealing with all those variations Negotiating with data owners and data platforms Communication protocols and data movement Data translation Distributed operations Scale, reliability, coping with change, failure recovery, security, privacy … Currently solved incrementally on a per application basis OGSA-DAI to the rescue

7. OGSA-DAI Vision 3 OGSA-DAI will provide a generic framework Standard components for dealing with common requirements  Query, update, bulk load, schema edit, metadata access, transactions, … Agreed protocols and standard agreements with DBMS Established mechanisms for handling owner policy requirements Provides data translation Supports distributed operations Delivers  Scale, reliability, coping with change, failure recovery, security, privacy … Costs of developing a robust OGSA-DAI amortised over many applications Development of Applications significantly accelerated Accepted de facto standard by DBMS vendors and data owners Efficiency from direct support and system or resource knowledge Collaborating International Community Extending the Framework E.G. the work of OGSA-WebDB

8. OGSA-DAI Vision 4 Progressively develop the functionality Progressively raise level of operations Progressively improve quality Scale, Performance, Reliability, Flexibility, Usability Adapt to changing context Support user community Training, Documentation, Consultation, Collaboration Place High Value on Users’ & Application Developers’ Ingenuity They know about the data resources They invent new ways of combining and analysing data Therefore Extensibility and Flexibility MUST always be supported

9. OGSA-DAI today 1919 Downloads so far from http://www.ogsadai.org.uk/ Release 4: April 04 from http://www.ogsadai.org.uk/ Release 5: October 04 from http://www.ogsadai.org.uk/ Releases 6&7: 2005: Many options — You must help us choose

10. Download analysis

11. And Now … Users’ Presentations What are you doing with our “baby” What do you like What do you want changed, extended or added What consensus is there Team Presentations More details of current and planned release The issues that face us  Choosing Interoperation Platforms  Choosing which requirements get priority Working Groups Collation of the Groups’ conclusions Ideas for Future User Group Activities URGENT Questions ?

12. Reserve Slides Follow

13. Database (Xindice, MySQL Oracle, DB2) Request to Registry for sources of data about “x” Registry responds with Factory handle Request to Factory for access to database Factory creates GridDataService Factory returns handle of GDS to client Client queries GDS with SQL, XPath, XQuery etc GDS interacts with database Query results returned XML SOAP/HTTP service creation API interactions Analyst Registry GDSR Factory GDSF Grid Data Service GDS Consumer OR delivered to consumer as XML OGSA-DAI Behaviours

14. Delivery Patterns

15. Statement Operations Receives statement Performs statement against a data resource Returns results Statement Component and Database Connections Data Resource StatementResults

16. … Name, versioning, configuration specification Update with data delivered with the script myimages id = 2 SELECT * FROM WHERE xyz response xyz Statement Document

17. Statement Architecture Engine/GDS-S JDBC ConnectionXMLDB Connection JDBC API Relational DB XML DB XML:DB API Statement Statement Activity Statement Handlers JDBC Statement Handlers XMLDB Statement Handlers

18. OGSA-DAI Security Grid Security Infrastructure Authentication The role mapper takes credentials and a database name and returns a dbRole dbRole contains username and password Role Mapper dbRole Credentials Database Name username password

19. 1a. Request to Registry for sources of data about “x” 1b. Registry responds with Factory handle 2a. Request to Factory for access to database 2c. Factory returns handle of GDS to client 3a. Client queries GDS with XPath, SQL, etc 3b. GDS interacts with database 3c. Results of query returned to client as XML SOAP/HTTP service creation API interactions RegistryFactory 2b. Factory creates GridDataService to manage access Grid Data Service Client XML / Relationa l database Data Access & Integration Services

20. GDTS 2 GDS 3 2 GDTS 1 S x S y 1a. Request to Registry for sources of data about “x” & “y” 1b. Registry responds with Factory handle 2a. Request to Factory for access and integration from resources Sx and Sy 2b. Factory creates GridDataServices network 2c. Factory returns handle of GDS to client 3a. Client submits sequence of scripts each has a set of queries to GDS with XPath, SQL, etc 3c. Sequences of result sets returned to analyst as formatted binary described in a standard XML notation SOAP/HTTP service creation API interactions Data Registry Data Access & Integration master Client Analyst XML database Relational database GDS GDTS 3b. Client tells analyst GDS 1 Future DAI Services? “scientific” Application coding scientific insights Problem Solving Environment Semantic Meta data Application Code