OGSA-DAI Scenarios and Requirements OGSA-DAI for Developers GridWorld 2006, Washington DC 11 September 2006

2 Outline l Let’s talk to the users u Who wants to use OGSA-DAI? l What do they want to use it for? l Why aren’t they using it right now? u Who is using OGSA-DAI? l What are they using it for? l How could they use it more effectively? u Who was using OGSA-DAI? l Why aren’t they using it now? l How to use OGSA-DAI productively

3 Requirements – Why l Why? u Learn more about the data access and integration challenges that other projects face u Use this information to inform the future development of OGSA-DAI u Associate requirements with projects and aid work prioritisation l Do what we think most users want VS doing what specific users want

4 Requirements - What l What? u Data l Structure, quantity and types of data resource u Queries l Types of queries that are performed against this data, query languages, typical size of result sets u Problems l Data access and integration problems faced u What can or could OGSA-DAI provide?

5 Requirements – who l AstroGrid u ( – distributed queries over large astronomy databases l Automed and ISpider u ( and ( – model- based data integration and Grid-based informatics platform for proteomics l CancerGrid u ( – storage and analysis of distributed data containing clinical trial and lab data l ESSC u ( – environmental and atmospheric simulations l Gold u ( – provides infrastructure for virtual organisations l NTRAC u ( – similar to CancerGrid

6 Users want… l Efficient bulk data transport u Between heterogeneous data resources u Required by application-level projects u Benefits higher-level middleware (DQP, data federation, etc.) l Data federation and distributed query processing across heterogeneous data resources

7 … and … l Asynchronous query model u Process large, long-running queries u Client can poll or be notified of the query status u Terminate queries at an intermediate stage l Data resource view creation and management u Provide different views of data resources to different users in a secure, DBMS- independent manner u Manage these views dynamically

8 …and also… l Security / certificate delegation u Access data from other networks with role- based access rules l Usability u Quick and easy installation, configuration and maintenance u Support deployment as a WAR u Reduce third-party dependencies or prerequisites

9 Now what… l Focus on high-priority requirements raised by projects l Continued scenario-driven development: u Project has a specific well-defined data access or integration scenario u Can OGSA-DAI support that scenario? l Yes? Almost? u What are OGSA-DAI’s limitations and how can these be addressed? l No? u What functionality is needed within OGSA-DAI? u Can we spare a developer to work with this project?

10 Usage scenarios l “I have a data-related problem and OGSA-DAI made things worse” u OGSA-DAI is not a solution to every data access and integration problem in existence l “OGSA-DAI is not as fast as JDBC” u Well, for starters OGSA-DAI uses JDBC… u Different products for different requirements Images from and

11 l OGSA-DAI is like any tool u It has strengths and weaknesses u There are scenarios where it will be helpful and where it will not u We have elicited these in conjunction with our users u We have begun to publish these on the WWW

12 A naïve usage

13 A more effective usage

14 A more effective usage l As the data no longer flows back through the application-specific service u Provide additional OGSA-DAI activities to do application-specific data processing u Configure the OGSA-DAI service to support these activities l OGSA-DAI provides the delivery activities out-of- the-box l Overhead of developing application-specific data processing is reduced u Especially if you wish to experiment l Different delivery options l Allowing clients to select the desired delivery option

15 Multiple distributed resources

16 Data federation

17 Exploiting OGSA-DAI activities l Preceding scenarios delegate much application-level functionality to OGSA-DAI so… l …why not implement all application-specific functionality as OGSA-DAI activities? u Potentially moves computation closer to data u Eliminates expensive data movement u Improved range of delivery methods u A customised OGSA-DAI service can expose only application-specific activities

18 What are your requirements? l Do get in touch with the OGSA-DAI team u Discuss OGSA-DAI matters u Discuss requirements of a specific project u Arrange visits and collaborations u Contribute your own extensions u Feedback and comments are always welcome! l Engage in discussions on OGSA-DAI user list