OGSA-DAI: Future Work and Wrap-up The OGSA-DAI Team

2 Goals for DAIT  Aim to deliver application mechanisms that: –Meet the data requirements of Grid applications Functionally, performance and reliability Reduce development cost of data centric Grid applications Provide consistent interfaces to data resources –Acceptable and supportable by database providers Trustable, imposed demand is acceptable, etc. Provide a standard framework that satisfies standard requirements  A base for developing higher-level services –Data federation –Distributed query processing –Data mining –Data visualisation

3 Current Release  R4 April 2004 –Provides Data Access components, an extensible framework for building applications and some integration components –Built on top of Globus Toolkit 3.2 –Supports relational, xml and some files MySQL, Oracle, DB2, SQL Server, Postgres, XIndice, CSV –Supports various delivery options SOAP, FTP, GridFTP, HTTP, files, , inter-service –Supports various transforms XSLT, ZIP, GZip –Supports message level security using X509 certificates –Client Toolkit library for application developers –GUI data browser (contributed by FirstDIG project) –Separate Distributed Query Processing components –Comprehensive documentation and tutorials

4 Roadmap / Workplan  Roadmap document available for comment: – –User feedback required to drive this document  Integrate parts of DQP into OGSA-DAI core –Addressing platform dependencies –Want to include XML data resources  Undertaking data integration requirements capture process –Talked to 11 UK and US projects so far  Move Computation to Data –Java mobile code

5 Release 5  R5 October 2004 (Interfaces) – OGSI –Built on Globus Toolkit –Re-engineered to separate interface layer from core OGSA-DAI functionality –Improved dependability and security integration –New file data resources representing flat files queried using full text searches (e.g. EMBL format) –Installation and Configuration Wizard, including “all-in-one installer” –Improved Data Browser which allows XPath querying. –Set of standard benchmarks. –JSP Quick View interface. –Support for other databases (e.g. Access, Exist, HSQL)

6 Release 5  Major refactoring exercise –Improve distinction between interface and core –Allow future support for different interfaces  Listened to major user concerns –Adding indexed text file access to support the bioinformatics community –Improved robustness and usability –Addressed bugs as they were reported –Hope to add Activity Configuration, requested by projects developing custom activities  Last release primarily targeted at the OGSI/GT3 platform

7 WS-I Technical Preview  A limited functionality evaluation version –An OGSA-DAI “Data Service” combining the metadata, configuration and perform document capabilities of the OGSI-based GDSF and GDS services. –Access to service metadata provided by a partial implementation of the WS-ResourceProperties specification. –Example clients are provided for testing and coding reference.  Caveats/Issues: –No registry component, no support for third party delivery. –Security may be available (based on OMII WS-Security plug-in for Axis). –Document schema and interfaces WILL change. –The WSDL is based on the OGSI-based WSDL from OGSA-DAI –Will not be supported to same level as main release for now.  Also released with OMII middleware distribution (Oct)

8 WS-RF Technical Preview  An evaluation version OGSA-DAI based on the Globus Toolkit 4.0 beta implementation of WSRF. –Provides an amalgamation of the capabilities of the OGSI-based GDSF and GDS services (the metadata and configuration aspects of the GDSF and the metadata and perform document processing aspects of the GDS). –Access to multiple data resources from a single service provided by data resource identifiers specified by a client within the WS-Addressing endpoint reference to a data service. –Access to service metadata (database schemas, request status, etc) provided by an implementation of the WS-ResourceProperties specification. –A WSRF version of the GridDataTransport portType supporting asynchronous data delivery between data services.  Caveats/Issues: –This preview of OGSA-DAI WSRF does not support data service security. –Document schema and interfaces WILL change. –Will not be supported to same level as main release.  Will also be released as part of the Globus Toolkit 4 beta

9 OGSA-DAI Project Webpage  Background News & Events Software Releases Documentation Support Training Courses Links

10 FAQ, Support, Mailing List  Frequently Asked Questions – –Updated as common problems become clear  Support for OGSA-DAI releases – –Use to report problems  Users mailing list – –General discussion of OGSA-DAI, data and the Grid

11 Second Users Group Meeting  Held alongside GGF12 in Brussels –~40 attendees, 12+ projects represented  Users Group Chair appointed –Prof. Beth Plale, Indiana University  User presentations from –AstroGrid (UK), GeneGrid (UK), eDiamond (UK), INWA (UK), LEAD (USA) –Also representatives from BioGrid (Japan), BRIDGES (UK), caBIG (USA), GEDDM (UK), GEON (USA), myGrid (UK), SIMDAT (EU),…  A separate independent body to engage with users and feedback to developers in a formal way –Mailing list and website being setup

12 Take Home Messages  There are plenty of Research Challenges –Data integration and workflow on a global scale –Data resource and schema heterogeneity –Dynamic variability Authorisation, Resources, Data & Schema, Performance –Some massive data –Metadata for discovery –Provenance tracking  Grasp the theoretical & practical challenges –Working in open & dynamic systems –Incorporate all computation –Welcome “code” visiting your data

13 Future DAI requires fundamental CS  What architecture best enables integration of data & computation? –Common conceptual models –Common planning & optimisation –Common enactment of workflows –Common debugging  What Fundamental CS is needed? –Trustworthy code & trustworthy evaluators –Decomposition and recomposition of applications –Metadata  Is there an evolutionary path?  Are web services a distraction?

14 Conclusions  Still early days –Standardisation process not stabilising quickly enough –Infrastructure still developing and prone to change  OGSA-DAI acting as an enabler –Showing people what can be done –However is it cracking a nut with a sledge hammer? –Evolving and improving with each release  Usage patterns are similar –Call for people to work together to solve similar problems  Some problems are not OGSA-DAI specific –Metadata, time zones, security, …  Data discovery perceived to be important –Is this in the scope of what OGSA-DAI should be doing?  Please try it out! –It’s free and supported –Make suggestions, extend functionality, contribute to DAIS-WG

15 Links  OGSA-DAI Webpage –  Globus Toolkit 3.2 –  Database Access and Integration Services (DAIS-WG) –  Grid Technology Repository –  ELDAS - Enterprise-Level Data Access Services (Eldas) –  Web Services Choreography –