1. Service-Based Distributed Query Processing on the Grid M.Nedim Alpdemir Department of Computer Science University of Manchester

2. Service-based approaches Virtualisation of Resources A convenient cooperation model for Distributed Systems (e.g. Grid) facilitate Leads to

3. Context Computational Complexity Data Complexity

4. Web Services Are Not Enough Lack facilities for: Computational resource description. Computational resource discovery. Application staging. Grid Services combine: Web Services for service description and invocation. Grid middleware for computational resource description and utilisation.

5. Open Grid Services Architecture (OGSA) OGSA services are described using WSDL. OGSA service instances are: Created dynamically by factories. Identified through Grid Service Handles. Self describing through Service Data Elements. Stateful, with soft state lifetime management. Current status: Globus 3 beta release in June 2003: www.globus.org. Supports service instances and access to other Globus services. Core database services from OGSA-DAI project tracking Globus releases.

6. Grid Database Service (GDS) Databases are made available on the Grid through integration with other Grid services, and provision of standard interfaces Build upon OGSA to deliver high-level data management functionality for the Grid. Seek to provide two classes of components: Data access components Data integration components

7. GDS interactions

8. Distributed Query Processing DQP involves a single query referencing data stored at multiple sites. The locations of the data may be transparent to the author of the query. select p.proteinId, Blast(p.sequence) from protein p, proteinTerm t where t.termId = GO:0005942 and p.proteinId = t.proteinId J. Smith, A. Gounaris, P. Watson, N. Paton, A. Fernandes, R. Sakellariou, Distributed Query Processing on the Grid, 3rd Int. Workshop on Grid Computing, Springer-Verlag, 279-290, 2002.

9. Mutual Benefit The Grid needs DQP: Declarative, high- level resource integration with implicit parallelism. DQP-based solutions should in principle run faster than those manually coded. DQP needs the Grid: Systematic access to remote data and computational resources. Dynamic resource discovery and allocation.

10. 1.1.1 GDQ PortType: Operations and Messages A Service-Based DQP Architecture

11. Service-based DQP framework Service-based in two orthogonal sense: data storageanalysis services Supports querying over data storage and analysis resources made available as services Constructionexecution Construction of distributed query plans and their execution over the grid are factored out as services Uses the emerging standard for GDSs to provide consistent access to database metadata and to interact with databases on the Grid. A query may refer to database (GDS) and computational services.

12. Extends OGSA & OGSA-DAI … By adding a new port type and two new services ( and their corresponding factories) : Grid Distributed Query (GDQ) Port Type importSchema operation GDQSR importSchema(GDQDataSourceList GDSL) GDSL : A document containing: the list of Data Sources. The items on this list should contain the handles of the GDS Factories, along with an instance creation document for each factory. And/or a set of WSDL URLs for the analysis services to be used

13. Continued … Grid Distributed Query Service (GDQS) Wraps an existing query compiler/optimiser system which compile, optimise, partition and schedule distributed query execution plans Obtains and maintains metadata and computational resource information required for above Grid Query Evaluator Service (GQES) Each GQES instance is an execution node and is dynamically created by the GDQS on the node it is scheduled to run A GQES is in charge of a partition of the query execution plan assigned to it by the GDQS and is responsible for dispatching the partial results to other GQESs.

14. Setting up a GDQS Set-up strategy depends on the life-time model of GDQS and GDSs GDQS instance is created per-client But it can serve multiple-queries This model avoids complexity of multi-user interactions while ensures that the set-up cost is not high Setup phase involves: Importing schemas of participating data sources Importing WSDL documents of participating analysis services Collecting computational resource metadata (implicit)

15. Issues in Initialisation Q: When is a GDQS bound to a particular GDS? A: When the schema of the GDS is imported. Q: What is the lifespan of a GDS used by a GDQS? A: The GDS is kept alive until the GDQS expires. Q: Are GDSs shared by multiple GDQSs? A: No. Q: When is a GQES created? A: When a query is about to be evaluated that needs it. Q: What is the lifespan of a GQES? A: It lasts only as long as a single query. Q: Is a GQES shared among several queries or GDQSs? A: No.

16. Importing Schemas

17. http://130.88.198.203:8080/ogsa/services/ogsadai/GridDataServiceFactory myDataResource http://www.ebi.ac.uk/collab/mygrid/service0/axis/services/urn:srs?WSDL An example of data source import list

18. select p.proteinId, Blast(p.sequence) from proteins p, proteinTerms t where t.termId = GO:0005942 and p.proteinId = t.proteinId http://ogsadai.org.uk/GDTService/my/GDT/GSH An example of a Query Document

19. Logical Optimiser Physical Optimiser PartitionerScheduler Evaluator OQL Parser Single-node Optimiser Multi-node optimiser Query Compilation

20. Plan is expressed using a logical algebra. Heuristic-based application of equivalence laws. Multiple equivalent plans generated. scan (protein) scan termID=… (proteinTerm) reduce join (proteinId) op_call (Blast) reduce Logical Optimisation

21. Plan is expressed using a physical algebra. Logical operators replaced with physical operators. Cost-based ranking of plans. table_scan (protein) index_scan termID=… (proteinTerm) reduce hash_join (proteinId) op_call (Blast) reduce Physical Optimisation

22. Partitioning Plan is expressed in a parallel algebra. Parallel algebra = physical algebra + exchange. Exchange operators are placed where data movement may be required. table_scan (protein) index_scan termID=… (proteinTerm) reduce hash_join (proteinId) op_call (Blast) reduce exchange

23. Scheduling Partitions are allocated to Grid nodes; partitions may be merged during scheduling. Expressed by decorating parallel algebra expression. Heuristic algorithm considers memory use, network costs. table_scan (protein) table_scan termID=S92 (proteinTerm) reduce hash_join (proteinId) op_call (Blast) reduce exchange 3,4 1 12

24. Query installation: GQESs created for partitions as required. Partitions sent to GQESs. Query evaluation: Partitions evaluated using iterator model. Pipelined and partitioned parallelism. Results conveyed to client. Query Evaluation

25. ... http://mach1.cs.man.ac.uk:8080/ogsa/services/ogs adai/GQESFactory/GQES1... string proteinTerms.GOproteinID string proteinTerms.term proteinTerms http://mach1.cs.man.ac.uk:8080/ogsa/services/ogs adai/GridDataServiceFactoryP2R1/GDS1 EQ...... http://mach1.cs.man.ac.uk:8080/ogsa/services/ogsadai/GQESFactory/GQES1... string proteinTerms.GOproteinID string proteinTerms.term proteinTermsDataResource http://mach1.cs.man.ac.uk:8080/…/GridDataServiceFactoryP2R1/GDS1 EQ... An example of a query sub-plan passed to a GQES

26. Interactions of SB-DQP components

28. Summary DQP on the Grid provides: The normal benefits of DQP. Some added benefits from a Grid setting. The Grid specifically enables: Runtime computational resource discovery. Dynamic creation of remote evaluators. Authentication/Transport services. Access to non-database services.

29. Features of Our GDQS Low cost of entry: Imports source descriptions through GDSs. Imports service descriptions as WSDL. Throw-away GDQS: Import sources on a task-specific basis. Discard GDQS when task completed. Builds on parallel database technology: Implicit parallelism. Pipelined + partitioned parallel evaluation. Public release in July 2003.

30. The SB-DQP Team Manchester: Nedim Alpdemir Anastasios Gounaris Alvaro Fernandes Norman Paton Rizos Sakellariou Newcastle: Arijit Mukherjee Jim Smith Paul Watson