1. Writing Perform Documents EPCC, University of Edinburgh Amy Krause ( a.krause@epcc.ed.ac.uk) Tom Sugden (tom@epcc.ed.ac.uk) First International Summer School on Grid Computing, Vico Equense, Italy

2. 2OGSA-DAI R3 Tutorial for the International Summer School on Grid Computing, Vico Equense, Italy Talk Outline  Introduction and Motivation  Structure of perform documents  Database queries and updates  Parameterisation  Result transformations  Delivery mechanisms

3. 3OGSA-DAI R3 Tutorial for the International Summer School on Grid Computing, Vico Equense, Italy Introduction  Interface to GDS is document- based  GridDataService::perform takes a perform document as its input parameter  Generates a response document

4. 4OGSA-DAI R3 Tutorial for the International Summer School on Grid Computing, Vico Equense, Italy Motivation  To access and integrate with data resources using OGSA-DAI: –Write perform documents OR –Write client utilities to generate perform documents

5. 5OGSA-DAI R3 Tutorial for the International Summer School on Grid Computing, Vico Equense, Italy What is a perform document?  A perform document contains a series of activities for a GDS to perform –Query a database –Transform the results –Deliver the transformed results  Expressed in XML

6. 6OGSA-DAI R3 Tutorial for the International Summer School on Grid Computing, Vico Equense, Italy XML Structure  Root element  Top-level elements correspond to activities Performs an SQL query against the data resource Transforms XML data from one structure to another Delivers data to a URL.

7. 7OGSA-DAI R3 Tutorial for the International Summer School on Grid Computing, Vico Equense, Italy Perform Document Validation  Each activity element must validate against an activity schema <xsd:element name="sqlParameter“ minOccurs="0" maxOccurs="unbounded">... <xsd:element name="expression" minOccurs="1" maxOccurs="1">... <xsd:element name="webRowSetStream" minOccurs="1" maxOccurs="1">...

8. 8OGSA-DAI R3 Tutorial for the International Summer School on Grid Computing, Vico Equense, Italy Example Performs a simple SQL query and returns the results in the response document. select * from littleblackbook where id=10

9. 9OGSA-DAI R3 Tutorial for the International Summer School on Grid Computing, Vico Equense, Italy Database Queries  OGSA-DAI R3 includes the following activities for querying databases –sqlQueryStatement Queries a JDBC compliant database –xPathStatement Queries an XML:DB compliant database  Future releases to support –XQuery for XML database –Querying file systems and files  Examples provided: – \examples\GDSPerform\JDBC\query – \examples\GDSPerform\XMLDB\xpath

10. 10OGSA-DAI R3 Tutorial for the International Summer School on Grid Computing, Vico Equense, Italy Example: sqlQueryStatement Performs a simple SQL query and returns the results within the response document. select * from littleblackbook where id=10

11. 11OGSA-DAI R3 Tutorial for the International Summer School on Grid Computing, Vico Equense, Italy Example: xPathStatement Performs a simple XPath statement and delivers the results within the response document. subCollection --> someResourceId -->... --> /entry[@id<100]

12. 12OGSA-DAI R3 Tutorial for the International Summer School on Grid Computing, Vico Equense, Italy Database Updates  OGSA-DAI R3 includes the following activities for updating databases –sqlUpdateStatement Updates a JDBC compliant database using SQL notation –xUpdateStatement Updates an XML:DB compliant database using XUpdate notation  Examples provided: – \examples\GDSPerform\JDBC\update – \examples\GDSPerform\XMLDB\xupdate

13. 13OGSA-DAI R3 Tutorial for the International Summer School on Grid Computing, Vico Equense, Italy Database Management  OGSA-DAI R3 includes the following activities for database management –relationalResourceManagement Creates and drops databases within a JDBC compliant DBMS –xmlCollectionManagement Creates, removes and lists collections within an XML:DB compliant DBMS –xmlResourceManagement Creates, removes and lists the resources contained in collections within XML:DB compliant DBMS  Examples provided – \examples\GDSPerform\XMLDB\collectionManager – \examples\GDSPerform\XMLDB\resourceManager

14. 14OGSA-DAI R3 Tutorial for the International Summer School on Grid Computing, Vico Equense, Italy Example: xmlCollectionManagement Performs a simple XML:DB collection management operation. subCollection --> -->

15. 15OGSA-DAI R3 Tutorial for the International Summer School on Grid Computing, Vico Equense, Italy Transformation Activities  OGSA-DAI R3 includes the following activities for transforming data –xslTransform Transforms XML data from one structure to another using an XML Stylesheet Language Transformation –gzipCompression Compresses data using the GZIP format –zipArchive Archives and compresses data using the ZIP format  Examples provided – \examples\GDSPerform\XMLDB\transform

16. 16OGSA-DAI R3 Tutorial for the International Summer School on Grid Computing, Vico Equense, Italy Example: xslTransformActivity somewhere/transform.xsl /entry[@id<100] Transforms the results of a query into HTML

17. 17OGSA-DAI R3 Tutorial for the International Summer School on Grid Computing, Vico Equense, Italy Parameterisation  Parameterisation allows data to be inserted into a query or update expression FrogSpecimens 5000 10000 select * from ? where id>? and id<=?  Currently supported for SQL query and update –sqlQueryStatement –sqlUpdateStatement

18. 18OGSA-DAI R3 Tutorial for the International Summer School on Grid Computing, Vico Equense, Italy Using Parameters  sqlParameter references the output stream of another activity –dataStore activity –deliverFrom activity  references the position of the parameter insert into MyTable values ?

19. 19OGSA-DAI R3 Tutorial for the International Summer School on Grid Computing, Vico Equense, Italy Usings Data Stores  Parameter values can be specified using the dataStore activity FrogSpecimens 5000 10000  sqlParameter references the itemCursor name

20. 20OGSA-DAI R3 Tutorial for the International Summer School on Grid Computing, Vico Equense, Italy Example: sqlQueryStatement with Parameters FrogSpecimens 5000 10000 select * from ? where id>? And id

21. 21OGSA-DAI R3 Tutorial for the International Summer School on Grid Computing, Vico Equense, Italy Delivery Activities  Activity output can be delivered: –Synchronously in the response document (default behaviour) –Asynchronously using delivery activities  OGSA-DAI R3 includes the following asynchronous delivery activities: –deliverToGDT and deliverFromGDT Delivers data to and from a GDS using the Grid Data Transport port type. –deliverToGFTP and deliverFromGFTP Delivers data between GDS and GFTP location in both directions. –deliverToURL and deliverFromURL Delivers data between GDS and URL in both directions (ftp, http, https, file)

22. 22OGSA-DAI R3 Tutorial for the International Summer School on Grid Computing, Vico Equense, Italy Delivery Structure Delivery - Push  Input Stream: fromLocal – references the name of another activity  Output Stream: toURL, toGFTP, toGDT – where to push the data to Delivery – Pull  Input Stream: fromURL, fromGFTP, fromGDT – where to pull the data from  OutputStream: toLocal – references the name of another activity

23. 23OGSA-DAI R3 Tutorial for the International Summer School on Grid Computing, Vico Equense, Italy /entry[@id<100] file://c:/ogsadai/resultsets/newresultset.txt Example Query with Asynchronous Delivery to URL

24. 24OGSA-DAI R3 Tutorial for the International Summer School on Grid Computing, Vico Equense, Italy Practicals  Practical exercises to try out: –Synchronous request –Parameterisation –Transformations –Asynchronous delivery http://192.167.1.106:8080/workshop  Further examples – /examples/GDSPerform