1. Grid-enabling OGC Web Services Andrew Woolf, Arif Shaon STFC e-Science Centre Rutherford Appleton Lab

2. Outline OGC background Profiling approach Refactoring approach Grid-enabling WPS

3. OGC background Key architectural frameworks –Reference Model for Open Distributed Processing (RM-ODP, ISO 10746- {1,2,3,4}) –ISO TC211 Domain Reference Model (ISO 19101)

4. Enterprise viewpoint –roles, activities, policies (incl. VO) Information viewpoint –semantics of information and information processing (static, invariant, dynamic schema) Computational viewpoint –interfaces and computational objects (cf. CORBA IDL, WSDL portTypes) Engineering viewpoint –distribution infrastructure (e.g. web services, WSRF vs OGSI) Technology viewpoint –choices of technology (e.g. app servers, DBMS) RM-ODP

5. ISO 19101 ‘Domain Reference Model’ A geospatial dataset… …consists of features and related objects… …in a defined logical structure… …delivered through services… …and described by metadata.

6. services SOA WSRF OGCWSDL metadata resource discovery catalogue data access integration information XML registries ‘Spatial Data Infrastructure’

7. Summary metadatadataservices Enterprise discoverydownload, viewprocess Information ISO 19115 ISO 19139 OGC Geography Markup Language (ISO 19136) OWS Common Computational OGC Catalogue Service for the Web (CSW) OGC Web Feature Service (ISO 19142) OGC Web Coverage Service OGC Web Map Service (ISO 19128) OGC Web Processing Service (WPS) Engineering Technology

8. Grid-enabling OGC: profiling Use existing specifications Incorporate ‘Grid’ functionality through convention, e.g. –specific parameters for security tokens Implementation uses Grid middleware at backend

9. Grid-enabling OGC: refactoring Problems: –tight coupling: service ↔ data data providers have to be service providers – latency general approach for asynchronous – service coherence data → (view, download, process) – workflow abstraction abstraction: data, operations – service consistency data resource: WPS vs. W{M,F,C}S

10. Grid-enabling OGC: refactoring WSRF: –separates ‘service’ and ‘stateful resource’ upon which service acts –WS-Resource (‘implied resource pattern’): identity (WS-Addressing) lifetime (WS-ResourceLifetime) state/properties (WS-ResourceProperties) –publish-subscribe mechanism for state changes (WS-Notification)

11. Grid-enabling OGC: refactoring Existing OGCWSRF-refactored

12. Grid-enabling WPS WPS provides standardised means of publishing, discovery and use of geospatial processes on the web Fundamentally, both WPS and Grid provide a remote interface for invoking computational processes

13. Grid-enabling WPS JSDL provides a standardised description of a computational job and its required resources Similarities with WPS: –process description information (Identifier, JobIdentification/JobName) –process outputs and inputs (DataInputs, POSIXApplication/Input,Argument,Output, DataStaging/Source) –sufficient to produce a valid JSDL document Differences with WPS: –resource requirements (filesystem parameters, disk space, operating system, CPU, etc.) –data staging instructions for input and output

14. Grid-enabling WPS WPS+JSDL as interface to encapsulated Grid resource JSDL consumed by compliant implementation, e.g. GridSAM Two broad options: –conceptually distinct data input parameters (through a specific ‘JSDL’ parameter) –“ordinary” data input parameters

15. Grid-enabling WPS Three approaches to distinct JSDL data input parameters: –URL to a complete pre-created JSDL document, or valid snippet http://foo.bar.1/wps?version=1.0.0&request=Execute&service=WPS&Identifier=WeatherGener ator&DataInput=other_inputs=xxx;JSDL=http://www.foo.com/myfoo.jsdl@Format=text/xml &storeExecuteResponse=true&status=true –URL-encoded JSDL document or snippet http://foo.bar.1/wps?version=1.0.0&request=Execute&service=WPS&Identifier=WeatherGener ator&DataInput=other_inputs=xxx;JSDL=%3CJobDefinition%3E%3CJobDescription%3E% 3CResources%3E%3CTotalDiskSpace%3E%3CExact%3E5%3C%2FExact%3E%3C%2FT otalDiskSpace%3E%3CTotalCPUCount%3E%3CExact%3E1%3C%2FExact%3E%3C%2F TotalCPUCount%3E%3C%2FResources%3E%3C%2FJobDescription%3E%3C%2FJobD efinition%3E@Format=text/xml@Schema=http://server/jsdl.xsd&storeExecuteResponse= true&status=true –Microformat http://foo.bar.1/wps?version=1.0.0&request=Execute&service=WPS&Identifier=WeatherGener ator&DataInput=other_inputs=xxx;JSDL=[TotalDiskSpace=5;TotalCPUCount=1]@Format= text/kvp&storeExecuteResponse=true&status=true

16. Grid-enabling WPS JSDL parameters encoded as individual WPS DataInput parameters http://foo.bar.1/wps?version=1.0.0&request=Execute&servic e=WPS&Identifier=WeatherGenerator&DataInput=other_inp uts=xxx;TotalDiskSpace=5;TotalCPUCount=1&storeExecu teResponse=true&status=true BUT.... –JSDL parameters are not inputs to the WPS process, but rather specify required resources

17. Grid-enabling WPS Security –OGC security is subject of considerable ongoing work –for now, used lightweight (and dodgey) solution: MyProxy parameters embedded in request secure transport layer

18. Grid-enabling WPS Implementation: –UK National Grid Service –BADC Trajectory Service –part of OWS-6 testbed –WPS 1.0.0-compliant

19. Grid-enabling WPS Grid-enabled WPS Pylons Framework GridSAM Client UK NGS GridSAM Service Grid WPS SOAP/ Proxy Layer MyProxy Server Execute Process (SOAP) Verify Cert. Job Id Query/ Response WPS Response

20. Grid-enabling WPS

21. Conclusions OGC adopts a sophisticated architectural approach to distributed geospatial data/processing –NB: OGC-OGF MoU, OGF-28 (Munich Mar. 2010) Grid-enabling OGC: –‘Profiling’ approach –‘Refactoring’ approach Example: grid-enabled WPS profile

22. More info... OWS-6 WPS Grid Processing Profile Engineering Report –http://portal.opengeospatial.org/files/?artifact_id=34977http://portal.opengeospatial.org/files/?artifact_id=34977 OWS-6 interactive demonstration –http://www.opengeospatial.org/pub/www/ows6/index.htmlhttp://www.opengeospatial.org/pub/www/ows6/index.html WPS Change Request for controlling & checking asynch processes –https://portal.opengeospatial.org/files/?artifact_id=36732https://portal.opengeospatial.org/files/?artifact_id=36732