or some relevant web services What is a Web Service or some relevant web services (for) Developing Applications for Real-time Environmental Data Bryan Lawrence Head, NCAS British Atmospheric Data Centre P.I. The NERC DataGrid.

Outline Question: What is a Web Service? Answer: Acronym Soup! Fundamentals (XML+) SOAP and REST WSDL OGC Web Services WFS, WCS, WFS UNIDATA LDM NDG Web Services

What is a web service? Googling gives: emergingtech.ittoolbox.com: Answers.Com: Web-based applications that dynamically interact with other Web applications using open standards that include XML, UDDI and SOAP. (Yuck: UDDI?) emergingtech.ittoolbox.com: A Service-Oriented Architecture (SOA) is a collection of services or software agents that communicate freely with each other. Sub-topic definition: Web Services protocols and standards are the technology that promote the sharing and distribution of information and business data. A protocol is a standard method for transmitting data through a network. There are many different specialized protocols to accommodate the many kinds of data that might be transmitted.

SOAP Simple Object Access Protocol Horrible looking XML specification, but lots of SOAP pacakges available which are (generally easy to use), eg: from SOAPpy import SOAPproxy url=http://services.xmethods.net:80/soap/servlet/rpcrouter’ n=‘urn:xmethods-Temperature’ server=SOAPProxy(url,namespace=n) temperature=server.getTemp(‘27502’) Returns the temperature at U.S. zipcode 27502 (from Dive Into Python, Mark Pilgrim) (One knew a priori that a method named getTEmp would return a float given an argument of a zip code, provided one used the right namespace – How?)

WSDL :Web Services Description Language >>>from SOAPpy import WSDL >>>Server=WDSL.Proxy(‘path/to/WSDL’) >>>server.method.keys() [u’doGoogleSearch’,u’doGetCachedPage’ …] >>>callInfo=server.methods[‘doGoogleSearch’] >>>for arg in callInfo.inparams: . . . print arg.name, arg.type key (u’http://www.w3.org/2001/XMLSchema’,u’string’) . . . Define expected messages for a service, and their (input or output parameters). An interface will group together a number of messages (operations) ABSTRACT PROPERTIES Bind an Interface via a definition to a specific transport (e.g. HTTP) and messaging (e.g. SOAP) protocol The network location where the service is implemented , e.g. http://localhost:8080

REST’ful Web Services REST (representational state transfer) is an approach for getting information content from a Web site by reading a designated (ok, you need a URI) Web page that contains an XML (Extensible Markup Language) file that describes and includes the desired content. That’s it! You use HTTP (get,put,…) to a URI, with XML as the payload …

Web Services the confusion: (Tim Ewald): “Today, I see three camps: 1) It's all about SOAP messages …, 2) It's all about WSDL, 3) It's all about XML/HTTP. Microsoft is definitely in the first camp, focusing on doing things with SOAP whether or not they are ultimately represented as XML or a binary data rep. IBM and Iona are in the second camp, focusing on describing in WSDL, whether or not they are ultimately represented as SOAP or a binary protocol. Lots of developers and some big companies like Amazon are in the XML/HTTP camp, focusing on getting data from one place to another in a way they can consume it. [Tim Ewald: http://pluralsight.com/blogs/tewald/archive/2004/10/18/2875.aspx]

Standards Dimensions of interoperability for earth science data use access discovery ISO / OGC

OGC web services Data access services, e.g.: Live Access Server, NDG DataExtractor GADS DODS/OPeNDAP OGC web services Web Map Service Web Feature Service Web Coverage Service [Open Geospatial Consortium (OGC): International consortium of nearly 300 companies, government agencies and universities participating in a consensus process to develop publicly available geoprocessing specifications]

OGC web services Web Map Service (WMS) Three operations: GetCapabilities (required) GetMap (required) GetFeatureInfo (optional) Typically support HTTP GET binding, SOAP being considered GetCapabilities request <Service> metadata for service as a whole (ISO 19115 compliant) <Capability> metadata describes request bindings, exceptions, and: <Layer>s: title, name, SRS, bounding box, keywords, style; nested layers inherit from parent eg, GLOBE program WMS server GetCapabilities: http://viz.globe.gov/viz-bin/wmt.cgi?REQUEST=GetCapabilities&SERVICE=WMS

OGC web services e.g.: ERA40 re-analysis surface air temperature, 2001-04-27 deegree open-source WMS modified with netCDF connector Overlaid with rainfall from globe.digitalearth.gov WMS server

OGC web services Web Coverage Service (WCS) – cf DODS/OPeNDAP/GADS A geographic coverage is raw (typically gridded) data WCS similar to WMS, but provides data, not images (cf DODS/OPeNDAP, GADS) Operations: GetCapabilities DescribeCoverage GetCoverage Coverages supplied in well-known-binary format: GeoTIFF, HDF-EOS, DTED, NITF, GML, netCDF???

OGC web services Web Feature Service (WFS) Provides access to geographic feature instances Features are defined by application schema compliant with the Geography Markup Language (GML) Operations: DescribeFeatureType (required): returns XML schema for the feature GetFeature (required): allows retrieval of features in XML compliant to the feature schema. Selected features may be constrained through Xpath expressions Transaction and LockFeature (optional): for inserting, updating and deleting feature instances GetCapabilities: describes capabilities of WFS (as for WMS) Typically used for vector data (eg cruise tracks), but can be any application schema

OGC web services FOSS implementations server: client: deegree geoserver mapserver client: quickWMS www.wmsviewer.com List maintained on NDG wiki at http://ndg.nerc.ac.uk/kwiki/ndg/index.cgi?OtherProjects

OGC web services Summary Web Map Service (WMS, ISO 19128) GetCapabilities GetMap Web Feature Service (WFS) DescribeFeatureType GetFeature Transaction, LockFeature Web Coverage Service (WCS) DescribeCoverage GetCoverage Catalog Service for the Web (CSW) – very new! REST, not SOAP (currently)

NDG: CSML Climate Science Modelling Language (CSML, a GML application schema) http://ndg.nerc.ac.uk/csml Defines seven features of interest! CSML feature type Description Examples TrajectoryFeature Discrete path in time and space of a platform or instrument. ship’s cruise track, aircraft’s flight path PointFeature Single point measurement raingauge measurement ProfileFeature Single ‘profile’ of some parameter along a directed line in space. wind sounding, XBT, CTD, radiosonde GridFeature Single time-snapshot of a gridded field gridded analysis field PointSeriesFeature Series of single datum measurements tidegauge, rainfall timeseries ProfileSeriesFeature Series of profile-type measurements vertical or scanning radar, shipborne ADCP, thermistor chain timeseries GridSeriesFeature Timeseries of gridded parameter fields. numerical weather prediction model, ocean general circulation model

CSML ProfileSeriesFeature ProfileFeature GridFeature

NERC DataGrid; web services coming soon Discovery Service (here now) Data Extractor (next week) Vocab Service (next month) Attribute Authority (next couple of months) Data Provider (next couple of months) DeliveryBroker (later this year) Browse Service (a long way off) All Open Source, documented, clear APIs

NERC DataGrid: Discovery Service

NDG: dataExtractor and GeoSplat

UNIDATA Unidata (www.unidata.ucar.edu) provide data feeds using their Local Data Manager (LDM) technology to push data around the U.S. (and further afield) in the Internet Data Distribution system (IDD). Users to specify in advance which data should be delivered to their local systems. The IDD then delivers the data as soon as they are available. The IDD is a data subscription service, implemented in such a way that delivery (and often processing) are triggered by external events. WMO considering the use of LDM for some data movement.

Unidata: LDM LDM Delivery Scalable Hierarchical fan out Relying on relay sites with appropriate resources Flexible New data products can be introduced from any node in the system

UNIDATA IDS Topology

Summary Web services not just about SOAP and WSDL … We should use web service standards, WS mechanisms optimised for environmental applications Currently need to extend GML (CSML) Options for data push if UK community want it based on LDM (or similar) technologies … NDG can (and is) building a range of environmental web services