Using python client libraries to access remote servers via Web Coverage Services Dominic Lowe, BADC Ben Domenico, Unidata/UCAR* and the GALEON Team * Work partially supported by US NSF Atmospheric Science Division

Working Together on A Mosaic for Atmospheric Data Ostia Antica circa 7 BC This presentation describes and draws on the work* of many collaborating individuals and institutions * Unidata’s contribution supported by the U.S. National Science Foundation

Acronym Glossary GALEON (Geo-interface for Air, Land, Environment, Oceans NetCDF) FES (Fluid Earth Systems, mainly the data systems of the atmospheric and ocean sciences) ations/acronyms/glossary.htmlhttp:// ations/acronyms/glossary.html

Outline GALEON Background OWSLib – OGC Web Services Library Simple Exercises for WCS Using OWSLib Clients Participating Institutions and Servers Future Directions References

Reconciling Disparate Data Models: Different Ways of Thinking about Data To the GIS (solid earth and societal impacts) community, the world is: features –A collection of static features (e.g., roads, lakes, plots of land) with geographic footprints on the Earth (surface). features discrete objects database. –The features are discrete objects with attributes which can be stored and manipulated conveniently in a database. To the fluids (atmosphere and oceans) communities, the world is: parameters continuous functions –A set of parameters (e.g., pressure, temperature, wind speed) which vary as continuous functions in 3-dimensional space and time. parameters equations. –The behavior of the parameters in space and time is governed by a set of equations. –Data are simply discrete points in the mathematical function space.

Taking Advantage of Web Services for Data System Interoperability GIS Client Applications FES Client Applications OpenGIS Protocols: WMS, WFS, WCS, CSW OGC or proprietary GIS protocols OGC or THREDDS, OPeNDAP, ADDE. FTP… protocols GIS Server GIS Servers Hydrologic, demographic, infrastructure, societal impacts, … datasets THREDDS Server FES Servers Satellite, radar, forecast model output, … datasets

Traditional GIS view Attributes in DBMS tables Features as points, lines, polygons

Typical NetCDF Visualization

OGC GALEON Phase 1: –Many successful experiments with WCS 1.0 –Gridded data Phase 2: –WCS 1.1 (and later) plus CS-W for catalogs –Non-gridded data collections via other Services (WFS, SOS) –Dearth of client implementations slows progress OGC Network Rather than Interoperability Experiment

Apply GIS Tools To Atmospheric Science Data

OWSLib – OGC Web Service utility library is a lightweight package for working with OGC map, feature, and coverage services using Python provides a common API for accessing service metadata and wrappers for GetCapabilities, GetMap, GetCoverage and GetFeature requests facilitates implementation of simple WCS clients as python scripts

OWSLib – Design Goals Instantiate an OGC web service proxy (ala xmlrpclib) Make a GetCapabilities request and marshal the response into a Python structure that models OWS Common metadata Proxy GetMap/GetFeature/GetCoverage/Get* requests to the service with Pythonic wrappers for those operations Note that OWS common is an initiative to separate out common elements of OGC web services -- likely important as GALEON attempts to serve non-gridded data

OWSLib History Originally developed by Sean Gillies as part of the Python Cartographic Library. Version 0.2 contained support for accessing WMS version and WFS servers. Version 0.3 developed by Sean Gillies & Dominic Lowe introduced support for WCS versions and Version 0.3 also introduced a harmonized OWS Common-like API to service metadata that is common across WCS, WMS and WFS. This API is a 'pythonic' interpretation of OWS-Common which provides easy access to the OWS-common metadata.

OWSLib Design Decisions Lightweight Easy to use Modelled on OWS-common but not verbatim; “pythonic beats pedantic as long as significant meaning is not lost” Use python idioms: e.g. convenient dictionary like access to coverage metadata: wcs['temperature'] Harmonize metadata between services where not harmonized (and where possible) in line with intentions of OWS common.

Rudimentary WCS Exercises via OWSLib Python Scripts Use getCapabilities for list of coverages Extract name of first coverage in list If multiple times are available, use the initial time Use getCoverage to request first coverage Use defaults wherever possible Focus on whether protocol works Worry about “correctness” of the returned coverages later

WCS Sites US Pacific Fisheries Environmental Lab US Unidata THREDDS Data Server US USGS Coast Environmental US NCDC NOMADS Netherlands KNMI Geoservices Netherlands ARGOSS UK BADC US NSIDC US George Mason U. Italy U. of Florence CNR US NNEW Weather.aero

General Results 3 servers returned a coverage on the first try 2 required minor changes to the default parameter list The rest required interactions between Dominic, Ben, and the WCS site administrator 2 cases of missing required parameters 2 cases uncovered bugs in python client library which were subsequently fixed All cases working in the end

Future Enhancements Include more servers in the exercise Augment exercise in the direction of a true test Retrieve representative coverages Retrieve representative times Use bounding box other than default Examine returned coverage Track evolution of WCS specification

Initial WCS Server URLs ' /1day', version='1.0.0' ' US_40km/conduit/NCEP-NAM-CONUS_40km-conduit_best.ncd', version='1.0.0' ' version='1.0.0' ' / /narrmon-a_221_ _0000_000.grb' As noted in the list given above, some servers can be made to work with alterations to the default parameter list, e.g.: –' version='1.0.0' –' version='1.0.0'

More Inforamtion OWSLib wiki: Subversion Repository: svn co Python Cheeseshop: GALEON Wiki page: GALEON%20WCS%20"Tests" GALEON%20WCS%20"Tests"