1. vers. 20070524 national spatial data infrastructure training program Serving Your Data Via WFS Serving Transportation Data Through the NSDI

2. national spatial data infrastructure training program Serving Your Data Via WFS 1 Learning Objectives After completing this Module, the student will be able to Describe the three fundamental operations of a Web Feature Service (WFS) Explain several considerations in configuring a WFS system to serve local data as FDS-compliant GML data Use a web browser to examine the capabilities of a WFS over the internet, and retrieve data from it

3. national spatial data infrastructure training program Serving Your Data Via WFS 2 Primary Topic This Module provides guidance on how to serve your data to the Internet via a Web Feature Service (WFS), using the mapping you’ve created to an FDS-compliant model, as presented in Module 4 There are only two essential notions: convert your data to a form that is consistent with the FDS serve the data through an OGC-compliant Web Feature Service (WFS)

4. national spatial data infrastructure training program Serving Your Data Via WFS 3 Outline of this Module This Module addresses the following topics: an overview of the Web Feature Service (WFS) and the OGC WFS protocol considerations for automating transformation of your data retrieving and viewing FDS-compliant GML data from a working WFS

5. national spatial data infrastructure training program Serving Your Data Via WFS 4 Characteristics of the WFS Protocol The WFS protocol, and OGC specifications in general, do NOT constrain: Data format Custodial or management policies Hardware or software platform They DO define: Lightweight suites of web service operations Encodings for data transport Various means for services to describe their contents and capabilities

6. national spatial data infrastructure training program Serving Your Data Via WFS 5 What Exactly Does a WFS Do? The Web Feature Service (WFS) is simply a well- specified interface: it provides a known mechanism for interacting with its underlying system The underlying system might be a suite of GIS programs, a simple data handling system, or any other system that holds or can distribute geospatial data

7. national spatial data infrastructure training program Serving Your Data Via WFS 6 How WFS Get The Data They Serve The WFS specification describes only the interface that a compliant web service presents on the Internet The way that data are stored or acquired, and the additional capabilities provided by the underlying system, are completely independent of the WFS interface itself The WFS server may directly retrieve the data it serves from a database system, or it may accept data via a functional or object-oriented interface, a remote procedure call, or some other programmatic means

8. national spatial data infrastructure training program Serving Your Data Via WFS 7 WFS and Other Services WFS is one of several data service specifications published by the OGC This document describes WFS version 1.1.0 The specification document may be found at http://www.opengeospatial.org/standards/wfs http://www.opengeospatial.org/standards/wfs OGC services include the Web Map Service (WMS) and the Web Coverage Service (WCS) as well as the WFS These services share several characteristics, including the “W*S” pattern of interaction between client and server

9. national spatial data infrastructure training program Serving Your Data Via WFS 8 OGC Data Services (“W*S”) Pattern The OGC data services follow a typical pattern of interaction with client software: Client asks server, “what data and operations do you provide?” via the GetCapabilities operation Server responds with a Capabilities document that describes what it has Client may parse the Capabilities, feed available choices to the user interface or other programmatic components Client issues requests for given data type (e.g., layer, feature, bands of a gridded coverage) within the bounding box of interest

10. national spatial data infrastructure training program Serving Your Data Via WFS 9 OGC Data Services (“W*S”) Pattern What can you do? Here… read this. GetCapabilities … … … Capabilities Document Great! Give me data GetFeature (or similar operation) Data Here you are…. 1 4 3 2

11. national spatial data infrastructure training program Serving Your Data Via WFS 10 Capabilities Document The Capabilities Document is an XML document that describes the service instance Its form is described in XML Schema per the OGC specification that defines the service; the WFS Capabilities document contains: a element that provides general metadata for the service as a whole  e.g., name, human-readable title, online resource  geographic area for which the service can provide data an metadata element  operations such as and  URL prefix for each operation elements that describe the data held by the WFS system  name, title  default Spatial Reference System ( ) - supports EPSG codes  enclosing rectangle ( ) of layer

12. national spatial data infrastructure training program Serving Your Data Via WFS 11 Web Feature Server Operations Basic WFS operations for data retrieval: GetCapabilities DescribeFeatureType GetFeature Transactional WFS also has operations that support remote update of its datastore: Transaction LockFeature

13. national spatial data infrastructure training program Serving Your Data Via WFS 12 Two Kinds of WFS Requests WFS can be queried by two mechanisms, both using the HTTP protocol One involves sending a request document to the server via an HTTP POST message The other sends an HTTP GET message, with the parameters specified in a Key-Value Pair (KVP) format

14. national spatial data infrastructure training program Serving Your Data Via WFS 13 One Kind of WFS Response The server’s response depends only on the content of the request, not the form of the HTTP message The response is a GML document whose form and content are specified by the WFS Standard KML, JSON, and other encodings may also be available

15. national spatial data infrastructure training program Serving Your Data Via WFS 14 … returns a GML 3 application schema for that feature type. … against a feature type in a local data model … WFS A WFS DescribeFeatureType query through a WFS interface… WFS Returns GML Schemas

16. national spatial data infrastructure training program Serving Your Data Via WFS 15 … yields a GML 3 encoding of features of that type within the given area … against a particular feature type within a given geographic area WFS A WFS GetFeature query through a WFS interface… WFS Automatically Generates GML Features

17. national spatial data infrastructure training program Serving Your Data Via WFS 16 WFS Fundamental Operations The three fundamental operations that every WFS supports can be summarized as follows: GetCapabilities: retrieves the Capabilities document that describes the service, the area it covers, the operations it supports, and the types of data it contains DescribeFeature: retrieves application schemas for one or more of the feature types available from the WFS GetFeature: retrieves feature data for the area requested

18. national spatial data infrastructure training program Serving Your Data Via WFS 17 Query Examples Using HTTP POST The next three slides depict these three operations, supported by all WFS servers, with requests submitted via HTTP POST

19. national spatial data infrastructure training program Serving Your Data Via WFS 18 WFS GetCapabilities Operation Client Opaque Feature Store Web Feature Server WFS Capabilities Document (response document only) CubeWerx Web Feature Service Web Feature Server maintained by X. Query Roads Road Segments EPSG:4269 -179 -85 179 83 WFS Request GetCapabilities WFS Response Capabilities Document

20. national spatial data infrastructure training program Serving Your Data Via WFS 19 WFS Request DescribeFeatureType WFS DescribeFeatureType Operation Client Opaque Feature Store Web Feature Server WFS WFS Response Schema for the FeatureType DescribeFeatureType (XML request document) <DescribeFeatureType version="1.1.0“ service="WFS" xmlns="http://www.opengis.net/wfs" xmlns:trans="http://www.fgdc.gov/framework/073004/transportation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> trans:RoadSeg <xs:element name="RoadSeg" type="trans:RoadSegType" substitutionGroup="gml:_Feature"/> … XML Schema of the FeatureType (response document)

21. national spatial data infrastructure training program Serving Your Data Via WFS 20 WFS Request GetFeature WFS GetFeature Operation Client Opaque Feature Store Web Feature Server WFS WFS Response GetFeature (XML request document) Feature data in GML (response document) <GetFeature version="1.1.0“ service="WFS“ xmlns:trans="http://www.fgdc.gov/framework/073004/transportation" maxFeatures=“3”> -122.5 37 -122 38 <wfs:FeatureCollection xmlns="http://www.cubewerx.com/cw" xmlns:cw="http://www.cubewerx.com/cw" xmlns:trans="http://www.fgdc.gov/framework/073004/transportation" … 2005-06-01T00:00:00Z -122.3525 37.4976 -122.3524 37.4973 - 122.3522 …

22. national spatial data infrastructure training program Serving Your Data Via WFS 21 OGC Filter Encoding Specification An expressive, system-neutral filtering capability is offered by the OGC’s Filter Specification, described in detail in OGC document 04-095 Filters of this sort can only be used within a request that uses an HTTP POST message The Filter Specification is much the same as any other query language, and supports a variety of operators, including: Spatial operators (equals, disjoint, touches, within, overlaps, crosses, intersects, contains, within, beyond, bbox) Comparison operators (propertyIsEqualTo, propertyIsNotEqualTo, propertyIsLessThan, propertyIsGreaterThan, etc) Logical operators (And, Or, Not) Arithmetic operators (Add, Sub, Mul, Div)

23. national spatial data infrastructure training program Serving Your Data Via WFS 22 Queries Using HTTP GET The WFS interface provides for queries that require only a single command line encoded within the URL for an HTTP GET Queries of this type take the form of a string of Key- Value Pairs (KVP), i.e. an ampersand-separated list of parameter names and their values, as is typical for cgi-style web interfaces A web browser is all that is needed to execute such queries

24. national spatial data infrastructure training program Serving Your Data Via WFS 23 KVP-Encoded GetCapabilities Request The following KVP-encoded GetCapabilities request returns a Capabilities document for the USGS WFS that provides the agency’s Framework Data service http://frameworkwfs.usgs.gov/framework/wfs/wfs.cgi?datastore=Framewor k&service=WFS&version=1.1.0&request=GetCapabilities Note the parameters in the KVP-encoded section “datastore” is an implementation-specific parameter, and should be considered part of the base URL “service” and “version” are standard parameters for any WFS, or in fact any of the W*S services “request” is also a part of every KVP-encoded W*S request in this case, the operation requested is “GetCapabilities”

25. national spatial data infrastructure training program Serving Your Data Via WFS 24 KVP-Encoded GetFeatureInfo Request The following request will return the XML Schema for a RoadSeg: http://frameworkwfs.usgs.gov/framework/wfs/wfs.cgi?d atastore=Framework&service=WFS&version=1.1.0&re quest=DescribeFeatureType&TYPENAME=RoadSeg Note that the service type, service version, and requested operation are present as they are in the GetCapabilities request The one additional parameter specifies the name of the data type for which to provide the schema

26. national spatial data infrastructure training program Serving Your Data Via WFS 25 KVP-Encoded GetFeature Request The following KVP-encoded URL retrieves RoadSeg data from the USGS Framework WFS, within a bounding box defined by a southwest corner of 122.5º west longitude, 37º north latitude, and a northwest corner of 121º west longitude, 38º north latitude Like the other requests, this one describes the service and type of the data for which it is querying The bounding box is a filter that defines the geographic region in which to search for the features http://frameworkwfs.usgs.gov/framework/wfs/wfs.cgi?datastore=Framework &service=WFS&version=1.1.0&request=GetFeature&BBOX=-122.5,37,- 121.0,38&MAXFEATURES=10&TYPENAME=RoadSeg

27. vers. 20070524 national spatial data infrastructure training program Your Data and WFS

28. national spatial data infrastructure training program Serving Your Data Via WFS 27 Recall: How WFS Get Their Data As noted earlier: The WFS specification describes only the interface that a compliant web service presents on the Internet The way that data are stored or acquired, and the additional capabilities provided by the underlying system, are completely independent of the WFS interface itself The WFS server may directly retrieve the data it serves from a database system, or it may accept data via a functional or object-oriented interface, a remote procedure call, or some other programmatic means

29. national spatial data infrastructure training program Serving Your Data Via WFS 28 Data Harmonization and the WFS In Module 4, you learned how to develop a mapping of your data model to an information model that is compatible with the FDS Transportation Model To use this mapping in practical installations, an appropriate Transformation Utility must be implemented in software to apply the mapping to an automated transformation process The Transformation Utility accepts (or retrieves) input data that conform to your local Transportation Model, and delivers the equivalent FDS-compliant data to a WFS, via whatever interface the WFS system provides

30. national spatial data infrastructure training program Serving Your Data Via WFS 29 Data Transformation Utility It is possible that the WFS system you use provides its own utilities or programming environments for data transformation If a separate transformation utility is required, it can be written in essentially any programming language. Popular choices include: C: a traditional compiled language C++: based upon C, adds object-oriented capabilities Java: a semicompiled object-oriented language Perl: a scripting language frequently used for text processing Python: an object-oriented scripting language Ruby: an object-oriented scripting language with specializations for use with web services (Ruby on Rails) XSLT: explicitly designed to facilitate conversions between models

31. national spatial data infrastructure training program Serving Your Data Via WFS 30 What the Transformation Utility Must Do The Data Transformation Utility must accept (or retrieve) data from your local data store apply the mapping: transform the data to a form compliant with the FDS Transportation Model hand off the data to the WFS depending upon your architecture, the Transformation Utility may also need to respond to requests from the WFS system to retrieve and transform data Internet Local Data Store Transformation Utility System WFS Interface

32. national spatial data infrastructure training program Serving Your Data Via WFS 31 Form of the Transformation Utility The transformation utility may take many forms, including: a script that runs in the context of your Geographic Information System, and performs an appropriate transformation as data are retrieved an external script or compiled program called as a function by the WFS system - uses a vendor-specific library to pass data back and forth an external process that runs asynchronously in coordination with the WFS - communicates via some interprocess channel (e.g., pipes, messaging, remote procedure call…), also possibly using a vendor- supplied library an independent application that converts the entire native data store and transfers it to an independent, FDS-compliant data store - runs updates on demand, or is programmed to run at regular intervals The means by which the Transformation Utility passes data to the WFS depends upon the interfaces that your WFS system provides, and the time at which you want the data conversion to be done

33. national spatial data infrastructure training program Serving Your Data Via WFS 32 Example Configurations The following three slides illustrate some of the possibilities for configuring your Transformation Utility in the context of your data storage system and your WFS system

34. national spatial data infrastructure training program Serving Your Data Via WFS 33 Example 1: On-The-Fly Transformation Transformation Utility Local Data Store Web Feature Server Local Data Local Model FDS-Compliant Data FDS-Compliant GML Data WFS Web Client Data are managed and stored according to the local information model When the WFS receives a request for data, it triggers a retrieval, using the local data model, from the data store. The Transformation Utility changes it to FDS- compliant form, and the WFS serves it as GML.

35. national spatial data infrastructure training program Serving Your Data Via WFS 34 Example 2: Transformation A Priori Transformation Utility Local Data Store Web Feature Server Local Data Local Model FDS-Compliant Data WFS Web Client FDS-Compliant Data Store FDS-Compliant GML Data Data may be converted and stored in FDS- compliant form in a batch process when the WFS is implemented, and also subsequent to data entry or modification. The WFS uses only the FDS-compliant data store.

36. national spatial data infrastructure training program Serving Your Data Via WFS 35 Example 3: Using Features of the WFS Transformation Utility Local Data Store Local Data Local Model FDS-Compliant Data FDS-Compliant GML Data WFS Web Client System With WFS Interface The WFS may be one of several interfaces to a larger GIS system, which may include the data store as well as internal capabilities for data transformation, or hooks for access to an external transformation utility.

37. national spatial data infrastructure training program Serving Your Data Via WFS 36 Other Strategies There are many ways to structure a system that converts your local data to FDS-compliant data, and presents a WFS interface to the web The foregoing examples are very general possibilities, and there are others The ultimate decision depends upon what systems and tools you already have as much as on the new features you wish to implement

38. national spatial data infrastructure training program Serving Your Data Via WFS 37 RDBMS and Object Data Stores Among the many types of data stores used by GIS systems, RDBMS and object data stores are among the more common The information model you use to transfer data, or to store transformed data, should mirror the organization of the Tables in the FDS transportation standard documents If your system uses an RDBMS to store the data, you will typically wish to create a table for each data type, with a column for each attribute, precisely as in the FDS documents If your system uses an object data model, it generally makes sense to create a class for each feature type, and store the features’ attributes as properties (data members, instance variables, …) of the class The details, as always, depend upon the particular requirements, capabilities, and programming interfaces of your system

39. national spatial data infrastructure training program Serving Your Data Via WFS 38 Summary: Considerations for WFS Summary of considerations for implementing WFS for your transportation data: what WFS product to use for your system?  built into existing GIS / Transportation Data System?  external product?  ( listings of OGC-compliant products, including WFS, may be found at http://www.opengeospatial.org/resource/products ) http://www.opengeospatial.org/resource/products is a Transformation Utility required? (or is your data m odel already FDS-compliant?) translate data a priori or on-the-fly? for a priori translation: what storage solution to use for transformed data? what technology to use for implementing the Transformation Utility?  is support built into your GIS system? what are the detailed interface and functional requirements for your Transformation Utility?

40. national spatial data infrastructure training program Serving Your Data Via WFS 39 The Data Transformation Process The data transformation process itself can be programmed using the same paradigm as for mapping your local information model to the FDS Transportation Model Recall from Module 4 that there are three cases to consider: perfect matches tolerable matches semantic mismatches

41. national spatial data infrastructure training program Serving Your Data Via WFS 40 Transforming Perfect Matches Perfect semantic matches between your data elements and components of the FDS Transportation Model should be the simplest to transform in software A straightforward way to accomplish this in code: define and instantiate the target structure or object (representing the FDS model) using the structures, interfaces, and libraries or tools (if any) for your WFS system then assign the value of each attribute from the local data entity to the corresponding attribute in the target (FDS compliant) data entity

42. national spatial data infrastructure training program Serving Your Data Via WFS 41 Transforming Tolerable Matches “Tolerable” matches should also be straightforward to program The process is similar to the case for perfect semantic matches: defining the target structures or objects assign values to them from the local datastore It may also be desirable to add new information, such as an annotation, to the target object

43. national spatial data infrastructure training program Serving Your Data Via WFS 42 Transforming Semantic Mismatches Entities that do not map directly to any object within the FDS Transportation model require more programming The same fundamental strategy applies nonetheless: 1.define the target object or structure using the tools and paradigms of the WFS system 2.assign values one by one to the attributes of the target structure The assignment step may require computation that implements the details of your data mapping

44. national spatial data infrastructure training program Serving Your Data Via WFS 43 Recall: The WFS Generates GML Unless you are creating your own WFS capability, it is unlikely that you will need to convert your data to GML: that is what the WFS does

45. national spatial data infrastructure training program Serving Your Data Via WFS 44 Exercises The rest of this material provides three exercises based on the KVP-encoded WFS requests discussed earlier in this module There is also a link to more information about using the Web Feature Service These exercises demonstrate a process that you can use to exercise your own WFS once it is configured and populated with the standard- compliant form of your data

46. national spatial data infrastructure training program Serving Your Data Via WFS 45 Exercise: Get a Capabilities Document Retrieve the Capabilities Document from a working WFS server: Access the URL of the USGS Framework WFS  http://frameworkwfs.usgs.gov/framework/wfs/wfs.cgi?datastore =Framework&service=WFS&version=1.1.0&request=GetCapa bilities http://frameworkwfs.usgs.gov/framework/wfs/wfs.cgi?datastore =Framework&service=WFS&version=1.1.0&request=GetCapa bilities View the result in your web browser The browser may interpret the XML in the document as markup, and render it incorrectly To see the full XML, select the “View Source” option in your browser Note the sections of the document that describe the Service, The Operations it supports, and the Feature Types it contains

47. national spatial data infrastructure training program Serving Your Data Via WFS 46 Contents of Capabilities Document - 1 The Capabilities Document Contains several major sections, listed on this and the next two slides: The WFS_Capabilities element contains all of the others represents the entire document ServiceIdentification and ServiceProvider sections identify name and characteristics of the service identify and provide contact information for the organization running the service

48. national spatial data infrastructure training program Serving Your Data Via WFS 47 Contents of Capabilities Document - 2 OperationsMetadata section lists all the operations performed by this service provides base URL, parameter names, and valid parameter values for each operation this information can be used to formulate requests for the operation * note that RoadSeg is a valid parameter for TypeName in the DescribeFeature operation FeatureTypeList a listing of feature types available from this service provides a geographic bounding box for each type contains all instances of the given feature type * note that RoadSeg is one of the FeatureTypes

49. national spatial data infrastructure training program Serving Your Data Via WFS 48 Contents of Capabilities Document - 3 SupportsGMLObjectTypeList lists the GML types and profiles supported by this service Filter_Capabilities shows filtering (selection) options supported by this service

50. national spatial data infrastructure training program Serving Your Data Via WFS 49 Exercise: DescribeFeatureType The following URL will return the XML Schema for a RoadSeg: http://frameworkwfs.usgs.gov/framework/wfs/wfs.cgi?d atastore=Framework&service=WFS&version=1.1.0&re quest=DescribeFeatureType&TYPENAME=RoadSeg Click on the link, view source in your browser Compare the GML definition of the RoadSeg to the tabular definition in FDS 7c, Table 1

51. national spatial data infrastructure training program Serving Your Data Via WFS 50 Contents of Feature Type Document - 1 The Feature Type document returned by the query contains the application schema for RoadSeg note that the structure it contains is an “xs:schema” - an XML schema object The first portions of the structure define namespaces and so forth, and then state that RoadSeg is an extension of the GML AbstractFeatureType Each object in the center portion of the document corresponds to a line in the description of RoadSeg in FDS 7c, Table 1

52. national spatial data infrastructure training program Serving Your Data Via WFS 51 Contents of Feature Type Document - 2 As an exercise, find the XML elements that correspond to each item in the section of FDS 7c Table 1 that describes RoadSeg Note that in the XML document, the optional elements have a “minoccurs” value of 0; the mandatory elements do not

53. national spatial data infrastructure training program Serving Your Data Via WFS 52 Exercise: Get Feature Data From a WFS Click on the following URL to acquire feature data: ….and select your browser’s “View Source” option as before Examine the contents of the GML instance document returned http://frameworkwfs.usgs.gov/framework/wfs/wfs.cgi?datastore=Framework &service=WFS&version=1.1.0&request=GetFeature&BBOX=-122.5,37,- 121.0,38&MAXFEATURES=10&TYPENAME=RoadSeg

54. national spatial data infrastructure training program Serving Your Data Via WFS 53 Contents of Feature Data Document Note that the structure contained by the document is a GML FeatureCollection Note the various parts of the FeatureCollection: the XML header, namespace, and schemalocation definitions the spatial envelope - a bounding box - within which all the features contained in the collection are located the list of features - in this case Road Segments - included in the FeatureCollection Compare the attributes for each object with the definition for RoadSeg in FDS 7c, Table 1 are all the mandatory attributes present? which optional ones are present? note the substantial number of Extended Attribute elements included in the representation of each Road Segment

55. national spatial data infrastructure training program Serving Your Data Via WFS 54 Additional Information on Using WFS There is an excellent tutorial on using WFS at http://www.ogcnetwork.net/wfstutorial http://www.ogcnetwork.net/wfstutorial The tutorial uses the same server as the previous exercises

56. national spatial data infrastructure training program Serving Your Data Via WFS 55 Conclusion Having completed this Module, you should have some understanding of how a WFS works several approaches to configuring a WFS for serving your data in standard-compliant form how to access the capabilities and contents of a WFS This Module, combined with Modules 1 through 4, should give you the basic information necessary to formulate a plan for serving your data, in FDS- compliant form, through the National Spatial Data Infrastructure