1. CUAHSI WaterOneFlow Web Services By Tim Whiteaker CE 394K.2 Hydrology 1 February 2007

2. Outline What are Web Services What is WaterOneFlow What is WaterML How do we use HIS Analyst to access WaterOneFlow

3. Programs Use Functions Functions are pieces of code that perform a specific task Program Math library Add Multiply SquareRoot Functions: What’s the square root of 16? 164 result = Math.SquareRoot(16)

4. Web Services Let Computers Share Functions Program Math library Add Multiply SquareRoot Functions: What’s the square root of 16 The answer is 4 Local programs take advantage of remote resources Works just like local function result = Service.SquareRoot(16) Service Client

5. How Do Web Services Work SOAP (Simple Object Access Protocol) is a protocol for exchanging messages over a network WSDL (Web Services Description Language) is a language for describing what a web service can do

6. How Do Web Services Work Client Server SOAP WSDL (This is what I can do) SOAP is like speaking the same language WSDL is like a contract

7. CUAHSI Web Services (WaterOneFLow) Standard mechanism for flow of hydrologic data between hydrologic data servers (databases) and users. Provides data access to –USGS NWIS –EPA STORET –NCEP North American Model (NAM) –NASA MODIS –Daymet –NWS ASOS http://water.sdsc.edu/waterOneFlow/

8. Water Data Web Sites

9. NWISWeb site output # agency_cd Agency Code # site_no USGS station number # dv_dt date of daily mean streamflow # dv_va daily mean streamflow value, in cubic-feet per-second # dv_cd daily mean streamflow value qualification code # # Sites in this file include: # USGS 02087500 NEUSE RIVER NEAR CLAYTON, NC # agency_cdsite_nodv_dtdv_vadv_cd USGS020875002003-09-011190 USGS020875002003-09-02649 USGS020875002003-09-03525 USGS020875002003-09-04486 USGS020875002003-09-05733 USGS020875002003-09-06585 USGS020875002003-09-07485 USGS020875002003-09-08463 USGS020875002003-09-09673 USGS020875002003-09-10517 USGS020875002003-09-11454 Time series of streamflow at a gaging station

10. Observation Stations Ameriflux Towers (NASA & DOE)NOAA Automated Surface Observing System USGS National Water Information SystemNOAA Climate Reference Network Map for the US

11. WaterOneFlow Consistent Query FormatConsistent Data Format

12. WaterOneFlow Methods (Almost) the same for all networks Methods –GetSiteInfo – Give me info about a site –GetVariableInfo – Give me info about a variable –GetValues – Give me a time series for a variable at a site

13. GetSiteInfo Input –Network name and site code –Authorization token Output –Site name –Site location –Variables measured at the site Variable code Period of record response = service.GetSiteInfo("NWIS:08158000", "") where

14. GetVariableInfo Input –Network name and variable code –Authorization token Output –Variable name –Variable description –Variable units response = service.GetVariableInfo("NWIS:00060", "") what

15. GetValues Input –Network name and location (e.g., site code) –Network name and variable code –Start datetime –End datetime –Authorization token Output –Site info –Variable info –Time series response = service.GetValues("NWIS:08158000", "NWIS:00060", _ "2006-12-31T01:00:00", "2006-12-31T08:00:00", "") where what when

16. The “Where” in GetValues Works for sites, or geographic locations NetworkName:SiteCode –NWIS:08158000 GEOM:POINT(Longitude Latitude) –GEOM:POINT(-113 35) GEOM:BOX(WLon SLat,ELon NLat) –GEOM:BOX(-108 45,-107 46)

17. Locations Variable Codes Date Ranges Extract – Transform – Load GetSiteInfo GetVariableInfo GetValues WaterOneFlow Web Service Client STORET NAM NWIS Data Repositories Data EXTRACT TRANSFORM LOAD WaterML

18. WaterML – The Output of WaterOneFlow Standardized for all data sources Formats –Object –XML (for programs that can’t handle objects) Defined in XML

19. XML – A Primer The Extensible Markup Language (XML) is a W3C-recommended general-purpose markup language that supports a wide variety of applications. – Wikipedia XML represents data XML is both human and machine readable Mansfield Dam

20. XML Structure OR element start tagend tag Example – Let’s describe a streamflow site element name = “site”

21. Elements can have Children Mansfield Dam element child element element text

22. Elements can have Attributes Mansfield Dam 08154510 attribute nameattribute value

23. XML Nesting Mansfield Dam 08154510 30.39 97.91

24. WaterML XML for describing water data Defined at WaterOneFlow returns data in WaterML format http://water.sdsc.edu/waterOneFlow/documentation/schema/

25. Point Observations Information Model Data Source Network {Value, Time, Qualifier} USGS Streamflow gages Neuse River near Clayton, NC Discharge, stage (Daily or instantaneous) 206 cfs, 13 August 2006 A data source operates an observation network A network is a set of observation sites A site is a point location where one or more variables are measured A variable is a property describing the flow or quality of water A value is an observation of a variable at a particular time A qualifier is a symbol that provides additional information about the value Sites Variables Values http://www.cuahsi.org/his/webservices.html WaterML Response SiteInfoResponseType VariablesResponseType TimeSeriesResponseType

26. WaterML Key Elements Response Types –SiteInfo –Variables –TimeSeries Key Elements –site –sourceInfo –seriesCatalog –variable –timeSeries values –queryInfo GetValues GetVariableInfo GetSiteInfo

27. SiteInfoResponseType Namespaces queryInfo site Network Sites Variables

28. queryInfo Parameters sent to service URLs called (if external resource)

29. queryInfo – Another Example user parameters query URL

30. site siteInfo – info about the site seriesCatalog – catalogs of variables measured

31. siteInfo Name Site Code Location

32. geoLocation geogLocation – geographic coordinates –LatLon point –LatLon box localSiteXY – projected coorindates European Petroleum Survey Group i.e., NAD 83, etc.

33. seriesCatalog nameweb service location original data source series (variables)

34. series variable – what is measured valueCount – how many measurements variableTimeInterval – when is it measured

35. variable variableCode – global identifier variableName units Sites Variables Values

36. variableTimeInterval TimeIntervalType – date range TimeSingleType – single measurement TimePeriodRealTimeType – last n days

37. ISO Time International Organization for Standardization 1998-03-01T14:30:00 = March 1, 1998, at 2:30 in the afternoon

38. SiteInfo Response, Recap queryInfo site name code location seriesCatalog variables what how many when

39. VariablesResponseType variable – same as in series element Code, name, units Sites Variables Values

40. Additional variable Children variableDescription valueType (enumeration) –Field Observation –Sample –Model Simulation Result –Derived Value generalCategory – e.g., “water quality” sampleMedium – e.g., “water column” -- Many are optional

41. TimeSeriesResponseType queryInfo timeSeries –sourceInfo – “where” –variable – “what” –values Sites Variables Values

42. sourceInfo SiteInfoType –Same as siteInfo element –code, name, location DataSetInfoType –For data continuous in space –LatLonPointType –LatLonBoxType

43. variable Same as previous variable element code, name, units, etc.

44. values Each time series value recorded in value element Timestamp, plus metadata for the value, recorded in element’s attributes ISO Timevalue qualifier

45. value Metadata Examples qualifiers censorCode (lt, gt, nc) qualityControlLevel (Raw, QC’d, etc.) methodID offset –offsetValue –offsetUnitsAbbreviation –offsetDescription –offsetUnitsCode

46. TimeSeries Response, Recap queryInfo location variable values

47. Conclusions about WaterML Consistent Format Includes lots of information How do I use it? HIS Analyst

48. Desktop apps that use WaterOneFlow Examples for various application environments HydroObjects – for apps that aren’t web service enabled Workbook available at: http://www.cuahsi.org/his/docs/HIS-workbook-20061130.pdf

49. HIS Workbook Ingesting NWIS Data into Excel Ingesting STORET Data into Excel Ingesting Weather and Streamflow Data into ArcGIS Plotting MODIS Data with Matlab Ingesting NWIS Data using VB.Net Ingesting NWIS Data Using Java

50. Excel Demo

51. Weather Downloader

52. Historical Weather Data Forecasted Weather Data Streamflow Other

54. Recap Computers share functionality through Web Services HIS Analyst ClientWaterOneFlow Web Service GetSiteInfo GetVariableInfo GetValues WaterML Time Series Repository