XML Via SOAP & Local Degrib Applications Arthur A. Taylor (arthur.taylor@noaa.gov) http://www.weather.gov/mdl/degrib/ Meteorological Development Laboratory NWS NDFD Technical Workshop 2006 November 2, 2006

Why Use XML Existing technology Facilitates additional processing Extensible Markup Language is widely used (W3C specification) De Facto standard for internet data exchange Infrastructure exists (validators, XML Schema, etc.) Facilitates additional processing XML tools (XSLT, SAX, DOM) facilitate value added applications Web application developers familiar with tools Schema precisely defines a language Reaches new users

Communicating Process XML on the Move Data Broker (UDDI) WSDL Internet SOAP Request SOAP Request Data Consumer (Client) Provider (Server) SOAP Response SOAP Response Communicating Process Discovery (UDDI) Description (WSDL/Schema) Messaging (SOAP) Transport (HTTP)

The SOAP server The experimental (becoming operational Dec. 4) XML via ‘Simple Object Access Protocol’ server provided by the NWS does the following: Once an hour: Download the NDFD GRIB2 files Convert the GRIB2 files to data cubes Upon request from the SOAP server: PHP code parses the request degrib_DP probes the data cubes PHP code reformats it into XML Note: Sometime after Dec. 4, all formatting will be done by degrib C code.

SOAP Server Lessons Learned “If you build it, they will come”

Motivation Why add DWML capability to degrib? Alleviate pressure on the XML server allows whole sale users to use the GRIB2 files to create XML, using their own machines, for as many points and as often as they desire. Speed up the server Migrates from a scripted language (PHP) to a compiled language (C) Provide the source code for the logic behind DWML (c:\ndfd\degrib18\src\dwmllib\xmlparse.c) Encourage the use of DWML

Overview To create NDFD DWML on a PC we need to Download the NDFD GRIB2 to our PC Probe the data at a given location To enhance the speed of probing, we can also Convert the data to a data cube Probe the data cube at a given location

Attaining software To get the degrib program, go to: http://www.weather.gov/mdl/degrib/register.php For MS-Windows, use “degrib-install.exe” By default this installs to c:\ndfd\degrib18 For UNIX, use “degrib-src.tar.gz” By default this expand to “/degrib”. To compile it: cd /degrib/src make –f makefile.* (* is replaced by your operating system)

Get Data Option 1: Use “tkdegrib” or “tcldegrib web.tcl” Advantage: In addition to downloading the data, it checks for data corruption, by performing an inventory. From a command Prompt: cd c:\ndfd\degrib18\bin tcldegrib web.tcl –dataSet ndfd –sector midatlan –day d17 –variable all Option 2: Use your own “URL get” program To determine the locations of the NDFD GRIB2 files: http://www.weather.gov/mdl/degrib/dataloc.php http://weather.noaa.gov/pub/SL.us008001/ST.opnl/DF.gr2/DC.ndfd/AR.conus/

Probe Data For more details on probe options, see: http://www.weather.gov/mdl/degrib/tutorial.php#outputXML http://www.weather.gov/mdl/degrib/txtview.php?file=degrib.txt&dir=base From a command prompt cd c:\ndfd\degrib18\data\ndfd\midatlan ..\..\..\bin\degrib . –P –pnt 38.98,-77.07 –XML 1 –geoData c:\ndfd\degrib18\geodata The –geoData option supplies a directory to look in for information about the point’s timezone For multiple points, either provide multiple –pnt options, or use the “-pntFile” option

Flavors of DWML -XML 1 (or time-series) creates the time series product User can choose to format any combination of the following: maxt, mint, pop12, temp, wdir, wspd, td, sky, qpf, snow, wx, waveh, apt, rhm -XML 1 -Icon 1 will return just the icon information. Requires: temp, wspd, sky, wx Formats: icons -XML 2 (or glance) is the glance product Requires: maxt, mint, temp, wspd, sky, wx Formats: maxt, mint, sky, wx, icons -XML 3 (or 12-hourly) is the 12 hour summary product Requires: maxt, mint, pop12, wdir, wspd, sky, wx Formats: maxt, mint, pop12, wx, icons -XML 4 (or 24-hourly) is the 24 hour summary product

Convert to data cube The data cube consists of uncompressed data, which allows faster access than the GRIB2 files Method 1 (use c:\ndfd\degrib18\bin\MakeData.tcl): cd c:\ndfd\degrib18\bin tcldegrib MakeData.tcl c:\ndfd\degrib18\data\ndfd\midatlan c:\ndfd\degrib18\cube\ndfd\midatlan cd ..\cube\ndfd\midatlan Method 2: Manual method cd c:\ndfd\degrib18\data\ndfd\midatlan ..\..\..\bin\degrib maxt.bin –Data –Cube –Index maxt.ind –msg all Repeat for all GRIB (*.bin) files.

Probe the data cube From a command prompt either: ..\..\..\bin\degrib . –DP –pnt 38.98,-77.07 –XML 1 –geoData c:\ndfd\degrib18\geodata ..\..\..\bin\degrib_DP . –DP –pnt 38.98,-77.07 –XML 1 –geoData c:\ndfd\degrib18\geodata -DP is for probing a data cube, while -P is for a GRIB file degrib_DP is compiled with just the data cube probing code so it is smaller than degrib

Credits I’d like to acknowledge: John Schattel for developing DWML Paul Hershberg for converting the PHP code to C.