1. 7 Nov 20072007 Geospatial Interoperability Summit Iowa Environmental Mesonet: Using Open Source GIS Tools and Web Services to Disseminate Environmental Data Daryl Herzmann Iowa State University

2. 7 Nov 20072007 Geospatial Interoperability Summit Data, data, everywhere data

3. 7 Nov 20072007 Geospatial Interoperability Summit

4. 7 Nov 20072007 Geospatial Interoperability Summit The Desperate Challenge Disparate Formats –Grib –METAR –Text CSV –Free form text –ADAS III –SHEF –Excel (gasp!) Disparate Network Protocols –HTTP –FTP –LDM IDD (tcp) –TCP stream –Modems The data comprising the IEM is made available to us in the following formats and network protocols.

5. 7 Nov 20072007 Geospatial Interoperability Summit Why an Open Source solution? 1.They are the best tools out there. UNIX mentality -> Do something small and do it well. 2.Transparency. We are a public institution doing public funded research. 3.Excellent Support Public bug trackers Community forums IRC Motivated developers

6. 7 Nov 20072007 Geospatial Interoperability Summit Open Source Solution PostGIS / PostgreSQL Spatial Database Map Server Mapping Engine + Scripting Red Hat Linux Operating System Python Data processing and network services GDAL Swiss Army Knife of all things GIS PHP Dynamic Website

7. 7 Nov 20072007 Geospatial Interoperability Summit Key Component: PostGIS Early IEM code (2001) was a series of flat files in Shapefile format and database of observations. This was a disaster. PostGIS provided the opportunity to manage data, metadata, and the GIS data in one location. –Network scalability –Transaction / stored procedure support –Speed / functionality with merge of GEOS library

8. 7 Nov 20072007 Geospatial Interoperability Summit Look at all of my data! 1 billion observations 1 terabyte of data –Archives of NEXRAD, GOES Satellite, precipitation datasets, web cameras!, Weather Service products, ……. Now what? Swim in all of my data?

9. 7 Nov 20072007 Geospatial Interoperability Summit Here’s an idea, lets create a webpage where they can download the data! Daryl: –Creates webpage End user: –Finds my page –Fills out form –Saves data, attempts to figure out format –Attempts to find geo- referencing for their GIS. –Hopefully loads data into app –Commits numerous errors along the way

10. 7 Nov 20072007 Geospatial Interoperability Summit The Open Geospatial Consortium (OGC) Way

11. 7 Nov 20072007 Geospatial Interoperability Summit

12. 7 Nov 20072007 Geospatial Interoperability Summit In 2002, we built OGC services, but they didn’t come…. Why? –Where were the cool clients? –ESRI’s OGC support wasn’t there yet –It all felt slow as compared with local disk access –A bit ahead of its time

13. 7 Nov 20072007 Geospatial Interoperability Summit In 2007, they have come! Thanks to cool clients –Google Maps –OpenLayers –ESRI Interop Extension –Map Server and others ~ 2 million hits per day (~20 “maps” per second)

14. 7 Nov 20072007 Geospatial Interoperability Summit Supporting the Load 12 systems comprise IEM web farm Various levels of redundancy, but no spatial database redundancy yet Roughly 30 GB in, 60 GB out each day Linux IPVS provides redundant entry point and load balancing

15. 7 Nov 20072007 Geospatial Interoperability Summit Available OGC Services

16. 7 Nov 20072007 Geospatial Interoperability Summit http://mesonet.agron.iastate.edu/ogc/ CONUS GOES (Infrared, Visible) CONUS NEXRAD Base Reflectivity, Storm Total Precip, 1 Hour Precip NWS issued advisories,watches, and warnings Iowa Rainfall Estimates Iowa Road Conditions IEM Data Services (temp, winds, etc)

17. 7 Nov 20072007 Geospatial Interoperability Summit Raw NEXRAD Composites Have a large amount of noise due to ground clutter. A simple GDAL script comparing radar echos to its elevation quickly gets rid of most noise.

18. 7 Nov 20072007 Geospatial Interoperability Summit Cleaned Composite

19. 7 Nov 20072007 Geospatial Interoperability Summit Some IEM developed GIS webapps

20. 7 Nov 20072007 Geospatial Interoperability Summit

21. 7 Nov 20072007 Geospatial Interoperability Summit

22. 7 Nov 20072007 Geospatial Interoperability Summit

23. 7 Nov 20072007 Geospatial Interoperability Summit

24. 7 Nov 20072007 Geospatial Interoperability Summit

25. 7 Nov 20072007 Geospatial Interoperability Summit

26. 7 Nov 20072007 Geospatial Interoperability Summit 4 May 2003, heavy rains on freshly tilled soils 25 Sep 2005, heavy rains on mature crops

27. 7 Nov 20072007 Geospatial Interoperability Summit GIS + webcams IEM developed software to have the cameras automatically track the sun and the moon. Can track NEXRAD identified storms as well! Done using FOSS GIS tools.

28. 7 Nov 20072007 Geospatial Interoperability Summit What others have done.

29. 7 Nov 20072007 Geospatial Interoperability Summit

30. 7 Nov 20072007 Geospatial Interoperability Summit Lessons Learned Adopting OGC services can greatly reduce the complexity of serving out your data and using your own data! Performance hasn’t been an issue, most of the overhead is with HTTP. For Map Server, try to limit the number of mapfiles you maintain!

31. 7 Nov 20072007 Geospatial Interoperability Summit Future Work Increase Metadata! Help support NetCDF in GDAL (?) Work with end users on getting IEM data into their platform.

32. 7 Nov 20072007 Geospatial Interoperability Summit Time for Questions? Daryl Herzmann 515.294.5978 akrherz@iastate.edu