O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY Making Usable Data Available and Available Data Usable: The Application of Geospatial Standards in Near Real-Time Sensor Networks Daniel Getman Aaron Myers Geographic Information Science and Technology Group Oak Ridge National Laboratory April 20, 2007

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY What is SensorNet  Much more than what is presented here  Stand Alone Sensor Networks Mobile Sensor Systems Weigh Station Systems Port Systems Military Base Systems  Distributed Sensor Networks Connecting stand alone systems in a larger network  Overarching Concepts Interoperability/Standards Real Time Data Distribution and Alerting Ubiquitous Data Access Development of Global Scale Sensor Networks

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY Describing the Stand Alone Systems  Multiple Sensors  Chemical – Radiation – Biological – Weather  Video with real time analysis capabilities  Multiple Viewers  ArcMap  Google Earth  Browser based viewers  Integration with Models  HPAC  Data Distribution Capabilities  Mobile systems communicating with operations centers and other systems

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY Defining The Goals  Minimum System Goal  Provision of near real-time data within the stand alone system while also providing access to data to external users including Operation Centers and other emergency response personnel  Maximum System Goal  Develop an open standards based system within which a variety of sensors, viewers, and complete stand alone systems can plug-and-play can use and share near real-time sensor data  Stretch Goal  Make the openness of the system transparent to all users

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY Standards Used in the Stand Alone Systems  Open Geospatial Consortium (OGC)  Sensor Web Enablement (SWE) Includes Sensor Observation Service (SOS), Sensor Alert Services (SAS), and Sensor Planning Service (SPS)  Web Feature Service (WFS)  Geography Markup Language (GML)  U.S. Department of Homeland Security (DHS)  ANSI N42.42 (Radiation Data Representation)  Organization for the Advancement of Structured Information Standards (OASIS)  Common Alerting Protocol (CAP)  National Institute of Standards and Technology (NIST)  IEEE 1451 (Sensor Plug-n-play) Courtesy: David Resseguie, ORNL

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY Why Standards are Great  Standards are great once they are adopted  Data sharing is not only easier but can be easily automated  Everything is less expensive…eventually  Creativity is stimulated in application development

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY Why Standards Can Be Difficult to Use  Querying differences  Data Size Differences  Data Structure Differences Traditional Database Query WFS Database Query

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY WFS and SQL Queries  Traditional SQL Query SELECT Col1, Att1, Val1 FROM TABLE1 WHERE Val2 = “SomeValue”  WFS Query //snet:Val2 SomeValue

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY Simple WFS Performance Tests

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY Practical Difficulties  Clients can have conflicting expectations  Things that are difficult with Standards Based Systems Legacy Software and Data –Few software packages will natively access standards based data sources Familiar/Robust/Highly Functional Interface –It has to be as good as the proprietary system Near Real-Time Dynamic Data –Lots of queries, some very specific (alerting) Database Design Limitations –Designed for support of standard rather than support of user activities –Large Data Capacity can be problematic »Keeping all of the data without impeding system speed Data Integration –Leveraging of existing proprietary data

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY Idealistic SensorNet System

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY Where does this leave the ISS?

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY The Duct Tape Approach  Implement standards where the technology and client requirements allow  Implement component based solutions that handle the places in the system where standards either do not exist or are not fully implemented  Plan to remove the duct tape when the standards are implemented and adopted

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY Slightly More Realistic SensorNet System

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY Tale of Two Deployments  JFHQ SNAPS in Washington DC  Mobile SensorNet System (Actually in a trailer)  JOC System designed to receive data from Mobile and Base systems  GPS enabled Radiation, Chemical, Weather, Video  HPAC Integration  SRRPP at the Port of Charleston  Land and Marine based SensorNet Systems  Vehicle and Officer GPS Tracking  GPS enabled Radiation, Video

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY JFHQ SNAPS Deployment

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY SRRPP Deployment

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY Lessons Learned with the SMRSS  Intrinsic problems include  Data is duplicated  Two conversion processes to maintain  Performance Issues  Both WFS and ArcSDE are bottlenecks compared to direct database interaction. Together, they can really hinder performance  The conversion to ArcSDE caused a large number of reads on the WFS. This had a negative impact on the overall performance of the system. More reads per second means less writes per second  Inserting data into ArcSDE is significantly slower than inserting data directly into a database We found that ArcObjects inserts were faster than SDE API inserts Querying against the data tables directly in Oracle to gather data used in the conversion process, rather than going through SDE, is a good shortcut  Conversion process can be used for other tasking that would otherwise require an additional WFS call Alerting and interfacing with other systems (cameras for example)

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY Realistic Idealistic SensorNet System No pounding the WFS with SDE Inserts Single Conversion Utility Everyone Gets What They Want

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY Evolution of the RISS Times They Are A-Changin'

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY Evolution of the RISS

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY New & Improved Idealistic SensorNet System

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY - Questions and Comments - Daniel Getman

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY Abstract  The benefits associated with the use of geospatial standards in application development, including vendor neutrality and application interoperability, are often gained at the expense of system performance and the integration of existing proprietary applications that users have already invested in and are familiar with. Determining an appropriate balance of these two strategies can be a difficult calculation. Efforts to effectively bring these concepts together within the same system design can be hampered by issues associated with latency, scalability, conflicting software requirements, and demanding processing requirements. This balance has been experimented with, and achieved to some extent, within two projects undertaken within SensorNet. The process of designing, implementing, and deploying these systems will be described while outlining the issues, balancing the pros and cons, and providing insight into the pitfalls of attempting to combine the two goals of making usable data available and available data usable.