Using Topic-Based Publish/Subscribe for Managing Real Time GPS Streams Marlon Pierce, Galip Aydin, Zhigang Qi Community Grids Lab Indiana University 1

California Real Time Network Network Data Rates Message Format TimeRYOASCIIGML CRTN GPS Site Positions (9 Stations) 1 second1.5KB4.03KB48.7KB 1 hour5.31MB14.18MB171.31MB 1 day127.44MB340.38MB4.01GB 1 month3.8GB9.97GB123.3GB 1 year45.8GB119.67GB1.41TB Entire SCIGN Network (250 stations) 1year1.23TB16.18TB160TB Continuous GPS Stations (CGPS) are depicted as triangles while the Real-Time stations are represented as circles. Image is obtained from SOPAC GPS Explorer at How does one manage all the data generated by the 85 stations? How can you get just the data you want? Note this is fundamentally different from traditional request/response style Web Services.

RDAHMM: GPS Time Series Segmentation Slide Courtesy of Robert Granat, JPL  Complex data with subtle signals is difficult for humans to analyze, leading to gaps in analysis  HMM segmentation provides an automatic way to focus attention on the most interesting parts of the time GPS displacement (3D) length two years. Divided automatically by HMM into 7 classes. Features: Dip due to aquifer drainage (days ) Hector Mine earthquake (day 626) Noisy period at end of time series

Use Case - GPS Sensors  GPS is used to identify long-term tectonic deformation and static displacements. SCIGN has 250 Real-Time GPS Stations.  SOPAC GPS networks:  8 networks for 80 stations produce 1Hz high resolution data.  Socket based real-time binary-RYO format access is available.  We developed filters to provide multiple format (RYO, ASCII, GML) real- time streaming access.  OHIO principle (a general principle required by DOD) and chain of filters.  Our Architecture  Uses publish/subscribe based NaradaBrokering for managing real-time GPS streams  Utilizes topics for hierarchical organization of the sensors  Deploys successive data filters ranging from format translators to data analysis codes  Could potentially be used to run RDAHMM clones to monitor state changes in the entire GPS network  We are partner in a pioneering project to use the real-time GPS data for the first time in this context. 4

Processing Real-Time GPS Streams 5 ryo2nb Raw Data RYO Ports NB Server ryo2asciiascii2gmlascii2pos Single Station Displacement Filter Station Health Filter RDAHMM Filter Scripps RTD Server Scripps RTD Server ryo2nb Raw Data ryo2asciiascii2pos Single Station RDAHMM Filter A Complete Sensor Message Processing Path, including a data analysis application. /SOPAC/GPS/CRTN01/RYO /SOPAC/GPS/CRTN01/ASCII /SOPAC/GPS/CRTN01/POS /SOPAC/GPS/CRTN01/DSME GPS Networks

Application Integration with Real-Time Filters Station Monitor Filter records real-time positions for 10 minutes and calculates position changes Graph Plotter Application creates visual representation of the positions. RDAHMM Filter records real-time positions for 10 minutes and invokes RDAHMM application which determines state changes in the XYZ signal. Graph Plotter Application creates visual representation of the RDAHMM output. 6

GIS Grid Usage Model – Earthquake Science  Supporting geophysical repositories and real-time sensors is essential  To analyze a typical earthquake it is important to access to precise measurements of the initial earthquakes and aftershocks  To support earthquake forecasting and the time and spatial positions of the forecasts  PI can be used with existing data  RDAHMM can be used with the real-time data  Earth Science field is moving from a previously data poor field to a data rich world. We will have thousands of sensors spread around the world. (i.e. GPS sensors, InSAR satellites) 7

Recording and Replaying Sensor Streams  Filters can be used to record and replay scenarios, such as Earthquakes in GPS case.  We developed RYO Recorder and RYO Publisher Filters.  The RYO Recorder creates daily archives of the GPS Streams.  RYO Publisher can be used to play daily or certain segments of the records.  We replayed the 2004 Southern California Earthquake using Parkfield GPS network archive  These filters are used in the performance and scalability tests. 8

SensorGrid Performance Tests  Two Major Goals: System Stability and Scalability  Ensuring stability of the distributed Filter Services for continuous operation.  Finding the maximum number of publishers (sensors) and clients that can be supported with a single broker.  Investigate if system scales for larger number of sensors and clients. 9

Test Methodology  The test system consists of a NaradaBrokering server and a three-filter chain for publishing, converting and receiving RYO messages.  We take 4 timings to calculate mean end-to-end delivery times of GPS measurements.  The tests were run at least for 24 hours.  GridFarm servers are used in these tests. T transfer = (T 2 – T 1 ) + (T 4 – T 3 ) 10 NB Server 1 RYO To ASCII Converter RYO Publisher Simple Filter

1- System Stability Test  The basic system with three filters and one broker.  The figure shows average results for every 30 minutes.  The average transfer time shows the continuous operation does not degrade the system performance. 11

NB Server RYO To ASCII Converter Simple Filter RYO Publisher 1 RYO Publisher 2 RYO Publisher n 2 – Multiple Publishers Test  We add more GPS networks by running more publishers.  The results show that 1000 publishers can be supported with no performance loss. This is an operating system limit. 12 Topic 1A Topic 1B Topic 2 Topic n

3 – Multiple Clients Test  We add more clients by running multiple Simple Filters which subscribe to the same ASCII topic.  The system can support as many as 1000 clients with very low performance decrease. 13 NB Server RYO To ASCII Converter Simple Filter 1 RYO Publisher 1 Simple Filter 2 Simple Filter n Topic 1A Topic 1B 1000 Clients

Extending Scalability  The limit of the basic system appears to be 1000 clients or publishers.  This is due to an Operating System restriction of open file descriptors (1024 for Red Hat Linux) which can be increased by changing OS parameters.  To overcome this limit we create NaradaBrokering networks with linking multiple brokers. NB supports scalable linkage of the brokers for building tree like architectures.  We run 2 brokers to support 1500 clients.  Number of brokers can be increased indefinitely, so we can potentially support any number of publishers and subscribers. 14

4 – Multiple Brokers Test  NaradaBrokering allows creation of Broker networks.  We create a two-broker network.  Messages published to first broker can be received from the second broker.  We take timings on each broker.  We connect 750 clients to each broker and run for 24 hours. We chose 750 clients to stay well below the saturation limit.  The results show that the performance is very good and similar to single broker test. 15 NB Server 1 NB Server 2 RYO To ASCII Converter Simple Filter 1 RYO Publisher Topic 1A Topic 1B Simple Filter 2 Simple Filter 750 Simple Filter 751 Simple Filter 752 Simple Filter 1500 Topic 1B NB Server 2

4 – Multiple Brokers Test 16

Real-Time Filters Test Results  The RYO Publisher filter runs at 1Hz and publishes 24-hour archive of the CRTN_01 GPS network, which contains 9 GPS stations.  The single broker configuration can support 1000 clients or publishers (GPS networks individual stations).  The system can be scaled up by creating NaradaBrokering broker networks.  Message order was preserved in all tests. 17

Acknowledgement  Galip Aydin: Designer and implementer  ZhiGang Qi: SensorGrid Performance Tests  We thank Prof. Yehuda Bock and his group at SIO for their help with real-time GPS data streams.  Robert Granat, JPL: RDAHMM developer 18