1. Real-Time GIS Use Cases and Implementation Patterns
Derrick Burke

2. Overview Background Real-Time Data Formats Updating Real-Time Data
Products / Patterns / Architecture / Components

3.
Building Live Data Feeds using Python

4. Building Live Data Feeds using Python

5. Building Live Data Feeds using Python

6. Real-Time Data, what do we mean?
Frequently Changing Content!
Real-Time Data
Changes moment-by-moment
Trickle or burst
No set intervals
Near Real-Time Data
Periodic or Batch changes
Bulk updates typical
Timed intervals

7. Typical Data Formats Real-Time Data Near Real-Time Data Sensors
File based
Vehicles
Shapefile
Manufacturing / Industrial
Field equipment
Raster / Spatial Analysis
SCADA
NetCDF
BIN
Internet Resources
Streaming (Twitter)
Bulk / Batch
Wide Range of Formats
Heavy Geoprocessing / Enrichment
CSV, XML, KML, RSS, GeoRSS, …
Data Transformation
Data Aggregation (multiple sources)

8. Available Products GeoEvent Extension (GEE)
Selected Content
OSIsoft – PI Integrator for ArcGIS
Aggregated Live Feed (ALF) methodology

9. Update Patterns (which product do I choose?)
Source / Function
GEE
ALF
Sensors Networks X *
Vehicle Tracking
Raster
Weather Data
Shapefile
File GDB
XML / KML
CSV
JSON
RSS / GeoRSS
BIN
NetCDF
Internet Streams
Geoprocessing
GeoFence
Update Patterns (which product do I choose?)
SCADA System w/Alerts
GeoEvent Ext. / OSIsoft
Raster Weather Data
ALF – ArcPy w/Spatial Analyst
Vehicle Tracking
GeoEvent Ext. w/GeoFence
Shapefile Processing
ALF – ArcPy
* = Depends on format and / or architecture

10. ArcGIS Server GeoEvent Extension
Data Bridge
Update / Output
Feature Service
Expose Stream Service
ArcGIS Online
Others
HA w/Clusters

11. GeoEvent Extension Components
Input Connector – Read
OSIsoft – PI Integrator
Decision – Filter
Compute – Processor
Output Connector – Write

12. GeoEvent Extension (cont)
GeoFence Alerting
Polygon(s)
Spatial Filter
Position Tracking
Inside / Outside
Enter / Exit

13. Aggregated Live Feed (ALF) Methodology
Python Solution
GP w/ArcPy
Update / Output
Read-Only File GDBs
Feature Services
Others…Analysis
Flexible & Extensible
HA w/Redundancy

14. ALF Methodology – NDFD Precipitation, Raster Processing

15. ALF Methodology – Cloud / Distributed Feed
Internet
Log Files
Work
FileGDB
Feed Routine
(Aggregator)
Deployment
Logic
Data
Archive
Zip File
Feed Aggregator
ALFprocessor.py
Data Deployment
ALFdeployer.py
Internet
Work
Folder
Download and
Deployment
Logic
Live
FileGDB
ArcGIS Server
Published
Map Service
Amazon S3
Common Storage
(External or Internal)

16. ALF Feed Construction – What’s required
Pre-requisites
Components
Python
ALFlib.py
ArcPy
Function / Class Library
Others (.py, 7zip, degrib, …)
ALFprocessor.py
Optional
Feed Processor +
‘<feed file>.cfg’ (data processing logic)
IDE – PyScripter, VS, …
ALFdeployer.py
Archive Distribution

17. Questions & Comments? AGOL Community Page (samples, scripts)
GeoEvent Extension Resources:
Integrator-for-Esri-ArcGIS/System-Requirements
Live Data Feeds :
added/

18. Rate This Session

19. Thank You!
@derrickburke