ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors Ming Zhao Software Developer ArcGIS GeoEvent Processor for Server mzhao@esri.com Patrick Hill Software Developer ArcGIS for Military Solutions Team patrick_hill@esri.com

ArcGIS GeoEvent Processor for Server Integrates and exploits real-time data Integrates real-time streaming data into ArcGIS Performs continuous processing and real-time analytics Sends updates and alerts to those who need it where they need it GeoEvent Services ws:// Outputs GeoEvent Processor ArcGIS Server Inputs ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Receiving real-time data Connectors You can easily integrate real-time data with ArcGIS by using an input connector. GeoEvent Processor Inputs Outputs GeoEvent Services Esri Gallery Twitter Instagram VMF Cursor-on-Target CAP TAIP (Trimble) NMEA RabbitMQ RAP (Sierra Wireless) GeoMessage ActiveMQ Partner Gallery GNIP Geofeedia OSIsoft ASDI (FAA) exactEarth Zonar NetworkFleet CompassCom Valarm Harris Receive RSS Receive text from a TCP Socket Receive text from a UDP Socket Receive JSON on a REST endpoint Receive Features on a REST endpoint Receive JSON on a Web Socket Receive JSON on external Web Socket ws:// http:// Poll an ArcGIS Server for Features Poll an external website for JSON Out of the Box Watch a folder for new .csv files Watch a folder for new .json files .csv .json ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Applying real-time analytics GeoEvent Services A GeoEvent Service configures the flow of GeoEvents, the Filtering and GeoEvent Processing steps to perform, what input(s) to apply them to, and what outputs(s) to send the results to. ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Sending real-time data Connectors You can easily send updates and results to those who need it where they need it using an output connector. GeoEvent Processor Inputs Outputs GeoEvent Services Twitter Esri Gallery ActiveMQ RabbitMQ Hadoop MongoDB CESIUM Partner Add a feature Update a feature Send an email Send an instant message im Send a text message Publish JSON to a Web Socket ws:// Out of the Box Push JSON to an external Web Socket ws:// Publish text on a TCP Socket Publish text on a UDP Socket Publish JSON to an external website http:// Publish on a REST endpoint Write to a .csv file .csv Write to a .json file .json ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Extending GeoEvent Processor Software Development Kit (SDK) You can create your own custom connectors and processors using the GeoEvent Processor Software Development Kit (SDK). GeoEvent Processor Inputs Outputs GeoEvent Services Your connectors Your connectors Your processors ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

GeoEvent Processor SDK api: JavaDoc content associated with GeoEvent Processor SDK lib: Contains library used to build connectors (and processors) repository: Local maven repository samples: Sample connectors (and processors) GeoEvent Processor Developer Guide ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Connectors ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Connectors What Does a Connector Do? Connectors are used to create Inputs and Outputs, hiding the technical details It might be very specific Get latest earthquakes from USGS Or more general Connect to an RSS feed ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

National Weather Service Creating Inputs Configured using Connectors RSS Connector National Weather Service Connectors JSON over REST Receive Text via TCP US Geological Survey GeoEvent Processor Processor Inputs GeoEvent Services Live Traffic Services Storms Earthquakes Outputs Traffic ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Connector The connector helps the user by Providing default values How Does it Help The connector helps the user by Providing default values Re-label properties to be appropriate to the context Move properties under an “advanced” area to discourage modification Completely hide properties that the user should not see ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Input (Receive text from a TCP Socket) Connector How Does it Help By choosing a Connector, the user implicitly selects components from the GeoEvent Processor that know HOW to move data (Transport) WHAT the data looks like (Adapter) Example Input Input (Receive text from a TCP Socket) GeoEvent Start Stop Byte [ ] GeoEvent Services TCP Transport Text Adapter ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Exploring a Connector Demo ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Transport ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Transport Provides a Transport Definition Instantiates new Transports What makes up a Transport? Transport Service Provides a Transport Definition Instantiates new Transports Transport Definition Defines transport metadata and properties Transport Implements transport application logic ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Transport Behavior Transports are given Properties defining behavior A “ByteListener” where the bytes should be sent Transport is started by the server and it sends bytes to the receiver Transport is stopped by the server and it stops sending bytes ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Transport Lifecycle Transports Transports have a lifecycle that determines if they are producing data. Server calls start() Transport reads properties and starts STARTING Internal error occurs STOPPED STOPPED STARTED STARTED STOPPING ERROR Transport closes resources Server calls stop() ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Outbound Transports Transports Outbound Transports accept arrays of bytes from the Adapter and transmit them. Occasionally the destination for the bytes depends on content in the GeoEvent. The Transport has the option of “looking back” at the GeoEvent that generated the bytes, and using it to route the data. ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Transport Code Walkthrough Starting the Transport ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Transport Code Walkthrough Receiving Data ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Transport Code Walkthrough Applying Properties Stopping Transport ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Properties ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Property Definition Properties Transports and Adapters request properties through their “Definition” class. Each requested property has a Name Description Type (String, Integer, Float, …) Default Value ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Advanced Topics – More on Properties Properties can be Mandatory or Optional. Some properties are dependent on others Example: “Compression Algorithm” depends on “Compressed = True” Some properties only accept values from a list of Allowed Values “Compression Algorithm” : [ Run Length Encoding, Zip, LZW ] ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Adapter ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Adapter Provides an Adapter Definition Instantiates new Adapters What makes up an Adapter? Adapter Service Provides an Adapter Definition Instantiates new Adapters Adapter Definition Defines adapter metadata and properties Adapter Implements adapter application logic ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Adapter Behavior Adapters are given Adapters are DATA DRIVEN Properties defining behavior A “GeoEventListener” where the GeoEvents should be sent Adapters are DATA DRIVEN No start/stop calls The adapter is handed a byte array and pushes any generated GeoEvents to the Listener ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Adapter Code Walkthrough Adapter Properties ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Adapter Code Walkthrough Adapter Properties ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Adapter Code Walkthrough – Parsing Data ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Installing and Using Custom Adapter and Transport Demo Installing and Using Custom Adapter and Transport ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Processors ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Extending GeoEvent Processor What is a Processor? Software component written in Java User configurable Performs specific actions on GeoEvents Runs continuously inside of server processing environment Shares lifecycle with GeoEvent Service GeoEvent Processor Inputs Outputs GeoEvent Services 783 2323 5122 7138 9102 7827 4913 2981 1326 802 ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Extending GeoEvent Processor GeoEvent Processors You can perform continuous analytics on GeoEvents as they are received using a processor. GeoEvent Processor Inputs Outputs GeoEvent Services Out of the Box Field Enricher Field Reducer Track Gap Detector Incident Detector Field Mapper Geotagger Field Calculator You can create your own processors. Track Idle Detector Esri Gallery ETA Calculator Service Area Buffer Ellipse Range Fan Visibility Query Report Slope Calculator SDK Volume Control ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Anatomy of a Processor What makes up a Processor? Processor Service Instantiates processor definition, processor and applies configuration Processor Definition Defines processor metadata, properties and GeoEvent Definition(s) Processor Implements processor lifecycle ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Processor Definition Step 1: Properties and GeoEvent Definition(s) ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Processor Definition Identification Name Domain Version Description Step 1: Metadata Identification Name Domain Version Description Label Contact Information ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Processor Step 2: Lifecycle Review GeoEvent Processor GeoEvent Service Stop Start Processor Creation Shutdown Initialization Processing… ERROR Validation ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Property Definitions & Spatial API Demo Property Definitions & Spatial API Rangefan Calculator ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Processor Creation Initialization Validation Step 2: Lifecycle ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Processor Processing Shutdown Service Start / Stop Step 2: Lifecycle ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Processor Service Create new Java class Step 3: Definition Create new Java class Extend GeoEventProcessorServiceBase class Creates processor definition Creates processor instance Applies configuration ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Processor Service Step 4: Configuration Declare processor service as a service implementing GeoEventProcessorService interface from GeoEvent Processor SDK ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Processor Service and Processor Demo Processor Service and Processor Rangefan Calculator ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Putting it Together Rangefan Dashboard Demo ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Connectors – Recipe for creating inputs/outputs Review Connectors – Recipe for creating inputs/outputs Transport – Moves raw data in/out of the GeoEvent Processor Adapter – Converts raw data to GeoEvents and back Properties – Used to configure an input/output for a specific use case ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

GeoEvent Processor Developer Guide in the SDK Additional Resources Developer Guide in the SDK Forum http://forums.arcgis.com/forums/257-GeoEvent-Processor Resource Center – Includes Tutorials http://pro.arcgis.com/share/goeevent-processor Browse the GitHub projects ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Offering ID: 1233 Please fill out the session survey: Thank you… Please fill out the session survey: Offering ID: 1233 Online – www.esri.com/ucsessionsurveys Paper – pick up and put in drop box ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

Questions / Feedback? To learn more: http://pro.arcgis.com/share/geoevent-processor Ming Zhao | Software Developer ArcGIS GeoEvent Processor for Server mzhao@esri.com Patrick Hill | Software Developer ArcGIS for Military Solutions Team patrick_hill@esri.com ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors

ArcGIS GeoEvent Processor for Server: Extending with New Processors and Connectors