NSF Industry-University Cooperative Research Center for Advanced Knowledge Enablement NOA Inc DBA TerraFly Inc IBM Naphtali Rishe Control and mapping of airborne video sensors

GIS & Data GIS Analytics GIS & Data for South America Moving Objects, Modeling Data Transmission Predictive Spatial Modeling for Diseases Moving Video Streaming Aerial Imaging from Balloons

The Challenge Securing a vast border region with limited human and capital resources Border Control Irregular Immigration Crime and Illegal Trafficking Persistent, live surveillance combined with intelligence and operations is often needed, but not available

Our Solution: CARMEL-TerraFly Users are able to: Select a geographic area of interest Retrieve multimedia data from mobile sensors in the area View streaming videos in real time (vehicles, people, animals, etc.) Easily set temporal, geographic, textual and structural constraints Browse relevant video clips synchronized with map display of the ground projection of the area video-recorded MAP querying and visualization is provided by TerraFly® (

CARMEL-TerraFly Demo Video: Web Simulator: User Interface Solid trapezoids are current projections of cameras’ view. The dotted trapezoid is synchronized with the playback of the selected camera. CARMEL: Geo-monitoring of stationary, mobile and airborne video streams

CARMEL CARMEL Location Traces Surveillance paths for real-time viewing of streaming videos in the path of interest, perform end-to-end tracking of moving objects and make historical comparisons CARMEL: Geo-monitoring of desired surveillance paths Surveillance Path Area Overview: Red trace – already traveled Yellow trace - enroute

CARMEL-TerraFly Location Trace Monitoring & Alerts Quickly and easily zoom into surveillance areas and paths for close monitoring and review, with data fusion and alert capabilities such as scheduled arrivals and departures. Time-sensitive and detailed monitoring to help pinpoint what you need to know Surveillance Path Scheduled Arrival Expected Cargo & Activities

CARMEL Persistent Monitoring Using Mobile Sensors Our ability to effectively visualize moving objects extends to providing real- time, geolocated video and data streams from mobile sensors, such as drones or boats, and allowing fusion and tracking via multiple moving sensors. Real-time monitoring of moving objects and use of mobile, geolocated sensors Real-time, geolocated video surveillance via moving sensors (e.g., drones, balloons, etc.) Maintain detailed and aerial overview of the area of interest

Border Surveillance Scenario In an area along the El Paso/Mexico border that is known for illegal immigration crossings as well as drug smuggling, a border patrol officer is conducting routine surveillance by viewing a video stream from a drone. During the surveillance, the officer sees unusual activity on the river’s edge. Surveillance Path

Border Surveillance Scenario The officer is able to see what is occurring in the area in real time: A car parked in an area on the Mexican side of the border that provides easy access to the river. A person running between the river and highway on the US side of the border. Car parked by the river Person running to the highway

Border Surveillance Scenario To determine whether this is an illegal crossing or drug smuggling operation, the officer is able to: “rewind” surveillance video of the area to see that it is indeed an illegal crossing of the boarder call on-the-ground units for immediate dispatch to the area. Person running to the river

Border Surveillance Scenario Because the surveillance is in real time, the officer is also able to keep constant track of the individuals who cross the border illegally to help the on-the-ground officers apprehend the suspect. Follow the running person

Border Surveillance Scenario The officer is also able to: search for historical videos of the same area to determine if this location has been used in the past for illegal crossings, Find a new pathway into the river at this location, which indicates that this is a new illegal crossing point. New Pathway Historic VideoCurrent Video