Smart UNattended airborne sensor Network for detection of vessels used for cross border crime and irregular entrY FP7 SUNNY Project UMS 2014 William Martin BMT Group Exponor, Porto - 30 th May 2014

Overview The SUNNY project will develop novel and comprehensive solutions for intelligent surveillance of maritime borders Detection of illegal entry using a heterogeneous network of sensors carried by Unmanned Aerial Systems (UASs) –Smuggling –Irregular Immigration The surveillance of maritime borders is prioritised and sea scenarios will be used in the project

Project Partners

Motivation Illegal incursions and border crossing are causing social and economic problems for EU member states; especially Mediterranean countries with Southern EU Maritime Borders Strengthening border security for immigration control and fighting cross‐border crime is a key objective in the EU Internal Security Strategy 2 nd Quarter 2011 Frontex Risk Analysis Network Report: Italian islands in the Central Mediterranean identified as a hotspot for illegal border ‐ crossing into the EU migrants from N.Africa and other sub ‐ Saharan countries in small sea vessels, the majority of which in Q2 had been forcibly departed from Libya

Concept First-tier sensors patrol large border areas to provide situation awareness and detect suspicious targets: –Medium altitude; –Long-endurance; –Autonomous. From this information, smaller second- tier sensors collect further evidence via focused surveillance: –track targets; –threat evaluation; –more accurate target recognition. Ground station coordinates and communicates with UAS network Two-tier, intelligent heterogeneous UAS sensor network providing both large field and focused surveillance capabilities:

Objectives Develop and integrate an airborne sensor network for the purpose of maritime surveillance and situation awareness –To enhance EU ‐ border security The SUNNY system will implement: –Fast, near real ‐ time detection and tracking of people, vehicles and ships; –UAS mounted sensors: infrared, hyper-spectral and radar; –On ‐ board, real ‐ time data processing; –Adapted data link solutions from existing data link technology; –Innovative data fusion and focus-of-attention techniques; –An effective and intuitive human ‐ machine interface.

User requirements SUNNY has questioned Navy and Customs in four Southern EU countries –Portugal, Greece, Spain, Italy Place most importance on preventing & detecting Drug traffic & Illegal immigration –Then human trafficking and weapon smuggling Wind and visibility are considered the main limitations to safety. Rain is secondary. All groups consider UAS use important, yet most do not use any –Preference for Small (~2h), Tactical (~3h) and Endurance (~12h) Want to equip end-users –Autonomous usage (no externals) –Field training with real platforms – preferential –Integration with older systems or partial replacement is preferred

IMINT The Imagery Intelligence (IMINT) standards were developed for observation and surveillance using satellites and manned aircraft –allow Imagery Analysts to detect, classify and identify objects, units or situations The IMINT methods are: –”Pre-Mission Preparation”; –“Route to Target”; –“Data Collection”; –”Transfer of Primary Imagery to Exploitation”; –”Collect Collateral For Information Exploitation”; –and “Exploitation” (analysis and target identification). The methods have since evolved to account for unmanned vehicles and on-board results exploitation – leading to a more concise set of standards for SUNNY

Threat processing Sensors and real-time on-board processing integrated on UAS –low weight, low cost, high resolution that can operate under darkness, snow, rain, etc. Imaging sensors generate RGB, infrared and hyper-spectral images, and use radar information to detect, discriminate and track objects of interest Coupled sensor processing using on- board processors produce preliminary detection results

Threat processing Novel identification algorithms, independent of target and terrain knowledge, analyse UAS data; –e.g. using deviations of colours, shapes and textures between suspicious objects and their surroundings UAS information is combined with: –target models; –other data, e.g. AIS; –environment knowledge; –global situation; –impact assessment. A 3D visualisation UI will show: –assets (e.g. borders): –sensors (e.g. UASs); –agents (e.g. vessels).

Data compression All UAS activities will use civilian frequencies and protocols on an adapted data-link –limitations in bandwidth / range Raw UAS sensor data cannot be concisely represented for human operators Human operator assisted decisions will require small timescales –data must be filtered to a logical format reducing non-relevant information Solution: Automatic data pre ‐ processing and compression on ‐ board UASs achieves data reduction between the vehicle and ground stations –compression must not reduce accuracy or image resolution; –the proper selection of relevant information and images must occur

Sensors summary Developing multi ‐ band IR cameras for agile platform use –Low power consumption with a high temperature range Hyper-spectral sensors utilise characteristic spectral signatures –includes information about fine colour and chemical differences –on board data processing is mandatory to reduce the very high raw data output Developing compact, low power CW radar technologies for both UAS tiers –image processing will compensate for image blur due to relative motion of target –3D radar imaging will enable more robust object recognition than existing radar sensors

Example Example scenario for the SUNNY system: Tier 1 UAS detects a target vessel using Long-Wavelength Infrared sensors Target detection supported by CW radar sensors Ground control validates target and sends in Tier 2 UASs Tier 2 Hyper-spectral (VisNIR ) sensors can identify the number of persons on-board Visibility degrades between Tier 1 detection and Tier 2 deployment: CW radar sensors allow continued operation Thermal LWIR hyper-spectral sensors can enable night-time operation Threat evaluation is made – relevant authorities are notified to take action

Conclusion SUNNY intends to develop a new tool for collecting real ‐ time information in operational maritime scenarios SUNNY represents an advancement due to the following features: –A two ‐ tier, intelligent UAS sensor network; –Novel sensors and on ‐ board data processing, integrated on UAS systems; –The exploitation and adaptation of emerging standard wireless technologies and architectures. SUNNY will deliver pre ‐ processed information with meaningful decision support tools –to enhance and support the capabilities of border surveillance personnel

QUESTIONS & COMMENTS

EXTRA SLIDES

SUNNY and EUROSUR "The use of UAVs in the framework of EUROSUR is excluded for the time being. Their potential future use depends on a number of pre-conditions, such as their full integration into civil/non-segregated airspace (planned for 2016 at the earliest) and further operational testing, showing that this technology is at least as cost-effective and efficient as manned surveillance planes (which currently is not the case). This means that any assessment whether the results of the FP7 SUNNY project can be used for border surveillance respectively in the framework of EUROSUR can only be carried out after the SUNNY project has been finished in 2017."

Infrared sensors Developing multi ‐ band IR cameras for agile platforms –Long-Wavelength Infrared (LWIR) information for detection; –Visible Near (VisNIR) and Short-Wavelength Infrared (SWIR) for identification. Low power consumption with a high temperature range LWIR uncooled IR camera engines are already used in static surveillance –advantages of lower weight and less energy consumption Integration of IR camera with UAS still requires: –mechanical and electrical integration, –communication and data link connections, –data processing, analysis and visualization connections (VisNIR: nm, LWIR: 8-15 µm, SWIR: µm)

Hyper-spectral detection Hyper-spectral (HS) imaging is uniquely able to both detect targets and to identify them, by using their characteristic spectral signatures –Improvement over traditional RGB and broadband thermal imaging Spectral signatures include information about small colour differences and particularly about chemical differences –VisNIR region can discriminate between vegetation and man-made targets in day-time operations; –Thermal LWIR enables night-time operation. On board data processing is mandatory to reduce the very high raw data output –Integration of multiple-core GPU HS imagery fusion, from multiple UASs, to generate wider, improved situational awareness

Radar sensors Innovative CW miniSAR sensor for both UAS tiers Features new signal and radar imaging processing for enhanced target identification –will extend monitoring operations in reduced visibility conditions; –contributes to the detection and identification phases. Developing compact, low power CW radar technologies Innovative ISAR image processing to compensate for image blur due to relative motion of target Developing innovative 3D radar imaging, enabling more robust object recognition than existing radar sensors (SAR = Synthetic Aperture Radar; ISAR = Inverse-SAR; CW = Continuous Wave)