Institute for Software Integrated Systems Vanderbilt University Wireless Sensor Net ISIS Akos Ledeczi Senior Research Scientist

Copyright © , Vanderbilt University 2 Outline  Countersniper system:  WSN-based static deployment  Soldier-wearable system for sniper localization and weapon classification  Sensor node localization:  Radio Interferometric Positioning  Radio Interferometric Tracking  Doppler-shift based Tracking

Copyright © , Vanderbilt University 3 Countersniper System

Copyright © , Vanderbilt University 4 time t2t2 t1t1 t4t4 t3t3 d1d1 f(x,y)f(x,y) ? d3d3 d4d4 d2d2 t 2 – d 2 /v t 3 – d 3 /v t 1 – d 1 /v t 4 – d 4 /v Shot (x 1,y 1,T 1 ) Shot (x 2,y 2,T 2 ) Echo (x 3,y 3,T 1 ) f(x,y) = [max number of ticks in window] = 3 Shot time estimate T sliding window Patented Sensor Fusion Identifies echoes and resolves multiple simultaneous shots

Copyright © , Vanderbilt University 5 Experiments at McKenna MOUT site at Ft. Benning NORTH B1 Church  Sep 2003: Baseline system  Apr 2004: Multishot resolution  60 motes covered a 100x40m area  Network diameter: ~7 hops  Used blanks and Short Range Training Ammunition (SRTA)  Hundreds of shots fired from ~40 different locations  Single shooter, operating in semiautomatic and burst mode in 2003  Up to four shooters and up to 10 shots per second in 2004  M-16, M-4, no sniper rifle  Variety of shooter locations (bell tower, inside buildings/windows, behind mailbox, behind car, …) chosen to absorb acoustic energy, have limited line of sight on sensor networks  1 meter average 3D accuracy (0.6m in 2D)  Hand placed motes on surveyed points (sensor localization accuracy: ~ 0.3m)

Copyright © , Vanderbilt University 6 Shooter Localization VIDEO

Copyright © , Vanderbilt University 7 DARPA IPTO ASSIST: Soldier-Wearable Shooter Localization and Weapon Classification System Muzzle blast Shockwave Zigbee & Bluetooth Microphones 3-axis compass Optional laptop display PDA display Zigbee Bluetooth Zigbee

Copyright © , Vanderbilt University 8 Latest Sensor Board  Detect TOA and AOA of acoustic shockwave and muzzle blast using a single board  New acoustic sensor board:  4 acoustic channels w/ high-speed AD converters  FPGA for signal processing  3-axis digital compass  Bluetooth  LEDs for on-board display  MicaZ connectivity

Copyright © , Vanderbilt University 9 Architecture  PC/PDA (Java/Ewe)  User interface  Local/central sensor fusion  Location information from external GPS  Sensor Board (VHDL/assembly)  Custom DSP IP cores (detection)  Soft processor macros (digital compass, debug & test interface)  Communication bridge  Shared memory paradigm  Mote (nesC/TinyOS):  Data sharing across nodes  Time synchronization  Application Configuration & Management (from a central point)

Copyright © , Vanderbilt University 10 Sensor Fusion  Localization: Single sensor: simple analytical formula to compute shooter location based on Time of Arrival (ToA) and Angle of Arrival (AoA) of both shockwave and muzzle blast.  Localization: Multi-sensor: all available detections are utilized in a multiresolution search of a discrete multi-dimensional consistency function. Consistency function specifies how many observations agree on a given point in space and time.  Online caliber estimation based on measured ballistic shockwave length and miss distance given by the computed trajectory estimate.  Online weapon classification based on estimated caliber and muzzle velocity that is computed using the projectile velocity over the sensor web and the estimated range.

Copyright © , Vanderbilt University 11 Single Sensor Results  Localization rate for single sensors:  range < 130m: 42%  Range < 80m: 61%  Percentage of shots not localized by at least one single sensors alone (range < 150m): 13%  Accuracy:  0.9 degree in azimuth  5 m in range Blue dots: sensors Black squares: targets Black line: trajectory estimate Black dot: shooter position estimate White arrows: single sensor shooter estimates

Copyright © , Vanderbilt University 12 Multi-Sensor Results Localization Results Classification Results  Independent evaluation by NIST at Aberdeen  Shots between 50 and 300 m w/ 6 different weapons (3 calibers)  Trajectory was highly accurate  Big range error at >200m was due to a bug in the muzzle blast detection  Caliber estimation was almost perfect (rates are relative to localized shots, not all shots).  Classification for 4 out of 6 six weapons were excellent  At longer ranges it started to degrade as it needs range estimate, i.e. muzzle blast detections  M4 and M249 was too similar to each other and the test was the first time the system encountered these weapons Sensors located o surveyed points with practically no position error. Manual orientation and then automatic calibration used. No mobility.

Copyright © , Vanderbilt University 13 Radio Interferometric Localization  12000m 2 area  16 XSM motes on the ground  Minimum node distance 25m  3 anchor nodes  Took 50 minutes  Average loc error < 4cm  Maximum loc error 12cm  Maximum “range” 170m

Copyright © , Vanderbilt University 14 Ranging  Interferometric range: d ABCD =d AD −d BD +d BC −d AC  Phase offset measurements: d ABCD mod λ (65cm < λ < 75cm)  Multiple measurements at different frequencies  Wavelength ambiguity d ABCD λd ABCD modulo λ

Copyright © , Vanderbilt University 15 Multipath  Ranging: minimum of discrepancy function (RMS error)  RF multipath may cause significant phase error  Global minimum may not be the correct solution  Solution: use interleaved ranging/localization

Copyright © , Vanderbilt University 16 Radio Interferometric Tracking -Use N “infrastructure nodes” at known locations to track M moving nodes using radio interferometric ranging -Tracked node is transmitter: more measurements, but single tracked object -Tracked node is receiver: less measurements, multiple tracked objects, but less accurate -actual distances between the nodes change as the tracked object moves -1 ranging measurement (20 channels) takes significant time (0.5 sec) -so, the q-range d ABCD changes significantly during the measurement period, rendering the set of Diophantine-like equations incorrect -estimate the mobile object’s velocity and compensate for these errors -to estimate the velocity, we measure Doppler shifts -benefits: -improved accuracy of localization -We can compute the velocity vector of the moving target also Solution: φ 1 CX = d AX – d BX mod λ 1 φ k CX = d AX – d BX mod λ k...

Copyright © , Vanderbilt University 17 Results: single tracked node  Vanderbilt Football Stadium  12 motes deployed at known positions  One extra node is tracked  The tracked node and one other are the transmitters, the rest are receivers  11 channels are measured, but only 4 consecutive ones are used at a time in the sensor fusion  No speed compensation  Consistency function based multiresolution search algorithm running on the base station finds location estimate  Accuracy: <1m  Update rate: ~1 per 3 seconds  Max speed: ~3m/s Note: Hard to establish ground truth

Copyright © , Vanderbilt University 18 Results: multiple tracked nodes  Vanderbilt Football Stadium  5 motes deployed at known positions  3 extra nodes are tracked  The tracked nodes are receivers  10 channels are measured and used concurrently in the sensor fusion  Speed compensation measuring Doppler shift  Analytical solution  Accuracy: <1m  Update rate: ~1 per 4 seconds  Max speed: ~2m/s Note: Even harder to establish ground truth 2 persons walking, one with two nodes in outstretched hands 3 persons walking

Copyright © , Vanderbilt University 19 ISIS+ORNL: Dirty Bomb Localization Outside the window Jumbotron: automatic camera feed Jumbotron/Screen: Tracking info inside Google Earth  Security is guard walking around the stadium with a cell-phone connected radiation detector and an Crossbow XSM mote.  His position is continuously tracked using a radio interferometric technique running on the motes.  A camera automatically tracks his position using the geolocation info from the mote network.  When the radiation level crosses a threshold the detector sends an alarm and the camera zooms in on the position.

Copyright © , Vanderbilt University 20 Dirty Bomb Localization VIDEO

Copyright © , Vanderbilt University Utilizing Doppler Effect  Single receiver allows us to measure relative speed.

Copyright © , Vanderbilt University Utilizing Doppler Effect  Multiple receivers allow us to calculate location and velocity of the tracked node.

Copyright © , Vanderbilt University Intriguing option: if we can utilize radio signals, no extra HW is required Can we Measure Doppler Shifts? Typ. freq Dopp. Shift 1 m/s) Acoustic signals1-5 kHz3-15 Hz Radio signals (mica2)433 MHz1.3 Hz Radio signals (telos)2.4 GHz8 Hz Solution: radio interfereometry

Copyright © , Vanderbilt University 430MHz 430MHz+300 Hz Measuring Doppler shift We use radio interferometry to measure Doppler frequency shifts with 0.2 Hz accuracy. T SiSi A 2 nodes T, A transmit sine MHz f T, f A Node S i receives interference signal (in stationary case) f i = f T – f A T is moving, f i is Doppler shifted f i = f T – f A + Δf i,T (one problem: we don’t know the value f T -f A accurately) 300Hz + Δf i,T Beat frequency is estimated using the RSSI signal.

Copyright © , Vanderbilt University Formalization Unknowns: Location, velocity of T, and f T -f A x=(x,y,v x,v y,f^) Knowns (constraints): Locations (x i,y i ) of nodes S i Doppler shifted frequencies f i c=(f 1,…,f n ) Function H(x)=c: f 4 = f T – f A + Δf 4 = f T – f A + v 4 /λ T Non-linear system of equations! We want to calculate both location and velocity of node T from the measured Doppler shifts.

Copyright © , Vanderbilt University Tracking Algorithm Doppler shifted frequencies Infrastructure nodes record Doppler shifted beat frequency. Extended Kalman filter Location & Velocity Calculate location and velocity using Kalman filter. Non-linear least squares NLS Location & Velocity Update EKF Updated Location & Velocity If maneuver is detected, calculate NLS solution and update EKF state. Show location on the screen. Maneuver detection Yes Run a simple maneuver detection algorithm. No Location & Velocity

Copyright © , Vanderbilt University Experimental Evaluation 27 Vanderbilt football stadium 50 x 30 m area 9 infrastructure XSM nodes 1 XSM mote tracked position fix in 1.5 seconds Non-maneuvering case

Copyright © , Vanderbilt University Experimental Evaluation 28 Vanderbilt football stadium 50 x 30 m area 9 infrastructure XSM nodes 1 XSM mote tracked position fix in 1.5 seconds Maneuvering case Only some of the tracks are shown for clarity.

Copyright © , Vanderbilt University 29 Questions? More information: