Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Slide 1 Sensor network based vehicle classification and license plate identification system Jan Frigo, Vinod Kulathumani Ed Rosten, Eric Raby Sean Brennan

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Scenario Facility monitoring detect suspicious vehicles entering secure area deployed at key access points / check posts or along length of a road Vehicle classes Personal such as car, SUV Heavy loads such as pickup trucks, Military vehicles such as ATV, hummers and huge log trucks Platforms Mica2 motes ARM processor stargates

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Slide 3 Objectives Classify vehicles with High reliability low latency low energy Extract license plate image

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Slide 4 Challenges Small scale deployment at each access point (< 10 units) Vehicles last in influence region for a short time (1-2 seconds) Spectral signature of vehicles changes with time Resource constrained devices Vehicles moving at variable speeds Environment Temperature Physical barriers (trees, winding road, environmental) System Power

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Slide 5 Seismic Acoutic Node Architecture Geophone Mica2 Stargate Microphone Network 900 MHz 2 GHz

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Seismic-Acoustic Field Experiment

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Slide 7 Frequency characteristics of seismic detection Geophone placed 50 ft away from road to avoid acoustic interference

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Slide 8 Seismic detection Sample at 100 Hz 16 bits samples using MDS320 board Estimate ‘energy’ of 12-25Hz band Haar wavelets up to level 2 Energy = average of coefficients of band 2 Haar wavelet computed on 128 samples every 10 ms 10 new samples each round 118 samples from previous round Compute variance of the energy on moving window of size 20 Use variance threshold to detect vehicle

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Slide 9 Seismic detection performance Seismic detection triggers acoustic sampling and / or processing Energy efficient Person walking 2 ft does not trigger detection Person thumping feet (running) < 10 ft away triggers detection Can be isolated using temporal characteristics

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Slide 10 Acoustic classification FFT used to obtain spectral characteristics Fixed point FFT implemented on stargate Classifier trained using FFTs computed on stargate Identify best feature vector characteristics to distinguish between vehicle classes Use Fisher linear discriminant analysis (FLDV) for classification Pairwise classifier Select order of classification that maximizes accuracy Input obtained vectors into stargate for classification

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Slide 11 Acoustic classification FFT computed every 1/8 of a second 512 samples FFT 12 samples from previous round 8Hz resolution Consider frequencies > 64 Hz Mic response varies at < 60 Hz Temporal variation in response < 60 Hz (probably due to wind) Closest 1.5 seconds of data used as training samples

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Slide 12 Mean FFT coefficients for Vehicle Classes

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Slide 13 Acoustic classification Classification order that maximizes accuracy Presence of vehicle Hummer vs car and truck Car vs truck Presence Use average ‘energy’ of Hz band Moving window variance (size 20) based detection Hz band less sensitive to high frequency chirp and wind noise

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Classification using FLDV Hummer vs car / truck Feature vector 1: ratio of energies of 80 – 112 Hz and 350 – 500 Hz bands Feature vector 2: ratio of energies of 250 – 300 Hz and Hz bands Ratios less sensitive to mic response and distance from road FLDV uses training samples to compute best projection vector

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Slide 15 Acoustic classification Integrating output Approaching vehicle characteristics differ from closest point Classifier operates for ~10 seconds as the vehicle approaches and passes node Classifier designed such that — Low probability of car being classified as truck or hummer at any instant — Low probability of truck being classified as hummer at any instant > 5 consecutive truck classifications within a test run-> vehicle is truck > 5 consecutive hummer classifications within a test run -> vehicle is hummer

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Slide 16 License Plate Recognition Classify Pixels Find and filter regions Find bounding boxes Extract and resample plates Send image Sending only license plates over the network requires very little bandwidth. Resampling license plate to fixed size reduces network usage when vehicles are close. Computationally expensive OCR is run on remote host Algorithms use integer arithmetic only

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Performance Vehicle classification Within 2 seconds of object passing zone Reliability > 90% 2.26 watts power consumption [when computing] Will last about 12 hours if continuously computing on 4.8V 4200mAH cell License plate recognition Detection accuracy > 95% Suppresses 90% of image content Latency 5.1 seconds [mainly for image capture] Higher power consumption [can perform 5582 trials on 4.8V 4200maH cell]

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Slide 18 License plate detection algorithm Combination of Viola jones detector [object detection] Decision tree classifier [license plate segmentation] Analyse using Haar Wavlet like features Efficient to compute using an integral image Integral image is also required for resampling license plate Computational time is independent of feature size Train decision tree classifier to recognize license plate pixels Decision trees are very efficient: — Only integer arithmetic required for evaluation — Tune the tree to rapidly reject most pixels very quickly

Operated by Los Alamos National Security, LLC for NNSA U N C L A S S I F I E D Slide 19 Ongoing and Future Work Increase power efficiency Embedded FPGA implementations for in situ computing Estimated 5 – 100x power savings and 30 – 100x speed up in run-time performance over COTS Node Architecture combination of — new ARM processor technology on next generation mezzanine board — Igloo FPGA on sensor board Low power analog acoustic circuitry Design of cooperative analog-digital signal processing systems Upto 300X power savings Identify optimal balance between nodal computation, in-network processing and central computation