1. VoxNet: An Interactive, Rapidly Deployable Acoustic Monitoring Platform Michael Allen¹, Lewis Girod², Ryan Newton², Samuel Madden², Travis Collier³, Daniel Blumstein³, Deborah Estrin³ ¹Coventry University ²CSAIL, MIT ³CENS, UCLA

2. 2008/23/4 2IPSN/SPOTS '08, St. Louis, MO Applications and Challenges of Distributed Acoustic Sensing Has traditional WSN complexities, in addition: Has traditional WSN complexities, in addition: High data rate = no streaming to sink High data rate = no streaming to sink Forces local, in network processing Forces local, in network processing * Distributed processing properties * Develop applications using a high level language * Easily push new applications into the network Without sacrificing performance VoxNet: integrated hardware and software platform for distributed acoustic sensing Captures, integrates previous experience in acoustic sensing networks Audio Input Detect Server Side AML DOA Send Raw Data Buffer >3sec? Run Local AML DOA N Send AML Output Y Sensor Node Server Side Event association sync4 Network Operators Sensor side Server side

3. 2008/23/4 3IPSN/SPOTS '08, St. Louis, MO How we got to VoxNet SenSys 2006 (Girod et al) Acoustic sensing Platform IPSN 2007 (Ali et al) Marmot localization application Wavescope: signal-oriented stream processing @ MIT VoxNet

4. 2008/23/4 4IPSN/SPOTS '08, St. Louis, MO VoxNet architecture In-field PDA Gateway Mesh Network of Deployed Nodes On-line operation Off-line operation and storage Compute Server Storage Server Internet or Sneakernet Control Console

5. 2008/23/4 5IPSN/SPOTS '08, St. Louis, MO Roadmap VoxNet platform VoxNet platform How it’s used How it’s used How it’s programmed How it’s programmed Performance results Performance results Network latency Network latency Runtime dynamics Runtime dynamics Ongoing work Ongoing work Conclusions Conclusions

6. 2008/23/4 6IPSN/SPOTS '08, St. Louis, MO Current VoxNet node: ENSBox V2 Based on Acoustic ENSBox prototype Based on Acoustic ENSBox prototype 4 channel mic. array at 48KHz, 802.11b 4 channel mic. array at 48KHz, 802.11b Linux, 400 MHz ARM/64MB/8GB Linux, 400 MHz ARM/64MB/8GB Smaller form factor, Internal battery Smaller form factor, Internal battery Low power supervisor Low power supervisor Always-on mote for duty cycling Always-on mote for duty cycling Example: ~5 days @ dawn/dusk sampling Example: ~5 days @ dawn/dusk sampling Rapidly deployable! Rapidly deployable! Field tested over several months Field tested over several months …in the rainforest (Chajul, Mexico) …in the rainforest (Chajul, Mexico) …and the mountains (Gothic, CO) …and the mountains (Gothic, CO) With thanks to Travis Collier

7. 2008/23/4 7IPSN/SPOTS '08, St. Louis, MO VoxNet on-line usage model Write program Node-side part Sink-side part Run program Optimizing compiler Disseminate to nodes // acquire data from source, assign to four streams (ch1, ch2 ch3, ch4) = VoxNetAudio(44100) // calculate energy freq = fft(hanning(rewindow(ch1, 32))) bpfiltered = bandpass(freq, 2500, 4500) energy = calcEnergy(bpfiltered) Development cycle happens in-field, interactively

8. 2008/23/4 8IPSN/SPOTS '08, St. Louis, MO Programming VoxNet Programming language: Wavescript Programming language: Wavescript High level, stream-oriented macroprogramming language High level, stream-oriented macroprogramming language Compiles down to a directed, graph of communicating ‘stream operators’ which data flow through (source to endpoint) Compiles down to a directed, graph of communicating ‘stream operators’ which data flow through (source to endpoint) Operates on stored OR streaming data Operates on stored OR streaming data User writes a script connecting operators User writes a script connecting operators Writes both node side + sink side separately Writes both node side + sink side separately User decides where processing occurs (node, sink) User decides where processing occurs (node, sink) Explicit, not automated processing partitioning Explicit, not automated processing partitioning Source Endpoint

9. 2008/23/4 9IPSN/SPOTS '08, St. Louis, MO Example – energy calculation // acquire data from source, assign to four streams (ch1, ch2 ch3, ch4) = VoxNetAudio(44100) // window ch1 data stream into 32 sample chunks, // passing through hanning then fft operators freq = fft(hanning(rewindow(ch1, 32))) // apply bandpass filter to freq bpfiltered = bandpass(freq, 2500, 4500) // calculate energy in bandpassed data energy = calcEnergy(bpfiltered) VoxNetAudio rewindowhanningfft ch1

10. 2008/23/4 10IPSN/SPOTS '08, St. Louis, MO Splitting up a logical program // acquire data from source, assign to four streams (ch1, ch2 ch3, ch4) = VoxNetAudio(44100) freq = fft(hanning(rewindow(ch1, 32))) //send data to sink toNet(“frequencies”,freq) // receive data from node stream freq2 = fromNet(“frequencies”) // apply bandpass filter to freq bpfiltered = bandpass(freq2, 2500, 4500) // calculate energy in bandpassed data energy = calcEnergy(bpfiltered) freq toNet(“frequencies”)fromNet(“frequencies”) send(“frequencies”)arrived(“frequencies”) Sink Node bandpass

11. 2008/23/4 11IPSN/SPOTS '08, St. Louis, MO Roadmap VoxNet platform VoxNet platform How it’s used How it’s used How it’s programmed How it’s programmed Validating VoxNet Validating VoxNet Network latency Network latency Runtime dynamics Runtime dynamics Ongoing work Ongoing work Conclusions Conclusions

12. 2008/23/4 12IPSN/SPOTS '08, St. Louis, MO Validating VoxNet Implementation of indicative application Implementation of indicative application Marmot localization (from IPSN ‘07) Marmot localization (from IPSN ‘07) Resource usage Resource usage Hand coded C vs. high level Wavescope Hand coded C vs. high level Wavescope Network trade-off Network trade-off Process locally or send raw data to sink? Process locally or send raw data to sink?

13. 2008/23/4 13IPSN/SPOTS '08, St. Louis, MO Example VoxNet application Detect and determine location of marmots Detect and determine location of marmots Crossing of DoA estimates Crossing of DoA estimates Obtain results on-line (scientist interaction) Obtain results on-line (scientist interaction) Event detection DoA processing and position estimation X Send 32KB per node, per detection NODESSINK

14. 2008/23/4 14IPSN/SPOTS '08, St. Louis, MO Hand-coded C vs. Wavescript 30% less CPU Extra resources mean that data can be archived to disk as well as processed (‘spill to disk’, where local stream is pushed to storage co-processor) EVENT DETECTOR DATA ACQUISITION ‘SPILL TO DISK’ 100% Hand-coded CWavescript

15. 2008/23/4 15IPSN/SPOTS '08, St. Louis, MO Detection data transmission latency Deployed network in Gothic, CO Deployed network in Gothic, CO Request all nodes to transmit data simultaneously Request all nodes to transmit data simultaneously Emulates the ‘worst case’ when all nodes detect the acoustic event Emulates the ‘worst case’ when all nodes detect the acoustic event Measure latency to transmit raw detection or processed data (32KB vs. 800B/node) Measure latency to transmit raw detection or processed data (32KB vs. 800B/node) Gathered single and multi-hop data Gathered single and multi-hop data Up to three hops Up to three hops Used single hop 800B data to help predict missing 800B multi-hop data Used single hop 800B data to help predict missing 800B multi-hop data

16. 2008/23/4 16IPSN/SPOTS '08, St. Louis, MO Detection data transfer latency Network latency will become a problem if scientist wants results in <5 seconds (otherwise animal might move position) 1 HOP2 HOPS3 HOPS 12

17. 2008/23/4 17IPSN/SPOTS '08, St. Louis, MO Processing trade-off Sending raw data to sink Sending raw data to sink DoA processing takes a short time (0.04s) DoA processing takes a short time (0.04s) Sending data takes a long time (32KB, up to 12s in expt.) Sending data takes a long time (32KB, up to 12s in expt.) Sending processed data to sink Sending processed data to sink DoA processing takes a long time (~2.2s) DoA processing takes a long time (~2.2s) Sending data takes a short time (800B, <1s) Sending data takes a short time (800B, <1s) Which is best to do to reduce overall position estimation latency? Which is best to do to reduce overall position estimation latency?

18. 2008/23/4 18IPSN/SPOTS '08, St. Louis, MO Local vs. sink processing trade-off AML PROCESSING TIME NETWORK LATENCY Send raw data, process at sink Process locally, send 800B DoA processing As hops from sink increases, benefit of processing DoA locally is clearly seen

19. 2008/23/4 19IPSN/SPOTS '08, St. Louis, MO Applying the trade-off at run-time Aim: process raw data locally when possible Aim: process raw data locally when possible Node has limited buffer space for data samples Node has limited buffer space for data samples Event detection can happen faster than real-time, so can catch up Event detection can happen faster than real-time, so can catch up Wavescript makes this easy to implement Wavescript makes this easy to implement Trivial, natural to split and merge multiple streams Trivial, natural to split and merge multiple streams Same on-node and sink DoA implementations Same on-node and sink DoA implementations AML DOA Send Raw Data Buffer >3sec? AML DOA N Send AML Output Y Node Sink Merge 2.2s 0.04s x y

20. 2008/23/4 20IPSN/SPOTS '08, St. Louis, MO Roadmap VoxNet platform VoxNet platform How it’s used How it’s used How it’s programmed How it’s programmed Performance results Performance results Network latency Network latency Runtime dynamics Runtime dynamics Ongoing work Ongoing work Conclusions Conclusions

21. 2008/23/4 21IPSN/SPOTS '08, St. Louis, MO Ongoing work VoxNet Platform Development VoxNet Platform Development NSF Funded UCLA/MIT collaboration (2008-2010) NSF Funded UCLA/MIT collaboration (2008-2010) Aiming for 10x improvement on lifetime Aiming for 10x improvement on lifetime Make platform smaller, lighter, cheaper Make platform smaller, lighter, cheaper Develop VoxNet application portfolio further Develop VoxNet application portfolio further Add classification, census to bioacoustics apps. Add classification, census to bioacoustics apps. New applications from different domains: Increases generality, validates applicability New applications from different domains: Increases generality, validates applicability Ongoing development of Wavescript compiler Ongoing development of Wavescript compiler

22. 2008/23/4 22IPSN/SPOTS '08, St. Louis, MO Conclusions VoxNet hardware is easy to deploy VoxNet hardware is easy to deploy Experience is invaluable in this respect Experience is invaluable in this respect Wavescript simplifies development Wavescript simplifies development Allows on-line development in the field Allows on-line development in the field Provides comparable operating performance Provides comparable operating performance Despite being high level language Despite being high level language Easy to implement network dynamics Easy to implement network dynamics Such as local processing adaptation Such as local processing adaptation

23. 2008/23/4 23IPSN/SPOTS '08, St. Louis, MO

24. 2008/23/4 24IPSN/SPOTS '08, St. Louis, MO The ENSBox platform solves lots of problems, but… Algorithm is still difficult to implement Algorithm is still difficult to implement Must code in C, lacks safety features Must code in C, lacks safety features Want convenient networking primitives Want convenient networking primitives Hard to debug, hard to choose parameters Hard to debug, hard to choose parameters Desired features Desired features Visibility in the field Visibility in the field Convenient way to install code Convenient way to install code Interactive use in the field Interactive use in the field  Motivation for Wavescope  Motivation for Wavescope

25. 2008/23/4 25IPSN/SPOTS '08, St. Louis, MO What is Wavescope? Idea: High performance dataflow system, but with semantics similar to stream databases Idea: High performance dataflow system, but with semantics similar to stream databases Safe, powerful, optimizable language Safe, powerful, optimizable language Support for asynchronous, dynamic, unreliable sensor net environment Support for asynchronous, dynamic, unreliable sensor net environment Initial Results: 1000x faster than existing stream db’s; Similar performance to hand-coded C; Better support for task dynamics than SDF Borealis Stanford STREAM TelegraphCQ Coral8, StreamBase Ptolemy StreamIT GnuRadio Simulink Audio0 Audio1 Audio2 Audio3 ProfileDetector sync + Dataflow Languages Streaming Databases

26. 2008/23/4 26IPSN/SPOTS '08, St. Louis, MO Wavescope Objectives Interactive “query” model Interactive “query” model Concurrent queries installed in node engine Concurrent queries installed in node engine UI tools provide visibility into query progress UI tools provide visibility into query progress Integration between backend and sensors Integration between backend and sensors Single program is split across both realms Single program is split across both realms Operates on stored and streaming data Operates on stored and streaming data Safe, fast, highly optimizable language Safe, fast, highly optimizable language High rate isochronous data, dynamic data flow High rate isochronous data, dynamic data flow Single language with modern properties Single language with modern properties Girod, et. al., CIDR 2007

27. 2008/23/4 27IPSN/SPOTS '08, St. Louis, MO Wavescript Functional language with mutation Functional language with mutation Strict, static typing with type inference Strict, static typing with type inference Partial evaluation at compile time Partial evaluation at compile time “meta-program” constructs data flow graph “meta-program” constructs data flow graph Why a new language? Why a new language? Expressive for this class of applications Expressive for this class of applications Optimizations Optimizations Newton, et. al., in submission to PLDI 2008

28. 2008/23/4 28IPSN/SPOTS '08, St. Louis, MO Contributions Developed platform capable of rapid deployment in realistic environments Developed platform capable of rapid deployment in realistic environments Platform = Hardware + software Platform = Hardware + software High level macroprogramming interface High level macroprogramming interface Compiles into a high performance distributed application Compiles into a high performance distributed application Abstracts the user from network details Abstracts the user from network details Defined an interactive usage model Defined an interactive usage model Run-time installable programs Run-time installable programs Run same program over live or stored data Run same program over live or stored data

29. 2008/23/4 29IPSN/SPOTS '08, St. Louis, MO Conclusions: VoxNet First deployment, platform using Wavescope approach on embedded hardware First deployment, platform using Wavescope approach on embedded hardware Deployed, validated with motivating application Deployed, validated with motivating application Improves on previous work Improves on previous work Less resource overhead Less resource overhead On-line interaction On-line interaction Data archival Data archival Dynamically adjust processing boundaries Dynamically adjust processing boundaries Built over solid, deployed, software framework (EmStar) Built over solid, deployed, software framework (EmStar) Less emphasis on debugging visibility Less emphasis on debugging visibility EmStar validated through application experience, deployment and iterative development over several years EmStar validated through application experience, deployment and iterative development over several years

30. 2008/23/4 30IPSN/SPOTS '08, St. Louis, MO Conclusions: Motivations and challenges Distributed acoustic sensing is ongoing Distributed acoustic sensing is ongoing High data rate sensing is a challenge in WSNs High data rate sensing is a challenge in WSNs In-network processing/data reduction is vital In-network processing/data reduction is vital A variety of motivating domains and applications A variety of motivating domains and applications Scientific, military, smart office Scientific, military, smart office Event detection, localization, classification Event detection, localization, classification A need for flexible, capable, general acoustic sensing platforms A need for flexible, capable, general acoustic sensing platforms Applications are hard, solutions need iteration Applications are hard, solutions need iteration Significant expense, effort is always expended in deployment Significant expense, effort is always expended in deployment Suitable high level programming abstraction for non-experts Suitable high level programming abstraction for non-experts Highly optimized for embedded platforms Highly optimized for embedded platforms Interactive - react to unforeseen situations Interactive - react to unforeseen situations We believe VoxNet meets these needs

31. 2008/23/4 31IPSN/SPOTS '08, St. Louis, MO Here’s how Interactive programming allows operational adjustments in the field Interactive programming allows operational adjustments in the field Make most of time present in field Make most of time present in field Programming environment allows abstraction of networking concerns (as far as possible) Programming environment allows abstraction of networking concerns (as far as possible) Platform architecture avoids problems of badly configured applications Platform architecture avoids problems of badly configured applications Record full data stream, replay data offline Record full data stream, replay data offline Accurate self-localization removes need for manual surveying* Accurate self-localization removes need for manual surveying* Deployment is always harder than you remember Deployment is always harder than you remember *not a direct contribution of VoxNet

32. 2008/23/4 32IPSN/SPOTS '08, St. Louis, MO Library: Spill to disk Save full data stream locally (on gumstix storage server) Save full data stream locally (on gumstix storage server) Regardless of current on-line processing Regardless of current on-line processing User can spill whatever data they require to named local stream User can spill whatever data they require to named local stream Multiple uses for disk archiving Multiple uses for disk archiving Process data in the field, after the fact Process data in the field, after the fact Off-line archival, indexing, annotation, sharing Off-line archival, indexing, annotation, sharing Data Acq. + Time sync (slauson) Chan 1 Chan 2 Chan 3 Chan 4 Application (slauson) Spill to disk (gumstix) Raw data stream

33. 2008/23/4 33IPSN/SPOTS '08, St. Louis, MO What’s new? First embedded Wavescope deployment First embedded Wavescope deployment Provides on-line and off-line operation Provides on-line and off-line operation Record all data or last hour’s data for reprocessing in the field Record all data or last hour’s data for reprocessing in the field Safe, high level macro-programming environment Safe, high level macro-programming environment Single Wavescript program is partitioned and distributed Single Wavescript program is partitioned and distributed Heterogeneous, multi-hop wireless network Heterogeneous, multi-hop wireless network Efficiency improvements (CPU, memory) Efficiency improvements (CPU, memory)

34. 2008/23/4 34IPSN/SPOTS '08, St. Louis, MO Backup: Named Network Streams // Send side: ToNet(“MyStream”, S); // Receive side: R = FromNet(“MyStream”); Default stream semantics: reliable message delivery, with bounded send buffer and head drop Default stream semantics: reliable message delivery, with bounded send buffer and head drop Implemented in Wavescript and C via Foreign Function Interface Implemented in Wavescript and C via Foreign Function Interface

35. 2008/23/4 35IPSN/SPOTS '08, St. Louis, MO Backup: Streams in Wavescript // Sensor side: (sendback,runlocal) = switch(Detections, fun (x) SensorBufferTimeRemaining() < 3.0); ToNet(“AMLStream”, AML(runlocal)); ToNet(“RawStream”, sendback); // Server side: AMLStream = FromNet(“AMLStream”); RawStream = FromNet(“RawStream”); AllAMLs = merge(AMLStream, AML(RawStream)); Note: switch is simple to implement in Wavescript. Note: switch is simple to implement in Wavescript. Related work: VanGo Related work: VanGo

36. 2008/23/4 36IPSN/SPOTS '08, St. Louis, MO Backup: Visualisation Visualiser allows scientist to ‘subscribe’ to any network stream and visualise it Visualiser allows scientist to ‘subscribe’ to any network stream and visualise it Future work – automatic marshalling, data types Future work – automatic marshalling, data types

37. 2008/23/4 37IPSN/SPOTS '08, St. Louis, MO Backup: One-hop data transfer

38. 2008/23/4 38IPSN/SPOTS '08, St. Louis, MO Back up: VoxNet node Four channel 48KHz audio, 802.11b network, internal battery (5-10hr) Four channel 48KHz audio, 802.11b network, internal battery (5-10hr) 400MHz processor, 64MB RAM 400MHz processor, 64MB RAM Linux based OS, with EmStar on top Linux based OS, with EmStar on top Targeted at science applications Targeted at science applications A standard acoustic platform A standard acoustic platform Rapidly deployable: Rapidly deployable: Attended Attended short-lived deployments short-lived deployments

39. 2008/23/4 39IPSN/SPOTS '08, St. Louis, MO Back up: VoxNet node Self-calibration/localization Self-calibration/localization Acoustic time of flight ranging (~75m) Acoustic time of flight ranging (~75m) Direction of arrival Direction of arrival Multilateration algorithm for position/orientation estimation Multilateration algorithm for position/orientation estimation Take advantage of capabilities required for acoustic sensing Take advantage of capabilities required for acoustic sensing Global time over network (10us) Global time over network (10us) 5cm 2D pos, 1.5 deg. orientation error in 80mx50m outdoor environment

40. 2008/23/4 40IPSN/SPOTS '08, St. Louis, MO What is VoxNet? Integrated hardware and software platform for distributed acoustic sensing Integrated hardware and software platform for distributed acoustic sensing Captures previous experience in deploying acoustic sensing networks Captures previous experience in deploying acoustic sensing networks Hardware Networked streams Time syncSelf-localization Control console Data acquisition MH Routing Data archival Node-sideSink-side Top level Wavescript application EmStar drivers App. Compilation App. Dissemination Networked streams MH Routing EmStar drivers Visualization +

41. 2008/23/4 41IPSN/SPOTS '08, St. Louis, MO Distributed Acoustic Sensing is hard Event detection Event detection Target localization Target localization Target tracking Target tracking Target classification Target classification Signal enhancement Signal enhancement Deployment Self-localization Time synchronization Programming model Network communication Operating faults Wide-band signals Channel noise Processing capability Real-time operation Scientific research Scientific research Smart offices Smart offices Military Military Domains GoalsChallenges

42. 2008/23/4 42IPSN/SPOTS '08, St. Louis, MO VoxNet network support Support flow of Wavescript streams between nodes Support flow of Wavescript streams between nodes In our case, sink is endpoint for programs In our case, sink is endpoint for programs Publish/subscribe named streams (“control”, “data”) Publish/subscribe named streams (“control”, “data”) Sits below Wavescope application Sits below Wavescope application Default network stream semantics: Default network stream semantics: Reliable message delivery Reliable message delivery Bounded send buffer, head drop Bounded send buffer, head drop Assumes IP connectivity (over multiple hops) Assumes IP connectivity (over multiple hops) User space implementation of Dynamic Source Routing User space implementation of Dynamic Source Routing

43. 2008/23/4 43IPSN/SPOTS '08, St. Louis, MO VoxNet node software stack Hardware Networked streams Time syncSelf-localization Control console Data acquisition MH Routing Data archival Node-sideSink-side Wavescript application Drivers App. Compilation App. Dissemination Networked streams MH Routing Drivers Visualization

44. 2008/23/4 44IPSN/SPOTS '08, St. Louis, MO Networked streams Special network operators allow logical streams to flow over network toward endpoint (sink) Special network operators allow logical streams to flow over network toward endpoint (sink) Publish/subscribe mechanism Publish/subscribe mechanism Underlying C implementation Underlying C implementation Default network stream semantics: Default network stream semantics: Reliable message delivery Reliable message delivery Bounded send buffer, head drop Bounded send buffer, head drop Assumes multi-hop IP connectivity Assumes multi-hop IP connectivity Used implementation of Dynamic Source Routing Used implementation of Dynamic Source Routing