1. http://www.cs.bu.edu/groups/wing/ If You Build It, They Will Come: The SN B ENCH Architecture Adam Bradley Azer Bestavros & Michael Long Computer Science Department Boston University

2. May 13, 2005The snBench Architecture2 Before we begin… Welcome To Summer!

3. May 13, 2005The snBench Architecture3 Sensorium Infrastructure @ BU Sensorium: A common space equipped with video sensors (VS) for ubiquitous recognition and tracking of activities therein Infrastructure:  Range of VS Elements  Programmable VS Network  Backend compute engines  Backend TByte storage  Mobile/wireless query units  Research Engineer

4. May 13, 2005The snBench Architecture4 snBench The Sensorium is the computer… Design/implement the programming and run-time infrastructure necessary for developers to specify and deploy truly distributed applications over a heterogeneous network of Sensing Elements (SEs) and Computing Elements (CEs) What sensors could I use and what functionality do I get from them?

5. May 13, 2005The snBench Architecture5 snBench: Goals  “Write Once, Run… Wherever”  Don’t program nodes… Program the network! Start with building blocks  Sensors  Stock algorithms (edge detect, face count, FFT)  Dynamic modules (novel algorithms) Glue together with high-level language  Conditionals, loops, functions… Pretend the network isn’t there  “Single System Image” Programming Paradigm

6. May 13, 2005The snBench Architecture6 snBench: Programming Cycle  Program Program specified by gluing together building blocks using SNAFU language  Compile SNAFU program is compiled to produce a plan of execution expressed in STEP  Map and Link Service Dispatcher decomposes STEP plans into smaller linked dispatch-able STEPs  Load and Execute STEP plans are dispatched (i.e., loaded) into SXE execution environments

7. May 13, 2005The snBench Architecture7 SNAFU: SNet Apps as FUnctions!  Functional specification language e.g., identify the face seen through camera 1 identify(facefind(snapshot(cam1)))

8. May 13, 2005The snBench Architecture8 SNAFU: SNet Apps as FUnctions!  Use “annotative” functions for constraints e.g., minimum acceptable resolution e.g., what streams can be “exposed” public(facecount(snapshot(private(cam1))))) facecount(resn(800x600x24b,snapshot(cam1)))

9. May 13, 2005The snBench Architecture9 SNAFU: SNet Apps as FUnctions!  Events in time captured by “triggers” e.g., motion detected at night triggers email email(“security@bu.edu”, trigger( (motion(snapshot(cam2)) && (2am<NOW<4am)), snapshot(cam2)))

10. May 13, 2005The snBench Architecture10 SNAFU: SNet Apps as FUnctions!  Three trigger types: trigger(p,a) : “Wait until p”  do { /* no-op */ } until (p); a; level_trigger(p,a) : “Whenever p”  while (true) { if (p) a; }; edge_trigger : “Whenever p becomes true”  while (true) { if (p) { a; while (p) {} } }; Value of trigger is value of a’s evaluation Level and edge triggers produce streams  a.k.a. “persistent queries” living in network

11. May 13, 2005The snBench Architecture11 SNAFU: SNet Apps as FUnctions!  Trigger QoS (Scheduling annotations) period(100ms, level_trigger(f(cam1), g(cam1)))

12. May 13, 2005The snBench Architecture12 SNAFU: SNet Apps as FUnctions!  Retrieving values from triggers: Non-blocking read - Last result Blocking read - Wait for next result Fresh read - Wait for a “from scratch” result  Recursion: LAST_TRIGGER_EVALUATION term  Last value produced by innermost(?) trigger term  Not sure what to call it… “head recursion” ?

13. May 13, 2005The snBench Architecture13 STEP: Sensorium Typed Exec Plan cam2 snapshot motion clock4am 2am << && edge_trigger email “security@bu.edu” “Instruction Set Architecture” is a DAG email(“security@bu.edu”, edge_trigger( (motion(snapshot(cam2)) && (2am<NOW<4am)), snapshot(cam2)))

14. May 13, 2005The snBench Architecture14 STEP: Sensorium Typed Exec Plan Notes:  Evaluation percolates up from leaf nodes toward roots.  Sensor nodes provide fresh data at all times.  Function nodes evaluate once then disable themselves.  Trigger nodes can re-enable children for re-evaluation. cam2 snapshot motion clock4am 2am << && edge_trigger email “security@bu.edu”

15. May 13, 2005The snBench Architecture15 snBench: Runtime Setting SNAFU-to-STEP compiler SXE: Sensorium eXecution Environment SD: Service Dispatcher SD, SXE SXE Compiler, Web Browser, SXE(?)

16. May 13, 2005The snBench Architecture16 Map, Link, and Dispatch SXE edge_trigger email “security@bu.edu” cam2 snapshot motion clock4AM 2AM << &&

17. May 13, 2005The snBench Architecture17 trigger email “security@bu.edu” Map, Link, and Dispatch SXE motion 2AM clock4AM << logical_and cam2 snapshot edge_trigger email “security@bu.edu” motion clock4AM 2AM << && cam2 snapshot

18. May 13, 2005The snBench Architecture18 Why snBench?  A Sensorium Research Catalyst Extrinsic research  Accessible framework for developing distributed sensing applications in heterogeneous networks Intrinsic research  Languages, type systems, compilers, graph algorithms, schedulers, QoS engines, naming, etc. to make the framework efficient/useful/flexible Supporting research  System context, supporting middleware/services

19. May 13, 2005The snBench Architecture19 Extrinsic Research  Flexible substrate for experimenting with distributed sensor applications Extensible w/ new algorithms (“opcodes”) Trade some speed for versatility and easy re-programmability/re-tasking  Abstracts away “the network” Vision researchers don’t need to understand communication protocols, RT schedulers, network resource reservation, etc.

20. May 13, 2005The snBench Architecture20 Extrinsic Research: Sensorium Target Applications Assistive Environments e.g. for home/hospice/elder care/… Safety Monitoring e.g. in factories/daycare/garages/subways… Intelligent Spaces e.g. for classrooms/meeting rooms/theaters/farms… Secure Facilities and Homeland Security Uses e.g. at airports/embassies/prisons/CS labs/… People Flow/Activity Studies e.g. at malls/interstates/…

21. May 13, 2005The snBench Architecture21 Intrinsic/Extrinsic Research Why limit ourselves to passive sensing? Control actuators with snBench programs! Multi-DOF Cameras Mobile Sensors Robots Control theory issues…

22. May 13, 2005The snBench Architecture22 Intrinsic Research  Scheduling/Linking/Binding How did we arrive at this partitioning? How did we arrive at these assignments? SXE edge_trigger email “security@bu.edu” cam2 snapshot motion clock4AM 2AM << &&

23. May 13, 2005The snBench Architecture23 edge_trigger email “security@bu.edu” cam2 snapshot motion clock4AM 2AM << && Scheduling: Processing Capacity SXE Load averages? Busy rate? STEP nodes/s? Realtime scheduler? ?

24. May 13, 2005The snBench Architecture24 Scheduling: Network Capacity SXE edge_trigger email “security@bu.edu” cam2 snapshot motion clock4AM 2AM << && SXE 100Mb/s 20% 512kb/s 75% 8Mb/s 95% 100Mb/s 40% 100Mb/s 10% ?

25. May 13, 2005The snBench Architecture25 Scheduling: Code Presence edge_trigger email “security@bu.edu” cam2 snapshot motion clock4AM 2AM << && SXE motion oflow delta [empty] delta facefind bodytrack email motion email [empty] snapshot delta ?

26. May 13, 2005The snBench Architecture26 Intrinsic Research SXE edge_trigger email “security@bu.edu” cam2 snapshot motion clock4AM 2AM << && SXE 100Mb/s 20% 512kb/s 75% 8Mb/s 60% 100Mb/s 40% 100Mb/s 10% ? motion email [empty] motion oflow delta [empty] delta facefind bodytrack email snapshot delta Scheduling = Constrained Graph embedding and optimization!

27. May 13, 2005The snBench Architecture27 More Scheduling Multiple simultaneous programs may share STEP sub-graphs: email(“security@bu.edu”, block( edge_trigger(motion(snapshot(cam2)) && (2am<NOW<4am), snapshot(cam2))) trigger( facerecognizer( block( edge_trigger(motion(snapshot(cam2)) && (2am<NOW<4am), snapshot(cam2))), facelibrary(“Adam Bradley”)), email(“adams.wife@gmail.com”, “He’s working late again.”))

28. May 13, 2005The snBench Architecture28 email(“security@bu.edu”, block( edge_trigger(motion(snapshot(cam2)) && (2am<NOW<4am), snapshot(cam2))) trigger( facerecognizer( block( edge_trigger( motion(snapshot(cam2)) && (2am<NOW<4am), snapshot(cam2))), facelibrary(“Adam Bradley”)), email(“adams.wife@gmail.com”, “He’s working late again.”)) More Scheduling: CSE Common Subexpression/Subgraph Elimination cam2 snapshot motion clock4AM 2AM << && edge_trigger email “security@bu.edu” cam2 snapshot motion clock4am 2am << && edge_trigger email “adams.wife@gmail.com” trigger “He’s working late again” facerecognizer facelibrary “Adam Bradley”

29. May 13, 2005The snBench Architecture29 trigger( facerecognizer( block( edge_trigger( motion(snapshot(cam2)) && (2am<NOW<4am), snapshot(cam2))), facelibrary(“Adam Bradley”)), email(“adams.wife@gmail.com”, “He’s working late again.”)) email(“security@bu.edu”, block( edge_trigger(motion(snapshot(cam2)) && (2am<NOW<4am), snapshot(cam2))) Common Subgraph More Scheduling: CSE Common Subexpression/Subgraph Elimination cam2 snapshot motion clock4AM 2AM << && edge_trigger email “security@bu.edu” cam2 snapshot motion clock4am 2am << && edge_trigger email “azer.wife@cs.bu.edu” trigger “He’s working late again” facerecognizer facelibrary “Azer Bestavros”

30. May 13, 2005The snBench Architecture30 trigger( facerecognizer( block( edge_trigger( motion(snapshot(cam2)) && (2am<NOW<4am), snapshot(cam2))), facelibrary(“Adam Bradley”)), email(“adams.wife@gmail.com”, “He’s working late again.”)) email(“security@bu.edu”, block( edge_trigger(motion(snapshot(cam2)) && (2am<NOW<4am), snapshot(cam2))) More Scheduling: CSE cam2 snapshot lights_are_on clock4AM 2AM << logical_and edge_trigger email “security@bu.edu” cam2 snapshot lights_are_on clock4AM 2AM << logical_and edge_trigger email “azers.wife@cs.bu.edu” trigger “He’s working late again” facerecognizer facelibrary “Azer Bestavros” Common Subexpression/Subgraph Elimination

31. May 13, 2005The snBench Architecture31 trigger( facerecognizer( block( edge_trigger( motion(snapshot(cam2)) && (2am<NOW<4am), snapshot(cam2))), facelibrary(“Adam Bradley”)), email(“adams.wife@gmail.com”, “He’s working late again.”)) email(“security@bu.edu”, block( edge_trigger(motion(snapshot(cam2)) && (2am<NOW<4am), snapshot(cam2))) More Scheduling: CSE cam2 snapshot lights_are_on clock4AM 2AM << logical_and edge_trigger email “security@bu.edu” cam2 snapshot lights_are_on clock4AM 2AM << logical_and edge_trigger email “azers.wife@cs.bu.edu” trigger “He’s working late again” facerecognizer facelibrary “Azer Bestavros” Common Subexpression/Subgraph Elimination SXE

32. May 13, 2005The snBench Architecture32 Intrinsic Research  Program Optimization  What to optimize? STEP Node count? Trigger count? (More or less?) let x = snapshot(cam2) in email(“security@bu.edu”, block( edge_trigger(motion(x) && (2am<NOW<4am), x))) let x = snapshot(cam2) in trigger( edge_trigger(motion(x) && (2am<NOW<4am), true), email(“security@bu.edu”, x))

33. May 13, 2005The snBench Architecture33 More Scheduling: Architecture  What if we can’t run SXE on a node?  STEP: a compile-able virtual ISA Embedded C dialects? Java ME?  Support alternate “linking” protocols Serial (base station) SN wireless protocols (e.g. 802.15.4)

34. May 13, 2005The snBench Architecture34 Intrinsic Research  Better Programming Languages  Alternate Execution Environments STEP SNAFU SnQL SnLOG SnC SXE J2ME Native C JXTA

35. May 13, 2005The snBench Architecture35 Intrinsic Research  Scheduling/Real-Time Type Systems  Resource Management Type Inference  Actualizers (SD is a “mini-ITM”) Static and dynamic “type checks” period(100ms, let x = period(75ms, edge_trigger(y, z)) in edge_trigger(f(x), g(x)) level_trigger( state(4MB, entropymeasure( bw(2MBps, deltastream(cam1)))), period_gt(250ms, signal(actuator1, true))

36. May 13, 2005The snBench Architecture36 Intrinsic Research  Intentional Naming Naming sensors by identity: “The webcam in Azer’s Office” Naming sensors by property: “Any two cams which see point X from perspectives >90 degrees apart” Naming sensors by dynamic attribute: “Any cam which sees Adam”  May depend upon existence of persistent queries. What p.q.’s should we instantiate to produce the highest odds of success at the lowest cost?  Wearables as part of the Sensorium?

37. May 13, 2005The snBench Architecture37 Intrinsic Research  How do you offer RealTime, QoS, etc. if you don’t know about the timing of atomic operations? Live profiling of new opcodes Speculative/probabilistic scheduling Run-time scheduler parameter changes

38. May 13, 2005The snBench Architecture38 Supporting Research  SXE Implementations and Internals SXE itself as high-performance service  Real-time scheduling  Self-profiling  Fast I/O delivery (Lives in kernel sandbox?) Programmer can upload their own functions  C++, Java, C#, Perl, Matlab, whatever…  Automatically generate glue to link with SXE  Sandboxing issues

39. May 13, 2005The snBench Architecture39 Current Status  Whitepaper SNAFU and STEP syntax (tentative) SNAFU type principles (tentative)  1G Implementation (work in progress) Java  SNAFU-to-STEP compiler  Service Dispatcher  SXE (with HTTP-based interface)  JAR-based extensibility Expect first live demo mid-summer

40. http://www.cs.bu.edu/groups/wing/ If You Build It, They Will Come: The SN B ENCH Architecture Adam Bradley Azer Bestavros & Michael Long Computer Science Department Boston University