1. JazzEnsemble: A Group Communication Middleware for MANET Roy Friedman Technion Israel

2. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK Motivation  Many envisioned application for MANET require group based interactions  Group communication toolkits are standard mechanism in wired clusters Provide consistent membership service Provide reliable and ordered multicast services  GC middleware from wired LANs do not work well in MANETs

3. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK Membership Maintenance View1 View2 View4 View3 P1 P2 P3 P4 P5 P6

4. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK Deficiencies in Cluster Based GCS  One slow node may slow down the entire calculation Due to multicast flow control Failure detection time tradeoff  Fast detection avoids these performance hick-ups, but causes in instability  No adequate routing  No power awareness  No Byzantine tolerance

5. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK Flow Control  Limits the number of outstanding concurrent messages  Vital for obtaining decent performance, since otherwise too many messages will be lost in collisions and buffer overflows … 1 2 3 4 5 15 16 17 18 19 20 Last ack received Last message sent Send window

6. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK Ensuring Reliable Delivery  Messages are buffered by receivers until it is known that all recipients got them This way, if one recipient did not get the message, and maybe the sender crashed/disconnected by then, then another receiver can forward the message  If one process does not ack messages, it causes a waste of buffer space

7. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK Failure Detection Evaluation  Area=300x300, range=50m, speed=0-2m/s, failure prob.=0.2

8. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK Failure Detection Evaluation  Area=300x300, range=50m, nodes=40, failure prob.=0.2

9. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK Fuzzy Group Membership  Assign a fuzziness level to each member  Various aspects of the group membership can treat “ fuzzy ” members in a special way For example, a message that was acked by all nodes whose fuzziness if above a certain threshold can be considered Fuzzy Stable:  The flow control can proceed past them  Buffer management can compress them or eliminate some of their copies from some nodes  Eventually, above some threshold, a member can be declared faulty and a new view is computed

10. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK Fuzzy Stability

11. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK Impact of Fuzziness on Flow Control

12. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK Impact of Fuzziness on Flow Control

13. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK The JazzEnsemble project  Adapting GC to MANETs The notion of fuzzy membership Multicast routing Multicast power awareness Byzantine fault-tolerance Positioning and location based routing An emulation layer that allows testing production code in an emulated environment http://www.cs.technion.ac.il/Labs/dsl/projects/JazzEnsemble/

14. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK The JazzEnsemble Project  Platforms Windows, Linux, MacOS, Solaris iPAQs (Windows CE, Linux) Ethernet, WiFi, etc

15. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK JazzEnsemble Demo Applications  All work in peer-to-peer serverless mode A distributed auctions system Shared white board + chat Tele-conference Video conference Multiplayer games  Car racing, puzzle solver

16. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK The Ensemble Legacy Top_Appl Total Virtual Synchrony Fragmentation Reliable FIFO Bottom Network Encrypt Authenticate Application Failure Detection Stability Detection Flow Control

17. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK Middleware Level Routing  Why in the middleware?  Observations regarding collaborative applications Most messages are multicast Pt2pt messages are typically acknowledgements for previous multicasts Heartbeats are exchanged in any case, which can be used to learn the number of neighbors

18. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK Routing in JazzEnsemble  We developed a probabilistic learning group communication routing protocol that is now part of JazzEnsemble  Multiple routing protocols are being added to the system E.g., we are implementing a connected dominating set (CDS) and minimal independent set (MIS) based overlays

19. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK Byzantine Resilience  Implemented a Byzantine tolerant stack for JazzEnsemble Involved analyzing all possible attacks on each layer and defending against it Added mute and verbose failure detectors that monitor each layer  Developed a new efficient Byzantine multicast protocol Sends very few messages when there are no Byzantine failures

20. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK Mute and Verbose Failure Detectors  Assumption: each message has a header part that can be locally expected and a data part that cannot Holds naturally for group communication micro-protocols  Mute failure is failure to send an expected message  Verbose failure means sending too many messages (or too frequent) based on their headers

21. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK Fuzzy Mute and Verbose FD in JazzEnsemble Top_Appl Byzantine Consensus Byzantine Virtual Synchrony Byzantine Fragmentation Byzantine Reliable FIFO Byzantine Bottom Network Application Fuzzy Based Failure Detection Byzantine Stability Detection Byzantine Flow Control Mute and Verbose FD

22. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK Positioning  Why should a device know its location? Geographic routing is more efficient Location based services Sometimes, it is a requirement of the application  How does a device learn its position GPS Cellular triangulation

23. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK Ad-hoc Positioning System  What if most nodes cannot find their position? No line of site to the sky No three base stations  Location propagation Nodes that can learn their position start acting as Stationary Emitters, allowing transitive learning of all nodes position

24. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK Methods for APS  Triangulation Requires knowing the distance from sender, and being within range of three other SEs  Average density based hop distance Does not work well if nodes are not spread evenly  Combining distance and angle of transmission Requires special hardware

25. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK RLISE  Relative Location with Incomplete Stationary Emitters Devices learn, remember and gossip partial information  Can be in UNKNOWN, CIRCLE, SEMI_ACCURATE, or ACCURATE state By exchanging partial information, they improve the accuracy of their position  Being implemented in JazzEnsemble

26. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK RLISE - Convergence  Area=100x100, range=20m, number of SEs=10

27. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK RLISE - Convergence

28. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK RLISE – Error Propagation

29. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK Future Work  Additional work on power aware and power save multicast protocols  Efficient ordering protocols  Mobility patterns based protocols  Further evaluation of the system  A service based architecture with Web-Services interfaces

30. © Roy Friedman MINEMA workshop 1/12/04 Lancaster, UK Maestro Membership Service (group daemon mode of Ensemble) Membership Notification Network Application Core Stack Video (Unreliable) Stack Audio (Reliable) Stack Text (Totally Ordered) Stack Group Manager