Fireflies: Scalable Support for Intrusion-Tolerant Network Overlays Håvard Johansen University of Tromsø Norway André Allavena Robbert van Renesse University of Waterloo Cornell University Canada USA

Overlay networks are easy targets Adds important Internet functionality –Routing, search, Skype, etc. Deployed on untrusted hosts –Malicious behavior is likely Fireflies provides scalable intrusion-tolerant group membership that supports robust overlay networks Malicious members cannot –Keep correct members out –Keep failed members in

Fireflies: a group membership protocol Gossip –Disseminates membership events Membership –Accuse members suspected of being failed –Rebut false accusations Failure detection (monitoring) –Adaptive pinging –Can make mistakes Failure Detection Gossip Membership other members

Problem: false accusations A false accusation does not imply that the accuser is Byzantine How to prevent Byzantine members from overloading the system with false accusations?

Assigning monitors 6 members: CA assigns identifiers 3 identifier spaces (rings) 1 23

Assigning monitors members: CA assigns identifiers 3 identifier spaces (rings) s-hash ( | #), # = 1, 2, 3

Assigning monitors members: CA assigns identifiers 3 identifier spaces (rings) s-hash ( | #), # = 1, 2, 3

Assigning monitors members: CA assigns identifiers 3 identifier spaces (rings) 3 pseudo-random permutations of the members

Assigning monitors 1 23 Monitor successors: monitors Monitored by predecessors: monitored by

Disabling predecessors 1 23 Prevents predecessor from accusing a member on a specific ring disables predecessor on ring 1 as part of rebutting a false accusation X

How many rings? Use 2t + 1 rings Choose t so that there are no more than t Byzantine predecessors for any member members disable t predecessors of their choice t + 1 predecessors remains enabled  at least 1 correct enabled predecessor … 2t + 1 rings 12342t+1

Calculating t P byz : bounded probability that a member is Byzantine min t:  > binominal.cdf( t; 2t+1; 1 - P byz )  = O( 1 / # members )

PlanetLab evaluation setup Configuration –t = 12 (25 monitoring rings) –Gossip rate = 1 gossip / 3.5 seconds Byzantine members: –aggressive attacks: accuse at any opportunity do not forward rebuttals –passive attacks: never accuse do not forward accusations

Protocol overhead on PlanetLab # members Byzantine: 20% (10% aggressive + 10% passive) bytes/sec

Applications Intrusion-Tolerant video streaming –Gossip mesh ideal for multicast dissemination –Based on Chainsaw randomized flooding protocol Intrusion-Tolerant software mirroring tool –Replicate GNU/Linux Debian APT repositories –Time critical patches

Conclusions Fireflies: a group membership protocol –Intrusion-tolerant –Scalable –Probabilistic guarantees Available on SourceForge: