1 The Design of a Robust Peer-to-Peer System Rodrigo Rodrigues, Barbara Liskov, Liuba Shrira Presented by Yi Chen Some slides are borrowed from the authors’

2 Talk Outline  Existing P2P systems  Motivation for a robust P2P system  The new P2P architecture  Conclusion and future work  Discussions

3 Existing P2P Systems  Started with explosion of file sharing apps  Existing P2P systems: All nodes have identical responsibilities All communication is symmetric Volunteer nodes join and leave system at any time  Advantages: Harness idle storage and network resources Automatic load balancing Self-organization Desirable properties scale as we add nodes

4 Motivation: Support More Applications  Current applications that use P2P storage systems Assume states are in secure place Use untrusted P2P storage system to publish data E.g. content sharing  What if we want to store the state of the application in a P2P system (e.g. support writes)? E.g. digital library archiving, mail service

5 Motivation: Better Fault-Tolerance  Current P2P systems tolerate failstop failures Faulty nodes stop  Current P2P systems does not handle Byzantine failures faulty node may behave arbitrary and malicious e.g. an attacker can join the system with multiple Ids, provide fake information

6 Motivation: Efficient Lookups Current approach: Chord  Every node maintains a routing table about O(logN) entries  Routing algorithm locates data with latency O(logN), N8 N51 N42 N1 N38 N32 N21 N14 Lookup K36 N8+1: N14 N8+2: N14 N8+8: N21 N8+4: N14 N8+16: N32 N8+32: N42

7 Talk Outline  Existing P2P systems  Motivation for a robust P2P system  The new P2P architecture Configuration Service P2P nodes  Conclusion and future work  Discussions

8 Architecture  Configuration Service (CS): can run on subset of P2P nodes or be separate from P2P nodes determines current configuration and propagate the information  P2P nodes (server machines) Configuration Service P2P Nodes

9 Configuration Service  Four main functions: Admission control Node monitoring Deciding on a new configuration Propagating configuration information

10 Admission Control  Prevents a single user controlling large fraction of the ID space  Hard to do in volunteer-based system  Approach: Maintain a list of authorities who are permitted to add nodes to the system Each new nodes inform CS its ID, IP and public key

11 Node Monitoring  How to determine a node should be evicted?  Failstop: ping nodes periodically  Byzantine: hard! Probing: must be indistinguishable from client requests Still, faulty nodes may lie in wait Rely on proactive recovery

12 Proactive Recovery  All nodes in the system are recovered proactively and frequently.  Approach: Each node has a watchdog timer to trigger recovery During a recovery, a node reboots from the saved correct state. Then, in order to bring the states up- to-date, it sends status message to replicas, who detect missing messages (if any) and retransmits them.

13 Proactive Recovery (cont.)  To ensure correctness, choose recovery frequency, such that: There are (3f+1) the number of replicas for a data item. For any moment, there are at most f faulty replicas (see algorithm in [2])  Byzantine faulty nodes may only present temporarily in the system

14 Assumptions of the model To apply proactive recovery, P2P nodes are server machines, such that less failure-prone dedicated to handle distributed applications. not constantly joining and leaving the system, and routing states changes infrequently still can run symmetric protocols, and distributed across a wide area

15 Deciding on a New Configuration  Include newly joined nodes  Evict potentially faulty nodes Byzantine faulty nodes only present temporarily, therefore do need to detect and remove them. Remove potential failstop nodes that are not reachable for certain time period  Produce new configuration periodically e.g. once every hour

16 Propagating Configuration  What to propagate? Disseminate entire config Assumption: few nodes joining/leaving Advantage: enable efficient lookup Overhead: Local storage of configuration information 100,000 nodes -> 14.7 MB Transmit using diffs and compressions  How to propagate? Will be discussed later

17 Talk Outline  Existing P2P systems  Motivation for a robust P2P system  The new P2P architecture Configuration Service P2P nodes  Conclusion and future work  Discussions

18 Two Layers on P2P Nodes  Lookup layer: Receive/maintain configuration info  Storage layer: Store/retrieve/transfer data items Upon configuration changes, need state transfer During lookup, ask lookup layer for the replicas that hold the data of interest, then contact the storage layer at the replicas directly

19 Two Types of Data  Immutable, self-verifying data. Use the hash of its content as ID  Mutable data Extend the data with a monotonically increasing version number Retrieval needs to make sure to be the latest version

20 Data Management of Storage Layer  Storage/Retrieval with static configuration Immutable data Mutable data  Storage/Retrieval during re-configuration State transfer Configuration propagation

21 Storage/Retrieval with Static Configuration  Need to ensure that 2f+1 replicas claim to have stored the data  Assume no concurrent writes to the same data items

22 Immutable Data  Adversary cannot forge them  Not updateable – never out-of-date  Write to and receive acknowledgement from at least 2f + 1 replicas  Read from only one replica, if the content hash matches the ID

23 Mutable data  Replay attack  Must include version number  Write to 2f+1 replicas One round-trip if version number is known Otherwise learn current version number by an extra round-trip  Read from 2f+1 replicas and choose the one with highest version number to guarantee correctness

24 Store / Retrieve Algorithms Write 2f+1 replicas Read 2f+1 replicas  In total n=3f+1 replicas  At any moment, at most f faulty replicas  Therefore read at least 1 correct replica

25 Storage/Retrieval during Re-configuration  When configuration changes, the set of replicas of a data item may change  Need to State transfer Bring the up-to-date version of data to P2P nodes that are responsible for them Configuration propagation Propagate new configuration to all nodes over some time

26 State Transfer  If a node receives a new configuration, and realizes it is responsible for new data items  Then it pulls the up-to-date data from replicas of last configuration Immutable data: read one replica if the content hash matches id Mutable data: read 2f+1 replicas to ensure correctness

27 Configuration Propagation  Nodes include id of the latest configuration they know in the messages they exchange  If a node detects a new configuration, it requests for a copy of that configuration

28 Talk Outline  Existing P2P systems  Motivation for a robust P2P system  The new P2P architecture  Conclusion and future work  Discussions

29 Conclusions  A hybrid system consists of a set of servers as P2P nodes, and a CS.  P2P nodes are sever machines that dedicated for the storage application  CS are responsible for computing and propagating the current configuration  A Byzantine fault-tolerant system by applying proactive recovery

30 Future Work  Implementation  Improve algorithms – delete, quotas, …  Eliminate rebooting requirement

31 Discussions  Compare its underlying assumptions with existing P2P systems  Proposed improvements over existing P2P systems Support more applications Better fault-tolerance Efficient lookups

32 Discussion: Assumptions  Compare its underlying assumptions with existing P2P systems All nodes have identical responsibilities? Yes All communication is symmetric? Yes Volunteer nodes join / leave system at any time? No Assume participating nodes are dedicated to handle distributed applications, and do not frequently join/leave Node join under admission control  Similar as a distributed system?

33 Discussion: Support More Applications  Current P2P systems use P2P storage to publish data, only useful for content sharing  This system supports state changes of the data (writes), useful for archiving Read/write 2f+1 replicas, big overhead?

34 Discussion: Better fault-tolerance  Current P2P systems: failstop fault-tolerant  This system: Byzantine fault-tolerant By applying proactive recovery Only suitable for server machines

35 Discussion: Efficient Lookups  Existing P2P systems: A node is responsible for the routing states of a subset of the nodes. Routing requires several lookups  This system: Every node stores the whole configuration, enables direct lookups Pros: efficient lookups Cons: big storage; big transfer; many routing state changes when nodes join/leave

36 Questions? Thank you!