1. 1 The Design of a Robust Peer-to-Peer System Gisik Kwon Dept. of Computer Science and Engineering Arizona State University Reference: SIGOPS European Workshop 25/09/2002 Rodrigo Rodrigues(MIT)

2. Scalable Computing Lab. Arizona State University 2 Motivation What if we wanted to store the state of the application in a P2P system? E.g., mail service, archiving Need a robust P2P storage system Current algorithms do not provide reliable service: – No admission control – Trust configuration information – Vulnerable to malicious routing – Tolerance to Byzantine faults

3. Scalable Computing Lab. Arizona State University 3 Architecture Configuration Service P2P Nodes Configuration Services(CS) – Selected statically(a subset of P2P nodes) or dynamically P2P nodes – set of servers, not clients Common to equip with – Co-processor, read-only disk, watch-dog timer, Cryptographic co- processor

4. Scalable Computing Lab. Arizona State University 4 Configuration Service Four main functions: – Admission control – Node monitoring – Deciding on a new configuration – Propagating configuration information CS nodes carry BFT protocol [CL99]

5. Scalable Computing Lab. Arizona State University 5 Admission Control Prevents single user controlling large fraction of the ID space Maintains the list of node – Hard to do in volunteer-based system We assume servers Nodes have public / private keys

6. Scalable Computing Lab. Arizona State University 6 Node Monitoring Detection fault node – Fail-stop failure: ping protocol – Byzantine failure: hard! proactively recovered frequently -> restart from the correct code in read-only disk -> copy state from other nodes All CS do its own monitoring for all P2P nodes

7. Scalable Computing Lab. Arizona State University 7 Configuration Information What to propagate? – Incomplete config is a limitation of p2p – Disseminate entire config – Transmit using diffs When to propagate? – Server machines: not very often (e.g., every hour) – Configs include start and expiration times Periodically deciding the new config – Using non-deterministic choices validation Send eviction request to other CS -> validation

8. Scalable Computing Lab. Arizona State University 8 Architecture – P2P Nodes Lookup layer – Periodically receive the latest config from CS – Notify to storage layer Storage layer – Uses Byzantine quorums to store data – Data placement algorithm assigns 3 f + 1 replicas to each data item Lookup Storage Application Lookup Storage Lookup Storage ClientServer

9. Scalable Computing Lab. Arizona State University 9 Correctness Condition “At any moment, any group of 3 f +1 replicas of a data item contains no more than f faulty replicas” Reconfiguration must be frequent enough to preserve this invariant

10. Scalable Computing Lab. Arizona State University 10 State Transfer During reconfiguration, set of replicas may change Replica finds out of new configuration: – Know old configuration – Pulls data items from old replicas (as seen before)