Autonomous Replication for High Availability in Unstructured P2P Systems Francisco Matias Cuenca-Acuna, Richard P. Martin, Thu D. Nguyen

Introduction Increasing connectivity is driving more dynamic and complex information sharing pattern −Web, file-sharing (e.g., Gnutella), collaborative computing (e.g., −Appearing in many other important computing domains −e.g., Grid and Web services Three emerging properties of these systems (P2P) −Large scale −Decentralized control −Computing takes place on a range of devices, not just servers New problems −Successful operation may depend on numerous resources spanning multiple administrative domains −Service and data availability is difficult to achieve

Introduction(contd.) Factors that affect availability −Lack of coordination in resource management −Wide range of peer behaviors −Expensive (impossible?) to create global views GOAL: provide predictable data availability in P2P systems −E.g., for file systems, we want to reason about minimum file availability Approach −Present explicit model of probabilistic availability to users −Allow users to specify the desired availability level to guide replication −Explicitly inform users when cannot achieve desired availability level −Use an autonomous replication algorithm to tolerate decentralized control −Use as little global state as possible Availability: % of time a piece of data is accessible

Pitfalls in P2P environments Wide range of node availability −Node MTTF no longer determined by hardware reliability but by users’ on-line behavior −Fixed number of replicas too wasteful −E.g., small number of replicas on highly available nodes equivalent to many replicas on low available nodes −Gnutella span from 0.1% to 100%, with an average of 24% −Also, we need to recreate replicas as nodes join and leave Long term dynamic membership −In fact, a fixed number doesn’t work at all because availability profile will likely change over time

Our Approach

Our approach   Use replication but −Vary number of replicas based on estimated file availability −Take advantage of nodes going offline as opposed to failing −Constantly monitor availability −Use erasure codes to minimize space requirements and spread file to more nodes

Internet The strategy Node A PlanetP Hoarded Set F1F1 F2F2 FiFi Excess Storage FjFj FkFk MembershipInfo. Gossiping Node B PlanetP Hoarded Set F3F3 F4F4 FxFx Excess Storage FyFy FzFz MembershipInfo. Gossiping PlanetP provides: - A Loosely synchronized community view - Propagates - Node av. Information - File placement as a Bloom Filter

Internet The strategy Node A PlanetP Hoarded Set F1F1 F2F2 FiFi Excess Storage FjFj FkFk MembershipInfo. Gossiping Node B PlanetP Hoarded Set F3F3 F4F4 FxFx Excess Storage FyFy FzFz MembershipInfo. Gossiping Advertise: - Availability 20% - Files F1, F2 - Fragments Fi, Fj, Fk

Internet The strategy Node A PlanetP Hoarded Set F1F1 F2F2 FiFi Excess Storage FjFj FkFk MembershipInfo. Gossiping Node B PlanetP Hoarded Set F3F3 F4F4 FxFx Excess Storage FyFy FzFz MembershipInfo. Gossiping Based on Node’s B view of F3: - Pick a random node - Create a new fragment for F3 - Push it  F3F3

Dealing with Decentralization Nodes replicate and evict autonomously −Reduce the use of global data and lack of central control All decisions are probabilistic −Weighted by availability estimates Target nodes control their own storage space −Protects system against greedy and faulty nodes Erasure codes plus −Use a modified version of Reed Solomon −Provide a large fragment space −Don’t re-create lost fragments −Prevents duplicates due to autonomous and misinformed decisions

Availability-based replacement Estimating file availability −Probability of finding an online copy or being able to reconstruct the file from the erasure coded fragments Evict fragments of files with “too much” availability −Note that “too much” is in comparison only to files in local excess storage (don’t have to know about all files in system) Why does it work? −Randomized placement decisions  local sample of file availabilities reflect global distribution −This approximation drives space allocation and allows files with insufficient availability to gain fragments

Evaluation

Test scenarios Evaluate three significantly different environments The file sharing environment −1000 nodes hosting a total of files −Node availability avg:24%, min:0.1%, 90th perc:75%, max:100% −Target 99.9% availability −10 minute refresh rate Sources −Saroiu et al. (Gnutella, Napster), DirectConnect at Rutgers OMNI −Centralized knowledge with no limitation on replica placement Base −What happens if you do not have availability estimates?

Availability Comparison

Effect of Av. Based Replacement

Conclusion Practical availability levels are achievable in spite of low node availability and decentralized environment −CO: 80% avg. av.  1X −FS: 24% avg. av.  6X −WG: 33% avg. av.  9X Having some global information is critical −But can do quite well with loosely synchronized data Our algorithm is competitive −Matches OMNI on avg. availability −Provides reasonable min av. −Validated under different environments −Resilient to malicious users −Allocates space fairly within limits

PlanetP Questions?