Download presentation
Presentation is loading. Please wait.
1
Autonomous Replication for High Availability in Unstructured P2P Systems Francisco Matias Cuenca-Acuna, Richard P. Martin, Thu D. Nguyen Department of Computer Science, Rutgers University April, 2003
2
Content Introduction Background: PlanetP Autonomous Replication Performance Evaluation Conclusions
3
Introduction (1) Peer-to-peer (p2p) computing becomes a powerful paradigm sharing information across the internet Problem of providing high availability for shared data Recent measurements suggest – Members of p2p communities may be offline more than they are online Providing practical availability, say 99-99.9% would be – Expensive storage-wise using traditional replication methods – Expensive bandwidth-wise as peers leave and rejoin the community
4
Introduction (2) Question: Is it possible to place replicas of shared files in such a way that – Files are highly available – Without requiring the continual movement of replicas to members currently online Propose a distributed replication algorithm – Decisions are made entirely autonomously by individual members – Only need a small amount of loosely synchronized global state
5
Introduction (3) Assumption – Files are replicated in their entirety only when a member hoards that file for disconnected operation – Otherwise, files are replicated using an erasure code – Study a very weakly structured system because tight coordination is difficult and costly in dynamic environment
6
Background: PlanetP (1) PlanetP is a publish/subscribe system – Support content-based search, rank and retrieval Members publish documents when they wish to share Publication of a document – Give PlanetP a XML snippet containing a pointer to the file – PlanetP indexes the XML snippet and the file – Local index is used to support content search
7
Background: PlanetP (2) Two major components to enable community-wide sharing – An gossiping layer Periodically gossip about changes to keep shared data weakly consistent – Content search, rank and retrieval service Two data structures to be replicated on every peer – Membership directory: Contains names and addresses of all current members – Global content index: Contains term-to-peer mapping
8
Background: PlanetP (3) To locate content – Users pose queries at a specific node – Identify the subset of target peers from the local copy of global content index – Query is passed to these target peers – The targets evaluate the query against their local indexes and return results (URLs for relevant documents) Results have shown that PlanetP can easily scale to sizes of several thousands
9
Autonomous Replication (1) Member’s hoard set – Members hoard some subset of the shared files entirely on their local storage – Members take responsibility for ensuring the availability Replicator – Member is trying to replicate an erasure-coded fragment of a file Target – Peer that the replicator is asking to store the fragment Replicator store – Excess storage space contributed by each member for replication
10
Autonomous Replication (2) Each file is identified by a unique ID Overall algorithm – Advertises the file IDs in its hoard set and the fragments in its replication store to the global index – Periodically estimates the availability of its hoarded files and the fragments (Estimating Availability) – Every Tr time units, increase the availability of a file that is not at a target availability (Randomized Replication) – The target peer saves the incoming fragment (Replacement Scheme)
11
Estimating Files Availability (1) Replicated in two manners – Entire copies – Erasure-coded file fragments H(f): set of peers hoarding a file f F(f): set of peers containing a fragment f A(f): availability of f – All nodes in H(f) are simultaneously offline – At least n-m+1 of the nodes in F(f) are offline
12
Estimating Files Availability (2) H(f) and F(f) do not intersect – Peer adds a file for which it is storing a fragment to its hoard set, it ejects the fragment immediately A(f) does not account for the possibility of duplicate fragments – n >> m
13
Randomized Replication (1) Erasure codes (Reed Solomon) provide data redundancy – Divide a file into m fragment and recode them into n fragment (m < n) Generate all n fragments => detect and regenerate specific lost fragments Disadvantages for highly dynamic environment – Member availability changes over time Necessary to change n => re-fragmenting and replication of some files – Peers leaving Accurate accounting of which peer is storing which fragment => regenerate fragment loss – Peers temporarily going offline Introducing duplicate fragments
14
Randomized Replication (2) Choose n >> m but do not generate all n fragments To increase the availability of a file – RANDOMLY generate an additional fragment from the set of n possible fragments Chance of having duplicate fragments is small if n is very large Not having any peer coordination
15
Replacement (1) A target peer receives a replication request If its store is full – Decide whether to accept the incoming fragment, OR – select other fragments to evict from its store Choose the fragments with the highest availability to make space – Deterministic algorithm => victimize fragments of the same file => drastic changes in the file’s availability Propose Weighted Random Selection Process
16
Replacement (2) Policy – Compute the average number of nines in the availability of the fragments – Incoming fragment’s number of nines > 10% of this average => reject incoming fragment – Lottery scheduling to select victim fragments Divide tickets into two subsets with 80:20 Each fragment is assigned an equal share of the smaller subset Fragments with availability above 10% of the average are given a portion of the larger subset
17
Replacement (3) Example – Target node has 3 fragments with availability 0.99, 0.9, 0.5 – Number of nines => 2, 1, 0.3 – Average availability in nines + 10% = 0.76 – If we have 100 tickets First fragment=67+6.6, Second fragment=13+6.6, Third fragment=0+6.6 – Chances of each fragment to be evicted First fragment=0.74, Second fragment=0.2, Third fragment=0.06 Use number of nines rather than the availability – It linearizes the differences between values
18
Replicators (1) Select fragment to replicate Similar to replacement, lottery scheduling is used – Favoring files with low availability Find target peer – Select a target randomly – If the target does not have sufficient space, select another target – Repeat this process for five times – If not success, randomly choose from these five targets
19
Experimental Environment (1) Event driven simulator Assumption – Members replicate files at synchronous intervals – Not simulate the detail timing of message transfers – Not simulate the full PlanetP gossiping protocol To account for the data staleness, reestimate file availability only once every 10 minutes – Use Reed Solomon code and m=10
20
Experimental Environment (2) Evaluate 3 different p2p communities File Sharing (FS) : very loosely coupled community sharing multimedia files Commercial (CO): corporate/university environment Worgroup (WG): distributed development group
21
Experimental Environment (3) Parameters – Per peer mean uptime and downtime – Peer arrival to and exit from the community as exponential arrival process – Number of file per node – Number of hoarded replicas per file – Amount of excess space on each node – File sizes BASE: nodes pushing and accepting replicas in the complete absences of information on file availability OMNI: replication is driven by a central agent to maximize the minimum file availability
22
Availability VS Excess Storage CO: small amount of excess space is required FS: 8X excess capacity for 3 nines availability File hoarding has little effect on file availability (0 replication from hoarding)(25% of the files have 2 hoarded replicas)(25% of the files have 5 hoarded replicas)
23
Overall Availability (CDF) (CO with 1X, 1.5X and 2X)(FS with 1X, 3X and 6X)(WG with 9X) CO: 2X excess storage, over 99% of files with 3 nines availability FS: around 6X excess storage for 3 nines availability WG: performs better if files and excess storage are uniformly distributed – Non-uniform => not easy for replicators to find free space on the subset of peers – Peers with the most files to replicate have the most excess storage
24
Against BASE (CDF: Availability for FS with 3X)(BASE: number of fragments)(REP: number of fragments) BASE: about 16% of files < a single nine availability Replacement policy can increase fairness BASE’s FIFO favors peers who are frequently online – They push their files to less available peers even if the latter’s content should be replicated more
25
Bandwidth Usage CO – REP: excess space from 1X to 2X => average number of files replicated per hour from 10 to 0 FS – REP: excess space from 1X to 3X => average number of files replicated per hour from 241 to 0 – BASE: 3X excess space => replicate 816 files per hour
26
Conclusions Address the question of increasing the availability of shared files Study a decentralized algorithm under a very loosely coordinated environment Achieve practical availability (99.9%) in a completely decentralized system with low individual availability Such availability levels do not require – Sophisticated data structures – Complex replication schemes – Excessive bandwidth
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.