1 Online Balancing of Range-Partitioned Data with Applications to P2P Systems Prasanna Ganesan Mayank Bawa Hector Garcia-Molina Stanford University

2 Motivation Parallel databases use range partitioning Parallel databases use range partitioning Advantages: Inter-query parallelism Advantages: Inter-query parallelism –Data Locality  Low-cost range queries  High thru’put Key Range

3 The Problem How to achieve load balance? How to achieve load balance? –Partition boundaries have to change over time –Cost: Data Movement Goal: Guarantee load balance at low cost Goal: Guarantee load balance at low cost –Assumption: Load balance beneficial !! Contribution Contribution –Online balancing -- self-tuning system –Slows down updates by small constant factor

4 Roadmap Model and Definitions Model and Definitions Load Balancing Operations Load Balancing Operations The Algorithms The Algorithms Extension to P2P Setting Extension to P2P Setting Experimental Results Experimental Results

5 Model and Definitions (1) Nodes maintain range partition (on a key) Nodes maintain range partition (on a key) –Load of a node = # tuples in its partition –Load imbalance σ = Largest load/Smallest load Arbitrary sequence of tuple inserts and deletes Arbitrary sequence of tuple inserts and deletes –Queries not relevant –Automatically directed to relevant node

6 Model and Definitions (2) After each insert/delete: After each insert/delete: –Potentially fix “imbalance” by modifying partitioning –Cost= # tuples moved Assume no inserts/deletes during balancing Assume no inserts/deletes during balancing –Non-critical simplification Goal: σ < constant always Goal: σ < constant always –Constant amortized cost per insert/delete –Implication: Faster queries, slower updates

7 Load Balancing Operations (1) NbrAdjust: Transfer data between “neighbors’’ NbrAdjust: Transfer data between “neighbors’’ [0,50) AB [50,100) [0,35)[35,100)

8 Is NbrAdjust good enough? Can be highly inefficient Can be highly inefficient –  (n) amortized cost per insert/delete ( n=#nodes ) AEDCBF

9 Load Balancing Operations (2) Reorder: Hand over data to neighbor and split load of some other node Reorder: Hand over data to neighbor and split load of some other node AEDCBF [0,10)[10,20)[20,30)[30,40)[40,50)[50,60)[40,60)[0,5)[5,10)

10 Roadmap Model and Definitions Model and Definitions Load Balancing Operations Load Balancing Operations The Algorithms The Algorithms Experimental Results Experimental Results Extension to P2P Setting Extension to P2P Setting

11 The Doubling Algorithm Geometrically divide loads into levels Geometrically divide loads into levels –Level i  Load in ( 2 i,2 i+1 ] –Will try balancing on level change Two Invariants Two Invariants –Neighbors tightly balanced  Max 1 level apart –All nodes within 3 levels  Guarantees σ ≤ i2i 2 i +1 2 i +2 Level i Level 0 Level 2 Level 1 Load Scale

12 The Doubling Algorithm (2) AEDCBF

13 The Doubling Algorithm (2) AEDCBF

14 The Doubling Algorithm (2) AEDCBF

15 The Doubling Algorithm: Case 2 AEDCBF Search for a blue node Search for a blue node –If none, do nothing!

16 The Doubling Algorithm: Case 2 ADCEBF Search for a blue node Search for a blue node –If none, do nothing!

17 The Doubling Algorithm (3) Similar operations when load goes down a level Similar operations when load goes down a level –Try balancing with neighbor –Otherwise, find a red node and reorder yourself Costs and Guarantees Costs and Guarantees –σ ≤ 8 –Constant amortized cost per insert/delete

18 From Doubling to Fibbing Change thresholds to Fibonacci numbers Change thresholds to Fibonacci numbers –σ ≤  3  4.2 –Can also use other geometric sequences –Costs are still constant FiFi F i+1 F i+2 = +

19 More Generalizations Improve σ to (1+  ) for any  >0 [BG04] Improve σ to (1+  ) for any  >0 [BG04] –Generalize neighbors to c-neighbors –Still constant cost O(1/  ) Dealing with concurrent inserts/deletes Dealing with concurrent inserts/deletes –Allow multiple balancing actions in parallel –Paper claims it is ok

20 Application to P2P Systems Goal: Construct P2P system supporting efficient range queries Goal: Construct P2P system supporting efficient range queries –Provide asymptotic performance a la DHTs What is a P2P system? A parallel DB with What is a P2P system? A parallel DB with –Nodes joining and leaving at will –No centralized components –Limited communication primitives Enhance load-balancing algorithms to Enhance load-balancing algorithms to –Allow dynamic node joins/leaves –Decentralize implementation

21 Experiments Goal: Study cost of balancing for different workloads Goal: Study cost of balancing for different workloads –Compare to periodic re-balancing algorithms (Paper) –Trade-off between cost and imbalance ratio (Paper) Results presented on Fibbing Algorithm (n=256) Results presented on Fibbing Algorithm (n=256) Three-phase Workload Three-phase Workload –(1) Inserts (2) Alternating inserts and deletes (3) Deletes Workload 1: Zipf Workload 1: Zipf –Random draws from Zipf-like distribution Workload 2: HotSpot Workload 2: HotSpot –Think key=timestamp Workload 3: ShearStress Workload 3: ShearStress –Insert at most-loaded, delete from least-loaded

22 Load Imbalance (Zipf) Time (x1000) Load Imbalance Growing PhaseSteady PhaseShrinking Phase

23 Load Imbalance (ShearStress)

24 Cost of Load Balancing

25 Related Work Karger & Ruhl [SPAA 04] Karger & Ruhl [SPAA 04] –Dynamic model, weaker guarantees Load balancing in DBs Load balancing in DBs –Partitioning static relations, e.g., [GD92,RZML02, SMR00] –Migrating fragments across disks, e.g., [SWZ93] –Intra-node data structures, e.g., [LKOTM00] Litwin et al. SDDS Litwin et al. SDDS

26 Conclusions Indeed possible to maintain well-balanced range partitions Indeed possible to maintain well-balanced range partitions –Range partitions competitive with hashing Generalize to more complex load functions Generalize to more complex load functions –Allow tuples to have dynamic weights –Change load definition in algorithms! * –Range partitioning is powerful Enables P2P system supporting range queries Enables P2P system supporting range queries –Generalizes DHTs with same asymptotic guarantees * Lots of caveats apply. Need load to be evenly divisible. No guarantees offered on costs. This offer not valid with any other offers. Etc, etc. etc.