1. Contention Resolution with Jobs of Heterogeneous Sizes
Michael A. Bender
Jeremy T. Fineman
Seth Gilbert
I’m Seth Gilbert. I’m going to talk about contention resolution when jobs have different sizes.
Basic setting: many processes, one shared resource. Each process wants to execute a job, which uses the resource, and the goal is to give each process a turn. The traditional solution for this problem is to use a randomized backoff protocol protocol.
Main points I want to make in this talk: (1) When jobs have different sizes, standard backoff protocols don’t work very well. In fact, even a small number of larger jobs can significantly slow down the system. (2) By using carefully designed hash functions to group the jobs by size, we can do better than standard backoff protocols.

2. Problem: Contention Resolution
n processors want to access a shared medium.
Each processor wants to run one job.
All processes start at the same time.
Coordinate access: each proc gets a turn.
Model: Simple Channel
If 1 proc talks, it succeeds.
If ≥2 procs talk, none succeed.
Feedback: success or failure.
Strategy: Try.
Backoff.
Try again. n
Traditionally, the goal has been to give each process a fixed amount of time to access the channel.
The goal of this talk, is to analyze randomized backoff when processors have different-sized jobs.

3. Exponential Backoff Protocol divides time into windows.
A packet tries once per window until it succeeds.
Makespan: n jobs Θ(n logn) rounds.
Really what is happening is guessing the volume.

4. How big is a job? Prior work: Unit-sized jobs
For example: [Aldous ’87], [Bender, Farach-Colton, He, Kuszmaul, Leiserson ’05], [Gereb-Graus, Tsantilas ’92], [Goldberg ’00], [Goldberg, Jerrum, Leighton, Rao ’93], [Greenberg, Leiserson ’89], [Greenberg, Winograd ’85], [Hastad, Leighton, Rogoff ’87], [MacKenzie, Plaxton, Rajamaron ’98], [Willard ’84]
Today: Heterogeneous-sized jobs
Examples:
Networks with variable-length packets
Wireless networks, variable-speed transceivers
Transactional memory
Obstruction-free Algorithms

5. Model Jobs: Simple Channel: All jobs start at the same time.
Number of jobs =n
Size of job i = ti
Volume V =  ti
Simple Channel:
If 1 job executes: success
If ≥2 job executes: failure
Learn of collision after executing
All jobs start at the same time.

6. Results Backoff Protocol Makespan (w.h.p.) Binary Exponential
Modified Exponential
Size-hashed 1 (mod)
Size-hashed 2

7. Results Backoff Protocol Makespan (w.h.p.) Binary Exponential
Modified Exponential
Size-hashed 1 (mod)
Size-hashed 2

8. Outline 1. Introduction 2. Exponential Backoff 3. Size-hashed Backoff
Contention resolution
Overview of results
2. Exponential Backoff
Lower Bound
3. Size-hashed Backoff
Basic strategy
Good hash functions
Overall algorithm
6-7

9. Exponential Backoff Theorem:
There exists a set of jobs for which binary exponential backoff has makespan with high probability.
n-1 small jobs of size 1
1 big job of size b = Θ(n /logn)

10. Heterogeneous Job Sizes
Example: two job sizes
Big job: (size = 4)
Small jobs: (size = 1)
Big jobs can block smaller jobs (and vice versa).

11. Proof Sketch Case 1: window-size < Θ (b) = Θ(n / logn)
Big job: too big! Does not run.
Small jobs: too many. No one finishes.
Case 2: window-size ≥ Θ (b) = Θ(n / logn)
Big job collides with small jobs for
rounds.

12. Proof: Case 2 W = 2b : Pr(collide) = 1/2 W = 4b : Pr(collide) = 1/4
W = 2ib : Pr(collide) = 1/2i
Pr(job j collides in W = 2b, 4b, …, ) = 1/b

13. Proof (Conclusion) Pr(job j collides in windows) = 1/b
Pr(no job collides in all windows) =
Thus, w.h.p. makespan >
Algebra…

14. Proof (Conclusion) Pr(job j collides in windows) = 1/b
Pr(no job collides in all windows) =
Thus, w.h.p. makespan >
Algebra…

15. Outline 1. Introduction 2. Exponential Backoff 3. Size-hashed Backoff
Contention resolution
Overview of results
2. Exponential Backoff
Lower Bound
3. Size-hashed Backoff
Basic strategy
Good hash functions
Overall algorithm
6-7

16. Detour: Unit-Sized Jobs
Can we do better?
Detour: Unit-Sized Jobs
Backoff-Backon Protocol
Backs off like exponential backoff
Backs on occasionally
Achieves makespan Θ(n)
[Greenberg, Leiserson ‘89] for FAT tree routing
[Gereb-Graus, Tsantilas ‘92] for optical switching
[Bender, Farach-Colton, He, Kuzmaul, Leiserson ’05] for on-line backoff.

17. Main Idea: Group jobs by size!
Size-hashed Backoff
Main Idea: Group jobs by size!
Job i: size t  job class  logt 
Backoff-Backon guarantees linear time if the jobs running are in the same job class.

18. Group By Size Assume volume: V logV job classes Θ(V)
[1,2]
[3,4]
[5,8]
[9,16]
[17, 32]
[33, 64]
[65, 128]
[129, 256]
Θ(V)
Running time: Θ(V logV)

19. Group By Size Assume volume: V logV job classes Θ( 2V /logV)
[1,2]
[3,4]
[5,8]
[9,16]
[17, 32]
[33, 64]
[65, 128]
[129, 256]
Θ( 2V /logV)
Running time: Θ(V)

20. Group By Size Maybe not… Do any jobs finish? Assume volume: V
logV job classes
Assume volume: V
[1,2]
[3,4]
[5,8]
[9,16]
[17, 32]
[33, 64]
[65, 128]
[129, 256]
If all the jobs are in the same class, then none finish. In other words, if all the volume is in one bucket, then the slot is too small!
Θ( 2V /logV)
Maybe not…
Do any jobs finish?

21. Do any job classes finish?
Group By Size
logV job classes
Assume volume: V
[1,2]
[3,4]
[5,8]
[9,16]
[17, 32]
[33, 64]
[65, 128]
[129, 256]
Wrong question. The right question…
Θ( 2V /logV)
Yes!
Do any job classes finish?

22. Do any job classes finish?
Fact: ≥ logV /2 job classes have volume < 2V /logV.
[Counting argument.]
≥ 1/2 of all job classes become empty.
Group By Size
log V job classes
Guess volume: V
[1,2]
[3,4]
[5,8]
[9,16]
[17, 32]
[33, 64]
[65, 128]
[129, 256]
Wrong question. The right question…
Θ( 2V /logV)
Yes!
Do any job classes finish?

23. Do any job classes finish?
Fact: ≥ logV /2 job classes have volume < 2V /logV.
[Counting argument.]
≥ 1/2 of all job classes become empty.
Group By Size
Guess volume: V
[1,2]
[3,4]
[5,8]
[9,16]
[17, 32]
[33, 64]
[65, 128]
[129, 256]
Wrong question. The right question…
Θ( 2V /logV)
Yes!
Do any job classes finish?

24. Next Step: Recurse Assume volume: V Θ( 4V /log V) Running time: Θ(V)
[1,2]
[3,4]
[5,8]
[9,16]
[17, 32]
[33, 64]
[65, 128]
[129, 256]
Wrong question. The right question…
Θ( 4V /log V)
Running time: Θ(V)

25. Next Step: Recurse Assume volume: V Θ( 4V /log V) [1,2] [3,4] [5,8]
[9,16]
[17, 32]
[33, 64]
[65, 128]
[129, 256]
Wrong question. The right question…
Θ( 4V /log V)

26. Next Step: Recurse Assume volume: V Θ( 8V /log V) [1,2] [3,4] [5,8]
[9,16]
[17, 32]
[33, 64]
[65, 128]
[129, 256]
Wrong question. The right question…
Θ( 8V /log V)

27. What next? Algorithm: Running time: O(V loglogV) For each job class:
Run backoff-backon for time Θ( 2V /logV).
For each non-empty job class:
Run backoff-backon for time Θ( 4V /logV)
Run backoff-backon for time Θ( 8V /logV)
Et cetera…
The obvious thing to do now is to recurse. After the first phase, we know that half the job classes are finished. So now we can just focus on the non-empty job classes. Since there are only half as many, we can run them for twice as long. … 2, 4, 8 Since there are logV job classes, after loglog iterations, we are done! Running time… Good idea, but there is a problem: we don’t know which job classes are empty.
But which job classes are empty??
* r = Θ(loglogV), for high probability in log V.

28. Which are empty? Randomly pair up job classes:
Analysis: ~ log V /4 (random) job classes run by themselves.
~ log V /4 non-empty job classes have volume < 4V/log V .
Constant fraction of job classes become empty.
Which are empty?
[1,2]
[3,4]
[5,8]
[9,16]
[17, 32]
[33, 64]
[65, 128]
[129, 256]
1/2 can finish. pr(1/2) choosing an empty/non-empty pair. pr(1/4) that choose
Randomly pair up job classes:
Success: one is non-empty, one is empty
Failure: both are non-empty or both are empty

29. Hash Functions Hash job classes to time slots:
[1,2]
[3,4]
[5,8]
[9,16]
[17, 32]
[33, 64]
[65, 128]
[129, 256]
1/2 can finish. pr(1/2) choosing an empty/non-empty pair. pr(1/4) that choose
Hash job classes to time slots:
Success: no collision; 1 non-empty job class in a slot
Failure: collision; > 1 non-empty job class in a slot

30. -Good Hash Functions Family Fm : job class  set of time slots
m : number of non-empty job classes
logV : total number of job classes
F is -Good if for every set of m job classes, at least m are assigned to their own time slot.

31. -Good Hash Functions Family Fm,η : job class  set of slots
Example: Identity function I : [1,logV ] [1,logV ] is 1-good.
Each job class is assigned to its own time slot
|Range (I)| = logV
Family Fm,η : job class  set of slots
m : number of non-empty job classes
η : total number of job classes
F is -Good if for every set of m job classes, at least m are assigned to their own time slot.

32. Probabilistic Method Theorem: There exists function Fm which is:
1/2-good
|Range(F)| = Θ(m loglogV)
For each set of Θ(m) slots, choose 1 at random
Claim 1: m balls (job classes) in Θ(m) bins (slots)
Pr(< m /2 bins have exactly 1 ball) ≤ δΘ(m)
Claim 2: If we have loglogV sets of Θ(m) bins, at least one of them will be good for each set of m job classes.

33. Size-Hashed Backoff Repeat until done: Guess V = 2V. m = logV
repeat loglogV times until m < 1 :
for m loglogV slots of size Θ(V /m) do
if Fm maps your job class to this time slot
then run Backoff-Backon.
m = m/2
Each phase reduces the number m of non-empty job classes by a constant fraction.

34. Running Time Θ(V loglogV ) Θ(V (loglogV )3) Θ(V (loglogV )2)
Repeat until done:
Guess V = 2V.
m = logV
repeat loglogV times until m < 1 :
for m loglogV slots of size Θ(V /m) do
if Fm maps your job class to this time slot
then run Backoff-Backon.
m = m/2
Θ(V (loglogV )2)
Θ(V (loglogV )3)
Θ(V loglogV )

35. Constructive Hash? Backoff Protocol Makespan (w.h.p.)
Binary Exponential
Modified Exponential
Size-hashed 1 (mod)
Size-hashed 2

36. Open Questions Linear time makespan? Less-simple channels?
Unit-sized jobs: O(V)
Heterogeneous-sized jobs: O(V (loglogV)3)
Less-simple channels?
What if processes can listen/snoop?
What if processes abort immediately on collisions?
On-line arrivals?

37. Conclusions Problem: Contention Resolution
Coordinating access to a shared resource.
Today: Heterogeneous-Sized Jobs
Summary of Results:
Exponential Backoff:
(Not as good as might be expected.)
Size-Hashed Backoff:
(Carefully grouping jobs does better.)

38. The End

39. Slow Exponential Backoff
Binary Exponential Backoff
Wi+1 = 2Wi
Window grows too fast!
Slow Exponential Backoff
Wi+1 = Wi + O(Wi /log Wi)
Every O(log W) rounds, the window size doubles.
Analysis: O(Vlog V) running time
When W>3V, constant fraction of jobs finish in each window, since constant fraction of “small” jobs can complete.

40. Detour: Unit-Sized Jobs
Can we do better?
Detour: Unit-Sized Jobs
Backoff-Backon Protocol achieves makespan Θ(n)
[Greenberg, Leiserson ‘89] for FAT tree routing
[Gereb-Graus, Tsantilas ‘92] for optical switching
[Bender, Farach-Colton, He, Kuzmaul, Leiserson ’05] for on-line backoff.
Guess a value of Wi = 2Wi-1
Backon with window Wi /2, Wi /4, Wi /8, …
Backoff...
Example:

41. Constructive Hash Function
Mod Hash Function
In phase m :
if m > sqrt(logV)/loglo then
F(i) = i
else
F(i) = {i mod 1, i mod 2, i mod 3, …}
Makespan: