1. Andrey Markov, public domain image
Numerical Algorithms
10/12/2017 5:24 PM
Presentation for use with the textbook, Algorithm Design and Applications, by M. T. Goodrich and R. Tamassia, Wiley, 2015
Chernoff Bounds
Andrey Markov, public domain image
© 2015 Goodrich and Tamassia
Chernoff Bounds

2. Markov’s Inequality
© 2015 Goodrich and Tamassia
Chernoff Bounds

3. Sums of Indicator Random Variables
Let X = X1 + X Xn be the sum of n independent 0-1 indicator random variables, such that Xi = 1 with probability pi.
Using the language of probability theory, X is a random variable from the binomial distribution.
Intuitively, X is equal to the number of heads one gets by flipping n coins such that the i-th coin comes up heads with probability pi.
Define the mean, or expected value of X, as follows:
© 2015 Goodrich and Tamassia
Chernoff Bounds

4. A Chernoff (upper) Bound
© 2015 Goodrich and Tamassia
Chernoff Bounds

5. A Chernoff (lower) Bound
© 2015 Goodrich and Tamassia
Chernoff Bounds

6. Application: Load Balancing
Suppose we have a set of n processors and a set of n jobs for them to perform, but no good way of assigning jobs to processors.
So we just assign jobs to processors at random.
What is a good high-probability upper bound on the number of jobs assigned to any processor?
We can answer this question using a Chernoff bound.
Let X be a random variable representing the number of jobs assigned to processor 1. Then X can be written as
X = X1 + X Xn,
where Xi is the 0-1 indicator random variable that job i is assigned to processor 1.
Thus, Pr(Xi = 1) = 1/n and μ = E[X] = 1.
© 2015 Goodrich and Tamassia
Chernoff Bounds

7. Applying a Chernoff Bound
Since the Xi’s are clearly independent, we can apply the Chernoff bound from Theorem to get, for any integer m > 1:
Thus, the probability that processor 1 has more than m jobs assigned to it by this random assignment is at most 1/n2. Therefore, the probability that any processor is assigned more than mjobs is at most n/n2 = 1/n.
In other words, the number of processors assigned to any processor is O(log n/ log log n) with high probability.
© 2015 Goodrich and Tamassia
Chernoff Bounds