Chernoff Bounds, and etc. Presented by Kwak, Nam-ju

Topics A General Form of Chernoff Bounds Brief Idea of Proof for General Form of Chernoff Bounds More Tight form of Chernoff Bounds Application of Chernoff Bounds: Amplification Lemma of Randomized Algorithm Studies Chebyshev’s Inequality Application of Chebyshev’s Inequality Other Considerations

A General Form of Chernoff Bounds Assumption Xi’s: random variables where Xi∈{0, 1} and 1≤i≤n. P(Xi=1)=pi and therefore E[Xi]=pi. X: a sum of n independent random variables, that is, μ: the mean of X

A General Form of Chernoff Bounds When δ >0

Brief Idea of Proof for General Form of Chernoff Bounds Necessary Backgrounds Marcov’s Inequality For any random variable X≥0, When f is a non-decreasing function,

Brief Idea of Proof for General Form of Chernoff Bounds Necessary Backgrounds Upper Bound of M.G.F.

Brief Idea of Proof for General Form of Chernoff Bounds Proof of One General Case (proof)

Brief Idea of Proof for General Form of Chernoff Bounds Proof of One General Case Here, put a value of t which minimize the above expression as follows:

Brief Idea of Proof for General Form of Chernoff Bounds Proof of One General Case As a result,

More Tight form of Chernoff Bounds The form just introduced has no limitation in choosing the value of δ other than that it should be positive. When we restrict the range of the value δ can have, we can have tight versions of Chernoff Bounds.

More Tight form of Chernoff Bounds When 0<δ<1 Compare these results with the upper bo und of the general case:

Application of Chernoff Bounds: Amplification Lemma of Randomized Algorithm Studies A probabilistic Turing machine is a nondeterministic Turing machine in which each nondeterministic step has two choices. (coin-flip step) Error probability: The probability that a certain probabilistic TM produces a wrong answer for each trial. Class BPP: a set of languages which can be recognized by polynomial time probabilistic Turing Machines with an error probability of 1/3.

Application of Chernoff Bounds: Amplification Lemma of Randomized Algorithm Studies However, even though the error probability is over 1/3, if it is between 0 and 1/2 (exclusively), it belongs to BPP. By the amplification lemma, we can construct an alternative probabilistic Turing machine recognizing the same language with an error probability 2-a where a is any desired value. By adjusting the value of a, the error probability would be less than or equal to 1/3.

How to construct the alternative TM? (For a given input x) Application of Chernoff Bounds: Amplification Lemma of Randomized Algorithm Studies How to construct the alternative TM? (For a given input x) Select the value of k. Simulate the original TM 2k times. If more than k simulations result in accept, accept; otherwise, reject. Now, prove how it works.

Xi’s: 1 if the i-th simulation produces a wrong answer; otherwise, 0. Application of Chernoff Bounds: Amplification Lemma of Randomized Algorithm Studies Xi’s: 1 if the i-th simulation produces a wrong answer; otherwise, 0. X: the summation of 2k Xi’s, which means the number of wrongly answered simulations among 2k ones. ε: the error probability X~B(2k, ε) μ=E[X]=2k ε

Application of Chernoff Bounds: Amplification Lemma of Randomized Algorithm Studies P(X>k): the probability that more than hal f of the 2k simulations get a wrong answe r. We will show that P(X>k) can be less tha n 2-a for any a, by choosing k appropriatel y.

Here we set δ as follows: Application of Chernoff Bounds: Amplification Lemma of Randomized Algorithm Studies Here we set δ as follows: Therefore, by the Chernoff Bounds,

To make the upper bound less than or eq ual to 2-a, Application of Chernoff Bounds: Amplification Lemma of Randomized Algorithm Studies To make the upper bound less than or eq ual to 2-a,

Here, we can guarantee the right term is po sitive when 0<ε<1/2. Application of Chernoff Bounds: Amplification Lemma of Randomized Algorithm Studies Here, we can guarantee the right term is po sitive when 0<ε<1/2.

Chebyshev’s Inequality For a random variable X of any probabilistic distribution with mean μ and standard deviation σ, To derive the inequality, utilize Marcov’s in equality.

Application of Chebyshev’s Inequality Use of the Chebyshev Inequality To Calculate 95% Upper Confidence Limits for DDT Contaminated Soil Concentrations at a Using Chebyshev’s Inequality to Determine Sample Size in Biometric Evaluation of Fingerprint Data Superfund Site.

Application of Chebyshev’s Inequality For illustration, assume we have a large body of text, for example articles from a publication. Assume we know that the articles are on average 1000 characters long with a standard deviation of 200 characters. From Chebyshev's inequality we can then deduce that at least 75% of the articles have a length between 600 and 1400 characters (k = 2).

Other Considerations The only restriction Markov’s Inequality impose is that X should be non-negative. It even doesn’t matter whether the standard deviation is infinite or not. e.g. a random variable X with P.D.F. it has a finite mean but a infinite standard deviation.

Other Considerations P.D.F. E[X] Var(x)

Conclusion Chernoff’s Bounds provide relatively nice upper bounds without too much restrictions. With known mean and standard deviation, Chebyshev’s Inequality gives tight upper bounds for the probability that a certain random variable is within a fixed distance from the mean of it.

Conclusion Any question?