Theory of Computational Complexity Probability and Computing Ryosuke Sasanuma Iwama and Ito lab M1

Chapter1 : Events and Probability 1.4 Application: A Randomized Min-Cut Algorithm Chapter2: Discrete Random Variables and Expectation 2.1 Random Variables and Expectation Linearity of Expectations Jensen’s Inequality 2.2 The Bernoulli and Binomial Random Variables 2.3 Conditional Expectation

1.4 A Randomized Min-Cut Algorithm A cut-set in a graph : a set of edges whose removal breaks the graph into two or more connected components. A min-cut problem : given a graph G=(V,E) with n vertices, to find a minimum cardinality cut-set in G. ← A cut set A minimum cut set →

A Randomized Min-Cut Algorithm Edge contraction : in contracting an edge {u,v} we merge the two vertices u and v into one vertex, eliminate all edges connecting u and v, and retain all other edges in the graph. The algorithm chooses and contracts an edge uniformly at random from remaining edges. The algorithm consists of n-2 iterations.

A Randomized Min-Cut Algorithm 1 1,3, , ,2,3,4 The cut set is { {3,5}, {4,5} }

Theorem 1.8 : The algorithm outputs a min-cut set with probability at least 2/n(n-1) proof k : the size of the min-cut set of G C : one of the min-cut sets We may conclude that, if the algorithm never chooses an edge of C in its n-2 iterations, then the algorithm returns C as the minimum cut-set. : event that the edge contracted in iteration i is not in C : event that no edge of C was contracted in the first i iterations

: event that the edge contracted in iteration i is not in C We need to compute. Firstly, we compute. Since the minimum cut-set has k edges, all vertices in G must have degree k or larger, then G has at least nk/2 edges. Since C has k edges,

In the first iteration, if an edge of C was not eliminated, the graph has at least k(n-1)/2 edges. Thus, Similarly,

Compute,

Assume that we run the randomized min-cut algorithm n(n-1)lnn times and output the minimum size cut-set found in all the iterations. The probability that the output is not a min-cut set is bounded by

Chapter1 : Events and Probability 1.4 Application: A Randomized Min-Cut Algorithm Chapter2: Discrete Random Variables and Expectation 2.1 Random Variables and Expectation Linearity of Expectations Jensen’s Inequality 2.2 The Bernoulli and Binomial Random Variables 2.3 Conditional Expectation

2.1 Random Variables and Expectation Def 2.1: A random variable X on a sample space Ω is a real-valued function on Ω; that is, A discrete random variable is a random variable that takes on only a finite or countably infinite number of values. “X=a” represents the set For example, let X be the sum of the two dice.

Def 2.2: Two random variables X and Y are independent if and only if for all values x and y. Similarly, random variables X 1,X 2,..., X k are mutually independent if and only if, for any subset I ⊆ [1,k] and any values x i, i ∈ I,

Def 2.3: The expectation of a discrete random variable X, denoted by E[X], is given by where the summation is over all values in the range of X. The expectation is finite if converges; otherwise, the expectation is unbounded. For example, let X be the sum of the two dice.

2.1.1 Linearity of Expectations Theorem 2.1 [Linearity of Expectations]: For any finite collection of discrete random variables X 1,X 2,...,X n with finite expectations, We prove the statement for two random variables. The general case can be proved by induction.

Proof:

For example, consider rolling two dice. Die 1: X 1, Die 2: X 2, X=X 1 +X 2, Y=X 1 +X 1 Applying the linearity of expectations, we have Linearity of expectations holds for any collection of random variables, even if they are not independent. 2

Lemma2.2: For any constant c and discrete random variable X, Proof: For c=0, obvious. For c≠0,

2.1.2 Jensen’s Inequality X X Suppose that we choose the length X of a side of a square uniformly at random from the range [1,99]. What is the expected value of the area? E[X ] = E[X] ? In fact, E[X ]=2500, E[X] = 9950/3 > More generally, we can prove that E[X ] ≧ (E[X]). This is an example of a more general theorem known as Jensen’s inequality. Jensen’s inequality shows that, for any convex function f, we have E[f(X)] ≧ f(E[X])

Def2.4: A function f : R → R is said to be convex if, for any x 1,x 2 and 0 ≦ λ ≦ 1, The following lemma is often a useful alternative to Definition 2.4. Lemma 2.3: If f is a twice differentiable function, then f is convex if and only if f”(x) ≧ 0. Theorem 2.4[Jensen’s Inequality]: If f is a convex function, then

Proof: We prove the theorem assuming that f has a Taylor expansion. Let μ = E[X]. By Taylor’s theorem there is a value c such that since f”(c)>0 by convexity. Taking expectations of both sides and applying linearity of expectations and Lemma 2.2 yields the result:

Chapter1 : Events and Probability 1.4 Application: A Randomized Min-Cut Algorithm Chapter2: Discrete Random Variables and Expectation 2.1 Random Variables and Expectation Linearity of Expectations Jensen’s Inequality 2.2 The Bernoulli and Binomial Random Variables 2.3 Conditional Expectation

2.2 The Bernoulli and Binomial Random Variables Suppose that we run an experiment that succeeds with probability p and fails with probability 1-p. Let Y be a random variable such that 1 if the experiment succeeds, 0 otherwise. Y = The variable Y is called a Bernoulli or an indicator random variable.

For example, if we flip a fair coin and consider the outcome “heads” a success, then the expected value of the corresponding indicator random variable is 1/2. Consider a sequence of n independent experiments, each of which succeeds with probability p. If we let X represent the number of successes in the n experiments, then X has a binomial distribution. Def2.5: A binomial random variable X with parameters n and p, denoted by B(n,p), is defined by the following probability distribution on j=0,1,2,...,n:

What is the expectation of a binomial random variable X? We can compute it directly from the definition. The linearity of expectations allows for a significantly simpler argument. Define a set of n indicator random variables X 1,...,X n, where X i =1 if the ith trial is successful and 0 otherwise. and,

Chapter1 : Events and Probability 1.4 Application: A Randomized Min-Cut Algorithm Chapter2: Discrete Random Variables and Expectation 2.1 Random Variables and Expectation Linearity of Expectations Jensen’s Inequality 2.2 The Bernoulli and Binomial Random Variables 2.3 Conditional Expectation

Just as we have defined conditional probability, it is useful to define the conditional expectation of a random variable. Def2.6: where the summation is over all y in the range of Y. For example, we consider rolling two dice. Let X 1, X 2 be the number that show on the die1, die2. Let X be the sum of the numbers on the two dice.

Consider E[X|X 1 =2]: Consider E[X 1 |X=5]: Lemma2.5 follows from Definition2.6.

Lemma2.5: For any random variables X and Y, where the sum is over all values in the range of Y and all of the expectations exist. Proof:

The following lemma is the linearity of conditional expectations. Lemma2.6: For any finite collection of discrete random variables X 1, X 2,...,X n with finite expectations and for any random variable Y, This lemma is proved by similar way to prove the linearity of expectations. The conditional expectation is also used to refer to the following random variable.

Def2.7: The expression E[Y|Z] is a random variable f(Z) that takes on the value E[Y|Z=z] when Z=z. E[Y|Z] is not a real value. It is actually a function of the random variable Z. For example, in the previous example of rolling two dice, If E[X|Z] is a random variable, then it makes sense to consider its expectation E[E[Y|Z]].

More generally, we have the following theorem. Theorem2.7: Proof: From Def2.7 we have E[Y|Z]=f(Z), where f(Z) takes on the value E[Y|Z=z] when Z=z. Hence The right-hand side equals E[Y] by Lemma2.5.

Chapter1 : Events and Probability 1.4 Application: A Randomized Min-Cut Algorithm Chapter2: Discrete Random Variables and Expectation 2.1 Random Variables and Expectation Linearity of Expectations Jensen’s Inequality 2.2 The Bernoulli and Binomial Random Variables 2.3 Conditional Expectation

Application of conditional expectations Consider a program that includes one call to a process S. Assume that each call to process S recursively spawns new copies of the process S, where the number of new copies is a binomial random variable with parameters n and p. We assume that these random variables are independent for each call to S. What is the expected number of copies of the process S generated by the program? S S S S SS S S S S... S

S S S S SS S S S S We introduce the idea of generations. The initial process S is in generation 0. Let Y i denote the number of S processes in generation i. Since we know that Y 0 =1, the number of processes in generation 1 has a binomial distribution. Thus, S Y 1 = 2Y 3 = 3Y 2 = 3Y 4 = 2

Suppose we knew that the number of processes in generation i-1 was y i-1, so Y i-1 = y i-1. Let Z k be the number of copies spawned by the kth process spawned in the (i-1)th generation for 1 ≦ k ≦ y i-1. Each Z k is a binomial random variable with parameters n and p. S S S S SS S S S S S Y 1 = 2Y 3 = 3Y 2 = 3Y 4 = 2 Z 1 = 2 Z 2 = 1 Z 1 = 1 Z 2 = 0 Z 3 = 2

Then

Applying Theorem 2.7, By induction on i, and using the fact that Y 0 = 1, we then obtain The expected total number of copies of process S generated by the program is given by

If np ≧ 1 then the expectation is unbounded; if np < 1, the expectation is 1/(1-np). Thus, the expected number of processes generated by the program is bounded if and only if the expected number of processes spawned by each process is less than 1.