1. PROBABILITY AND COMPUTING RANDOMIZED ALGORITHMS AND PROBABILISTIC ANALYSIS CHAPTER 1 IWAMA and ITO Lab. M1 Sakaidani Hikaru 1

2. About the Book  Chapter 1  Chapter 2  Chapter 3  Chapter 4  Chapter 5  Chapter 6  Chapter 7  Chapter 8  Chapter 9  Chapter 10  Chapter 11  Chapter 12  Chapter 13  Chapter 14 Core material 2

3. About the Book  Chapter 1  Chapter 2  Chapter 3  Chapter 4  Chapter 5  Chapter 6  Chapter 7  Chapter 8  Chapter 9  Chapter 10  Chapter 11  Chapter 12  Chapter 13  Chapter 14 Additional advanced material 3

4. About the Book  Chapter 1  Chapter 2  Chapter 3  Chapter 4  Chapter 5  Chapter 6  Chapter 7  Chapter 8  Chapter 9  Chapter 10  Chapter 11  Chapter 12  Chapter 13  Chapter 14 Review elementary probability theory 4

5. About the Book  Chapter 1  Chapter 2  Chapter 3  Chapter 4  Chapter 5  Chapter 6  Chapter 7  Chapter 8  Chapter 9  Chapter 10  Chapter 11  Chapter 12  Chapter 13  Chapter 14 More advanced topics 5

6. About the Book  Chapter 1  Chapter 2  Chapter 3  Chapter 4  Chapter 5  Chapter 6  Chapter 7  Chapter 8  Chapter 9  Chapter 10  Chapter 11  Chapter 12  Chapter 13  Chapter 14 6

7. CHAPTER 1 Events and probability 1.1 Verifying Polynomial Identities 1.2 Axioms of Probability 1.3 Verifying Matrix Multiplication 1.4 A Randomized Min-Cut Algorithm 7

8. CHAPTER 1 Events and probability 1.1 Verifying Polynomial Identities 1.2 Axioms of Probability 1.3 Verifying Matrix Multiplication 1.4 A Randomized Min-Cut Algorithm 8

9. Verifying Polynomial Identities ?  The problem of verifying the following identity  More generally… ? : canonical form 9

10. Verifying Polynomial Identities  First method (deterministic algorithm)  It takes multiplications ? 10

11. Verifying Polynomial Identities  There are at least two problems This approach is to write another program that does the same thing the first program to do.  1. If there is a bug in the first program, the same bug may occur in the checking program.  2. we would like to check the answer in less time than it takes to solve. 11

12. Verifying Polynomial Identities  Second method (randomized algorithm)  First, the algorithm chooses an integer uniformly at random in the range of {1, …, 100d} (d : degree of F(x))  If the algorithm decides  But this algorithm may give a wrong answer. → How can the algorithm give the wrong answer? 12

13. Verifying Polynomial Identities  If, the algorithm gives the correct answer.  If,  if, the algorithm gives the correct answer,  if, the algorithm gives the wrong answer. For this, must be a root of. has no more than roots. is chosen in the range of {1, …, 100d}. → the chance that the algorithm returns a wrong answer is no more than 1/100. 13

14. CHAPTER 1 Events and probability 1.1 Verifying Polynomial Identities 1.2 Axioms of Probability 1.3 Verifying Matrix Multiplication 1.4 A Randomized Min-Cut Algorithm 14

15. Axioms of Probability  Definition 1.1 A probability space has three components: 1. a sample space : the set of all possible outcomes; 2. a family of sets representing the allowable events : each set in is a subset of ; 3. a probability function satisfying Definition 1.2. 15

16. Axioms of Probability  Definition 1.2 A probability function is any function that satisfies the following conditions: 1. for any event ; 2. ; 3. for any finite or countably infinite sequence of pairwise mutually disjoint events 16

17. Axioms of Probability  Because events are sets, : the occurrence of both and : the occurrence of either or : the occurrence of an event that is in but not in 17

18. Axioms of Probability  Lemma 1.1 For any two events and  Proof 18

19. Axioms of Probability  Lemma 1.2 For any finite or countably infinite sequence of events 19

20. Axioms of Probability  Lemma 1.3 : inclusion-exclusion principle Let be any events. Then 20

21. Axioms of Probability  The randomized algorithm for verifying polynomial identities  Let represent the event that the algorithm failed to give the correct answer.  It may seem unusual to have an algorithm that can return the wrong answer. trade-off correctness speed 21

22. Axioms of Probability  The randomized algorithm for verifying polynomial identities  Let represent the event that the algorithm failed to give the correct answer.  The algorithm give the correct answer 99% of the time when the polynomials are not equivalent.  Can we improve this probability? 22

23. Axioms of Probability  One way to improve the probability  the algorithm chooses an integer uniformly at random in the range of {1, …, 1000d} → then the probability of a wrong answer is at most 1/1000  But the range of values we can use is limited by the precision available on the machine. 23

24. Axioms of Probability  Another approach  Repeat the algorithm multiple times The algorithm has one-sided error. The algorithm may be wrong only when it outputs If the algorithm finds in at least one round of the algorithm, we know. Only if there is equality for all runs, the algorithm outputs. 24

25. Axioms of Probability  Repeatedly choosing random numbers according to a given distribution is generally referred to as sampling.  In this case, we can choose random numbers in the range of {1, …,100} in two ways: 1. with replacement 2. without replacement 25

26. Axioms of Probability  The case where sampling is done with replacement  We repeat the algorithm k times.  The input polynomials are not equivalent.  What is the probability that the algorithm give a wrong answer? 26

27. Axioms of Probability  Definition 1.3 Two events and are independent if and only if More generally, events are mutually independent if and only if, for any subset, 27

28. Axioms of Probability  Let be the event that, on the ith run of algorithm, we choose a root such that.  The probability that the algorithm returns the wrong answer is given by 28

29. Axioms of Probability  The case where sampling is done without replacement  We repeat the algorithm k times.  The input polynomials are not equivalent.  What is the probability that the algorithm give a wrong answer? 29

30. Axioms of Probability  Definition 1.4 The conditional probability that event occurs given that event occur is When and are independent and, 30

31. Axioms of Probability  Let be the event that the random number chosen in the ith iteration of the algorithm is a root of.  The probability that the algorithm returns the wrong answer is given by 31

32. Axioms of Probability In the th iteration of the algorithm, there are only values out of the remaining choices 32

33. Axioms of Probability  The probability that the algorithm returns the wrong answer is given by 33

34. CHAPTER 1 Events and probability 1.1 Verifying Polynomial Identities 1.2 Axioms of Probability 1.3 Verifying Matrix Multiplication 1.4 A Randomized Min-Cut Algorithm 34

35. Verifying Matrix Multiplication  We want to verify whether ( are matrices) (we are working over the integers modulo 2.)  One way is to multiply and and compare the result to.  The simple matrix multiplication algorithm takes operations.  More sophisticated algorithm takes operations. 35

36. Verifying Matrix Multiplication  We use a randomized algorithm.  The algorithm chooses a random vector  If, then.  Otherwise, it returns that.  The algorithm can be done in time in the obvious way. 36

37. Verifying Matrix Multiplication  Theorem 1.4 If and if is chosen uniformly at random from, then 37

38. Verifying Matrix Multiplication  Lemma 1.5 Choosing uniformly at random is equivalent to choosing each independently and uniformly {0, 1}.  Proof If each is chosen independently and uniformly at random, then each of the possible vector is chosen with probability, giving the lemma. 38

39. Verifying Matrix Multiplication  Let.  Since it must have some nonzero entry. Let that entry be.  If, then. For, it must be the case that and 39

40. Verifying Matrix Multiplication  Instead of reasoning about the vector, we choose the in order, from down to.  Consider the situation just before is chosen.  The equality holds with probability at most ½.  The probability that is at most ½.  This idea is called the principle of deferred decisions. 40

41. Verifying Matrix Multiplication  Theorem 1.6 [Law of Total Probability] Let be mutually disjoint events in the sample space, and let. Then 41

42. Verifying Matrix Multiplication  Proof of Theorem 1.4 42

43. Verifying Matrix Multiplication  Theorem 1.4 If and if is chosen uniformly at random from, then 43

44. Verifying Matrix Multiplication  This algorithm has a one-sided error.  Run the algorithm multiple times.  The running time is  The probability of error is at most 44

45. Verifying Matrix Multiplication  Theorem 1.7 [Bay’s Law] Assume that are mutually disjoint sets such that. Then 45

46. Verifying Matrix Multiplication  Problem We are given three coins. Two of the coins are fair. One of the coins is biased, landing heads with probability 2/3. Flip each the coins. The first and second coins come up heads, and the third comes up tails. What is the probability that the first coin is the biased one? 46 ABC Pr(heads)1/2 2/3 Pr(tails)1/2 1/3 1st coin2nd coin3rd coin heads tails

47. Verifying Matrix Multiplication  Let be the event that the ith coin flipped is the biased one.  Let be the event that the three coin flips come up heads, heads, and tails.  Before flips, 47

48. Verifying Matrix Multiplication  We can compute the probability of the event conditioned on Applying Bay’s law, 48

49. Verifying Matrix Multiplication  In the matrix multiplication case, if we have no information, a reasonable prior assumption is that the identity is correct with probability ½.  Let be the event that the identity is correct.  Let be the event that the test returns the identity is correct. 49

50. Verifying Matrix Multiplication  If we run the randomized test ones and it returns that the matrix identity is correct, 50 ∵

51. Verifying Matrix Multiplication  We run the randomized test again and it returns that the identity is correct 51

52. Verifying Matrix Multiplication  In general : If our prior model is that and if the test returns that identity is correct, then  Thus, if all 100 calls return that the identity is correct, our confidence is at least 52

53. CHAPTER 1 Events and probability 1.1 Verifying Polynomial Identities 1.2 Axioms of Probability 1.3 Verifying Matrix Multiplication 1.4 A Randomized Min-Cut Algorithm 53