1. Administrative Sep. 25 (today) – HW3 (=QUIZ #1) due Sep. 27 – HW4 due Sep. 28 8am – problem session Oct. 2 Oct. 4 – QUIZ #2 (pages 45-79 of DPV)

2. Administrative You may work with one other person on homeworks, but you must each write up your solutions separately (without any written aid). If you work with another person, indicate who you worked with on your solution (thus you should each indicate each other). Homework rules: All cases of suspected dishonesty must be reported to the Board, either through a shortform resolution or by forwarding a case to the Board for a hearing. Faculty may not come to an understanding with a student on their own in a case of suspected dishonesty, but must use the short form resolution or submit a case. University Academic Honesty Policy:

3. Recurrences T(n) = a T(n/b) + f(n) If f(n) = O(n c-  ) then T(n) =  (n c ) If f(n) =  (n c ) then T(n) =  (n c.log n) If f(n) =  (n c+  ) then T(n)=  (f(n)) if a.f(n/b)  d.f(n) for some d n 0 c=log b a T(n) = 3 T(n/2) +  (n) T(n) =  (n log 3 ) T(n) = 2T(n/2) +  (n) T(n) =  (n.log n) 2

4. Finding the k-th smallest element k =  n/2  = MEDIAN Split(A[1..n],x) xx xx runs in time O(n)

5. Finding the k-th smallest element k =  n/2  = MEDIAN Split(A[1..n],x) xx xx j j  k  k-th smallest on left j<k  (k-j)-th smallest on right

6. Finding the k-th smallest element 6 3 1 8 7 2 6 1 8 5 8 9 1 3 2 631 8 7 26 1 8589 1 32

7. 6 3 1 8 7 2 6 1 8 5 8 9 1 3 2 1) sort each 5-tuple 631 8 7 26 1 8589 1 32

8. Finding the k-th smallest element 6 3 1 8 7 2 6 1 8 5 8 9 1 3 2 1) sort each 5-tuple

9. Finding the k-th smallest element 1 3 6 8 7 2 6 1 8 5 8 9 1 3 2 1) sort each 5-tuple

10. Finding the k-th smallest element 1 3 6 8 7 2 6 1 8 5 8 9 1 3 2 1) sort each 5-tuple

11. Finding the k-th smallest element 1 3 6 8 7 1 2 5 8 6 8 9 1 3 2 1) sort each 5-tuple

12. Finding the k-th smallest element 1 3 6 8 7 1 2 5 8 6 1 2 3 9 7 1) sort each 5-tuple TIME = ?

13. Finding the k-th smallest element 1 3 6 8 7 1 2 5 8 6 1 2 3 9 7 1) sort each 5-tuple TIME =  (n)

14. Finding the k-th smallest element 1 3 6 8 7 1 2 5 8 6 1 2 3 9 7 2) find median of the middle n/5 elements TIME = ?

15. Finding the k-th smallest element 1 3 6 8 7 1 2 5 8 6 1 2 3 9 7 2) find median of the middle n/5 elements TIME = T(n/5) We will use this element as the pivot

16. Finding the k-th smallest element 1 3 6 8 7 1 2 5 8 6 1 2 3 9 7 At least ? Many elements in the array are  X

17. Finding the k-th smallest element 1 2 3 9 7 At least ? Many elements in the array are  X 1 2 5 8 6 1 3 6 8 7

18. Finding the k-th smallest element 1 2 3 9 7 At least 3n/10 elements in the array are  X 1 2 5 8 6 1 3 6 8 7

19. Finding the k-th smallest element 1 2 3 9 7 At least 3n/10 elements in the array are  X 1 2 5 8 6 1 3 6 8 7 631 8 7 26 1 8589 1 32

20. Finding the k-th smallest element At least 3n/10 elements in the array are  X 631 8 7 26 1 8589 1 32 631 3 2 21 1 8589 6 87 XX XX

21. Finding the k-th smallest element At least 3n/10 elements in the array are  X 631 8 7 26 1 8589 1 32 631 3 2 21 1 8589 6 87 XX XX Recurse, time ?

22. Finding the k-th smallest element At least 3n/10 elements in the array are  X 631 8 7 26 1 8589 1 32 631 3 2 21 1 8589 6 87 XX XX Recurse, time  T(7n/10)

23. Finding the k-th smallest element 6 3 1 8 7 2 6 1 8 5 8 9 1 3 2 631 8 7 26 1 8589 1 32 1 3 6 8 7 1 2 5 8 6 1 2 3 9 7 1 3 6 8 7 1 2 5 8 6 1 2 3 9 7 63 1 87 2 618 58 913 2 63 1 32 2 1 1 85 8 968 7 XX XX recurse Split

24. Finding the k-th smallest element 6 3 1 8 7 2 6 1 8 5 8 9 1 3 2 631 8 7 26 1 8589 1 32 1 3 6 8 7 1 2 5 8 6 1 2 3 9 7 1 3 6 8 7 1 2 5 8 6 1 2 3 9 7 63 1 87 2 618 58 913 2 63 1 32 2 1 1 85 8 968 7 XX XX recurse  n) T(n/5)  n) T(7n/10)

25. Finding the k-th smallest element T(n)  T(n/5) + T(7n/10) + O(n)

26. Finding the k-th smallest element T(n)  T(n/5) + T(7n/10) + O(n) T(n)  d.n Induction step: T(n)  T(n/5) + T(7n/10) + O(n)  d.(n/5) + d.(7n/10) + O(n)  d.n + (O(n) – dn/10)  d.n

27. Why 5-tuples? 3 1 7 6 1 5 9 1 2 317165912 1 3 7 1 5 6 1 2 9 63 1 87 2 618 58 913 2 63 1 32 2 1 1 85 8 968 7 XX XX recurse  n) 1 3 7 1 5 6 1 2 9

28. Why 5-tuples? 3 1 7 6 1 5 9 1 2 317165912 1 3 7 1 5 6 1 2 9 63 1 87 2 618 58 913 2 63 1 32 2 1 1 85 8 968 7 XX XX recurse  n) T(2n/3) 1 3 7 1 5 6 1 2 9 T(n/3)

29. Why 5-tuples? T(n)  T(n/3) + T(2n/3) +  (n)

30. Why 5-tuples? T(n)  T(n/3) + T(2n/3) +  (n) T(n)  c.n.ln n Induction step: T(n) = T(n/3) + T(2n/3) +  (n)  c.(n/3).ln (n/3) + c.(2n/3).ln (2n/3) +  (n)  c.n.ln n - c.n.((1/3)ln 3+(2/3)ln 3/2)+  (n)  c.n.ln n

31. Quicksort(A[b..c]) Quicksort(A[b..i]); Quicksort(A[j..c]); Split(A[b..c],x) xx xx xx i j b c

32. Quicksort(A[b..c]) Worst-case running time? How to make the worst-case running time O(n.log n) ?

33. Quicksort(A[b..c]) if pivot = median then the worst-case running time satisfies T(n) = 2T(n/2) + O(n)

34. Quicksort(A[b..c]) Quicksort(A[b..i]); Quicksort(A[j..c]); Split(A[b..c],x) xx xx xx i j b c x = random element of A[b..c]

35. Finding the k-th smallest element k =  n/2  = MEDIAN Split(A[1..n],x) xx xx j j  k  k-th smallest on left j<k  (k-j)-th smallest on right

36. Finding the k-th smallest element Select(k,A[c..d]) Split(A[c..d],x) xx xx j j  k  k-th smallest on left j<k  (k-j)-th smallest on right x=random element from A[c..d]

37. Finite probability space set  (sample space) function P:  R + (probability distribution)  P(x) = 1 x 

38. Finite probability space set  (sample space) function P:  R + (probability distribution) elements of  are called atomic events subsets of  are called events probability of an event A is  P(x) xAxA P(A)=  P(x) = 1 x 

39. Examples 1. Roll a (6 sided) dice. What is the probability that the number on the dice is even? 2. Flip two coins, what is the probability that they show the same symbol? 3. Flip five coins, what is the probability that they show the same symbol? 4. Mix a pack of 52 cards. What is the probability that all red cards come before all black cards?

40. Union bound P(A  B)  P(A) + P(B) P(A 1  A 2  …  A n )  P(A 1 ) + P(A 2 )+…+P(A n ) LEMMA: More generally:

41. Union bound P(A 1  A 2  …  A n )  P(A 1 ) + P(A 2 )+…+P(A n ) Suppose that the probability of winning in a lottery is 10 -6. What is the probability that somebody out of 100 people wins? A i = i-th person wins somebody wins = ?

42. Union bound P(A 1  A 2  …  A n )  P(A 1 ) + P(A 2 )+…+P(A n ) Suppose that the probability of winning in a lottery is 10 -6. What is the probability that somebody out of 100 people wins? A i = i-th person wins somebody wins = A 1  A 2  …  A 100

43. Union bound P(A 1  A 2  …  A n )  P(A 1 ) + P(A 2 )+…+P(A n ) Suppose that the probability of winning in a lottery is 10 -6. What is the probability that somebody out of 100 people wins? P(A 1  A 2  …  A 100 )  100*10 -6 = 10 -4

44. Union bound P(A 1  A 2  …  A n )  P(A 1 ) + P(A 2 )+…+P(A n ) Suppose that the probability of winning in a lottery is 10 -6. What is the probability that somebody out of 100 people wins? P(A 1  A 2  …  A 100 )  100*10 -6 = 10 -4 P(A 1  A 2  …  A 100 ) = 1–P(A C 1  A C 2  …  A C 100 ) = 1-P(A C 1 )P(A C 2 )…P(A C 100 ) = 1-(1-10 -6 ) 100  0.99*10 -4

45. Independence Events A,B are independent if P(A  B) = P(A) * P(B)

46. Independence Events A,B are independent if P(A  B) = P(A) * P(B) “observing whether B happened gives no information on A” B A

47. Independence Events A,B are independent if P(A  B) = P(A) * P(B) “observing whether B happened gives no information on A” B A P(A|B) = P(A  B)/P(B) conditional probability of A, given B

48. Independence Events A,B are independent if P(A  B) = P(A) * P(B) P(A|B) = P(A)

49. Examples Roll two (6 sided) dice. Let S be their sum. 1) What is that probability that S=7 ? 2) What is the probability that S=7, conditioned on S being odd ? 3) Let A be the event that S is even and B the event that S is odd. Are A,B independent? 4) Let C be the event that S is divisible by 4. Are A,C independent? 5) Let D be the event that S is divisible by 3. Are A,D independent?

50. Examples A B C Are A,B independent ? Are A,C independent ? Are B,C independent ? Is it true that P(A  B  C)=P(A)P(B)P(C)?

51. Examples A B C Are A,B independent ? Are A,C independent ? Are B,C independent ? Is it true that P(A  B  C)=P(A)P(B)P(C)? Events A,B,C are pairwise independent but not (fully) independent

52. Full independence Events A 1,…,A n are (fully) independent If for every subset S  [n]:={1,2,…,n} P (  A i ) =  P(A i ) iSiS iSiS

53. Random variable set  (sample space) function P:  R + (probability distribution)  P(x) = 1 x  A random variable is a function Y :  R The expected value of Y is E[X] :=  P(x)* Y(x) x 

54. Examples Roll two dice. Let S be their sum. If S=7 then player A gives player B $6 otherwise player B gives player A $1 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12

55. Examples Roll two dice. Let S be their sum. If S=7 then player A gives player B $6 otherwise player B gives player A $1 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 -1, -1,-1,-1, -1, 6,-1,-1, -1, -1, -1 Expected income for B E[Y] = 6*(1/6)-1*(5/6)= 1/6 Y:

56. Linearity of expectation E[X  Y]  E[X] + E[Y] E[X 1  X 2  …  X n ]  E[X 1 ] + E[X 2 ]+…+E[X n ] LEMMA: More generally:

57. Linearity of expectation Everybody pays me $1 and writes their name on a card. I mix the cards and give everybody one card. If you get back the card with your name – I pay you $10. Let n be the number of people in the class. For what n is the game advantageous for me?

58. Linearity of expectation Everybody pays me $1 and writes their name on a card. I mix the cards and give everybody one card. If you get back the card with your name – I pay you $10. X 1 = -9 if player 1 gets his card back 1 otherwise E[X 1 ] = ?

59. Linearity of expectation Everybody pays me $1 and writes their name on a card. I mix the cards and give everybody one card. If you get back the card with your name – I pay you $10. X 1 = -9 if player 1 gets his card back 1 otherwise E[X 1 ] = -9/n + 1*(n-1)/n

60. Linearity of expectation Everybody pays me $1 and writes their name on a card. I mix the cards and give everybody one card. If you get back the card with your name – I pay you $10. X 1 = -9 if player 1 gets his card back 1 otherwise X 2 = -9 if player 2 gets his card back 1 otherwise E[X 1 +…+X n ] = E[X 1 ]+…+E[X n ] = n ( -9/n + 1*(n-1)/n ) = n – 10.

61. Expected number of coin-tosses until HEADS?

62. Expected number of coin-tosses until HEADS? 1/2 1 1/4 2 1/8 3 1/16 4 ….   n.2 -n = 2 n=1 

63. Expected number of dice-throws until you get “6” ?