1. Administrivia, Lecture 4 HW #2 assigned this weekend, due Thurs Week 4 HWs will be due Thurs of Weeks 2, 4, 6, 7, 9, 10 HW #1 solutions should be posted tonight (TA’s have compiled them) Reading for next week: Chapter 9.2-3 (linear-time selection), Chapter 33.3-4 (convex hull, closest- pair), Chapter 23 (minimum spanning trees)

2. Divide and Conquer for Sorting (2.3/1.3) Divide (into two equal parts) Conquer (solve for each part separately) Combine separate solutions Mergesort –Divide into two equal parts –Sort each part using Mergesort (recursion!!!) –Merge two sorted subsequences

3. Merging Two Subsequences x[1]-x[2]- … - x[K] y[1]-y[2]- … - y[L] if y[i] > x[j]  y[i+1] > x[j] < < K+L-1 edges = # (comparisons) = linear time

4. Merge Sort Execution Example 285 179652351423310861254450520 285 179652351 423 310 861254450520 285 179652351 423 310 861254450520 285 179652351 423 310 861254450520 310 179652351 423 285 861254450520 310 179652351 423 285 861254450520 310 179 652 351 423 285 861254450520 310 179351652 423 285 861254450520 310 179351652 423 285 861254450520 285 310351652 423 179 861254450520 285 310351652 423 179 861254450520 285 310351652 423 179 861 254450520 285 310351652 423 179 861 254450520 285 310351652 423 179 861 254450520 285 310351652 423 179 861 254 450520 285 310351652 423 179 861 254 450520 285 310351652 423 179 861 254450520 285 310351652 254 179 423450520861 254 285310351423179450520652861

5. Recursion Tree 1 2 3 4 5 6 7 8 1 3 5 82 4 6 7 1 53 8 5 13 8 4 72 6 7 4 6 2 log n n comparisons per level log n levels total runtime = n log n

6. Master Method (4. 3) Recurrence T(n) = a  T(n/b) + f(n) 1) If for some  > 0 then 2) If then 3) If for some  > 0 and a  f(n/b)  c  f(n) for some c < 1 then

7. Master Method Examples Mergesort T(n) = 2T(n/2) +  (n) Strassen (28.2) T(n) = 7T(n/2) +  (n 2 )

8. Quicksort (7.1-7.2/8.1-8.2) Sorts in place like insertion sort, unlike merge sort Divide into two parts such that –elements of left part < elements of right part Conquer: recursively solve for each part separately Combine: trivial - do not do anything Quicksort(A,p,r) if p < r then q  Partition (A,p,r) Quicksort (A,p,q) Quicksort (A,q+1,r) //divide //conquer left //conquer right

9. Divide = Partition PARTITION(A,p,r) //Partition array from A[p] to A[r] with pivot A[p] //Result: All elements  original A[p] have index  i x = A[p] i = p - 1 j = r + 1 repeat forever repeat j = j - 1 until A[j]  x repeat i = i +1 until A[i]  x if i < j then exchange A[i]  A[j] else return j

10. How It Works 976151651011 i j 9 76151651011 i j 9 761516510 i j 11 9 761516510 i j 1110 9 7615165 i j 1110 9 7615165 i j 1110 9 7615165 i j 1110 9 7615165 i j 1110 9 7615165 i j 11105*761516 9* i j 11105761516 9 i j 11105761516 9 i j 11105761516 9 i j 11105761516 9 i j 11105761516 9 i j 11105761516 9 i j 11105761516 9 i j 11105761516 9 i j 11105761516 9 i j 11105761516 9 i j 111057 6 1516 9 i j 111057 6 1516 9 ij 111057 6 1516 9 ij 11 1057 6 1516 9 11 leftright 9 76151651011 i j 1057 6 1516911 leftright 1057 6 1516911 leftright 10 5 7 6 1516911 leftright 10 5 7 6 1516911 leftright ji 10 5 7 6 1516911 leftright ji 10 5 7 6 1516911 leftright ji 10 5 7 6 1516911 leftright j i 10 5 7 6 1516911 leftright ji 1057 6 1516911 leftright 1056 7 1516911 leftright 1056 7 1516911 leftright 1056 7 15 16911 leftright 1056 7 15 16911 leftright jj 1056 7 15 16911 leftright ji 1056 7 11*169 15* leftright ji 1056 7 11169 15 leftright ji 1056 7 11169 15 leftright ji 1656 7 11109 15 leftright ji 1656 7 11109 15 leftright ji 1656 7 1110915 leftright 1656 7 9101115 leftright 1556 7 9101116 leftright 1556 7 9101116

11. Runtime of Quicksort Worst case: –every time nothing to move –pivot = left (right) end of subarray –O(n 2 ) 0123456789 123456789 8 0 9 89 n Recursion Tree of QSort

12. Runtime of Quicksort Best case: –every time partition in (almost) equal parts –no worse than in given proportion –O(n log n) Average case = ?

13. What is the DQ Recurrence for QSort? t(n) =  (n) + 1/n  j = 1 to n (t(j-1) + t(n-j)) pivots pivots equiprobable lengths of subproblems =  (n) + 2/n  k = 0 to n-1 t(k) Guess t(n)  O(n log n) t(n)   (n) + 2/n [  i=2 to n-1 c i log i]  c n log n – cn/2

14. Another QSort Analysis (Shift / Cancel) (1) T(n) = n-1 + 2/n  j = 1 to n-1 T(i), n  2, T(1) = 0 (2) T(n+1) = n+1 - 1 + 2/(n+1)  j = 1 to n T(i) (1)  (3) nT(n) = n(n-1) + 2  j = 1 to n-1 T(i) (2)  (4) (n+1)T(n+1) = (n+1)n + 2  j = 1 to n T(i) (4) - (3)  (n+1)T(n+1) – nT(n) = 2n + 2T(n)  T(n+1) = (n+2)/(n+1) T(n) + 2n/(n+1)  T(n+1)  (n+2)/(n+1) T(n) + 2

15. Shift / Cancel (cont.) Unroll: H n = 1 + ½ + 1/3 + …+ 1/n = ln n + + O(1/n)  = Euler’s constant = 0.577…  T(n)  2(n+1) (ln n +  – 3/2) + O(1)  T(n)  O(n log n)

16. Randomized Analysis of QSort Probability space –Set  of elementary events  experiment outcomes –Family F of subsets of , called events –Probability measure Pr, real-valued function on members of F –where A  , A  F  A c  F F closed under union, intersection For all A  F, 0  Pr(A)  1 Pr(  ) = 1 For disjoint events A 1, A 2,... Pr(U A i ) =  Pr(A i ) Random variable is a function from elements of  into  –e.g., event (X = x)  set of elements of  for which X assumes the fixed value x For integer-valued r.v. X, expectation E[X] =  i Pr(X=i)

17. Randomized Analysis of QSort (cont.) At each level, we compare each element to the splitter in its partition Def. A successful pivot p satisfies n/8 < p < 7n/8 Three facts –(1) E[  X i ] =  E[X i ] linearity of expectation –(2) If Pr(Heads) is q, expected # tosses up to and including first Head is 1/q // family size puzzle –(3) Pr(U i A i )   i Pr(A i ) prob of union  sum of probs Given that Pr(successful pivot) = 3/4, a single element e i participates in at most how many successful partitioning steps? –Ans: log 8/7 n

18. Randomized Analysis of QSort (cont.) What is number of partition steps expected between k th, (k+1) st successful pivots? –(2)  4/3 steps –(1)  4/3 log 8/7 n –Each element expects to participate in O(log n) pivots (comparisons) Define r.v.’s which give # comparisons for each element –(1)  get O(n log n) expected comparisons So, we have another analysis that gives us the O(n log n) expected complexity of QSort

19. How Badly Can We Deviate From Expectation? E.g., what is Pr (i th element sees 20 log n pivots)? –How many successes possible? –Know  log 8/7 n Pivots are independent (Bernoulli trials) –Pr(success) = 3/4, Pr(failure) = 1/4 –To see 20 log n pivots, need 20 log n - log n failures in 20 log n tries Can use (3) to bound probability of so many “bad” events, since in general events may not be independent –Can get: Pr(  20 n log n) is 1 - O(n -6 )

20. Selection Best pivot: median –exercise: analyze complexity when bad pivots chosen –too expensive  random splitter This leads to SELECTION: –select (L, k) returns k th - smallest element of L –e.g., k = |L|/2  median What is an efficient algorithm? –O(n log n)? –sorting D/Q Recursion: –N.B.: This if RSelect, below –recall pivot from QSort –if i<k, look in right part for (k-i) th smallest –if i>k, look in left part for k th smallest

21. Randomized Selection Worst case: –  (n 2 ) as in QSort analysis Suppose can guarantee “good” pivot –e.g., n/4  i  3n/4 –  subproblem size  3n/4 –Let s(n)  time to find good pivot –  t(n)  s(n) + cn + t([3n/4]) find pivot pivot, make subproblem solve subproblem

22. Randomized Selection Suppose further: S(n)  dn for some d; t(n)  (c+d)n + t([3n/4]) Claim: t(n)  kn for some k would follow –Constructive induction or “substitution” –Ind. Hyp.: t(m)  km for m  n-1 –Ind. Step: t(n)  (c+d)n + k(3n/4) = (c+d+3k/4)n  kn which we want to be equivalent to t(n)  kn –But this is true if k  4(c+d)

23. Break Celebrity Problem: Given n people and a “knows” relation, is there a celebrity? –Notation: Directed graph (and, let’s say that there can be 0, 1 or 2 directed edges between any two vertices (== people)) What is obvious algorithm? –Test each person’s celebrityhood Induction –Hyp: Can tell whether there is a celebrity among the first n-1 people –Induction Step: Have a celebrity among first n-1 people  two queries needed to verify whether known by n th person No celebrity among first n-1 people  check whether n th person is a celebrity (2(n-1) queries needed) (Else no celebrity)  (n 2 ) queries

24. Break (cont.) Celebrity Problem: Can you do better? –Hint: (n-1) + 2(n-1) queries suffice Why are we focusing on identifying the celebrity? Key idea: eliminate a non-celebrity with each query –K ij = 0 j not celebrity –K ij = 1 i not celebrity –(We’ve seen this “complement” idea in DQ-MAXMIN) Another question: Given the adjacency matrix of an undirected graph, describe a fast algorithm that finds all triangles in the graph. –Q: Why am I asking this now?

25. D/Q for Arithmetic Multiplying Large Integers –A = a 0 + r 1 a 1 +... + r n-1 a n-1, r  radix –“classic” approach   (n 2 ) work Can we apply D/Q? –Let n = 2s, r = 10  radix –AB = xz + 10 s (wz + xy) + 10 2s wy –T(n) = 4T(n/2) +  (n)  a = 4, b = 2 in Master Method –T(n)   (n 2 ) –Need to reduce # subproblems, i.e., want a < 4 Observation: r’ = (w+x)(y+z) = wy + (wz+xy) + xz –r’  (w+x)(y+z) –p  wy –q  xz –return 10 2s p + 10 s (r’-p-q) + q –T(n)  O(n log 2 3 ) = O(n 1.59 )

26. Matrix Multiplication A = […], B = […] are n x n matrices a 11, a 12, etc are n/2 x n/2 submatrices M = AB = […] –where m 11 = a 11 b 11 + a 12 b 21 etc. –Evaluation requires 8 multiplies, 4 adds T(n) = 8T(n/2) + O(n)   (n 3 ) Strassen: –p 1 = (a 21 + a 22 - a 11 )(b 22 - b 12 + b 11 ) –p 2 = a 11 b 11 –p 3 = a 12 b 21 –p 4 = (a 11 -a 21 )(b 22 -b 12 ) –p 5 = (a 21 +a 22 )(b 12 - b 11 ) –p 6 = (a 12 -a 21 +a 11 -a 22 )b 22 –p 7 = a 22 (b 11 +b 22 -b 12 -b 21 )

27. Strassen’s Matrix Multiplication p 1 = (a 21 + a 22 - a 11 )(b 22 - b 12 + b 11 ) p 2 = a 11 b 11 p 3 = a 12 b 21 p 4 = (a 11 -a 21 )(b 22 -b 12 ) p 5 = (a 21 +a 22 )(b 12 - b 11 ) p 6 = (a 12 -a 21 +a 11 -a 22 )b 22 p 7 = a 22 (b 11 +b 22 -b 12 -b 21 ) AB 11 = p 2 + p 3 AB 12 = p 1 + p 2 + p 5 + p 6 AB 21 = p 1 + p 2 + p 4 + p 7 AB 22 = p 1 + p 2 + p 4 + p 5 T(n)   (n 2.81 ) // 7 multiplies, 24 adds –Can get to 15 adds