1/20/20161 CS 3343: Analysis of Algorithms Review for final.

1/20/20161 CS 3343: Analysis of Algorithms Review for final

1/20/20162 Final Exam Closed book exam Coverage: the whole semester Cheat sheet: you are allowed one letter- size sheet, both sides Monday, May 4, 9:45 – 12:15pm Basic calculator (no graphing) allowed No cell phones!

1/20/20163 Final Exam: Study Tips Study tips: –Study each lecture –Study the homework and homework solutions –Study the midterm exams Re-make your previous cheat sheets

1/20/20164 Topics covered (1) By reversed chronological order: Graph algorithms –Representations –MST (Prim’s, Kruskal’s) –Shortest path (Dijkstra’s) –Running time analysis with different implementations Greedy algorithm –Unit-profit restaurant location problem –Fractional knapsack problem –Prim’s and Kruskal’s are also examples of greedy algorithms –How to show that certain greedy choices are optimal

1/20/20165 Topics covered (2) Dynamic programming –LCS –Restaurant location problem –Shortest path problem on a grid –Other problems –How to define recurrence solution, and use dynamic programming to solve it Binary heap and priority queue –Heapify, buildheap, insert, exatractMax, changeKey –Running time analysis

1/20/20166 Topics covered (3) Order statistics –Rand-Select –Worst-case Linear-time select –Running time analysis Sorting algorithms –Insertion sort –Merge sort –Quick sort –Heap sort –Linear time sorting: counting sort, radix sort –Stability of sorting algorithms –Worst-case and expected running time analysis –Memory requirement of sorting algorithms

1/20/20167 Topics covered (4) Analysis –Order of growth –Asymptotic notation, basic definition Limit method L’ Hopital’s rule Stirling’s formula –Best case, worst case, average case Analyzing non-recursive algorithms –Arithmetic series –Geometric series Analyzing recursive algorithms –Defining recurrence –Solving recurrence Recursion tree (iteration) method Substitution method Master theorem

1/20/20168 Review for finals In chronological order Only the more important concepts –Very likely to appear in your final Does not mean to be exclusive

1/20/20169 Asymptotic notations O: Big-Oh Ω: Big-Omega Θ: Theta o: Small-oh ω: Small-omega Intuitively: O is like  o is like <  is like   is like >  is like =

1/20/201610 Big-Oh Math: –O(g(n)) = {f(n):  positive constants c and n 0 such that 0 ≤ f(n) ≤ cg(n)  n>n 0 } –Or: lim n→∞ g(n)/f(n) > 0 (if the limit exists.) Engineering: –g(n) grows at least as faster as f(n) –g(n) is an asymptotic upper bound of f(n) Intuitively it is like f(n) ≤ g(n)

1/20/201611 Big-Oh Claim: f(n) = 3n 2 + 10n + 5  O(n 2 ) Proof: 3n 2 + 10n + 5  3n 2 + 10n 2 + 5n 2 when n > 1  18 n 2 when n > 1 Therefore, Let c = 18 and n 0 = 1 We have f(n)  c n 2,  n > n 0 By definition, f(n)  O(n 2 )

1/20/201612 Big-Omega Math: –Ω(g(n)) = {f(n):  positive constants c and n 0 such that 0 ≤ cg(n) ≤ f(n)  n>n 0 } –Or: lim n→∞ f(n)/g(n) > 0 (if the limit exists.) Engineering: –f(n) grows at least as faster as g(n) –g(n) is an asymptotic lower bound of f(n) Intuitively it is like g(n) ≤ f(n)

1/20/201613 Big-Omega f(n) = n 2 / 10 = Ω(n) Proof: f(n) = n 2 / 10, g(n) = n –g(n) = n ≤ n 2 / 10 = f(n) when n > 10 –Therefore, c = 1 and n 0 = 10

1/20/201614 Theta Math: –Θ(g(n)) = {f(n):  positive constants c 1, c 2, and n 0 such that c 1 g(n)  f(n)  c 2 g(n)  n  n 0  n>n 0 } –Or: lim n→∞ f(n)/g(n) = c > 0 and c < ∞ –Or: f(n) = O(g(n)) and f(n) = Ω(g(n)) Engineering: –f(n) grows in the same order as g(n) –g(n) is an asymptotic tight bound of f(n) Intuitively it is like f(n) = g(n) Θ(1) means constant time.

1/20/201615 Theta Claim: f(n) = 2n 2 + n = Θ (n 2 ) Proof: –We just need to find three constants c 1, c 2, and n 0 such that –c 1 n 2 ≤ 2n 2 +n ≤ c 2 n 2 for all n > n 0 –A simple solution is c 1 = 2, c 2 = 3, and n 0 = 1

1/20/201616 Using limits to compare orders of growth 0 lim f(n) / g(n) = c > 0 ∞ n→∞ f(n)  o(g(n)) f(n)  Θ (g(n)) f(n)  ω (g(n)) f(n)  O(g(n)) f(n)  Ω(g(n))

1/20/201617 Compare 2 n and 3 n lim 2 n / 3 n = lim(2/3) n = 0 Therefore, 2 n  o(3 n ), and 3 n  ω(2 n ) n→∞

1/20/201618 L’ Hopital’s rule lim f(n) / g(n) = lim f(n)’ / g(n)’ n→∞ If both lim f(n) and lim g(n) goes to ∞

1/20/201619 Compare n 0.5 and log n lim n 0.5 / log n = ? (n 0.5 )’ = 0.5 n -0.5 (log n)’ = 1 / n lim (n -0.5 / 1/n) = lim(n 0.5 ) = Therefore, log n  o(n 0.5 ) n→∞ ∞

1/20/201620 Stirling’s formula (constant)

1/20/201621 Compare 2 n and n! Therefore, 2 n = o(n!)

1/20/201622 More advanced dominance ranking

1/20/201623 General plan for analyzing time efficiency of a non-recursive algorithm Decide parameter (input size) Identify most executed line (basic operation) worst-case = average-case? T(n) =  i t i T(n) = Θ (f(n))

1/20/201624 Statement cost time__ InsertionSort(A, n) { for j = 2 to n { c 1 n key = A[j] c 2 (n-1) i = j - 1; c 3 (n-1) while (i > 0) and (A[i] > key) { c 4 S A[i+1] = A[i] c 5 (S-(n-1)) i = i - 1 c 6 (S-(n-1)) } 0 A[i+1] = key c 7 (n-1) } 0 } Analysis of insertion Sort

1/20/201625 Best case Array already sorted 1 ij sorted Key Inner loop stops when A[i] <= key, or i = 0

1/20/201626 Worst case Array originally in reverse order 1 ij sorted Inner loop stops when A[i] <= key Key

1/20/201627 Average case Array in random order 1 ij sorted Inner loop stops when A[i] <= key Key

1/20/201628 Find the order of growth for sums How to find out the actual order of growth? –Remember some formulas –Learn how to guess and prove

1/20/201629 Arithmetic series An arithmetic series is a sequence of numbers such that the difference of any two successive members of the sequence is a constant. e.g.: 1, 2, 3, 4, 5 or 10, 12, 14, 16, 18, 20 In general: Recursive definition Closed form, or explicit formula Or:

1/20/201630 Sum of arithmetic series If a 1, a 2, …, a n is an arithmetic series, then

1/20/201631 Geometric series A geometric series is a sequence of numbers such that the ratio between any two successive members of the sequence is a constant. e.g.: 1, 2, 4, 8, 16, 32 or 10, 20, 40, 80, 160 or 1, ½, ¼, 1/8, 1/16 In general: Recursive definition Closed form, or explicit formula Or:

1/20/201632 Sum of geometric series if r < 1 if r > 1 if r = 1

1/20/201633 Important formulas

1/20/201634 Sum manipulation rules Example:

1/20/201635 Recursive algorithms General idea: –Divide a large problem into smaller ones By a constant ratio By a constant or some variable –Solve each smaller one recursively or explicitly –Combine the solutions of smaller ones to form a solution for the original problem Divide and Conquer

1/20/201636 How to analyze the time-efficiency of a recursive algorithm? Express the running time on input of size n as a function of the running time on smaller problems

1/20/201637 Analyzing merge sort M ERGE -S ORT A[1.. n] 1.If n = 1, done. 2.Recursively sort A[ 1..  n/2  ] and A[  n/2  +1.. n ]. 3.“Merge” the 2 sorted lists T(n)T(n) Θ(1) 2T(n/2) f(n) Sloppiness: Should be T(  n/2  ) + T(  n/2  ), but it turns out not to matter asymptotically.

1/20/201638 Analyzing merge sort 1.Divide: Trivial. 2.Conquer: Recursively sort 2 subarrays. 3.Combine: Merge two sorted subarrays T(n) = 2 T(n/2) + f(n) +Θ(1) # subproblems subproblem size Work dividing and Combining 1.What is the time for the base case? 2.What is f(n) ? 3.What is the growth order of T(n) ? Constant

1/20/201639 Solving recurrence Running time of many algorithms can be expressed in one of the following two recursive forms or Challenge: how to solve the recurrence to get a closed form, e.g. T(n) = Θ (n 2 ) or T(n) = Θ(nlgn), or at least some bound such as T(n) = O(n 2 ) ?

1/20/201640 Solving recurrence 1.Recurrence tree (iteration) method - Good for guessing an answer 2.Substitution method - Generic method, rigid, but may be hard 3.Master method - Easy to learn, useful in limited cases only - Some tricks may help in other cases

1/20/201641 The master method The master method applies to recurrences of the form T(n) = a T(n/b) + f (n), where a  1, b > 1, and f is asymptotically positive. 1.Divide the problem into a subproblems, each of size n/b 2.Conquer the subproblems by solving them recursively. 3.Combine subproblem solutions Divide + combine takes f(n) time.

1/20/201642 Master theorem T(n) = a T(n/b) + f (n) C ASE 1: f (n) = O(n log b a –  )  T(n) =  (n log b a ). C ASE 2: f (n) =  (n log b a )  T(n) =  (n log b a log n). C ASE 3: f (n) =  (n log b a +  ) and a f (n/b)  c f (n)  T(n) =  ( f (n)). Key: compare f(n) with n log b a e.g.: merge sort: T(n) = 2 T(n/2) + Θ(n) a = 2, b = 2  n log b a = n  CASE 2  T(n) = Θ(n log n).

1/20/201643 Case 1 Compare f (n) with n log b a : f (n) = O(n log b a –  ) for some constant  > 0. : f (n) grows polynomially slower than n log b a (by an n  factor). Solution: T(n) =  (n log b a ) i.e., aT(n/b) dominates e.g. T(n) = 2T(n/2) + 1 T(n) = 4 T(n/2) + n T(n) = 2T(n/2) + log n T(n) = 8T(n/2) + n 2

1/20/201644 Case 3 Compare f (n) with n log b a : f (n) =  (n log b a +  ) for some constant  > 0. : f (n) grows polynomially faster than n log b a (by an n  factor). Solution: T(n) =  (f(n)) i.e., f(n) dominates e.g. T(n) = T(n/2) + n T(n) = 2 T(n/2) + n 2 T(n) = 4T(n/2) + n 3 T(n) = 8T(n/2) + n 4

1/20/201645 Case 2 Compare f (n) with n log b a : f (n) =  (n log b a ). : f (n) and n log b a grow at similar rate. Solution: T(n) =  (n log b a log n) e.g. T(n) = T(n/2) + 1 T(n) = 2 T(n/2) + n T(n) = 4T(n/2) + n 2 T(n) = 8T(n/2) + n 3

1/20/201646 Recursion tree Solve T(n) = 2T(n/2) + dn, where d > 0 is constant.

1/20/201647 Recursion tree Solve T(n) = 2T(n/2) + dn, where d > 0 is constant. T(n)T(n)

1/20/201648 Recursion tree Solve T(n) = 2T(n/2) + dn, where d > 0 is constant. T(n/2) dn

1/20/201649 Recursion tree Solve T(n) = 2T(n/2) + dn, where d > 0 is constant. dn T(n/4) dn/2

1/20/201650 Recursion tree Solve T(n) = 2T(n/2) + dn, where d > 0 is constant. dn dn/4 dn/2  (1) …

1/20/201651 Recursion tree Solve T(n) = 2T(n/2) + dn, where d > 0 is constant. dn dn/4 dn/2  (1) … h = log n

1/20/201652 Recursion tree Solve T(n) = 2T(n/2) + dn, where d > 0 is constant. dn dn/4 dn/2  (1) … h = log n dn

1/20/201653 Recursion tree Solve T(n) = 2T(n/2) + dn, where d > 0 is constant. dn dn/4 dn/2  (1) … h = log n dn

1/20/201654 Recursion tree Solve T(n) = 2T(n/2) + dn, where d > 0 is constant. dn dn/4 dn/2  (1) … h = log n dn …

1/20/201655 Recursion tree Solve T(n) = 2T(n/2) + dn, where d > 0 is constant. dn dn/4 dn/2  (1) … h = log n dn #leaves = n (n)(n) …

1/20/201656 Recursion tree Solve T(n) = 2T(n/2) + dn, where d > 0 is constant. dn dn/4 dn/2  (1) … h = log n dn #leaves = n (n)(n) Total  (n log n) …

1/20/201657 Substitution method 1.Guess the form of the solution: (e.g. using recursion trees, or expansion) 2.Verify by induction (inductive step). The most general method to solve a recurrence (prove O and  separately):

1/20/201658 Recurrence: T(n) = 2T(n/2) + n. Guess: T(n) = O(n log n). (eg. by recurrence tree method) To prove, have to show T(n) ≤ c n log n for some c > 0 and for all n > n 0 Proof by induction: assume it is true for T(n/2), prove that it is also true for T(n). This means: Fact: T(n) = 2T(n/2) + n Assumption: T(n/2)≤ cn/2 log (n/2) Need to Prove: T(n)≤ c n log (n) Proof by substitution

1/20/201659 Proof Fact: T(n) = 2T(n/2) + n Assumption: T(n/2)≤ cn/2 log (n/2) Need to Prove: T(n)≤ c n log (n) Proof: Substitute T(n/2) into the recurrence function => T(n) = 2 T(n/2) + n ≤ cn log (n/2) + n =>T(n) ≤ c n log n - c n + n => T(n) ≤ c n log n (if we choose c ≥ 1).

1/20/201660 Recurrence: T(n) = 2T(n/2) + n. Guess: T(n) = Ω(n log n). To prove, have to show T(n) ≥ c n log n for some c > 0 and for all n > n 0 Proof by induction: assume it is true for T(n/2), prove that it is also true for T(n). This means: Fact: Assumption: Need to Prove: T(n) ≥ c n log (n) Proof by substitution T(n) = 2T(n/2) + n T(n/2) ≥ cn/2 log (n/2)

1/20/201661 Proof Fact: T(n) = 2T(n/2) + n Assumption: T(n/2) ≥ cn/2 log (n/2) Need to Prove: T(n) ≥ c n log (n) Proof: Substitute T(n/2) into the recurrence function => T(n) = 2 T(n/2) + n ≥ cn log (n/2) + n =>T(n) ≥ c n log n - c n + n => T(n) ≥ c n log n (if we choose c ≤ 1).

1/20/201662 Quick sort Quicksort an n-element array: 1.Divide: Partition the array into two subarrays around a pivot x such that elements in lower subarray  x  elements in upper subarray. 2.Conquer: Recursively sort the two subarrays. 3.Combine: Trivial.  x x  x x x x ≥ x≥ x ≥ x≥ x Key: Linear-time partitioning subroutine.

1/20/201663 Partition All the action takes place in the partition() function –Rearranges the subarray in place –End result: two subarrays All values in first subarray  all values in second –Returns the index of the “pivot” element separating the two subarrays  x x  x x x x ≥ x≥ x ≥ x≥ x pr q

1/20/201664 Partition Code Partition(A, p, r) x = A[p];// pivot is the first element i = p; j = r + 1; while (TRUE) { repeat i++; until A[i] > x or i >= j; repeat j--; until A[j] < x or j < i; if (i < j) Swap (A[i], A[j]); else break; } swap (A[p], A[j]); return j; What is the running time of partition() ? partition() runs in O(n) time

1/20/201665 ij 6 6 10 5 5 8 8 13 3 3 2 2 11 x = 6 pr ij 6 6 10 5 5 8 8 13 3 3 2 2 11 ij 6 6 2 2 5 5 8 8 13 3 3 10 11 ij 6 6 2 2 5 5 8 8 13 3 3 10 11 ij 6 6 2 2 5 5 3 3 13 8 8 10 11 ij 6 6 2 2 5 5 3 3 13 8 8 10 11 3 3 2 2 5 5 6 6 13 8 8 10 11 qpr

1/20/201666 6 6 10 5 5 8 8 11 3 3 2 2 13 3 3 2 2 5 5 6 6 11 8 8 10 13 2 2 3 3 5 5 6 6 8 8 10 11 13 2 2 3 3 5 5 6 6 10 8 8 11 13 2 2 3 3 5 5 6 6 8 8 10 11 13

1/20/201667 Quicksort Runtimes Best case runtime T best (n)  O(n log n) Worst case runtime T worst (n)  O(n 2 ) Worse than mergesort? Why is it called quicksort then? Its average runtime T avg (n)  O(n log n ) Better even, the expected runtime of randomized quicksort is O(n log n)

1/20/201668 Randomized quicksort Randomly choose an element as pivot –Every time need to do a partition, throw a die to decide which element to use as the pivot –Each element has 1/n probability to be selected Partition(A, p, r) d = random(); // a random number between 0 and 1 index = p + floor((r-p+1) * d); // p<=index<=r swap(A[p], A[index]); x = A[p]; i = p; j = r + 1; while (TRUE) { … }

1/20/201669 Running time of randomized quicksort The expected running time is an average of all cases T(n) = T(0) + T(n–1) + dnif 0 : n–1 split, T(1) + T(n–2) + dnif 1 : n–2 split,  T(n–1) + T(0) + dnif n–1 : 0 split, Expectation

1/20/201670 Heaps In practice, heaps are usually implemented as arrays: 16 1410 8793 241 1614108793241

1/20/201671 Heaps To represent a complete binary tree as an array: –The root node is A[1] –Node i is A[i] –The parent of node i is A[i/2] (note: integer divide) –The left child of node i is A[2i] –The right child of node i is A[2i + 1] 16 1410 8793 241 1614108793241 A ==

1/20/201672 The Heap Property Heaps also satisfy the heap property: A[Parent(i)]  A[i]for all nodes i > 1 –In other words, the value of a node is at most the value of its parent –The value of a node should be greater than or equal to both its left and right children And all of its descendents –Where is the largest element in a heap stored?

1/20/201673 Heap Operations: Heapify() Heapify(A, i) { // precondition: subtrees rooted at l and r are heaps l = Left(i); r = Right(i); if (l A[i]) largest = l; else largest = i; if (r A[largest]) largest = r; if (largest != i) { Swap(A, i, largest); Heapify(A, largest); } }// postcondition: subtree rooted at i is a heap Among A[l], A[i], A[r], which one is largest? If violation, fix it.

1/20/201674 Heapify() Example 16 410 14793 281 1641014793281 A =

1/20/201675 Heapify() Example 16 410 14793 281 161014793281 A = 4

1/20/201680 Heapify() Example 16 1410 8793 241 1614108793241 A =

1/20/201681 Analyzing Heapify(): Formal T(n)  T(2n/3) +  (1) By case 2 of the Master Theorem, T(n) = O(lg n) Thus, Heapify() takes logarithmic time

1/20/201682 Heap Operations: BuildHeap() We can build a heap in a bottom-up manner by running Heapify() on successive subarrays –Fact: for array of length n, all elements in range A[  n/2  + 1.. n] are heaps (Why?) –So: Walk backwards through the array from n/2 to 1, calling Heapify() on each node. Order of processing guarantees that the children of node i are heaps when i is processed

1/20/201683 BuildHeap() // given an unsorted array A, make A a heap BuildHeap(A) { heap_size(A) = length(A); for (i =  length[A]/2  downto 1) Heapify(A, i); }

1/20/201684 BuildHeap() Example Work through example A = {4, 1, 3, 2, 16, 9, 10, 14, 8, 7} 4 13 216910 1487

1/20/201685 4 13 216910 1487

1/20/201686 4 13 1416910 287

1/20/201687 4 110 141693 287

1/20/201688 4 1610 14793 281

1/20/201689 16 1410 8793 241

1/20/201690 Analyzing BuildHeap(): Tight To Heapify() a subtree takes O(h) time where h is the height of the subtree –h = O(lg m), m = # nodes in subtree –The height of most subtrees is small Fact: an n-element heap has at most  n/2 h+1  nodes of height h CLR 7.3 uses this fact to prove that BuildHeap() takes O(n) time

1/20/201691 Heapsort Example Work through example A = {4, 1, 3, 2, 16, 9, 10, 14, 8, 7} 4 13 216910 1487 4132169101487 A =

1/20/201692 Heapsort Example First: build a heap 16 1410 8793 241 1614108793241 A =

1/20/201693 Heapsort Example Swap last and first 1 1410 8793 2416 1141087932416 A =

1/20/201694 Heapsort Example Last element sorted 1 1410 8793 2416 1141087932416 A =

1/20/201695 Heapsort Example Restore heap on remaining unsorted elements 14 810 4793 2116 Heapify 1481047932116 A =

1/20/201696 Heapsort Example Repeat: swap new last and first 1 810 4793 21416 1810479321416 A =

1/20/201697 Heapsort Example Restore heap 10 89 4713 21416 1089471321416 A =

1/20/201698 Heapsort Example Repeat 9 83 4712 101416 9834712101416 A =

1/20/2016101 Analyzing Heapsort The call to BuildHeap() takes O(n) time Each of the n - 1 calls to Heapify() takes O(lg n) time Thus the total time taken by HeapSort() = O(n) + (n - 1) O(lg n) = O(n) + O(n lg n) = O(n lg n)

1/20/2016102 HeapExtractMax Example 16 1410 8793 241 1614108793241 A =

1/20/2016103 HeapExtractMax Example Swap first and last, then remove last 1 1410 8793 24 16 141087932416 A = 1

1/20/2016104 HeapExtractMax Example Heapify 14 810 4793 21 793216 A = 16 14841

1/20/2016105 HeapChangeKey Example Increase key 16 1410 8793 241 1614108793241 A =

1/20/2016106 HeapChangeKey Example Increase key 16 1410 15793 241 161410793241 A = 15

1/20/2016107 HeapChangeKey Example Increase key 16 1510 14793 241 1610793241 A = 1415

1/20/2016108 HeapInsert Example HeapInsert(A, 17) 16 1410 8793 241 1614108793241 A =

1/20/2016109 HeapInsert Example HeapInsert(A, 17) 16 1410 8793 241 1614108793241 A = -∞ -∞ makes it a valid heap

1/20/2016110 HeapInsert Example HeapInsert(A, 17) 16 1410 8793 241 1610893241 A = 17 147 Now call changeKey

1/20/2016111 HeapInsert Example HeapInsert(A, 17) 17 1610 81493 241 1710893241 A = 7 7 1614

1/20/2016112 Heapify: Θ(log n) BuildHeap: Θ(n) HeapSort: Θ(nlog n) HeapMaximum: Θ(1) HeapExtractMax: Θ(log n) HeapChangeKey: Θ(log n) HeapInsert: Θ(log n)

1/20/2016113 Counting sort for i  1 to k do C[i]  0 for j  1 to n do C[A[ j]]  C[A[ j]] + 1 ⊳ C[i] = |{key = i}| for i  2 to k do C[i]  C[i] + C[i–1] ⊳ C[i] = |{key  i}| for j  n downto 1 doB[C[A[ j]]]  A[ j] C[A[ j]]  C[A[ j]] – 1 1. 2. 3. 4. Initialize Count Compute running sum Re-arrange

1/20/2016114 Counting sort A:A: 4 4 1 1 3 3 4 4 3 3 B:B: 12345 C:C: 1 1 0 0 2 2 2 2 1234 C':C': 1 1 1 1 3 3 5 5 for i  2 to k do C[i]  C[i] + C[i–1] ⊳ C[i] = |{key  i}| 3.

1/20/2016115 Loop 4: re-arrange A:A: 4 4 1 1 3 3 4 4 3 3 B:B: 3 3 12345 C:C: 1 1 1 1 3 3 5 5 1234 C':C': 1 1 1 1 3 3 5 5 for j  n downto 1 doB[C[A[ j]]]  A[ j] C[A[ j]]  C[A[ j]] – 1 4.

1/20/2016116 Analysis for i  1 to k do C[i]  0 (n)(n) (k)(k) (n)(n) (k)(k) for j  1 to n do C[A[ j]]  C[A[ j]] + 1 for i  2 to k do C[i]  C[i] + C[i–1] for j  n downto 1 doB[C[A[ j]]]  A[ j] C[A[ j]]  C[A[ j]] – 1  (n + k) 1. 2. 3. 4.

1/20/2016117 Stable sorting Counting sort is a stable sort: it preserves the input order among equal elements. A:A: 4 4 1 1 3 3 4 4 3 3 B:B: 1 1 3 3 3 3 4 4 4 4 Why this is important? What other algorithms have this property?

1/20/2016118 Radix sort Similar to sorting the address books Treat each digit as a key Start from the least significant bit 198099109123518183599 340199540380128115295 384700101594539614696 382408360201039258538 614386507628681328936 Most significant Least significant

1/20/2016119 Time complexity Sort each of the d digits by counting sort Total cost: d (n + k) –k = 10 –Total cost: Θ(dn) Partition the d digits into groups of 3 –Total cost: (n+10 3 )d/3 We work with binaries rather than decimals –Partition a binary number into groups of r bits –Total cost: (n+2 r )d/r –Choose r = log n –Total cost: dn / log n –Compare with dn log n Catch: faster than quicksort only when n is very large

1/20/2016120 Randomized selection algorithm R AND -S ELECT (A, p, q, i) ⊳ i th smallest of A[ p.. q] if p = q & i > 1 then error! r  R AND -P ARTITION (A, p, q) k  r – p + 1 ⊳ k = rank(A[r]) if i = k then return A[ r] if i < k then return R AND -S ELECT ( A, p, r – 1, i ) else return R AND -S ELECT ( A, r + 1, q, i – k )  A[r] A[r]  A[r] A[r]  A[r] A[r]  A[r] A[r] rpq k

1/20/2016121 Example pivot i = 6 7 7 10 5 5 8 8 11 3 3 2 2 13 k = 4 Select the 6 – 4 = 2nd smallest recursively. Select the i = 6th smallest: 3 3 2 2 5 5 7 7 11 8 8 10 13 Partition:

1/20/2016122 7 7 10 5 5 8 8 11 3 3 2 2 13 3 3 2 2 5 5 7 7 11 8 8 10 13 10 8 8 11 13 8 8 10 Complete example: select the 6 th smallest element. i = 6 k = 4 i = 6 – 4 = 2 k = 3 i = 2 < k k = 2 i = 2 = k Note: here we always used first element as pivot to do the partition (instead of rand-partition).

1/20/2016123 Intuition for analysis Lucky: C ASE 3 T(n)= T(9n/10) +  (n) =  (n) Unlucky: T(n)= T(n – 1) +  (n) =  (n 2 ) arithmetic series Worse than sorting! (All our analyses today assume that all elements are distinct.)

1/20/2016124 Running time of randomized selection For upper bound, assume i th element always falls in larger side of partition The expected running time is an average of all cases T(n) ≤ T(max(0, n–1)) + nif 0 : n–1 split, T(max(1, n–2)) + nif 1 : n–2 split,  T(max(n–1, 0)) + nif n–1 : 0 split, Expectation

1/20/2016125 Worst-case linear-time selection if i = k then return x elseif i < k thenrecursively S ELECT the i th smallest element in the lower part elserecursively S ELECT the (i–k)th smallest element in the upper part S ELECT (i, n) 1.Divide the n elements into groups of 5. Find the median of each 5-element group by rote. 2.Recursively S ELECT the median x of the  n/5  group medians to be the pivot. 3.Partition around the pivot x. Let k = rank(x). 4. Same as R AND - S ELECT

1/20/2016126 Developing the recurrence if i = k then return x elseif i < k thenrecursively S ELECT the i th smallest element in the lower part elserecursively S ELECT the (i–k)th smallest element in the upper part S ELECT (i, n) 1.Divide the n elements into groups of 5. Find the median of each 5-element group by rote. 2.Recursively S ELECT the median x of the  n/5  group medians to be the pivot. 3.Partition around the pivot x. Let k = rank(x). 4. T(n)T(n) (n)(n) T(n/5) (n)(n) T(7n/10 +3)

1/20/2016127 Solving the recurrence if c ≥ 20 and n ≥ 60 Assumption: T(k)  ck for all k < n if n ≥ 60

1/20/2016128 Elements of dynamic programming Optimal sub-structures –Optimal solutions to the original problem contains optimal solutions to sub-problems Overlapping sub-problems –Some sub-problems appear in many solutions

1/20/2016129 Two steps to dynamic programming Formulate the solution as a recurrence relation of solutions to subproblems. Specify an order to solve the subproblems so you always have what you need.

1/20/2016130 Optimal subpaths Claim: if a path start  goal is optimal, any sub-path, start  x, or x  goal, or x  y, where x, y is on the optimal path, is also the shortest. Proof by contradiction –If the subpath between x and y is not the shortest, we can replace it with the shorter one, which will reduce the total length of the new path => the optimal path from start to goal is not the shortest => contradiction! –Hence, the subpath x  y must be the shortest among all paths from x to y startgoalxy a b c b’ a + b + c is shortest b’ < b a + b’ + c < a + b + c

1/20/2016131 Dynamic programming illustration 3 9 12 3 2 52 2 4 23 3 6 33 1 2 32 53 33 3 23 39 3 62 37 4 46 31 3 3 121315 6 81315 9 111316 11 141720 17 1820 0 5 7 13 17 S G F(i-1, j) + dist(i-1, j, i, j) F(i, j) = min F(i, j-1) + dist(i, j-1, i, j)

1/20/2016132 Trace back 3 9 12 3 2 52 2 4 23 3 6 33 1 2 32 53 33 3 23 39 3 62 37 4 46 31 3 3 121315 6 81315 9 111316 11 141720 17 1820 0 5 7 13 17

1/20/2016133 Longest Common Subsequence Given two sequences x[1.. m] and y[1.. n], find a longest subsequence common to them both. x:x:ABCBDAB y:y:BDCABA “a” not “the” BCBA = LCS(x, y) functional notation, but not a function

1/20/2016134 Optimal substructure Notice that the LCS problem has optimal substructure: parts of the final solution are solutions of subproblems. –If z = LCS(x, y), then any prefix of z is an LCS of a prefix of x and a prefix of y. Subproblems: “find LCS of pairs of prefixes of x and y” x y m n z i j

1/20/2016135 Finding length of LCS Let c[i, j] be the length of LCS(x[1..i], y[1..j]) => c[m, n] is the length of LCS(x, y) If x[m] = y[n] c[m, n] = c[m-1, n-1] + 1 If x[m] != y[n] c[m, n] = max { c[m-1, n], c[m, n-1] } x y m n

1/20/2016136 DP Algorithm Key: find out the correct order to solve the sub-problems Total number of sub-problems: m * n c[i, j] = c[i–1, j–1] + 1if x[i] = y[j], max { c[i–1, j], c[i, j–1] } otherwise. C(i, j) 0 m 0 n i j

1/20/2016137 LCS Example (0) j 0 1 2 3 4 5 0 1 2 3 4 i X[i] A B C B Y[j]BBACD X = ABCB; m = |X| = 4 Y = BDCAB; n = |Y| = 5 Allocate array c[5,6] ABCB BDCAB

1/20/2016138 LCS Example (1) j 0 1 2 3 4 5 0 1 2 3 4 i A B C B BBACD 0 0 00000 0 0 0 for i = 1 to m c[i,0] = 0 for j = 1 to n c[0,j] = 0 ABCB BDCAB X[i] Y[j]

1/20/2016139 LCS Example (2) j 0 1 2 3 4 5 0 1 2 3 4 i A B C B BBACD 0 0 00000 0 0 0 if ( X i == Y j ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] ) 0 ABCB BDCAB X[i] Y[j]

1/20/2016143 LCS Example (6) j 0 1 2 3 4 5 0 1 2 3 4 i A B C B BBACD 0 0 00000 0 0 0 if ( X i == Y j ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] ) 00101 1 ABCB BDCAB X[i] Y[j]

1/20/2016146 LCS Example (14) j 0 1 2 3 4 5 0 1 2 3 4 i A B C B BBACD 0 0 00000 0 0 0 if ( X i == Y j ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] ) 10001 1211 112 1 22 1122 3 ABCB BDCAB X[i] Y[j]

1/20/2016147 LCS Algorithm Running Time LCS algorithm calculates the values of each entry of the array c[m,n] So what is the running time? O(m*n) since each c[i,j] is calculated in constant time, and there are m*n elements in the array

1/20/2016148 How to find actual LCS The algorithm just found the length of LCS, but not LCS itself. How to find the actual LCS? For each c[i,j] we know how it was acquired: A match happens only when the first equation is taken So we can start from c[m,n] and go backwards, remember x[i] whenever c[i,j] = c[i-1, j-1]+1. 2 23 2 For example, here c[i,j] = c[i-1,j-1] +1 = 2+1=3

1/20/2016149 Finding LCS j 0 1 2 3 4 5 0 1 2 3 4 i A B C BBACD 0 0 00000 0 0 0 10001 1211 112 1 22 1122 3 B X[i] Y[j] Time for trace back: O(m+n).

1/20/2016150 Finding LCS (2) j 0 1 2 3 4 5 0 1 2 3 4 i A B C BBACD 0 0 00000 0 0 0 10001 1211 112 1 22 1122 3 B BCB LCS (reversed order): LCS (straight order):B C B (this string turned out to be a palindrome) X[i] Y[j]

1/20/2016151 LCS as a longest path problem A B C B BBACD 1 1 11 1 1

1/20/2016152 LCS as a longest path problem A B C B BBACD 000000 000011 011112 011222 011123 1 1 11 1 1

1/20/2016153 Restaurant location problem 1 You work in the fast food business Your company plans to open up new restaurants in Texas along I-35 Towns along the highway called t 1, t 2, …, t n Restaurants at t i has estimated annual profit p i No two restaurants can be located within 10 miles of each other due to some regulation Your boss wants to maximize the total profit You want a big bonus 10 mile

1/20/2016154 A DP algorithm Suppose you’ve already found the optimal solution It will either include t n or not include t n Case 1: t n not included in optimal solution –Best solution same as best solution for t 1, …, t n-1 Case 2: t n included in optimal solution –Best solution is p n + best solution for t 1, …, t j, where j < n is the largest index so that dist(t j, t n ) ≥ 10

1/20/2016155 Recurrence formulation Let S(i) be the total profit of the optimal solution when the first i towns are considered (not necessarily selected) –S(n) is the optimal solution to the complete problem S(n-1) S(j) + p n j < n & dist (t j, t n ) ≥ 10 S(n) = max S(i-1) S(j) + p i j < i & dist (t j, t i ) ≥ 10 S(i) = max Generalize Number of sub-problems: n. Boundary condition: S(0) = 0. Dependency: i i-1 j S

1/20/2016156 Example Natural greedy 1: 6 + 3 + 4 + 12 = 25 Natural greedy 2: 12 + 9 + 3 = 24 5226663107 6798 3 324125 Distance (mi) Profit (100k) 6799 10 12 1426 S(i) S(i-1) S(j) + p i j < i & dist (t j, t i ) ≥ 10 S(i) = max 100 0 73412 dummy Optimal: 26

1/20/2016157 Complexity Time:  (nk), where k is the maximum number of towns that are within 10 miles to the left of any town –In the worst case,  (n 2 ) –Can be improved to  (n) with some preprocessing tricks Memory: Θ(n)

1/20/2016158 Knapsack problem Three versions: 0-1 knapsack problem: take each item or leave it Fractional knapsack problem: items are divisible Unbounded knapsack problem: unlimited supplies of each item. Which one is easiest to solve? Each item has a value and a weight Objective: maximize value Constraint: knapsack has a weight limitation We study the 0-1 problem today.

1/20/2016159 Formal definition (0-1 problem) Knapsack has weight limit W Items labeled 1, 2, …, n (arbitrarily) Items have weights w 1, w 2, …, w n –Assume all weights are integers –For practical reason, only consider w i < W Items have values v 1, v 2, …, v n Objective: find a subset of items, S, such that  i  S w i  W and  i  S v i is maximal among all such (feasible) subsets

1/20/2016160 A DP algorithm Suppose you’ve find the optimal solution S Case 1: item n is included Case 2: item n is not included Total weight limit: W wnwn Total weight limit: W Find an optimal solution using items 1, 2, …, n-1 with weight limit W - w n wnwn Find an optimal solution using items 1, 2, …, n-1 with weight limit W

1/20/2016161 Recursive formulation Let V[i, w] be the optimal total value when items 1, 2, …, i are considered for a knapsack with weight limit w => V[n, W] is the optimal solution V[n, W] = max V[n-1, W-w n ] + v n V[n-1, W] Generalize V[i, w] = max V[i-1, w-w i ] + v i item i is taken V[i-1, w] item i not taken V[i-1, w] if w i > w item i not taken Boundary condition: V[i, 0] = 0, V[0, w] = 0. Number of sub-problems = ?

1/20/2016162 Example n = 6 (# of items) W = 10 (weight limit) Items (weight, value): 2 4 3 3 5 6 2 4 6 9

1/20/2016163 0 0 0 0 0 0 00000000000 w012345678910 425 6 4 3 2 1 i 96 65 33 34 22 vivi wiwi max V[i-1, w-w i ] + v i item i is taken V[i-1, w] item i not taken V[i-1, w] if w i > w item i not taken V[i, w] = V[i, w] V[i-1, w]V[i-1, w-w i ] 6 wiwi 5

1/20/2016164 107400 1310764400 9633200 8653200 555532200 2222222200 00000000000 w0123456789 iwiwi vivi 122 243 333 456 524 669 max V[i-1, w-w i ] + v i item i is taken V[i-1, w] item i not taken V[i-1, w] if w i > w item i not taken V[i, w] = 2 4 3 6 5 6 7 5 9 6 8 10 911 8 3 1213 15

1/20/2016165 107400 1310764400 9633200 8653200 555532200 2222222200 00000000000 w0123456789 iwiwi vivi 122 243 333 456 524 669 2 4 3 6 5 6 7 5 9 6 8 911 8 3 1213 15 Item: 6, 5, 1 Weight: 6 + 2 + 2 = 10 Value: 9 + 4 + 2 = 15 Optimal value: 15

1/20/2016166 Time complexity Θ (nW) Polynomial? –Pseudo-polynomial –Works well if W is small Consider following items (weight, value): (10, 5), (15, 6), (20, 5), (18, 6) Weight limit 35 –Optimal solution: item 2, 4 (value = 12). Iterate: 2^4 = 16 subsets –Dynamic programming: fill up a 4 x 35 = 140 table entries What’s the problem? –Many entries are unused: no such weight combination –Top-down may be better

1/20/2016167 Longest increasing subsequence Given a sequence of numbers 1 2 5 3 2 9 4 9 3 5 6 8 Find a longest subsequence that is non- decreasing –E.g. 1 2 5 9 –It has to be a subsequence of the original list –It has to in sorted order => It is a subsequence of the sorted list Original list: 1 2 5 3 2 9 4 9 3 5 6 8 LCS: Sorted: 1 2 2 3 3 4 5 5 6 8 9 9 1 2 3 4 5 6 8

1/20/2016168 Events scheduling problem A list of events to schedule (or shows to see) –e i has start time s i and finishing time f i –Indexed such that f i < f j if i < j Each event has a value v i Schedule to make the largest value –You can attend only one event at any time Very similar to the new restaurant location problem –Sort events according to their finish time –Consider: if the last event is included or not Time e1e1 e2e2 e3e3 e4e4 e5e5 e6e6 e7e7 e8e8 e9e9

1/20/2016169 Events scheduling problem Time e1e1 e2e2 e3e3 e4e4 e5e5 e6e6 e7e7 e8e8 e9e9 V(i) is the optimal value that can be achieved when the first i events are considered V(n) = V(n-1) e n not selected e n selected V(j) + v n max { j < n and f j < s n s9s9 f9f9 s8s8 f8f8 s7s7 f7f7

1/20/2016170 Coin change problem Given some denomination of coins (e.g., 2, 5, 7, 10), decide if it is possible to make change for a value (e.g, 13), or minimize the number of coins Version 1: Unlimited number of coins for each denomination –Unbounded knapsack problem Version 2: Use each denomination at most once –0-1 Knapsack problem

1/20/2016171 Use DP algorithm to solve new problems Directly map a new problem to a known problem Modify an algorithm for a similar task Design your own –Think about the problem recursively –Optimal solution to a larger problem can be computed from the optimal solution of one or more subproblems –These sub-problems can be solved in certain manageable order –Works nicely for naturally ordered data such as strings, trees, some special graphs –Trickier for general graphs The text book has some very good exercises.

1/20/2016172 Unit-profit restaurant location problem Now the objective is to maximize the number of new restaurants (subject to the distance constraint) –In other words, we assume that each restaurant makes the same profit, no matter where it is opened 10 mile

1/20/2016173 A DP Algorithm Exactly as before, but p i = 1 for all i S(i-1) S(j) + 1 j < i & dist (t j, t i ) ≥ 10 S(i) = max S(i-1) S(j) + p i j < i & dist (t j, t i ) ≥ 10 S(i) = max

1/20/2016174 Greedy algorithm for restaurant location problem select t 1 d = 0; for (i = 2 to n) d = d + dist(t i, t i-1 ); if (d >= min_dist) select t i d = 0; end 5226663107 d 0 57915 0 69 0 10 0 7

1/20/2016175 Complexity Time: Θ(n) Memory: –Θ(n) to store the input –Θ(1) for greedy selection

1/20/2016176 Optimal substructure Claim 1: if A = [m 1, m 2, …, m k ] is the optimal solution to the restaurant location problem for a set of towns [t 1, …, t n ] –m 1 < m 2 < … < m k are indices of the selected towns –Then B = [m 2, m 3, …, m k ] is the optimal solution to the sub-problem [t j, …, t n ], where t j is the first town that are at least 10 miles to the right of t m 1 Proof by contradiction: suppose B is not the optimal solution to the sub-problem, which means there is a better solution B’ to the sub-problem –A’ = m i || B’ gives a better solution than A = m i || B => A is not optimal => contradiction => B is optimal m1m1 B’ (imaginary) A’ B m1m1 A m2m2 mkmk

1/20/2016177 Greedy choice property Claim 2: for the uniform-profit restaurant location problem, there is an optimal solution that chooses t 1 Proof by contradiction: suppose that no optimal solution can be obtained by choosing t 1 –Say the first town chosen by the optimal solution S is t i, i > 1 –Replace t i with t 1 will not violate the distance constraint, and the total profit remains the same => S’ is an optimal solution –Contradiction –Therefore claim 2 is valid S S’

1/20/2016178 Fractional knapsack problem 0-1 knapsack problem: take each item or leave it Fractional knapsack problem: items are divisible Unbounded knapsack problem: unlimited supplies of each item. Which one is easiest to solve? Each item has a value and a weight Objective: maximize value Constraint: knapsack has a weight limitation We can solve the fractional knapsack problem using greedy algorithm

1/20/2016179 Greedy algorithm for fractional knapsack problem Compute value/weight ratio for each item Sort items by their value/weight ratio into decreasing order –Call the remaining item with the highest ratio the most valuable item (MVI) Iteratively: –If the weight limit can not be reached by adding MVI Select MVI –Otherwise select MVI partially until weight limit

1/20/2016180 Example Weight limit: 10 1.5 2 1.2 1 0.75 1 $ / LB 966 425 654 333 342 221 Value ($) Weight (LB) item

1/20/2016181 Example Weight limit: 10 Take item 5 –2 LB, $4 Take item 6 –8 LB, $13 Take 2 LB of item 4 –10 LB, 15.4 itemWeight (LB) Value ($) $ / LB 5242 6691.5 4561.2 1221 3331 2430.75

1/20/2016182 Why is greedy algorithm for fractional knapsack problem valid? Claim: the optimal solution must contain the MVI as much as possible (either up to the weight limit or until MVI is exhausted) Proof by contradiction: suppose that the optimal solution does not use all available MVI (i.e., there is still w (w < W) pounds of MVI left while we choose other items) –We can replace w pounds of less valuable items by MVI –The total weight is the same, but with value higher than the “optimal” –Contradiction w w w w

1/20/2016183 Graphs A graph G = (V, E) –V = set of vertices –E = set of edges = subset of V  V –Thus |E| = O(|V| 2 ) 1 24 3 Vertices: {1, 2, 3, 4} Edges: {(1, 2), (2, 3), (1, 3), (4, 3)}

1/20/2016184 Graphs: Adjacency Matrix Example: 1 24 3 A1234 10110 20010 30000 40010 How much storage does the adjacency matrix require? A: O(V 2 )

1/20/2016185 Graphs: Adjacency List Adjacency list: for each vertex v  V, store a list of vertices adjacent to v Example: –Adj[1] = {2,3} –Adj[2] = {3} –Adj[3] = {} –Adj[4] = {3} Variation: can also keep a list of edges coming into vertex 1 24 3

1/20/2016186 Kruskal’s algorithm: example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 c-d: 3 b-f: 5 b-a: 6 f-e: 7 b-d: 8 f-g: 9 d-e: 10 a-f: 12 b-c: 14 e-h: 15

1/20/2016197 Time complexity Depending on implementation Pseudocode: sort all edges according to weights T = {}. tree(v) = v for all v. for each edge (u, v) if tree(u) != tree(v) T = T U (u, v); union (tree(u), tree(v)) Overall time complexity Naïve: Θ(nm) Better implementation: Θ(m log n) Θ(m log m) = Θ(m log n) m edges Avg time spent per edge Naïve: Θ (n) Better: Θ (log n) using set union

1/20/2016198 Prim’s algorithm: example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 abcdefgh ∞∞∞∞∞∞∞∞∞

1/20/2016199 Prim’s algorithm: example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 ChangeKey cbadefgh 00∞∞∞∞∞∞∞

1/20/2016200 Prim’s algorithm: example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 ExctractMin hbadefg ∞∞∞∞∞∞∞

1/20/2016201 Prim’s algorithm: example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 dbahefg 314∞∞∞∞∞ ChangeKey

1/20/2016202 Prim’s algorithm: example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 bgahef  14∞∞∞∞∞ ExctractMin

1/20/2016203 Prim’s algorithm: example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 beahgf 8810∞∞∞∞ Changekey

1/20/2016204 Prim’s algorithm: example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 efahg 10∞∞∞∞ ExtractMin

1/20/2016205 Prim’s algorithm: example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 feahg 5106∞∞ Changekey

1/20/2016206 Prim’s algorithm: example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 aegh 610∞∞ ExtractMin

1/20/2016207 Prim’s algorithm: example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 aegh 679∞ Changekey

1/20/2016208 Prim’s algorithm: example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 ehg 7∞9 ExtractMin

1/20/2016209 Prim’s algorithm: example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 gh 9∞ ExtractMin

1/20/2016210 Prim’s algorithm: example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 gh 9 Changekey

1/20/2016211 Prim’s algorithm: example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7 h ExtractMin

1/20/2016212 Prim’s algorithm: example a a b b f f c c e e d d g g h h 612 5 14 3 8 10 15 9 7

1/20/2016213 Complete Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; T = {}; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) T = T U (u, v); ChangeKey(v, w(u,v)); How often is ExtractMin() called? How often is ChangeKey() called? n vertices Θ(n) times Θ(n 2 ) times? Θ(m) times Overall running time: Θ(m log n) Cost per ChangeKey

1/20/2016214 Summary Kruskal’s algorithm –Θ(m log n) –Possibly Θ(m + n log n) with counting sort Prim’s algorithm –With priority queue : Θ(m log n) Assume graph represented by adj list –With distance array : Θ(n^2) Adj list or adj matrix –For sparse graphs priority queue wins –For dense graphs distance array may be better

1/20/2016215 abcdefghi ∞14750∞∞∞∞ 7 2 3 9 6 9 7 1 6 8 1 7 5 4 b e d c g a i h f 0 5 7 Dijkstra’s algorithm

1/20/2016216 abcdefghi 11 750∞∞∞∞ 7 2 3 9 6 14 9 7 1 6 8 1 7 5 4 b e d c g a i h f 0 5 7 11 Dijkstra’s algorithm

1/20/2016217 abcdefghi 911750∞∞∞∞ 7 2 3 9 6 14 9 7 1 6 8 1 7 5 4 b e d c g a i h f 0 5 7 11 9 Dijkstra’s algorithm

1/20/2016218 abcdefghi 91175012∞∞17 7 2 3 9 6 14 9 7 1 6 8 1 7 5 4 b e d c g a i h f 0 5 7 11 9 12 17 Dijkstra’s algorithm

1/20/2016219 abcdefghi 91175012∞2017 7 2 3 9 6 14 9 7 1 6 8 1 7 5 4 b e d c g a i h f 0 5 7 11 9 12 17 20 Dijkstra’s algorithm

1/20/2016220 abcdefghi 91175012∞1917 7 2 3 9 6 14 9 7 1 6 8 1 7 5 4 b e d c g a i h f 0 5 7 11 9 12 17 20 19 Dijkstra’s algorithm

1/20/2016221 abcdefghi 9117501218 17 7 2 3 9 6 14 9 7 1 6 8 1 7 5 4 b e d c g a i h f 0 5 7 11 9 12 17 20 19 18 Dijkstra’s algorithm

1/20/2016224 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; T = {}; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) T = T U (u, v); ChangeKey(v, w(u,v)); Overall running time: Θ(m log n) Cost per ChangeKey

1/20/2016225 Dijkstra’s Algorithm Dijkstra(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; T = {}; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and key[u]+w(u,v) < key[v]) T = T U (u, v); ChangeKey(v, key[u]+w(u,v)); Overall running time: Θ(m log n) Cost per ChangeKey Running time of Dijkstra’s algorithm is the same as Prim’s algorithm

1/20/2016226 Good luck with your final!

1/20/20161 CS 3343: Analysis of Algorithms Review for final.

Similar presentations

Presentation on theme: "1/20/20161 CS 3343: Analysis of Algorithms Review for final."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

1/20/20161 CS 3343: Analysis of Algorithms Review for final.

Similar presentations

Presentation on theme: "1/20/20161 CS 3343: Analysis of Algorithms Review for final."— Presentation transcript:

Similar presentations

About project

Feedback