Download presentation
Presentation is loading. Please wait.
Published byวิศิษฏ์ วอชิงตัน Modified over 5 years ago
1
CS 332: Algorithms Amortized Analysis Continued
Longest Common Subsequence Dynamic Programming David Luebke /5/2019
2
Administrivia Midterm almost graded Homework 4 assigned
Due: Tuesday 28 (after Thanksgiving break) David Luebke /5/2019
3
Review: MST Algorithms
In a connected, weighted, undirected graph, will the edge with the lowest weight be in the MST? Why or why not? Yes: If T is MST of G, and A T is a subtree of T, and (u,v) is the min-weight edge connecting A to V-A, then (u,v) T The lowest-weight edge must be in the tree (A=) David Luebke /5/2019
4
Review: MST Algorithms
What do the disjoint sets in Kruskal’s algorithm represent? A: Parts of the graph we have connected up together so far David Luebke /5/2019
5
Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ;
for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 14 17 8 25 5 21 13 1 David Luebke /5/2019
6
Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ;
for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 14 17 8 25 5 21 13 1? David Luebke /5/2019
7
Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ;
for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 14 17 8 25 5 21 13 1 David Luebke /5/2019
8
Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ;
for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T {{u,v}}; Union(FindSet(u), FindSet(v)); } 2? 19 9 14 17 8 25 5 21 13 1 David Luebke /5/2019
9
Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ;
for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 14 17 8 25 5 21 13 1 David Luebke /5/2019
10
Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ;
for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 14 17 8 25 5? 21 13 1 David Luebke /5/2019
11
Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ;
for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 14 17 8 25 5 21 13 1 David Luebke /5/2019
12
Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ;
for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 14 17 8? 25 5 21 13 1 David Luebke /5/2019
13
Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ;
for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 14 17 8 25 5 21 13 1 David Luebke /5/2019
14
Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ;
for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9? 14 17 8 25 5 21 13 1 David Luebke /5/2019
15
Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ;
for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 14 17 8 25 5 21 13 1 David Luebke /5/2019
16
Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ;
for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 14 17 8 25 5 21 13? 1 David Luebke /5/2019
17
Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ;
for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 14 17 8 25 5 21 13 1 David Luebke /5/2019
18
Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ;
for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 14? 17 8 25 5 21 13 1 David Luebke /5/2019
19
Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ;
for each v V MakeSet(v); sort E by increasing edge weight w for each (u,v) E (in sorted order) if FindSet(u) FindSet(v) T = T {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 14 17 8 25 5 21 13 1 David Luebke /5/2019
20
Review: Shortest-Path Algorithms
How does the Bellman-Ford algorithm work? How can we do better for DAGs? Under what conditions can we use Dijkstra’s algorithm? David Luebke /5/2019
21
Review: Running Time of Kruskal’s Algorithm
Expensive operations: Sort edges: O(E lg E) O(V) MakeSet()’s O(E) FindSet()’s O(V) Union()’s Upshot: Comes down to efficiency of disjoint-set operations, particularly Union() David Luebke /5/2019
22
Review: Disjoint Set Union
So how do we represent disjoint sets? Naïve implementation: use a linked list to represent elements, with pointers back to set: MakeSet(): O(1) FindSet(): O(1) Union(A,B): “Copy” elements of A into set B by adjusting elements of A to point to B: O(A) How long could n Union()s take? O(n2), worst case David Luebke /5/2019
23
Disjoint Set Union: Analysis
Worst-case analysis: O(n2) time for n Union’s Union(S1, S2) “copy” 1 element Union(S2, S3) “copy” 2 elements … Union(Sn-1, Sn) “copy” n-1 elements O(n2) Improvement: always copy smaller into larger How long would above sequence of Union’s take? Worst case: n Union’s take O(n lg n) time Proof uses amortized analysis David Luebke /5/2019
24
Amortized Analysis of Disjoint Sets
If elements are copied from the smaller set into the larger set, an element can be copied at most lg n times Worst case: Each time copied, element in smaller set 1st time resulting set size 2 2nd time 4 … (lg n)th time n David Luebke /5/2019
25
Amortized Analysis of Disjoint Sets
Since we have n elements each copied at most lg n times, n Union()’s takes O(n lg n) time Therefore we say the amortized cost of a Union() operation is O(lg n) This is the aggregate method of amortized analysis: n operations take time T(n) Average cost of an operation = T(n)/n David Luebke /5/2019
26
Amortized Analysis: Accounting Method
Charge each operation an amortized cost Amount not used stored in “bank” Later operations can used stored money Balance must not go negative Book also discusses potential method But we won’t worry about it here David Luebke /5/2019
27
Accounting Method Example: Dynamic Tables
Implementing a table (e.g., hash table) for dynamic data, want to make it small as possible Problem: if too many items inserted, table may be too small Idea: allocate more memory as needed David Luebke /5/2019
28
Dynamic Tables 1. Init table size m = 1
2. Insert elements until number n > m 3. Generate new table of size 2m 4. Reinsert old elements into new table 5. (back to step 2) What is the worst-case cost of an insert? One insert can be costly, but the total? David Luebke /5/2019
29
Analysis Of Dynamic Tables
Let ci = cost of ith insert ci = i if i-1 is exact power of 2, 1 otherwise Example: Operation Table Size Cost Insert(1) 1 1 1 David Luebke /5/2019
30
Analysis Of Dynamic Tables
Let ci = cost of ith insert ci = i if i-1 is exact power of 2, 1 otherwise Example: Operation Table Size Cost Insert(1) 1 1 1 Insert(2) 2 1 + 1 2 David Luebke /5/2019
31
Analysis Of Dynamic Tables
Let ci = cost of ith insert ci = i if i-1 is exact power of 2, 1 otherwise Example: Operation Table Size Cost Insert(1) 1 1 1 Insert(2) 2 1 + 1 2 Insert(3) 4 1 + 2 3 David Luebke /5/2019
32
Analysis Of Dynamic Tables
Let ci = cost of ith insert ci = i if i-1 is exact power of 2, 1 otherwise Example: Operation Table Size Cost Insert(1) 1 1 1 Insert(2) 2 1 + 1 2 Insert(3) 4 1 + 2 3 Insert(4) 4 1 4 David Luebke /5/2019
33
Analysis Of Dynamic Tables
Let ci = cost of ith insert ci = i if i-1 is exact power of 2, 1 otherwise Example: Operation Table Size Cost Insert(1) 1 1 1 Insert(2) 2 1 + 1 2 Insert(3) 4 1 + 2 3 Insert(4) 4 1 4 Insert(5) 8 1 + 4 5 David Luebke /5/2019
34
Analysis Of Dynamic Tables
Let ci = cost of ith insert ci = i if i-1 is exact power of 2, 1 otherwise Example: Operation Table Size Cost Insert(1) 1 1 1 Insert(2) 2 1 + 1 2 Insert(3) 4 1 + 2 3 Insert(4) 4 1 4 Insert(5) 8 1 + 4 5 Insert(6) 8 1 6 David Luebke /5/2019
35
Analysis Of Dynamic Tables
Let ci = cost of ith insert ci = i if i-1 is exact power of 2, 1 otherwise Example: Operation Table Size Cost Insert(1) 1 1 1 Insert(2) 2 1 + 1 2 Insert(3) 4 1 + 2 3 Insert(4) 4 1 4 Insert(5) 8 1 + 4 5 Insert(6) 8 1 6 Insert(7) 8 1 7 David Luebke /5/2019
36
Analysis Of Dynamic Tables
Let ci = cost of ith insert ci = i if i-1 is exact power of 2, 1 otherwise Example: Operation Table Size Cost Insert(1) 1 1 1 Insert(2) 2 1 + 1 2 Insert(3) 4 1 + 2 3 Insert(4) 4 1 4 Insert(5) 8 1 + 4 5 Insert(6) 8 1 6 Insert(7) 8 1 7 Insert(8) 8 1 8 David Luebke /5/2019
37
Analysis Of Dynamic Tables
Let ci = cost of ith insert ci = i if i-1 is exact power of 2, 1 otherwise Example: Operation Table Size Cost 1 2 3 4 5 6 7 Insert(1) 1 1 1 8 Insert(2) 2 1 + 1 9 2 Insert(3) 4 1 + 2 Insert(4) 4 1 Insert(5) 8 1 + 4 Insert(6) 8 1 Insert(7) 8 1 Insert(8) 8 1 Insert(9) 16 1 + 8 David Luebke /5/2019
38
Aggregate Analysis n Insert() operations cost
Average cost of operation = (total cost)/(# operations) < 3 Asymptotically, then, a dynamic table costs the same as a fixed-size table Both O(1) per Insert operation David Luebke /5/2019
39
Accounting Analysis Charge each operation $3 amortized cost
Use $1 to perform immediate Insert() Store $2 When table doubles $1 reinserts old item, $1 reinserts another old item Point is, we’ve already paid these costs Upshot: constant (amortized) cost per operation David Luebke /5/2019
40
Accounting Analysis Suppose must support insert & delete, table should contract as well as expand Table overflows double it (as before) Table < 1/2 full halve it: BAD IDEA (Why?) Better: Table < 1/4 full halve it Charge $3 for Insert (as before) Charge $2 for Delete Store extra $1 in emptied slot Use later to pay to copy remaining items to new table when shrinking table David Luebke /5/2019
41
Dynamic Programming Another strategy for designing algorithms is dynamic programming A metatechnique, not an algorithm (like divide & conquer) The word “programming” is historical and predates computer programming Use when problem breaks down into recurring small subproblems This lecture: a driving problem Next lecture: the algorithm David Luebke /5/2019
42
Dynamic Programming Example: Longest Common Subsequence
Longest common subsequence (LCS) problem: Given two sequences x[1..m] and y[1..n], find the longest subsequence which occurs in both Ex: x = {A B C B D A B }, y = {B D C A B A} {B C} and {A A} are both subsequences of both What is the LCS? Brute-force algorithm: For every subsequence of x, check if it’s a subsequence of y How many subsequences of x are there? What will be the running time of the brute-force alg? David Luebke /5/2019
43
LCS Algorithm Brute-force algorithm: 2m subsequences of x to check against n elements of y: O(n 2m) We can do better: for now, let’s only worry about the problem of finding the length of LCS When finished we will see how to backtrack from this solution back to the actual LCS Define c[i,j] to be the length of the LCS of x[1..i] and y[1..j] What is the length of LCS of x and y? David Luebke /5/2019
44
Finding LCS Length Theorem: What is this really saying?
David Luebke /5/2019
45
The End David Luebke /5/2019
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.