1. 204313 Algorithm Design and Analysis June 11, 20161 Algorithm Design and Analysis Pradondet Nilagupta Department of Computer Engineering This lecture note has been modified from lecture note for 23250 by Prof. Francis Chin, CS332 by David Luekbe

2. 204313 Algorithm Design and Analysis June 11, 20162 Greedy Method (Cont.)

3. 204313 Algorithm Design and Analysis June 11, 20163 TREE ● A tree is an undirected graph which is connected and acyclic. It is easy to show that if graph G ■ G (V,E) is connected. ■ G (V,E) is acyclic. ■ |E| =|V| -1

4. 204313 Algorithm Design and Analysis June 11, 20164 Spanning Trees ● A spanning tree in an undirected graph G(V,E) is a subset of edges that are acyclic and connect all the vertices in V. ● It follows from the above conditions that a spanning tree must consist of exactly n-1 edges. ● Now suppose that each edge has a weight associated with it: Say that the weight of a tree T is the sum of the weights of its edges;

5. 204313 Algorithm Design and Analysis June 11, 20165 Example: Spanning Tree 2 1 3 4 6 7 5 28 16 12 14 10 24 22 18 connected graph 2 1 3 4 6 7 5 28 16 14 10 24 18 2 1 3 4 6 7 5 28 16 14 2524 22 spanning tree cost = 129 spanning tree cost = 110

6. 204313 Algorithm Design and Analysis June 11, 20166 Definition ● The minimum spanning tree in a weighted graph G(V,E) is one which has the smallest weight among all spanning trees in G(V,E).

7. 204313 Algorithm Design and Analysis June 11, 20167 A Greedy Strategy Generic-MST(G, w) A = {} // Invariant: A is a subset of some MST while A does not form a spanning tree do find an edge (u, v) that is safe for A A = A + {(u, v)} // A remains a subset of some MST return A How to find a safe edge?

8. 204313 Algorithm Design and Analysis June 11, 20168 Example ● Example: spanning tree T {(a,b), (a,c), (b,d), (d,e), (d,f)} ● w(T) = 4+8+8+7+3 = 30 ● T * = {(a,b), (b,e), (c,e), (c,d), (d,f)} ● w(T * ) = 4+7+1+2+3 = 17 (minimum ?, why ? see note 1)

9. 204313 Algorithm Design and Analysis June 11, 20169 Method 1: Brute force ● Enumerate all spanning trees, evaluate their costs and find the minimum. ● Drawback - the number of spanning trees must be very large, how large? ● Let N s be the number of spanning trees for an n-node graph. ● lower bound, (the least value of N s ): n!/2 for a chain, why? ● upper bound, (the largest value of N s ): C e,n-1, where e = n(n-1)/2, i.e. O(n (n-1) )

10. 204313 Algorithm Design and Analysis June 11, 201610 Method 2: Removing redundant edges ● Edge of the largest weight is removed if the resultant graph remains connected. Repeat the edge removal until not possible. ● Correctness: If the largest-weight edge was not removed, it can always be replaced by an edge of lower weight because of the redundancy. ● Time complexity: ● sorting the edges - O(n 2 logn) or O(elogn) ● testing connectivity - O(n 2 ) or O(e) ● connectivity test for all edges - O(n 4 ) or O(e 2 )

11. 204313 Algorithm Design and Analysis June 11, 201611 Minimum Spanning Trees ● Theorem Given a spanning forest which is a subset of the MST, the minimum weight edge (u, v) emerging from a tree in the forest must be "safe", i.e., an edge in the MST.

12. 204313 Algorithm Design and Analysis June 11, 201612 Proof: ● Assume that the minimum weight edge (u, v) is not in MST. Let us add edge(u, v) to the MST to result a unique cycle which must contain another edge (u', v'), where u and u' belong to the some tree. As edge (u, v) is shorter than edge (u', v'), the weight of the new MST containing (u, v) and not (u', v') should be lower than the weight of the original MST. This contradicts our assumption that (u, v) is not in the MST.

13. 204313 Algorithm Design and Analysis June 11, 201613 Method 3: Ordered edge insertion (Krushal) ● The edge with minimum weight is added to the "forest" as long as it is not redundant, i.e., it connects two trees in the forest. Repeat the edge insertion until not possible. ● Correctness: The proof follows directly from the theorem and can be proved by induction. Initially, the forest has n trees each of which contains a single node of G. The edge with minimum weight edge which is not redundant, i.e., emerging from some trees in the forest, must be a minimum weight edge in the MST.

14. 204313 Algorithm Design and Analysis June 11, 201614 Time Complexcity ● Time Complexity: sorting the edges - O(n 2 logn) or O(elogn)

15. 204313 Algorithm Design and Analysis June 11, 201615 Minimum Spanning Tree (MST) Connect all the vertices Minimize the total edge weight // must be a tree Problem Select edges in a connected graph to f d a b c e g 2 7 5 7 1 4 1 3 4 4 5 2

16. 204313 Algorithm Design and Analysis June 11, 201616 Cut A cut of G = (V, E) is a partition (S, V-S) of V. Ex. S = {a, b, c, f}, V-S = {e, d, g} Edges {b, d}, {a, d}, {b, e}, {c, e} all cross the cut. The remaining edges do not cross the cut. f d a b c e g 2 7 5 7 1 4 1 3 4 4 5 2

17. 204313 Algorithm Design and Analysis June 11, 201617 Respected by a Cut Ex. S = {a, b, c, d} A1 = {(a, b), (d, g), (f, b), (a, f)} A2 = A1 + {(b, d)} The cut (S, V-S) respects A1 but not A2. f d a b c e g 2 7 5 7 1 4 1 3 4 4 5 2

18. 204313 Algorithm Design and Analysis June 11, 201618 Light Edge Ex. S = {a, b, c, d} Edge {b, e} has weight 3, “lighter” than the other edges {a, d}, {b, d}, and {b, e} that cross the cut. The edge that crosses a cut with the minimum weight. f d a b c e g 2 7 5 7 1 4 1 3 4 4 5 2 light edge

19. 204313 Algorithm Design and Analysis June 11, 201619 Light Edge Is Safe! Theorem Let (S, V-S) be any cut of G(V, E) that respects a subset A of E, which is included in some MST for G. Let (u, v) be a light edge crossing (S, V-S). Then (u, v) is safe for A; that is, A + {(u, v)} is also included in some MST. S V- S 4 2 6 u v A includes all red edges.

20. 204313 Algorithm Design and Analysis June 11, 201620 Corollary Let A be a subset of E that is included in some MST of G, and C be a connected component in the forest G =. If (u, v) is a light edge connecting C to some other component in G, then (u, v) is safe for A. A A u v C 7 4

21. 204313 Algorithm Design and Analysis June 11, 201621 Minimum Spanning Tree ● Problem: given a connected, undirected, weighted graph: 14 10 3 64 5 2 9 15 8

22. 204313 Algorithm Design and Analysis June 11, 201622 Minimum Spanning Tree ● Problem: given a connected, undirected, weighted graph, find a spanning tree using edges that minimize the total weight 14 10 3 64 5 2 9 15 8

23. 204313 Algorithm Design and Analysis June 11, 201623 Minimum Spanning Tree ● Which edges form the minimum spanning tree (MST) of the below graph? HBC GED F A 14 10 3 64 5 2 9 15 8

24. 204313 Algorithm Design and Analysis June 11, 201624 Minimum Spanning Tree ● Answer: HBC GED F A 14 10 3 64 5 2 9 15 8

25. 204313 Algorithm Design and Analysis June 11, 201625 Minimum Spanning Tree ● MSTs satisfy the optimal substructure property: an optimal tree is composed of optimal subtrees ■ Let T be an MST of G with an edge (u,v) in the middle ■ Removing (u,v) partitions T into two trees T 1 and T 2 ■ Claim: T 1 is an MST of G 1 = (V 1,E 1 ), and T 2 is an MST of G 2 = (V 2,E 2 ) (Do V 1 and V 2 share vertices? Why?) ■ Proof: w(T) = w(u,v) + w(T 1 ) + w(T 2 ) (There can’t be a better tree than T 1 or T 2, or T would be suboptimal)

26. 204313 Algorithm Design and Analysis June 11, 201626 Minimum Spanning Tree ● Thm: ■ Let T be MST of G, and let A  T be subtree of T ■ Let (u,v) be min-weight edge connecting A to V-A ■ Then (u,v)  T

27. 204313 Algorithm Design and Analysis June 11, 201627 Minimum Spanning Tree ● Thm: ■ Let T be MST of G, and let A  T be subtree of T ■ Let (u,v) be min-weight edge connecting A to V-A ■ Then (u,v)  T

28. 204313 Algorithm Design and Analysis June 11, 201628 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v);

29. 204313 Algorithm Design and Analysis June 11, 201629 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 14 10 3 64 5 2 9 15 8 Run on example graph

30. 204313 Algorithm Design and Analysis June 11, 201630 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v);     14 10 3 64 5 2 9 15 8 Run on example graph

31. 204313 Algorithm Design and Analysis June 11, 201631 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v);  0    14 10 3 64 5 2 9 15 8 Pick a start vertex r r

32. 204313 Algorithm Design and Analysis June 11, 201632 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v);  0    14 10 3 64 5 2 9 15 8 Red vertices have been removed from Q u

33. 204313 Algorithm Design and Analysis June 11, 201633 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v);  0  3  14 10 3 64 5 2 9 15 8 Red arrows indicate parent pointers u

34. 204313 Algorithm Design and Analysis June 11, 201634 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 14  0  3  10 3 64 5 2 9 15 8 u

35. 204313 Algorithm Design and Analysis June 11, 201635 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 14  0  3  10 3 64 5 2 9 15 8 u

36. 204313 Algorithm Design and Analysis June 11, 201636 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 14  08  3  10 3 64 5 2 9 15 8 u

37. 204313 Algorithm Design and Analysis June 11, 201637 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 10  08  3  14 10 3 64 5 2 9 15 8 u

38. 204313 Algorithm Design and Analysis June 11, 201638 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 10  08  3  14 10 3 64 5 2 9 15 8 u

39. 204313 Algorithm Design and Analysis June 11, 201639 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 102  08  3  14 10 3 64 5 2 9 15 8 u

40. 204313 Algorithm Design and Analysis June 11, 201640 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 102  0815 3  14 10 3 64 5 2 9 15 8 u

41. 204313 Algorithm Design and Analysis June 11, 201641 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 102  0815 3  14 10 3 64 5 2 9 15 8 u

42. 204313 Algorithm Design and Analysis June 11, 201642 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 1029 0815 3  14 10 3 64 5 2 9 15 8 u

43. 204313 Algorithm Design and Analysis June 11, 201643 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 1029 0815 3 4 14 10 3 64 5 2 9 15 8 u

44. 204313 Algorithm Design and Analysis June 11, 201644 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 529 0815 3 4 14 10 3 64 5 2 9 15 8 u

45. 204313 Algorithm Design and Analysis June 11, 201645 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 529 0815 3 4 14 10 3 64 5 2 9 15 8 u

46. 204313 Algorithm Design and Analysis June 11, 201646 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 529 0815 3 4 14 10 3 64 5 2 9 15 8 u

47. 204313 Algorithm Design and Analysis June 11, 201647 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 529 0815 3 4 14 10 3 64 5 2 9 15 8 u

48. 204313 Algorithm Design and Analysis June 11, 201648 Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 529 0815 3 4 14 10 3 64 5 2 9 15 8 u

49. 204313 Algorithm Design and Analysis June 11, 201649 Review: Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); What is the hidden cost in this code?

50. 204313 Algorithm Design and Analysis June 11, 201650 Review: Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; DecreaseKey(v, w(u,v));

51. 204313 Algorithm Design and Analysis June 11, 201651 Review: Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; DecreaseKey(v, w(u,v)); How often is ExtractMin() called? How often is DecreaseKey() called?

52. 204313 Algorithm Design and Analysis June 11, 201652 Review: Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q key[u] =  ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v  Adj[u] if (v  Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); What will be the running time? A: Depends on queue binary heap: O(E lg V) Fibonacci heap: O(V lg V + E)