1. 2016-1-9  2004 SDU Lecture 6- Minimum Spanning Tree 1.The Minimum Spanning Tree Problem 2.Greedy algorithms 3.A Generic Algorithm 4.Kruskal’s Algorithm 5.Prim’s Algorithm

2.  2004 SDU 2 An Application In the design of networking Given n computers, we want to connect them so that each pair of them can communicate with each other. Every foot of cable costs us $1 We want the cheapest possible network. In the design of electronic circuitry Given n pins that should be electrically equivalent by wiring them together To interconnect a set of n pins, n-1 wires can be used, each connecting two pins We want the least possible length of wire to save cost

3.  2004 SDU 3 Minimum Spanning Tree-- The Definition Given a connected undirected graph G = (V, E) and a weight w(e) to each edge e of G, a minimum spanning tree T of G is a spanning tree with minimum total edge weight Note: (1). A spanning tree of a graph is a tree that connects all vertices. (2). A connected undirected graph may have many different spanning trees (3). The minimum spanning tree may not be unique

4.  2004 SDU 4 Minimum Spanning Tree Problem 5 4 3 3 6 1 6 9 8 3 7 1 2 7 5 4 3 3 6 1 6 9 8 3 7 1 2 7 The Problem: Input: A connected undirected graph with weight (G, W). Output: A minimum spanning tree T for G.

5. Page 5 Greedy Algorithms A greedy algorithm always makes the choice that looks best at the moment. It makes a local optimal choice in the hope that this choice will lead to a globally optimal solution. Greedy algorithms yield optimal solutions for many (but not all) problems.  2004 SDU 5

6. Page 6 The 0-1 Knapsack problem The 0-1 knapsack problem: N items, where the i-th item is worth v i dollars and weight w i pounds. 11 p 3 p 4p 58 p 8p 88p  v i and w i are integers. 3$ 6 $ 35$ 8$ 28$ 66$ The thief can carry at most W (integer) pounds. How to take as valuable a load as possible.  An item cannot be divided into pieces. The fractional knapsack problem: The same setting, but the thief can take fractions of items. W may not be integer. W  2004 SDU 6

7. Page 7 Solve the fractional Knapsack problem: Greedy on the value per pound v i /w i.  Each time, take the item with maximum v i /w i.  If exceeds W, take fractions of the item. Example: (1, 5$), (2, 9$), (3, 12$), (3, 11$) and w=4. v i /w i : 5 4.5 4.0 3.667 First: (1, 5$), Second: (2, 9$), Third: 1/3 (3, 12$) Total W: 1, 3, 4. Can only take part of item  2004 SDU 7

8. Page 8 Proof of correctness: (The hard part) Let X = i 1, i 2, …i k be the optimal items taken. Consider the item j : (v j, w j ) with the highest v /w. if j is not used in X (the optimal solution), get rid of some items with total weight w j (possibly fractional items) and add item j. (since fractional items are allowed, we can do it.)  Total value is increased. Why?  One more item selected by greedy is added to X Repeat the process, X is changed to contain all items selected by greedy WITHOUT decreasing the total value taken by the thief.  2004 SDU 8

9. Page 9 The 0-1 knapsack problem cannot be solved optimally by greedy Counter example: W=10 2 1.8 Items found (6pounds, 12dollars), (5pounds, 9 dollar), 1.8 1 1 (5pounds, 9 dollars), (3pounds, 3 dollars), (3 pounds, 3 dollars) If we first take (6, 12) according to greedy algorithm, then solution is (6, 12), (3, 3) (total value is 12+3=15). However, a better solution is (5, 9), (5, 9) with total value 18. Note: To show that a statement does not hold, we only have to give an example.  2004 SDU 9

10. Greedy algorithm analysis Ingredients of greedy algorithm for solving a problem:  Greedy-choice property: a global optimal solution can be arrived at by making a locally optimal(greedy) choice. (fractional knapsack yes, 0- 1knapsack no )  Optimal substructure: an optimal solution to the problem contains within it optimal solutions to sub-problems.(both fractional and 0-1 knapsack yes) Correctness proof  Inductively prove that after each step of the greedy algorithm, its solution is at least as good as any other algorithm’s. (Huffman code)  Gradually transform any solution to the one found by the greedy algorithm. (Fractional knapsack)  Discover a simple “structural” bound asserting that any possible solution must have a certain value (lower bound). Then show that the greedy algorithm always achieves that bound. (Interval partitioning)  2004 SDU 10

11.  2004 SDU 11 Idea of the Generic Algorithm for MST It grows the minimum spanning tree one edge at a time.  The algorithm manages a set of edges A, maintaining the following loop invariant: –Prior to each iteration, A is a subset of some minimum spanning tree. An edge (u, v) is a safe edge for A if A  {(u, v)} is also a subset of a minimum spanning tree, that is,  (u, v) can be correctly added to A without violating the invariant

12. Genetic algorithm Kruskal’s and Prim’s algorithms are implementations of the generic algorithm on how to maintain A and find the safe edge (u, v) for A. At any time, edges of Kruskal’s form some trees, and (u,v) is the least edge between these trees Edges of Prim’s form one tree, and (u,v) is the least connecting the tree and other vertices. Safe edge is the greedy choice. At any time, the sub- problem is the graph obtained by contracting any component of A as a vertex. The problem has both greedy-choice property and optimal-substructure property.  2004 SDU 12

13.  2004 SDU 13 1.A cut (X, Y) of a graph G = (V, E) is a partition of the vertex set V into two sets X and Y = V \ X. 2.An edge (u, v)  E is said to cross the cut (X, Y) if u ∈ X and v ∈ Y. 3.A cut (X, Y) respects a set A of edges if no edge in A crosses the cut. 4.An edge is a light edge crossing a cut if its weight is the minimum of any edge crossing the cut. Related Notions Set of red vertices X and set of green vertices Y form a cut (X, Y) of the graph. Blue edges cross the cut (X, Y) (X, Y) respects a set A of edges consisting onlyof red or green edges.

14.  2004 SDU 14 Determine Safe Edges for A Theorem 23.1  Let G = (V, E) be a connected, undirected graph with real-valued weight function w defined on E. Let A be a subset of E that is included in some minimum spanning tree for G, let (X, Y) be any cut of G that respects A, and let (u, v) be a light edge crossing (X, Y). Then, edge (u, v) is safe for A.

15.  2004 SDU 15 The Proof of Theorem 23.1 Proof: Let T be a MST including A, and assume that T does not contain the light edge (u, v). We shall construct another MST T’ that includes A  {(u, v)}, showing that (u, v) is a safe edge for A. The edge (u, v) forms a cycle with the edges on the path from u to v in T. Since u and v are on opposite sides of the cut (X,Y), there is at least one edge in T on the path p that also crosses the cut. Let (x, y) be any such edge. The edge (x, y) is not in A since the cut respects A. Since (x, y) is on the unique path from u to v in T, removing (x, y) breaks T into two components. Adding (u, v) reconnects them to form a new spanning tree T’ = T-{(x, y)}  {(u, v)}.

16.  2004 SDU 16 A Corollary Corollary 23.2  Let G = (V, E) be a connected, undirected graph with real-valued weight function w defined on E. Let A be a subset of E that is included in some minimum spanning tree for G, and let C = (V C, E C ) be a connected component in the forest G A = (V, A). If (u, v) is a light edge connecting C to some other component in G A, then (u, v) is safe for A. Note: Kruskal’s and Prim’s algorithms are based on the corollary.

17.  2004 SDU 17 Idea of the Kruskal’s 1.Initialize the forest, each vertex as a tree, A  . 2.Find the least weight edge (u, v) that connects any two trees in the forest. 3.Add (u, v) to A and make the two tree as one tree, number of trees in the forest decreases 1. 4.Repeat step 2 and 3 until A forms a spanning tree.

18.  2004 SDU 18 Kruskal ’ s Algorithm

19.  2004 SDU 19 2 1 1 12 8 5 3 9 4 3 2 5 1 6 a b c d e f g i h (a, d):1(h, i):1(c, e):1(f, h):2(g, h):2 (b, c):3(b, f):3(b, e):4(c, d):5(f, g):5 (e, i):6(d, g):8(a, b):9(c, f):12 2 1 1 12 8 5 3 9 4 3 2 5 1 6 a b c d e f g i h (a, d):1(h, i):1(c, e):1(f, h):2(g, h):2 (b, c):3(b, f):3(b, e):4(c, d):5(f, g):5 (e, i):6(d, g):8(a, b):9(c, f):12

20.  2004 SDU 20 2 1 1 12 8 5 3 9 4 3 2 5 1 6 a b c d e f g i h (a, d):1(h, i):1(c, e):1(f, h):2(g, h):2 (b, c):3(b, f):3(b, e):4(c, d):5(f, g):5 (e, i):6(d, g):8(a, b):9(c, f):12 2 1 1 12 8 5 3 9 4 3 2 5 1 6 a b c d e f g i h (a, d):1(h, i):1(c, e):1(f, h):2(g, h):2 (b, c):3(b, f):3(b, e):4(c, d):5(f, g):5 (e, i):6(d, g):8(a, b):9(c, f):12

21.  2004 SDU 21 2 1 1 12 8 5 3 9 4 3 2 5 1 6 a b c d e f g i h (a, d):1(h, i):1(c, e):1(f, h):2(g, h):2 (b, c):3(b, f):3(b, e):4(c, d):5(f, g):5 (e, i):6(d, g):8(a, b):9(c, f):12 a 2 1 1 12 8 5 3 9 4 3 2 5 1 6 b c d e f g i h (a, d):1(h, i):1(c, e):1(f, h):2(g, h):2 (b, c):3(b, f):3(b, e):4(c, d):5(f, g):5 (e, i):6(d, g):8(a, b):9(c, f):12

22.  2004 SDU 22 2 1 1 12 8 5 3 9 4 3 2 5 1 6 a b c d e f g i h (a, d):1(h, i):1(c, e):1(f, h):2(g, h):2 (b, c):3(b, f):3(b, e):4(c, d):5(f, g):5 (e, i):6(d, g):8(a, b):9(c, f):12 2 1 1 12 8 5 3 9 4 3 2 5 1 6 a b c d e f g i h (a, d):1(h, i):1(c, e):1(f, h):2(g, h):2 (b, c):3(b, f):3(b, e):4(c, d):5(f, g):5 (e, i):6(d, g):8(a, b):9(c, f):12

23.  2004 SDU 23 2 1 1 12 8 5 3 9 4 3 2 5 1 6 a b c d e f g i h (a, d):1(h, i):1(c, e):1(f, h):2(g, h):2 (b, c):3(b, f):3(b, e):4(c, d):5(f, g):5 (e, i):6(d, g):8(a, b):9(c, f):12 2 1 1 12 8 5 3 9 4 3 2 5 1 6 a b c d e f g i h (a, d):1(h, i):1(c, e):1(f, h):2(g, h):2 (b, c):3(b, f):3(b, e):4(c, d):5(f, g):5 (e, i):6(d, g):8(a, b):9(c, f):12

24.  2004 SDU 24 2 1 1 12 8 5 3 9 4 3 2 5 1 6 a b c d e f g i h (a, d):1(h, i):1(c, e):1(f, h):2(g, h):2 (b, c):3(b, f):3(b, e):4(c, d):5(f, g):5 (e, i):6(d, g):8(a, b):9(c, f):12 2 1 1 12 8 5 3 9 4 3 2 5 1 6 a b c d e f g i h (a, d):1(h, i):1(c, e):1(f, h):2(g, h):2 (b, c):3(b, f):3(b, e):4(c, d):5(f, g):5 (e, i):6(d, g):8(a, b):9(c, f):12

25.  2004 SDU 25 Time Complexity Analysis 1. O(V 2 +ElgV) when disjoint-set data structure are not used. Keep two arrays: c[u]: The component name of vertex u v[c]: The vertices in component c, is a list FIND-SET(u): Checks c[u] //O(1), total  (E) Union (u, v): Append list c[v] at the end of list c[u], and for each vertex in c[v], define c[] as c[u]. //O(V), total O(V 2 ) 2.When use disjoint-set data structure with union-by-rank Keep two arrays: p[u]: parent of u in the min-tree rank[u]: height of tree rooted at u FIND-SET (u): if p[u] = u, return u; else FIND-SET (p[u]); // O(lgV) Union (u, v): if rank (FIND-SET(u)) > rank (FIND-SET(v)), p [FIND- SET(v)]=FIND-SET(u); if rank (FIND-SET(u)) < rank(FIND-SET(v)), p[FIND-SET(u)]=FIND-SET(v); else p[FIND-SET(u)]=FIND-SET(v), rank[FIND-SET(v)]++; //O(1), total  (V) So total time is: O(ElgV) u v

26. 1.At any time, A is a subset of a MST 2.In the end, A is a MST Proof of 1. By induction on |A|. Base case A 0, of course true. Suppose 1 is true for A k-1. Suppose the kth edge added to A is e k =(u,v). W.L.O.G, suppose u  C, C is a connected component of G(V, A k-1 ). Since (u, v) is the smallest remaining edge, it must be a light edge connecting C to all other components of G(V, A k-1 ), so, it is safe for A k-1, and hence A k is subset of a MST. Correctness

27. eiei Proof of 2. By 1, in the end, A is a subset of a MST T. By contradiction suppose that A is not a MST. Then, T \ A  . Suppose e i  T\A. e i must be weightier than each edge in A, since otherwise e i should have been added to A. Then, after all the edges in A have been chosen, there must be a time that e i is considered, since e i  A do not contain a cycle, e i should be added to A at that time. A contradiction. Correctness (cont.)

28.  2004 SDU 28 Idea of the Prim 1.Start from a vertex r, add r to a vertex set U which is initialized to empty. 2.Find the least weight edge (u, v), u  U, v  V-U, add (u, v) to A, and add v to U.  A is the loop invariant. 3.Repeat 2 until A forms a spanning tree or U = V.

29.  2004 SDU 29 Prim ’ s Algorithm //(  (u), u) is added to A u is added to U

30.  2004 SDU 30 2 1 1 12 8 5 3 9 4 3 2 5 1 6 r b c d e f g i h 2 1 1 8 5 3 9 4 3 2 5 1 6 r b c d e f g i h 2 1 1 8 5 3 9 4 3 2 5 1 6 r b c d e f g i h 2 1 1 8 5 3 9 4 3 2 5 1 6 r b c d e f g i h

31.  2004 SDU 31 2 1 1 12 8 5 3 9 4 3 2 5 1 6 r b c d e f g i h 2 1 1 8 5 3 9 4 3 2 5 1 6 r b c d e f g i h a 2 1 1 8 5 3 9 4 3 2 5 1 6 b c d e f g i h r 2 1 1 8 5 3 9 4 3 2 5 1 6 r b c d e f g i h

32.  2004 SDU 32 2 1 1 12 8 5 3 9 4 3 2 5 1 6 r b c d e f g i h r 2 1 1 8 5 3 9 4 3 2 5 1 6 b c d e f g i h

33.  2004 SDU 33 Time Complexity Running time  Using binary min-heap –EXTRACT-MIN: return and delete root of the heap, put the last leaf as the new root, and search down to find its proper position. //O(lgV), total O(VlgV) –key(v)  w(u,v): check array p[v] to find v’s position in the heap, then change its key value, and search back to find its proper position in the heap. //O(lgV), total O(ElgV) –Total: O(E lg V)  Using Fibonacci heap O(E+VlgV)

34. 1: each time we add a safe edge (a light edge crossing (U, Q)). 2: Notice that since G is connected, at any time there is at least one vertex in Q whose key value is less than . Thus, from the second iteration, at each iteration, we always can add an edge into A. After V iterations, we get V-1 edges. Correctness

35.  2004 SDU 35