1. Minimum Spanning Trees Definition of MST Generic MST algorithm Kruskal's algorithm Prim's algorithm Binary Search Trees1

2. 2 Problem: Laying Telephone Wire Central office

3. 3 Wiring: Naïve Approach Central office Expensive!

4. 4 Wiring: Better Approach Central office Minimize the total length of wire connecting the customers

5. Definition of MST Let G=(V,E) be a connected, undirected graph. For each edge (u,v) in E, we have a weight w(u,v) specifying the cost (length of edge) to connect u and v. We wish to find a (acyclic) subset T of E that connects all of the vertices in V and whose total weight is minimized. Since the total weight is minimized, the subset T must be acyclic (no circuit). Thus, T is a tree. We call it a spanning tree. The problem of determining the tree T is called the minimum-spanning-tree problem. Binary Search Trees5

6. Application of MST: an example In the design of electronic circuitry, it is often necessary to make a set of pins electrically equivalent by wiring them together. To interconnect n pins, we can use n-1 wires, each connecting two pins. We want to minimize the total length of the wires. Minimum Spanning Trees can be used to model this problem. Binary Search Trees6

7. Electronic Circuits: Binary Search Trees7

8. Electronic Circuits: Binary Search Trees8

9. 9 Here is an example of a connected graph and its minimum spanning tree: a b h cd e fg i 4 87 9 10 14 4 2 2 6 1 7 11 8 Notice that the tree is not unique: replacing (b,c) with (a,h) yields another spanning tree with the same minimum weight.

10. Growing a MST(Generic Algorithm) Set A is always a subset of some minimum spanning tree. This property is called the invariant Property. An edge (u,v) is a safe edge for A if adding the edge to A does not destroy the invariant. A safe edge is just the CORRECT edge to choose to add to T. Binary Search Trees10 GENERIC_MST(G,w) 1A:={} 2while A does not form a spanning tree do 3find an edge (u,v) that is safe for A 4A:=A ∪ {(u,v)} 5return A

11. How to find a safe edge We need some definitions and a theorem. A cut (S,V-S) of an undirected graph G=(V,E) is a partition of V. An edge crosses the cut (S,V-S) if one of its endpoints is in S and the other is in V-S. An edge is a light edge crossing a cut if its weight is the minimum of any edge crossing the cut. Binary Search Trees11

12. Binary Search Trees12 V-S↓ a b h cd e fg i 4 87 9 10 14 4 2 2 6 1 7 11 8 S↑S↑ ↑ S ↓ V-S This figure shows a cut (S,V-S) of the graph. The edge (d,c) is the unique light edge crossing the cut.

13. Theorem 1 ( Let G=(V,E) be a connected, undirected graph with a real- valued weight function w defined on E. Let A be a subset of E that is included in some minimum spanning tree for G. Let (S,V-S) be any cut of G such that for any edge (u, v) in A, {u, v}  S or {u, v}  (V-S). Let (u,v) be a light edge crossing (S,V-S). Then, edge (u,v) is safe for A. (The proof is not required) Proof: Let T opt be a minimum spanning tree.(Blue) A --a subset of T opt and (u,v)-- a light edge crossing (S, V-S) If (u,v) is NOT safe, then (u,v) is not in T. (See the red edge in Figure) There MUST be another edge (u ’, v ’ ) in T opt crossing (S, V-S).(Green) We can replace (u ’,v ’ ) in T opt with (u, v) and get another treeT ’ opt Since (u, v) is light, T ’ opt is also optimal. Binary Search Trees13 1 2 1 11 11 1

14. Corollary.2 Let G=(V,E) be a connected, undirected graph with a real- valued weight function w defined on E. Let A be a subset of E that is included in some minimum spanning tree for G. Let C be a connected component (tree) in the forest G A =(V,A). Let (u,v) be a light edge (shortest among all edges connecting C with others components) connecting C to some other component in G A. Then, edge (u,v) is safe for A. (For Kruskal ’ s algorithm) Binary Search Trees14

15. Prim's algorithm (basic part) MST_PRIM(G,w,r) 1.A={} 2.S:={r} (r is an arbitrary node in V) 3. Q=V-{r}; 4. while Q is not empty do { 5 take an edge (u, v) such that (1) u  S and v  Q (v  S ) and (u, v) is the shortest edge satisfying (1) 6 add (u, v) to A, add v to S and delete v from Q } Binary Search Trees15

16. Prim's algorithm MST_PRIM(G,w,r) 1for each u in Q do 2key[u]:=∞ 3parent[u]:=NIL 4 key[r]:=0; parent[r]=NIL; 5Q  V[Q] 6while Q!={} do 7u:=EXTRACT_MIN(Q); if parent[u]  Nil print (u, parent[u]) 8for each v in Adj[u] do 9if v in Q and w(u,v)<key[v] 10thenparent[v]:=u 11key[v]:=w(u,v) Binary Search Trees16

17. Grow the minimum spanning tree from the root vertex r. Q is a priority queue, holding all vertices that are not in the tree now. key[v] is the minimum weight of any edge connecting v to a vertex in the tree. parent[v] names the parent of v in the tree. When the algorithm terminates, Q is empty; the minimum spanning tree A for G is thus A={(v,parent[v]):v ∈ V-{r}}. Running time: O(||E||lg |V|). When |E|=O(|V|), the algorithm is very fast. Binary Search Trees17

18. Binary Search Trees18 a b h cd e fg i 4 87 9 10 14 4 2 2 6 1 7 11 8 a b h cd e fg i 4 87 9 10 14 4 2 2 6 1 7 11 8 The execution of Prim's algorithm (moderate part) the root vertex

19. Binary Search Trees19 a b h cd e fg i 4 87 9 10 14 4 2 2 6 1 7 11 8 a b h cd e fg i 4 87 9 10 14 4 2 2 6 1 7 11 8

20. Binary Search Trees20 a b h cd e fg i 4 879 10 14 4 2 2 6 1 7 11 8 a b h cd e fg i 4 87 9 10 14 4 2 2 6 1 7 11 8

21. Binary Search Trees21 a b h cd e fg i 4 87 9 10 14 4 2 2 6 1 7 11 8 a b h cd e fg i 4 87 9 10 14 4 2 2 6 1 7 11 8

22. Adaptable Heap: An adaptable heap is a heap that allows to change the key value of an entry in heap. – How to find the entry? – After changing the key value, how to make sure that the heap order holds. We use array representation to store a complete binary tree. Binary Search Trees22

23. Adaptable Heap: Entry: (key, id, value), where key is a real number (double), value is a string (String) and id is an integer (int) indicating the entry. Here id uniquely determine the entry. The value of id is from 1 to n if there are n entries in the priority queue. For MST application, id is the name of a vertex in a graph. Value is the name of the other vertex of the edge that we will added into A. Binary Search Trees23

24. Adaptable Heap: class ArrayNode { double key; String value; // value stored at this position int id; // identificsation of the entry id=1,2, 3, …, n add necessary methods here } public class MyAdaptableHeap{ protected ArrayNode T[]; // array of elements stored in the tree protected int maxEntries; // maximum number of entries protected int numEntries; //num of entries in the heap protected int location[]; // an array with size=T.length. here location[id] stores the rank of entry id in T[]. Add necessary methods here. In particular, we need Replace(id, k) (The key of entry id is updated to be key=k and we assume that the entry is already in the heap.) Binary Search Trees24

25. Adaptable Heap: Binary Search Trees25 11/6 13/4 12/7 15/314/1 16/2 17/5 1112131415 12345067 1617 46537 12345067 12 Array- based rep of the complete binary tree T[] location Replace (id, k) =replace (4, 25): go to location[4] to find the rank=3 of entry 4 and goto T[rank]=T[3] and update T[3]=25. After change T[3] from 13 to 25, the binary tree is not a heap any more.

26. The algorithms of Kruskal and Prim The two algorithms are elaborations of the generic algorithm. They each use a specific rule to determine a safe edge in line 3 of GENERIC_MST. In Kruskal's algorithm, – The set A is a forest. – The safe edge added to A is always a least-weight edge in the graph that connects two distinct components. In Prim's algorithm, – The set A forms a single tree. – The safe edge added to A is always a least-weight edge connecting the tree to a vertex not in the tree. Binary Search Trees26

27. Kruskal's algorithm (basic part) 1(Sort the edges in an increasing order) 2A:={} 3while E is not empty do { 3 take an edge (u, v) that is shortest in E and delete it from E 4 if u and v are in different components then add (u, v) to A } Note: each time a shortest edge in E is considered. Binary Search Trees27

28. Kruskal's algorithm MST_KRUSKAL(G,w) 1A:={} 2for each vertex v in V[G] 3do MAKE_SET(v) 4sort the edges of E by nondecreasing weight w 5for each edge (u,v) in E, in order by nondecreasing weight 6do if FIND_SET(u) != FIND_SET(v) 7thenA:=A ∪ {(u,v)} 8UNION(u,v) 9return A (Disjoint set is discussed in Chapter 21, Page 498) Binary Search Trees28

29. Disjoint-Set Keep a collection of sets S 1, S 2,.., S k, – Each S i is a set, e,g, S 1 ={v 1, v 2, v 8 }. Three operations – Make-Set(x)-creates a new set whose only member is x. – Union(x, y) – unites the sets that contain x and y, say, S x and S y, into a new set that is the union of the two sets. – Find-Set(x)-returns a pointer to the representative of the set containing x. – Each operation takes O(log n) time. Binary Search Trees29

30. Our implementation uses a disjoint-set data structure to maintain several disjoint sets of elements. Each set contains the vertices in a tree of the current forest. The operation FIND_SET(u) returns a representative element from the set that contains u. Thus, we can determine whether two vertices u and v belong to the same tree by testing whether FIND_SET(u) equals FIND_SET(v). The combining of trees is accomplished by the UNION procedure. Running time O(|E| lg (|E|)). ) Binary Search Trees30

31. Binary Search Trees31 a b h cd e fg i 4 87 9 10 14 4 2 2 6 1 7 11 8 The execution of Kruskal's algorithm (Moderate part) The edges are considered by the algorithm in sorted order by weight. The edge under consideration at each step is shown with a red weight number.

32. Binary Search Trees32 a b h cd e fg i 4 87 9 10 14 4 2 2 6 1 7 11 8 a b h cd e fg i 4 87 9 10 14 4 2 2 6 1 7 11 8

33. Binary Search Trees33 a b h cd e fg i 4 87 9 10 14 4 2 2 6 1 7 11 8 a b h cd e fg i 4 87 9 10 14 4 2 2 6 1 7 11 8

34. Binary Search Trees34 a b h cd e fg i 4 87 9 10 14 4 2 2 6 1 7 11 8 a b h cd e fg i 4 87 9 10 14 4 2 2 6 1 7 11 8

35. Binary Search Trees35 a b h cd e fg i 4 87 9 10 14 4 2 2 6 1 7 11 8 a b h cd e fg i 4 87 9 10 14 4 2 2 6 1 7 11 8

36. Binary Search Trees36 a b h cd e fg i 4 87 9 10 14 4 2 2 6 1 7 11 8 a b h cd e fg i 4 87 9 10 14 4 2 2 6 1 7 11 8

37. Disjoint Sets Data Structure

38. Disjoint Sets Some applications require maintaining a collection of disjoint sets. A Disjoint set S is a collection of sets where Each set has a representative which is a member of the set (Usually the minimum if the elements are comparable)

39. Disjoint Set Operations Make-Set(x) – Creates a new set where x is it’s only element (and therefore it is the representative of the set). Union(x,y) – Replaces by one of the elements of becomes the representative of the new set. Find(x) – Returns the representative of the set containing x

40. Analyzing Operations We usually analyze a sequence of m operations, of which n of them are Make_Set operations, and m is the total of Make_Set, Find, and Union operations Each union operations decreases the number of sets in the data structure, so there can not be more than n-1 Union operations

41. Disjoint-Set Forests Maintain A collection of trees, each element points to it’s parent. The root of each tree is the representative of the set We use two strategies for improving running time – Union by Rank – Path Compression c fba d

42. Make Set MAKE_SET(x) p(x)=x rank(x)=0 x

43. Find Set FIND_SET(d) if d != p[d] p[d]= FIND_SET(p[d]) return p[d] (Compression) c fba d

44. Union UNION(x,y) link(findSet(x), findSet(y)) link(x,y) if rank(x)>rank(y) then p(y)=x else p(x)=y if rank(x)=rank(y) then rank(y)++ c fba d w x c fba d w x

45. Analysis In Union we attach a smaller tree to the larger tree, results in logarithmic depth. O(log n). – Whenever height of a tree increases by 1, size doubles. Path compression can cause a very deep tree to become very shallow

46. The length of the Tree is O(log n) If n=1, it is obvious. Assume that it is true for n<=k. Let us consider a tree T of size k+1. Assume that T is constructed from T1 and T2. Then rank(T1)<=log (n1) and rank(T2)<=log (n2). Let n*=mim{n1, n2}. If rank(T1)  rank(T2), then rank(T)=max(rank(T1), rank(T2)}<=max{log (T1), log (T2)}<log (n1+n2)=log n. If rank(T1)=rank(T2), rank(T)=rank(T1)+1<=1+log n*=log (2n*)<=log(n) (note n=n1+n2).