1. 1 22c:31 Algorithms Union-Find for Disjoint Sets

2. Equivalence Relations Relation R : For every pair of elements (a, b) in a set S, a R b is either true or false. If a R b is true, then a is related to b. An equivalence relation satisfies: 1.(Reflexive) a R a 2.(Symmetric) a R b iff b R a 3.(Transitive) a R b and b R c implies a R c 2

3. Equivalence Classes Given a set of things… { grapes, blackberries, plums, apples, oranges, peaches, raspberries, lemons, bananas } …define the equivalence relation All citrus fruit is related, all berries, all stone fruits, … …partition them into related subsets { grapes }, { blackberries, raspberries }, { oranges, lemons }, { plums, peaches }, { apples }, { bananas } Everything in an equivalence class is related to each other. 3

4. Determining equivalence classes Idea: give every equivalence class a name –{ oranges, limes, lemons } = “like-ORANGES” –{ peaches, plums } = “like-PEACHES” –Etc. To answer if two fruits are related: –FIND the name of one fruit’s e.c. –FIND the name of the other fruit’s e.c. –Are they the same name? 4

5. Building Equivalence Classes Start with disjoint, singleton sets: –{ apples }, { bananas }, { peaches }, … As you gain information about the relation, take UNION of sets that are now related: –{ peaches, plums }, { apples }, { bananas }, … E.g. if peaches R limes, then we get –{ peaches, plums, limes, oranges, lemons } 5

6. Disjoint Union - Find Maintain a set of pairwise disjoint sets. –{3,5,7}, {4,2,8}, {9}, {1,6} Each set has a unique name, one of its members –{3,5,7}, {4,2,8}, {9}, {1,6} 6

7. Union Union(x,y) – take the union of two sets named x and y –{3,5,7}, {4,2,8}, {9}, {1,6} –Union(5,1) {3,5,7,1,6}, {4,2,8}, {9}, 7

8. Find Find(x) – return the name of the set containing x. –{3,5,7,1,6}, {4,2,8}, {9}, –Find(1) = 5 –Find(4) = 8 8

9. Example 9 S {1,2,7,8,9,13,19} {3} {4} {5} {6} {10} {11,17} {12} {14,20,26,27} {15,16,21}. {22,23,24,29,39,32 33,34,35,36} Find(8) = 7 Find(14) = 20 S {1,2,7,8,9,13,19,14,20 26,27} {3} {4} {5} {6} {10} {11,17} {12} {15,16,21}. {22,23,24,29,39,32 33,34,35,36} Union(7,20)

10. Implementing Disjoint Sets n elements, Total Cost of: m finds,  n-1 unions Target complexity: O(m+n) i.e. O(1) amortized per operation. O(1) worst-case for find as well as union would be great, but… Known result: find and union can be done practically in O(1) time 10

11. Implementing Disjoint Sets Observation: trees let us find many elements given one root… Idea: if we reverse the pointers (make them point up from child to parent), we can find a single root from many elements… Idea: Use one tree for each equivalence class. The name of the class is the tree root. 11

12. Up-Tree for Uinon/Find 12 1234567 Initial state 1 2 3 45 6 7 Intermediate state Roots are the names of each set.

13. Find Operation Find(x) follow x to the root and return the root 13 1 2 3 45 6 7 Find(6) = 7

14. Union Operation Union(i,j) - assuming i and j roots, point i to j. 14 1 2 3 45 6 7 Union(1,7)

15. Simple Implementation Array of indices 15 1 2 3 4 5 6 7 0107750 1 2 3 4 5 6 7 up Up[x] = 0 means x is a root.

16. Union 16 void Union(int[] up, int x, int y) { //precondition: x and y are roots// Up[x] := y } Constant Time!

17. Exercise Design Find operator –Recursive version –Iterative version 17 int Find(int[] up, int x) { //precondition: // x is in the range 1 to size ??? }

18. A Bad Case 18 123n … 1 23n Union(1,2) 1 2 3n Union(2,3) Union(n-1,n) … … 1 2 3 n :::: Find(1) n steps!!

19. Now this doesn’t look good  Can we do better? Yes! 1.Improve union so that find only takes O(log n) Union-by-size Reduces complexity to Θ(m log n + n) 2.Improve find so that it becomes even better! Path compression Reduces complexity to almost O(m + n) 19

20. Weighted Union –Always point the smaller tree to the root of the larger tree 20 1 2 3 45 6 7 W-Union(1,7) 2 4 1

21. Example Again 21 123n 1 23n Union(1,2) 1 2 3 n Union(2,3) Union(n-1,n) … … :::: 1 2 3n … Find(1) constant time …

22. Analysis of Weighted Union With weighted union an up-tree of height h has weight at least 2 h. Proof by induction –Basis: h = 0. The up-tree has one node, 2 0 = 1 –Inductive step: Assume true for all h’ < h. 22 h-1 Minimum weight up-tree of height h formed by weighted unions T1T1 T2T2 TW(T 1 ) > W(T 2 ) > 2 h-1 Weighted union Induction hypothesis W(T) > 2 h-1 + 2 h-1 = 2 h

23. Analysis of Weighted Union Let T be an up-tree of weight n formed by weighted union. Let h be its height. n > 2 h log 2 n > h Find(x) in tree T takes O(log n) time. Can we do better? 23

24. Worst Case for Weighted Union 24 n/2 Weighted Unions n/4 Weighted Unions

25. Example of Worst Cast (cont’) 25 After n -1 = n/2 + n/4 + …+ 1 Weighted Unions Find If there are n = 2 k nodes then the longest path from leaf to root has length k = log 2 (n). log 2 n

26. Elegant Array Implementation 26 1 2 3 45 6 7 2 4 1 0 2 10 1 7750 4 1 2 3 4 5 6 7 up weight

27. Weighted Union 27 void W_Union(int i,j){ //Pre: i and j are roots// int wi = weight[i]; int wj = weight[j]; if (wi < wj) { up[i] = j; weight[j] = wi + wj; } else { up[j] =i; weight[i] = wi + wj; }

28. Path Compression On a Find operation point all the nodes on the search path directly to the root. 28 1 2 3 45 6 7 1 23456 7 PC-Find(3) 89 10 89

29. Self-Adjustment Works 29 PC-Find(x) x

30. Exercise: Draw the result of Find(e) 30 fha b c d e g i

31. Path Compression Find 31 int PC_Find(int i) { int r = i; while (up[r] != 0) //find root r = up[r]; if (i != r) { //compress path// int k = up[i]; while (k != r) { up[i] = r; i = k; k = up[k]; } return r; }

32. Interlude: A Really Slow Function Ackermann’s function is a really big function A(x, y) with inverse  (x, y) which is really small. How fast does  (x, y) grow?  (x, y) = 4 for x far larger than the number of atoms in the universe (2 300 )  shows up in: –Computation Geometry (surface complexity) –Combinatorics of sequences 32

33. A More Comprehensible Slow Function log* x = number of times you need to compute log to bring value down to at most 1 E.g. log* 2 = 1 log* 4 = log* 2 2 = 2 log* 16 = log* 2 2 2 = 3 (log log log 16 = 1) log* 65536 = log* 2 2 2 2 = 4 (log log log log 65536 = 1) log* 2 65536 = …………… = 5 Take this:  (m,n) grows even slower than log* n !! 33

34. Disjoint Union / Find with Weighted Union and Path Compression Worst case time complexity for a W-Union is O(1) and for a PC-Find is O(log n). Time complexity for m  n operations on n elements is O(m log* n) –Log * n < 7 for all reasonable n. Essentially constant time per operation! 34

35. Amortized Complexity For disjoint union / find with weighted union and path compression. –average time per operation is essentially a constant. – worst case time for a PC-Find is O(log n). An individual operation can be costly, but over time the average cost per operation is not. 35

36. Cute Application Build a random maze by erasing edges. 36

37. Cute Application Pick Start and End 37 Start End

38. Cute Application Repeatedly pick random edges to delete. 38 Start End

39. Desired Properties None of the boundary is deleted Every cell is reachable from every other cell. There are no cycles – no cell can reach itself by a path unless it retraces some part of the path. 39

40. A Cycle, not allowed 40 Start End

41. A Good Solution 41 Start End

42. A Hidden Tree 42 Start End

43. Number the Cells 43 Start End 123456 789101112 131415161718 192021222324 252627282930 313233343536 We have disjoint sets S ={ {1}, {2}, {3}, {4},… {36} } each cell is unto itself. We have all possible edges E ={ (1,2), (1,7), (2,8), (2,3), … } 60 edges total.

44. Basic Algorithm S = set of sets of connected cells E = set of edges 44 While there is more than one set in S pick a random, unused edge (x,y) from E u := Find(x); v := Find(y); if u  v { Union(u,v); remove (x, y) from E } else mark (x, y) as “used”; All remaining members of E form the maze

45. Example Step 45 Start End 123456 789101112 131415161718 192021222324 252627282930 313233343536 S {1,2,7,8,9,13,19} {3} {4} {5} {6} {10} {11,17} {12} {14,20,26,27} {15,16,21}. {22,23,24,29,30,32 33,34,35,36} Pick (8,14)

46. Example 46 S {1,2,7,8,9,13,19} {3} {4} {5} {6} {10} {11,17} {12} {14,20,26,27} {15,16,21}. {22,23,24,29,39,32 33,34,35,36} Find(8) = 7 Find(14) = 20 S {1,2,7,8,9,13,19,14,20 26,27} {3} {4} {5} {6} {10} {11,17} {12} {15,16,21}. {22,23,24,29,39,32 33,34,35,36} Union(7,20)

47. Example 47 Start End 123456 789101112 131415161718 192021222324 252627282930 313233343536 S {1,2,7,8,9,13,19 14,20,26,27} {3} {4} {5} {6} {10} {11,17} {12} {15,16,21}. {22,23,24,29,39,32 33,34,35,36} Pick (19,20)

48. Example at the End 48 Start End 123456 789101112 131415161718 192021222324 252627282930 313233343536 S {1,2,3,4,5,6,7,… 36}

49. Problem 6-6 Suppose we are given the minimum spanning tree T of a given graph G (with n vertices and m edges) and a new edge e = (u; v) of weight w that we will add to G. Give an efficient algorithm to find the minimum spanning tree of the graph G + e. Your algorithm should run in O(n) time to receive full credit, although slower but correct algorithms will receive partial credit.

50. Problem 6-21 6-21. [5] Let G = (V,E) be a weighted acyclic directed graph with possibly negative edge weights. Design a linear-time algorithm to solve the single-source shortest-path problem from a given source v.

51. Short Paths in a DAG If we update the edges in topologically sorted order, we correctly compute the shortest paths. Reason: the only paths to a vertex come from vertices before it in the topological sort. 0146 132 9 s

52. Code for Shortest Paths in DAG 1DIJKSTRA(G, w, s) // DAG, weights, start vertex 2 for each vertex v in V[G] do 3d[v]   4  [v]  NIL 5d[s]  0 6Q  the topological order of V[G] starting at s 7while Q is not empty do 8u = EXTRACT-FIRST(Q) 9 for each vertex v in Adj[u] 10 d[v] = min{ d[v], d[u]+w(u, v)} 11 update  [v] if necessay

53. Shortest Paths in a DAG Theorem: For any vertex u in a DAG, if all the vertices before u in a topological sort of the dag have been updated, then d[u] is  (s,u), the shortest distance from s to u. Proof: By induction on the position of a vertex in the topological sort. Base case: d[s] is initialized to 0. Inductive hypothesis: Assume all vertices before u have been updated, and for all such vertices v, d[v] is  (s,v), the shortest distance from s to v. (continued)

54. Proof, Continued Inductive case: Some edge (v,u) where v is before u, must be on the shortest path to u, since there are no other paths to u. When v was updated, we set d[u] to d[v]+w(v,u) =  (s,v) + w(v,u) =  (s,u) Running time:  (V+E), same as topological sort