1. 15-211 Fundamental Data Structures and Algorithms Klaus Sutner April 20, 2004 Equivalence and Union-Find

2. Announcements  HW7 make sure to get your games in...  Quiz 3: Thursday April 22  Final Exam on Tuesday May 4, 5:30 pm  review session April 29

3. Chameleon Island

4. On a tropical island there are three kinds of chameleons perambulating themselves: red, green and blue. If a red and green chameleon meet, they both change color to blue, likewise for red/blue and green/blue. Initially there are 12 red, 13 green and 14 blue chameleons. Can the chameleons turn into a homogeneous population?

5. Brute Force We can compute this to death: use a digraph with nodes (r,g,b) and edges (r,g,b)  (r-1,g-1,b+2) (r,g,b)  (r-1,g+2,b-1) (r,g,b)  (r+2,g-1,b-1) provided that the numbers are non-negative. How many nodes are there? The starting configuration is (12,13,14) so the total number of animals is n = 39.

6. Reachability The number of nodes is C(39+2,2) = 820. We can simply use DFS or BFS to compute the nodes reachable from (12,13,14) and check if we run into one of (39,0,0), (0,39,0), (0,0,39). It turns out, we don't. OK but rather crude. Is there a more elegant solution? How about invariants?

7. Invariants If we suspect that some configuration cannot occur, we can try to prove this by finding some property P such that: - P holds on the initial configuration, - P is preserved in every single transition of the system, - P does not hold on the specific target configuration. Your favorite method: Induction.

8. Information Hiding For the chameleons, the key observation is that modulo 3 the three types of edges are all the same: (r,g,b)  (r+2,g+2,b+2) mod 3 Note that this quotient operation preserves paths, so it suffices to observe (0,1,2)  (2,0,1)  (1,2,0)  (0,1,2) Of course, we lose a lot of information but this is enough to answer the original question.

9. Equivalence Relations There is an important idea hiding here: identify objects that are distinct but share some property. Modeled by a binary relation ~ on some carrier set A. reflexivex ~ x symmetricx ~ y  y ~ x transitivex ~ y  y ~ z  x ~ z

10. Examples congruence modulo m polygons of same area people of same age reachable in a undirected graph programs with same input/output behavior

11. Classes and Quotients Equivalence class of x: [x] = { y | y ~ x } Quotient: A/~ = { [x] | x in A } Index of ~: cardinality of A/~ Note that equivalence classes form a partition of A. In fact, partitions and equivalence relations are essentially the same.

12. Kernel Relations Given any function f : A  B we can form a relation K(f) by defining x K(f) y iff f(x) = f(y). Note that K(f) is always an equivalence relation. If R = K(f) we say that f is a (kernel) representation for R. (As opposed to list of pairs, adjacency matrix, adjacency matrix, … ).

13. Everybody is a Kernel Claim: All equivalence relations are of this form. In fact we can choose a function f : A  A. This is intuitively clear: we map all x in an equivalence class to some special member of that class. (Take a course in set theory if you want to know why there are problems with this.)

14. Computational Aspects The last observation allows us to represent an equivalence relation on [n] = {1,2,...,n} compactly: Instead of n 2 bits for a Boolean matrix representation we only need n integers for an array representing f. We can still check if two elements are equivalent in O(1) time. What is a good choice for the function f?

15. The Canonical Representation f(x) = min( z | z ~ x ) For example, if x ~ y iff x = y mod 3 on [10] we get x 1 2 3 4 5 6 7 8 9 10 f(x) 1 2 3 1 2 3 1 2 3 1

16. Index x 1 2 3 4 5 6 7 8 9 10 f(x) 1 1 3 1 5 1 1 5 3 1 Question: How does one compute the index of R from a kernel representation for R?

17. Refinement Suppose we have two equivalence relations R and S on [n] both given by their canonical kernel function. How do we compute their intersection x int(R,S) y iff x R y and x S y In other words, we want to compute the canonical representation for T = int(R,S). Example 1 2 3 4 5 6 7 8 R1 1 3 1 5 3 3 1 S1 1 3 3 5 5 5 1 T1 1 3 4 5 6 6 1

18. Code initialize H hashmap; for x = 1,...,n do if( (R[x],S[x]) is undefined ) then T[x] = H( (R[x],S[x]) ) = x; else T[x] = H( (R[x],S[x]) ) Expected linear time. Could also replace H by a n  n array (interesting if the initialization cost can be amortized).

19. Small Machines

20. Recall: Finite State Machines Recall that a finite state machine is essentially a lookup table with one entry for each symbol/state combination, plus an initial state and some final states.

21. An Experiment Think of the finite state machine as a black box. Suppose you can perform the following experiment as often as you wish: - reset the machine to some state p, - feed some string to the machine, and - observe whether the resulting state is final. Of course, you are not allowed to open up the machine. Which states could be distinguished from each other by this experiment?

22. A Black Box Call p and q (behaviorally) equivalent if they cannot be distinguished. Claim: 1. We can distinguish final from non-final states. 2. If we can distinguish p and q and d(p',a) = p and d(q',a) = q then we can also distinguish p' and q'.

23. Who Cares? If two states are equivalent, we may as well collapse them into a single state. More precisely, we can replace the state set Q by Q/~. The latter may be much smaller, so we can build potentially smaller machines. Fact: One can show that the smallest possible finite state machine (for a given language) can be obtained this way.

24. Example a b b a,b a b a

25. Computing Behavioral Equiv. How do we actually compute the behavioral equivalence relation ~? Refine partitions. Initially only distinguish between F and Q – F. Then refine the partition as follows: Suppose we have an equivalence relation E. Define E' by p E' q iff p E q and for all symbols s: d(p,s) E d(q,s).

26. Computing Behavioral Equiv. But that's just a intersection operation: Define p E s q iff d(p,s) E d(q,s). Then E' = int( E, E a, E b,... ). When E' = E for the first time we have E = ~. Can be computed in O( k n 2 ) steps where n is the number of states and k the number of input symbols.

27. Example 1 a b b a,b a 2 34 5 6 1 2 3 4 5 6 init1 1 3 3 1 1 a1 1 1 1 1 1 b3 3 1 1 1 1 1 1 3 3 5 5 a1 1 5 5 5 5 b3 3 5 5 5 5 1 1 3 3 5 5

28. Dynamic Equivalence Relations

29. Recall: Mazes  Think about a grid of rooms separated by walls.  Each room can be given a name. abcd hgfe ijkl ponm Randomly knock out walls until we get a good maze.

30. The Party Problem You arrive at a party. As usual, there are separate groups of people standing around. In each group people talk to each other, but they don't talk to anyone outside of the group. You scan the groups, find someone that you know and join the corresponding group. If someone in another group knows you too, the two groups merge. How do we figure out the groups given a list of “is- friend-of” relations. The list is revealed step by step, we don't have access to the whole list from the start.

31. Dynamic E-Relations So far we have only dealt with static equivalence relations: the whole relation is given from the start and we can represent it by the canonical kernel function. Often that is not the case: all we have is knowledge about some equivalent pairs (x,y) of elements. The corresponding equivalence relation is thus given implicitely. This is really a closure problem: we have some (arbitrary) relation R and we want to compute the least equivalence relation eqc(R) that contains R.

32. Say What? R is arbitrary. We want S such that - x R y implies x S y - S is reflexive, symmetric and transitive - S is the coarsest such relation. Thus x S y only if this is forced by R and the equivalence condition. We do not frivolously identify elements.

33. Transitivity Making S reflexive and symmetric is no problem: we can just make R reflexive and symmetric. The difficult part is transitivity: Whenever there is a chain x 1 R x 2 R x 3... x n-1 R x n we need to set x 1 S x n.

34. Static is Easy If R is static this an old problem: Think of R as a graph and use DFS/BFS or Warshall. But what to do when the pairs in R pop up one after the other?

35. Kernel Schmernel Suppose we have the canonical kernel representation f for S. If we get another pair (x,y), how can we update S? If already f(x) = f(y) we're OK. But otherwise we have to scan the whole array to update the the entries affected by setting x equivalent to y. Takes time linear in n. Problem: Our representation is too uptight.

36. Fixed Points We need to relax the conditions on f a little. But how? Let FP(f,x) be the element z such that f(z) = z f k (x) = z for some k. Needless to say, fixed points do not exist in general, but we will make sure that f is constructed properly so that there is no problem.

37. FP versus EQ Let's say that f represents relation R if x R y iff FP(f,x) = FP(f,y). Clearly R has to be an equivalence relation. Note that the canonical kernel function would work here. But the whole point is that many other functions also work. And that makes it much easier to update. Also note: a query “x R y?” is no longer O(1) but a priori only O(n).

38. Testing Equivalence To test whether x is equivalent to y we do x' = FP(f,x); y' = FP(f,y); return ( x' == y' ); Running time is clearly O(n). But if we use a “good” f it can be close to O(1).

39. Updating Equivalence Suppose we are told that x is equivalent to y. To update, do the following: x' = FP(f,x); y' = FP(f,y); if( x' != y' ) then f[x'] = y' or f[y'] = x'; Picking the right alternative will be important for running time.

40. Union-Find In the world of programming the key operations are called - find(x) return the fixed point - union(x,y) union the classes of x and y So far, this is clever but not too exciting: both operations may be linear in n. We need to be more careful about how to perform the union operation. Note that our definition of representation gives us a lot of leeway.

41. Example {1} {2} {3} {4} {5} {6} {7} {1} {2,3} {4} {5} {6} {7} {1} {2,3,4} {5} {6} {7} {1} {2,3,4} {5,6} {7} {1} {2,3,4,5,6} {7} union(2,3) union(3,4) union(5,6) union(6,3) {1} {2,3,4,5,6} {7} union(2,6)

42. Think Tree It is helpful to think of the representing function f as a rooted tree. 1 3 0 4 2 5 1 3 0 4 2 5

43. Keeping the Trees Shallow If we think of f as a collection of rooted trees it is natural to try to keep the depth of these trees small. Several plausible strategies: Union by depth: attach more shallow tree to deeper one. Union by size: attach smaller tree to larger one.

44. A Trick: Path Compression Since we have to traverse a path from a node to the root we might as well smash all the nodes on that path up to the root. E.g., find(0) would produce: 1 3 0 4 2 5 1 0 4 2 5 3

45. How Hard to Implement? One might wonder how hard it is to code all these tricks (without union by size/depth and path compressions the code is nearly trivial). Also, what is the actual payoff in the end? As it turns out, the code is really simple, and the payoff is tremendous.

46. The Code

47. All the code class UnionFind { int[] u; UnionFind(int n) { u = new int[n]; for (int i = 0; i < n; i++) u[i] = -1; } int find(int i) { int j,root; for (j = i; u[j] >= 0; j = u[j]) ; root = j; while (u[i] >= 0) { j = u[i]; u[i] = root; i = j; } return root; } void union(int i,int j) { i = find(i); j = find(j); if (i !=j) { if (u[i] < u[j]) { u[i] += u[j]; u[j] = i; } else { u[j] += u[i]; u[i] = j; } }

48. The UnionFind class class UnionFind { int[] u; UnionFind(int n) { u = new int[n]; for (int i = 0; i < n; i++) u[i] = -1; } int find(int i) {... } void union(int i,int j) {... } }

49. Iterative find int find(int i) { int j, root; for (j = i; u[j] >= 0; j = u[j]); root = j; while (u[i] >= 0) { j = u[i]; u[i] = root; i = j; } return root; }

50. union by size void union(int i,int j) { i = find(i); j = find(j); if (i != j) { if (u[i] < u[j]) { u[i] += u[j]; u[j] = i; } else { u[j] += u[i]; u[i] = j; } }

51. Time bounds  Variables  M operations.N elements.  Algorithms  Simple forest representation Worst: find O(N). mixed operations O(MN). Average: tricky  Union by height; Union by size Worst: find O(log N). mixed operations O(M log N). Average: mixed operations O(M) [see text]  Path compression in find Worst: mixed operations: “nearly linear” [analysis in 15-451]