1. Chapter 8 Disjoint Sets and Dynamic Equivalence

2. Equivalence Relations
An equivalence relation R has three properties:
reflexive: for any x, xRx is true
symmetric: for any x and y, xRy implies yRx
transitive: for any x, y, and z, xRy and yRz implies xRz
Example equivalence relations?
Born in the same year as
Live on same island as
In same department, where two people are related if they are in the same department
This relationship is an equivalence relation if someone was in only one department
An example of a relation in which town a is related to town b if traveling from a to b by road is possible.
This relationship is an equivalence relation if the roads are two-way.
Disjoint Sets 2

3. Example
Building electrical network for a new community. Want to make sure everybody has access to electricity with minimal cost.
Union – add wiring between two homes.
Find – Are you in same community as power supply?
Disjoint Sets 3

4. Disjoint Set Union/Find ADT
Union/Find ADT operations
union
find
create
destroy
Disjoint set equivalence property: every element of a Disjoint set structure belongs to exactly one set
Dynamic equivalence property: the sets can change after execution of a union
{1,4,8}
{7}
{6}
{5,9,10}
{2,3}
find(4)
Name of 8
union(2,6)
{2,3,6}
Disjoint Sets 4

5. Disjoint Set Union/Find More Formally
Given a set U = {a1, a2, … , an}
Maintain a partition of U, a set of subsets of U {S1, S2, … , Sk} such that:
each pair of subsets Si and Sj are disjoint:
together, the subsets cover U:
Union(a, b) creates a new subset which is the union of a’s subset and b’s subset
NOTE: outside agent decides when/what to union – Abstract Data Type (ADT) is just the bookkeeper
Find(a) returns a unique name for a’s subset
NOTE: set names are arbitrary! We only care that:
Find(a) == Find(b)  a and b are in the same subset
Note: we’ll assume that a union is actually passed the names of the subsets. So, there are no hidden finds inside a union.
Disjoint Sets 5

6. How implement? Union(1,4) Union(2,3)
Group ID 1 4 2 3 5 6 7 8 9 10 11
Union(1,4)
Union(2,3)
Every other element it its own set.
How long to find?
How long to union?
Disjoint Sets 6

7. Linked Lists?
How long to find?
How long to union?
Disjoint Sets 7

8. Up-Tree Intuition
Finding the representative member of a set is somewhat like the inverse of finding whether a given item exists in a set. Instead of using trees with pointers from each node to its children; let’s use trees with a pointer from each node to its parent. Note we never need to know the members of a set – just what set something is in.
Disjoint Sets 8

9. Question With this representation, how would you do a union?
Could you find out who is in the set and point them all to the parent (directly)?
Would you want to do that? (The doorbell rings…)
Disjoint Sets 9

10. Union-Find Up-Tree Data Structure
Each subset is an up-tree with its root as its representative member
All members of a given set are nodes in that set’s up-tree
It data isn’t linear, Hash table maps input data to the node associated with that data a c g h d b f i k e
Up-trees are not necessarily binary!
Disjoint Sets 10

11. Union-Find Up-Tree Data Structure can be stored as an array** b c d e f g h i j k a c g h d b f i k e
Up-trees are not necessarily binary!
Disjoint Sets 11

12. Find – name is at root find(f) find(e) a c g h d k b f i e
O(depth) e
Just traverse to the root!
runtime:
Disjoint Sets 12

13. Union union(a,c) a c g h d b f i k e
Just have one root point to the other!
runtime:
Disjoint Sets 13

14. Example (1/11)
union(b,e) a b c d e f g h i a b c d f g h i e

15. Example (2/11)
union(a,d) a b c d f g h i e a b c f g h i d e

16. Example (3/11)
union(a,b) a b c f g h i d e a c f g h i b d e

17. Example (4/11) find(d) = find(e) No union! f g h a b c i d e
While we’re finding e,
could we do anything else?

18. Example (5/11)
union(h,i) a c f g h i a c f g h b d i b d e e

19. Example (6/11)
union(c,f) a c f g h a c g h b d i b d f i e e

20. Example (7/11) union(e,f) find(e) find(f) union(a,c) f g h a b c i d e
Could we do a
better job on this union?

21. Example (8/11) find(f) find(i) union(c,h) c g h c g a f i a f h b d b

22. Example (9/11) find(e) = find(h) and find(b) = find(c)
So, no unions for either of these. 11 c g a f h b d i e

23. Example (10/11) a b c d e f find(d) find(g) union(c, g) g h i c g g c12 h i c g g c a f h a f h b d i b d i e e

24. Example (11/11)
find(g) = find(h)
So, no union. g c a f h b d i e

25. Disjoint set data structuref g h a b c i d e
A forest of up-trees can easily be stored in an array.
Also, if the node names are integers or characters, we can use a very simple, perfect hash.
Nifty storage trick! -1 1 2 7
0 (a) 1 (b) 2 (c) 3 (d) 4 (e) 5 (f) 6 (g) 7 (h) 8 (i)
up-index:
Disjoint Sets 25

26. How can we make find cheaper?
Disjoint Sets 26

27. Room for Improvement: Smart Union
Always makes the root of the larger tree the new root
Cuts down on the number of nodes at the lower level f g h a b c i d e c g h a g h a f i b d c i
I use weighted because it makes the analysis a bit easier; your book suggests height-based union, and that’s probably fine. b d e f
Could we do a
better job on this union? e
Smart union!
Disjoint Sets 27

28. Union by size
A simple improvement to the previous algorithm is to make the smaller tree (in terms of number of nodes) a subtree of the larger, breaking ties by any method.
This is called union-by-size.
How could we keep track of the number of nodes in a set?
Disjoint Sets 28

29. Disjoint Sets 29

30. Example shows the worst case tree possible after 15 union by size operations.
Disjoint Sets 30

31. Another idea – union by height
Another implementation is union-by-height in which we keep track of the height of the trees and perform union operations by making a shallower tree a subtree of the deeper tree.
Disjoint Sets 31

32. Union by height Code new runtime of union: new runtime of find:
UnionSets( int root1, int root2 )
{ assertIsRoot(root1);
assertIsRoot(root2);
if (s[root2] < s[root1]) s[root1]=root2;
else
{if(s[root1] == s[root2]) s[root1]--;
// since the rank is stored as a negative,
this makes the value have higher rank.
s[root2] = root1;
} // else
} //UnionSets
new runtime of union:
Union still takes constant time.
Find hasn’t changed, so it still takes O(depth)
But what is the maximum depth now?
new runtime of find:
Disjoint Sets 32

33. Weighted Union Find Analysis
Finds with weighted union are O(max up-tree height)
But, an up-tree of height h with weighted union must have at least 2h nodes
 2max height  n and
max height  log n
So, find takes O(log n)
Base case: h = 0, tree has 20 = 1 node
Induction hypothesis: assume true for h < h
A merge can only increase tree height by one over the smaller tree. So, a tree of height h-1 was merged with a larger tree to form the new tree. Each tree then has  2h-1 nodes by the induction hypotheses for a total of at least 2h nodes. QED.
When a node gets deeper, it only gets one deeper, and its up-tree must have been the smaller one in the merge.
That means that its tree doubled in size.
How many times can a tree double in size?
Log n.
So find takes O(log n).
Disjoint Sets 33

34. could we do a little tidying up?
Room for Improvement: Simple idea with enormous consequences Path Compression
Points everything along the path (that you were searching anyway) of a find to the root
Reduces the height of the entire access path to 1 f g h a b c i d e a c f g h i b d e
Path compression!
While we’re finding e,
could we do a little tidying up?
Disjoint Sets 34

35. Analogy This is a little bit like the doorbell analogy.
The doorbell rings. The person at the door is trick or treating. You get the items from the basement, but leave them near the door – so you don’t have to fetch them again. You place yourself near the door so you don’t have to travel to the door again.
Disjoint Sets 35

36. Path Compression Example
find(e) c g c g a f h a f h b e b
Note that everything along the path got pointed to c.
Note also that this is definitely not a binary tree! d d i i e
Disjoint Sets 36

37. Complexity of Weighted Union + Path Compression
Ackermann created a function A(x, y) which grows very fast! Inverse Ackermann function (x, y) grows very slooooowwwly … Single-variable inverse Ackermann function is called log* n How fast does log n grow? log n = 4 for n = 16 Let log(k) n = log (log (log … (log n))) Then, let log* n = minimum k such that log(k) n  1 How fast does log* n grow? log* n = 4 for n = log* n = 5 for n = (20,000 digit number!) How fast does (x, y) grow? Even sloooowwwer than log* n (x, y) = 4 for n far larger than the number of atoms in the universe (2300)
Disjoint Sets 37

38. Complexity of Weighted Union + Path Compression
Tarjan proved that m weighted union and find operations on a set of n elements have worst case complexity O(m(m, n))
This is essentially amortized constant time
In some practical cases, weighted union or path compression or both are unnecessary because trees do not naturally get very deep.
Disjoint Sets 38

39. Summary Disjoint Set Union/Find ADT
Simple ADT, simple data structure, simple code
Complex complexity analysis, but extremely useful result: essentially, constant time!
Lots of potential applications
Object-property collections
Index partitions: e.g. parts inspection
To say nothing of maze construction
In some applications, it may make sense to have meaningful (non-arbitrary) set names
Disjoint Sets 39