1. CSE 326: Data Structures: Set ADT
Lecture 18: Friday, Feb 21, 2003

2. Today’s Outline Making a “good” maze Disjoint Set Union/Find ADT
Up-trees
Weighted Unions
Path Compression

3. What’s a Good Maze? A bunch of adjacent rooms.
Just enough walls so that there’s one path from any one room to any other room
especially from the start to the finish
at least a bit random so it’s hard to figure out.

4. What’s a Good Maze? Connected Just one path between any two rooms
Random
A bunch of adjacent rooms.
Just enough walls so that there’s one path from any one room to any other room
especially from the start to the finish
at least a bit random so it’s hard to figure out.

5. The Maze Construction Problem
Given:
collection of rooms: V
connections between rooms (initially all closed): E
Construct a maze:
collection of rooms: V = V
designated rooms in, iV, and out, oV
collection of connections to knock down: E  E
such that one unique path connects every two rooms
Here’s a formalism which tries to approach that.

6. The Middle of the Maze
So far, a number of walls have been knocked down while others remain.
Now, we consider the wall between A and B.
Should we knock it down? When should we not knock it? A B
Let’s say we’re in the middle of knocking out walls to make a good maze.
How do we decide whether to knock down the current wall?

7. Maze Construction Algorithm
While edges remain in E
Remove a random edge e = (u, v) from E
How can we do this efficiently?
If u and v have not yet been connected
add e to E
mark u and v as connected
How to check connectedness efficiently?

8. Equivalence Relations
An equivalence relation R must have three properties
reflexive:
symmetric:
transitive:
Connection between rooms is an equivalence relation
Why?

9. Equivalence Relations
An equivalence relation R must have three properties
reflexive: for any x, xRx is true
symmetric: for any x and y, xRy implies yRx
transitive: for any x, y, and z, xRy and yRz implies xRz
Connection between rooms is an equivalence relation
any room is connected to itself
if room a is connected to room b, then room b is connected to room a
if room a is connected to room b and room b is connected to room c, then room a is connected to room c

10. Disjoint Set Union/Find ADT
name of set
Union/Find operations
{1,4,8}
{7}
{6}
{5,9,10}
{2,3}
find(4) 8
union(3,6)
{2,3,6}
create(v)
destroy(s)
union(s1, s2)
find(v)

11. Disjoint Set Union/Find ADT
Disjoint set partition property: every element of a DS U/F structure belongs to exactly one set with a unique name
Dynamic equivalence property: Union(a, b) creates a new set which is the union of the sets containing a and b

12. Example Construct the maze on the right
Initial (the name of each set is underlined):
{a}{b}{c}{d}{e}{f}{g}{h}{i}
Randomly select edge 1 a d b e c f g h i 3 2 4 11 10 1 7 9 6 8 12 5
Order of edges in blue

13. Example, First Step {a}{b}{c}{d}{e}{f}{g}{h}{i} find(b)  b
find(e)  e
find(b)  find(e) so:
add 1 to E
union(b, e)
{a}{b,e}{c}{d}{f}{g}{h}{i} a b c 3 2 4 11 10 1 7 9 6 8 12 5 d e f g h i
Order of edges in blue

14. Example, Continued {a}{b,e}{c}{d}{f}{g}{h}{i} a d b e c f g h i3 2 4 11 10 7 9 6 8 12 5
{a}{b,e}{c}{d}{f}{g}{h}{i}
Let’s do three more steps.
Merge a and d
merge a,d and b,e
check on d,e: same set! No merge.
Order of edges in blue

15. Up-Tree Intuition
Finding the representative member of a set is somewhat like the opposite of finding whether a given key exists in a set.
So, instead of using trees with pointers from each node to its children; let’s use trees with a pointer from each node to its parent.

16. Up-Tree Union-Find Data Structure
Each subset is an up-tree with its root as its representative member
All members of a given set are nodes in that set’s up-tree a c g h d b f i e
Up-trees are not necessarily binary!

17. Find find(f) find(e) a b c a c g h d e f d b f i g h i e10 c a c g h d e 7 f 11 9 8 d b f i
O(depth) g 12 h i e
Just traverse to the root!
runtime:

18. Union union(a,c) a b c a c g h d e f d b f i g h i e10 c a c g h d e f 11 9 8 d b f i
O(1) g 12 h i e
Just hang one root from the other!
runtime:

19. The Whole Example (1/11) a b c d e f union(b,e) g h i a b c d e f g h3 b 10 c
The Whole Example (1/11) 2 1 6 d 4 e 7 f 11 9 8
union(b,e) g 12 h 5 i a b c d e f g h i a b c d f g h i e

20. The Whole Example (2/11) a b c d e f union(a,d) g h i a b c d f g h i3 b 10 c
The Whole Example (2/11) 2 6 d 4 e 7 f 11 9 8
union(a,d) g 12 h 5 i a b c d f g h i e a b c f g h i d e

21. The Whole Example (3/11) a b c d e f union(a,b) g h i a b c f g h i d10 c
The Whole Example (3/11) 6 d 4 e 7 f 11 9 8
union(a,b) g 12 h 5 i a b c f g h i d e a c f g h i b d e

22. The Whole Example (4/11) a b c d e f find(d) = find(e) No union! g h i10 c
The Whole Example (4/11) 6 d 4 e 7 f 11 9 8
find(d) = find(e)
No union! g 12 h 5 i f g h a b c i d e
While we’re finding e,
could we do anything else?

23. The Whole Example (5/11) a b c d e f union(h,i) g h i a c f g h i a c10 c
The Whole Example (5/11) 6 d e 7 f 11 9 8
union(h,i) g 12 h 5 i a c f g h i a c f g h b d i b d e e

24. The Whole Example (6/11) a b c d e f union(c,f) g h i a c f g h a c g10 c
The Whole Example (6/11) 6 d e 7 f 11 9 8
union(c,f) g 12 h i a c f g h a c g h b d i b d f i e e

25. The Whole Example (7/11) a b c d e f find(e) find(f) union(a,c) g h i10 c
The Whole Example (7/11) d e 7 f
find(e)
find(f)
union(a,c) 11 9 8 g 12 h i f g h a b c i d e
Could we do a
better job on this union?

26. The Whole Example (8/11) a b c d e f find(f) find(i) union(c,h) g h i10 c
The Whole Example (8/11) d e f
find(f)
find(i)
union(c,h) 11 9 8 g 12 h i c g h c g a f i a f h b d b d i e e

27. The Whole Example (9/11) a b c d e f10 c
The Whole Example (9/11) d e f
find(e) = find(h) and find(b) = find(c)
So, no unions for either of these. 11 9 g 12 h i c g a f h b d i e

28. The Whole Example (10/11) a b c d e f find(d) find(g) union(c, g) g h12 h i c g g c a f h a f h b d i b d i e e

29. The Whole Example (11/11) a b c d e f find(g) = find(h) So, no union.
And, we’re done! g 12 h i g a b c c d e f a f h g h i b d i
Ooh… scary!
Such a hard maze! e

30. Nifty storage trick
A forest of up-trees can easily be stored in an array.
Node names  integers with a hash table a c g h b d f i e -1 1 2 7
0 (a) 1 (b) 2 (c) 3 (d) 4 (e) 5 (f) 6 (g) 7 (h) 8 (i)
up-index:

31. Implementation ID find(Object x) { assert(HashTable.contains(x));
typedef ID int;
ID up[10000];
ID find(Object x) {
assert(HashTable.contains(x));
ID xID = HashTable[x];
while(up[xID] != -1) {
xID = up[xID]; }
return xID;
Find runs in time linear in the maximum depth of an up-tree.
What is the maximum depth?
runtime: O(depth)

32. Implementation runtime: O(1) ID union(Object x, Object y) {
typedef ID int;
ID up[10000];
ID union(Object x, Object y) {
ID rootx = find(x);
ID rooty = find(y);
assert(rootx != rooty);
up[y] = x; }
Find runs in time linear in the maximum depth of an up-tree.
What is the maximum depth?
runtime: O(1)

33. Room for Improvement: Weighted Union
Always makes the root of the larger tree the new root
Often cuts down on height of the new up-tree f g h a b c i d e c g h a g h
I use weighted because it makes the analysis a bit easier; your book suggests height-based union, and that’s probably fine. a f i b d c i b d e f
Could we do a
better job on this union? e
Weighted union!

34. Weighted Union Code new runtime of union: new runtime of find:
typedef ID int;
ID union(Object x, Object y) {
rx = Find(x);
ry = Find(y);
assert(rx != ry);
if (weight[rx] > weight[ry]) {
up[ry] = rx;
weight[rx] += weight[ry]; }
else {
up[rx] = ry;
weight[ry] += weight[rx];
new runtime of union:
Union still takes constant time.
Find hasn’t changed, so it still takes O(depth)
But what is the maximum depth now?
new runtime of find:

35. Weighted Union Find Analysis
Finds with weighted union are O(max up-tree height)
But, an up-tree of height h with weighted union must have at least 2h nodes
, 2max height  n and
max height  log n
So, find takes O(log n)
Base case: h = 0, tree has 20 = 1 node
Induction hypothesis: assume true for h < h
and consider the sequence of unions.
Case 1: Union does not increase max height. Resulting tree still has  2h nodes.
Case 2: Union has height h’= 1+h, where h = height of each of the input trees. By induction hypothesis each tree has  2h-1 nodes, so the merged tree has at least 2h nodes. QED.
When a node gets deeper, it only gets one deeper, and its up-tree must have been the smaller one in the merge.
That means that its tree doubled in size.
How many times can a tree double in size?
Log n.
So find takes O(log n).

36. Alternatives to Weighted Union
Union by height
Ranked union (cheaper approximation to union by height)
See Weiss chapter 8.

37. Room for Improvement: Path Compression
Points everything along the path of a find to the root
Reduces the height of the entire access path to 1 a c f g h i a c f g h i b d b d e e
While we’re finding e,
could we do anything else?
Path compression!

38. Path Compression Example
find(e) c g c g a f h a f h
Note that everything along the path got pointed to c.
Note also that this is definitely not a binary tree! b e b d d i i e

39. Path Compression Code runtime: ID find(Object x) {
assert(HashTable.contains(x));
ID xID = HashTable[x];
ID hold = xID;
while(up[xID] != -1) {
xID = up[xID]; }
while(up[hold] != -1) {
temp = up[hold];
up[hold] = xID;
hold = temp;
return xID;
runtime:
Find clearly still takes O(depth), right?
But how deep can a tree get now?

40. Digression: Inverse Ackermann’s
Let log(k) n = log (log (log … (log n)))
Then, let log* n = minimum k such that log(k) n  1
How fast does log* n grow?
log* (2) = 1
log* (4) = 2
log* (16) = 3
log* (65536) = 4
log* (265536) = 5 (a 20,000 digit number!)
log* ( ) = 6
k logs
Let’s talk about some funky doped horses at the silicon downs.

41. Complex Complexity of Weighted Union + Path Compression
Tarjan (1984) proved that m weighted union and find operations with path commpression on a set of n elements have worst case complexity O(m log*(n))
actually even a little better!
For all practical purposes this is amortized constant time
OK, now, your book shows a proof that find now takes amortized O(log*n)
but actually, it’s even better than that: O(alpha(m,n)) time.
Basically amortized 1.
Can anyone think why the alpha doesn’t matter even in any good theoretical sense?
Remember B-Trees: they showed us that our model was bogus for large data sets. By the time alpha gets near 4, our data sets are already larger than we can even imagine.

42. To Do
Read Chapter 8
Graph Algorithms
Weiss Ch 9
Coming Up