2. Data Structures & Algorithms Union-Find Example Richard Newman

3. Steps to Develop an Algorithm  Define the problem – model it  Determine constraints  Find or create an algorithm to solve it  Evaluate algorithm – speed, space, etc.  If algorithm isn’t satisfactory, why not?  Try to fix algorithm  Iterate until solution found (or give up)

4. Dynamic Connectivity Problem Given a set of N elements Support two operations: Connect two elements Given two elements, is there a path between them?

5. Example Connect (4, 3) Connect (3, 8) Connect (6, 5) Connect (9, 4) Connect (2, 1) Are 0 and 7 connected (No) Are 8 and 9 connected (Yes) 01234 56789

6. Example (con’t) Connect (5, 0) Connect (7, 2) Connect (6, 1) Connect (1, 0) Are 0 and 7 connected (Yes) Now consider a problem with 10,000 elements and 15,000 connections…. 01234 56789

7. Modeling the Elements Various interpretations of the elements:  Pixels in a digital photo  Computers in a network  Socket pins on a PC board  Transistors in a VLSI design  Variable names in a C++ program  Locations on a map  Friends in a social network …… Convenient to just number 0 to N-1 Use as array index, suppress details

8. Modeling the Connections Assume “is connected to” is an equivalence relation  Reflexive: a is connected to a  Symmetric: if a is connected to b, then b is connected to a  Transitive: if a is connected to b, and b is connected to c, then a is connected to c

9. Connected Components  A connected component is a maximal set of elements that are mutually connected (i.e., an equivalence set) 01234 56789 {0} {1,2} {3,4,8,9} {5,6} {7}

10. Implementing the Operations Recall – connect two elements, and answer if two elements have a path between them  Find: in which component is element a?  Union: replace components containging elements a and b with their union  Connected: are elements a and b in the same component?

11. Example Union(1,6) 01234 56789 {0} {1,2} {3,4,8,9} {5,6} {7} 01234 56789 {0} {1,2,5,6} {3,4,8,9} {7} Components?

12. Union-Find Data Type Goal: Design an efficient data structure for union-find  Number of elements can be huge  Number of operations can be huge  Union and find operations can be intermixed public class UF UF int(N); voidunion(int a, int b); intfind(int a); booleanconnected(int a, int b); ;;

13. Dynamic Connectivity Client  Read in number of elements N from stdin  Repeat: –Read in pair of integers from stdin –If not yet connected, connect them and print out pair read input int N while stdin is not empty read in pair of ints a and b if not connected (a, b) union(a, b) print out a and b ;;

14. Quick-Find  Data Structure –Integer array id[] of length N –Interpretation: id[a] is the id of the component containing a 01234 56789 i: 0 1 2 3 4 5 6 7 8 9 id[i]: 0 1 1 4 4 5 5 7 4 4

15. Quick-Find  Data Structure –Integer array id[] of length N –Interpretation: id[a] is the id of the component containing a  Find: what is the id of a?  Connected: do a and b have the same id?  Union: Change all the entries in id that have the same id as a to be the id of b.

16. Quick-Find Union(1,6) 01234 56789 i: 0 1 2 3 4 5 6 7 8 9 id: 0 1 1 4 4 5 5 7 4 4 i: 0 1 2 3 4 5 6 7 8 9 id: 0 5 5 4 4 5 5 7 4 4 It works – so is there a problem? Well, there may be many values to change, and many to search!

17. Quick-Find  Quick-Find operation times –Initialization takes time O(N) –Union takes time O(N) –Find takes time O(1) –Connected takes time O(1)  Union is too slow – it takes O(N 2 ) array accesses to process N union operations on N elements

18. Quadratic Algos Do Not Scale!  Rough Standards (for now) –10 9 operations per second –10 9 words of memory –Touch all words in 1 second (+/- truism since 1950!)  Huge problem for Quick-Find:  10 9 union commands on 10 9 elements  Takes more than 10 18 operations  This is 30+ years of computer time! 

19. Quadratic Algos Do Not Scale!  They do not keep pace with technology  New computer may be 10x as fast  But it has 10x as much memory  Want to solve problems 10x as big  With quadratic algorithm, it takes… … 10 x as long!!! 

22. Quick-Union  Data Structure –Integer array id[] of length N –Interpretation: id[a] is the parent of a –Component is root of a = id[id[…id[a]…]] (fixed point) 01 2 3 9 5 6 7 84 i: 0 1 2 3 4 5 6 7 8 9 id[i]: 0 1 1 3 3 5 5 7 3 4

23. Quick-Union  Data Structure – What is root of tree of a? – Do a and b have the same root? – Set id of root of b’s tree to be root of a’s tree 01 2 3 9 5 6 7 84 i: 0 1 2 3 4 5 6 7 8 9 id[i]: 0 1 1 3 3 5 5 7 3 4  –Find: –Connected: –Union:

24. Quick-Union 01 2 3 9 5 6 7 84 i: 0 1 2 3 4 5 6 7 8 9 id[i]: 0 1 1 3 3 5 5 7 3 4 –Find 9 –Connected 8, 9: –Union 7,5 5 Only ONE value changes! = FAST

25. Quick-Union  Quick-Union operation times (worst case) –Initialization takes time O(N) –Union takes time O(N) (must find two roots) –Find takes time O(N) –Connected takes time O(N)  Now union AND find are too slow – it takes O(N 2 ) array accesses to process N operations on N elements

26. Quick-Find/Quick-Union  Observations:  Problem with Quick-Find is unions –May take N array accesses –Trees are flat, but too expensive to keep them flat!  Problem with Quick-Union –Trees may get tall –Find (and hence, connected and union) may take N array accesses

27. Weighted Quick-Union  Make Quick-Union trees stay short!  Keep track of tree size  Join smaller tree into larger tree –May alternatively do union by height/rank –Need to keep track of “weight” b a a b Quick-Union may do this But we always want this

28. Weighted Quick-Union  Weighted Quick-Union operation times –Initialization takes time O(N) –Union takes time O(1) (given roots) –Find takes time O(depth of a) –Connected takes time O(max {depth of a, b})  Proposition: Depth of any node x is at most lg N Pf: What causes depth of x to increase?

29. Weighted Quick-Union  Proposition: Depth of any node x is at most lg N Pf: What causes depth of x to increase? Only union! And if x is in smaller tree. So x’s tree must at least double in size each time union increases x’s depth Which can happen at most lg N times. (Why?)

30. Next – Lecture 3  Read Chapter 2  Empirical analysis  Asymptotic analysis of algorithms  Basic recurrences