Trees Featuring Minimum Spanning Trees HKOI Training 2005 Advanced Group Presented by Liu Chi Man (cx)

2 Prerequisites  Asymptotic complexity  Set theory  Elementary graph theory  Priority queues

3 Assumptions  In the coming slides, we only talk about simple graphs  Extensions to multigraphs and pseudographs should be straightforward if you understand the materials well

4 Notations  A graph G = (V, E)  V is the set of vertices  E is the set of edges  |V| is the number of vertices in V  Sometimes we use V for convenience  |E| is the number of edges in E  Sometimes we use E for convenience

5 Uh… a long way to go  What is a Tree?  Disjoint Sets  Minimum Spanning Trees  Various Tree Topics

6 What is a Tree? A tree in Pokfulam, Hong Kong. (Photographer: cx)

7 What is a Tree?  In graph theory, a tree is an acyclic, connected graph  Acyclic means “with no cycles”

8 What is a Tree?  Properties of a tree:  |E| = |V| - 1  |E| =  (|V|)  Adding an edge between a pair of non- adjacent vertices creates exactly one cycle  Removing any one edge from the tree makes it disconnected

9 What is a Tree?  The following four conditions are equivalent:  G is connected and acyclic  G is connected and |E| = |V| - 1  G is acyclic and |E| = |V| - 1  Between any pair of vertices in G, there exists a unique path  G is a tree if at least one of the above conditions is satisfied

10 Other Properties of Trees  A tree is bipartite  A tree with at least two vertices has at least two leaves (vertices of degree 1)  A tree is planar

11 ZzZZzzz… bored to death?  What is a Tree?  Disjoint Sets  Minimum Spanning Trees  Various Tree Topics

12 The Union-Find Problem  Initially there are N objects, each in its own singleton set  Let’s label the objects by 1, 2, …, N  So the sets are {1}, {2}, …, {N}  There are two kinds of operations:  Merge two sets (Union)  Find the set to which an object belongs  Let there be a sequence of M operations

13 The Union-Find Problem  An example, with N = 4, M = 5  Initial: {1}, {2}, {3}, {4}  Union {1}, {3}  {1, 3}, {2}, {4}  Find 3. Answer: {1, 3}  Union {4}, {1,3}  {1, 3, 4}, {2}  Find 2. Answer: {2}  Find 1. Answer {1, 3, 4}

14 Disjoint Sets  We usually use disjoint-set data structures to solve the union-find problem  A disjoint set is simply a data structure that supports the Union and Find operations  How should a set be named?  Each set has its own representative element

15 Disjoint Sets  Implementation: Naïve arrays  Set[x] stores the representative of the set containing x, for 1 ≤ x ≤ N  :  Slight modifications give  U is the size of the union  Worst case is O(MN) for all M operations

16 Disjoint Sets  An example, with N = 4 x1234 Set[x]1234 Initial: x1234 Set[x]1214 Union 1 3: x1234 Set[x]1212 Union 2 4: x1234 Set[x]1111 Union 1 2: Find 4: Answer = 2 Find 4: Answer = 1

17 Disjoint Sets  Why is it slow?  Consider Union 1 2 on the table: x Set[x] x

18 Disjoint Sets  Idea for improvement:  When doing a Union, always update the set of smaller size  The overall running time can be shown to be O(M lgN)

19 Disjoint Sets  Implementation: Forest  A forest is a set of trees  Each set is represented by a rooted tree, with the root being the representative element Two sets: {1, 3, 5} and {2, 4, 6, 7}.

20 Disjoint Sets  Ideas  Find  Traverses from the vertex up to the root  Union  Merges two trees in the forest  That is, the root of one tree becomes a child of the other tree

21 Disjoint Sets  An example, with N = 4 Initial: Union 1 3: Union 2 4: Find 4:

22 Disjoint Sets  An example, with N = 4 (cont.) Union 1 2: Find 4:

23 Disjoint Sets  How to represent the trees?  Leftmost-Child-Right-Sibling (LCRS)?  Too complex!  We just need to traverse from bottom to top – parent-finding should be supported  An array Parent[] is enough  Parent[x] stores the parent vertex of x if x is not the root of a tree  Parent[x] stores x if x is the root of a tree

24 Disjoint Sets  The worst case is still O(MN ) overall  To improve the worst case time complexity, we employ two techniques  Union-by-rank  Path compression

25 Disjoint Sets  Union-by-rank  Recall that in our previous naïve array implementation, the worst case time bound can be improved by always updating the smaller set in a Union  Here, we make the “smaller” tree a subtree of the “larger” tree  To keep the resulting tree “small”  To compare the “sizes” of trees, we assign a rank to every tree in the forest

26 Disjoint Sets  Union-by-rank (cont.)  The rank of the tree is the height of the tree when it is created  A tree is created either at the beginning, or after a Union  That means we need to keep track of the trees’ ranks during the Unions

27 Disjoint Sets  Path compression  See also the solution for Symbolic Links (HKOI2005 Senior Final)  When we carry out a Find, we traverse from a vertex to the root  Why don’t we move the vertices closer to the root at the same time?  Closer to the root means shorter query time

28 Disjoint Sets  An example, with N = 7  Now we Find The root is

29 Disjoint Sets  We ignore the effect of path compression on tree ranks  Union-by-rank alone gives a running time of O(M lgN)  Together with path compression, the improved running time is O(M  (N))   is the inverse of the Ackermann's function   (n) is almost constant

30 When can I have my lunch?  What is a Tree?  Disjoint Sets  Minimum Spanning Trees  Various Tree Topics

31 Minimum Spanning Trees  Given a connected graph G = (V, E), a spanning tree of G is a graph T such that  T is a subgraph of G  T is a tree  T contains every vertex of G  A connected graph must have at least one spanning tree

32 Minimum Spanning Trees  Given a weighted connected graph G, a minimum spanning tree T* of G is a spanning tree of G with minimum total edge weight  Application: Minimizing the total length of wires needed to connect up a collection of computers

33 Minimum Spanning Trees  Two algorithms  Kruskal’s algorithm  Prim’s algorithm

34 Kruskal’s Algorithm  Algorithm  Initially T is an empty set  While T has less than |V|-1 edges Do  Choose an edge with minimum weight such that it does not form a cycle with the edges in T, and add it to T  The edges in T form a MST

35 Kruskal’s Algorithm  Algorithm (Detailed)  T is an empty set  Sort the edges in G by their weights  For (in ascending weight) each edge e Do  If T  {e} does not contain a cycle Then  Add e to T  Return T

36 Kruskal’s Algorithm  How to check if a cycle is formed?  Depth-first search (DFS)  Time complexity is O(E 2 )  Alternatively, a disjoint set!  The sets are connected components

37 Kruskal’s Algorithm  Algorithm (disjoint-set)  T is an empty set  Create sets {1}, {2}, …, {V}  Sort the edges in G by their weights  For (in ascending weight) each edge e Do  Let e connect vertices x and y  If Find(x)  Find(y) Then  Add e to T, then Union(Find(x), Find(y))  Return T

38 Kruskal’s Algorithm  The improved time complexity is O(ElgV)  In fact, the bottleneck is sorting!

39 Prim’s Algorithm  Algorithm  Initially T is a set containing any one vertex in V  While T has less than |V|-1 edges Do  Among those edges connecting a vertex in T and a vertex in V-T, choose one with minimum weight, and add it (with its two end vertices) to T  T is a minimum spanning tree

40 Prim’s Algorithm  Algorithm (Detailed)  T    Cost[x]   for all x  V  Marked[x]  false for all x  V  Pred[x]   for all x  V  Pick any u  V, and set Cost[u] to 0  Repeat |V| times  Let w be a vertex of minimum Cost among all unMarked vertices  Marked[w]  true  T  T  Pred[x]  For each unMarked vertex v such that (w, v)  E Do  If weight(w, v) < Cost[v] Then  Cost[v]  weight(w, v)  Pred[x]  { (w, v) }  Return T

41 Prim’s Algorithm  Time complexity: O(V 2 )  If a (binary) heap is used to accelerate the Find-Min operations, the time complexity is reduced to O(ElgV)  O(VlgV+E) if a Fibonacci heap is used

42 Minimum Spanning Trees  The fastest known algorithm runs in O(E  (V,E))

43 MST Extensions  Second-best MST  We don’t want the best!  Online MST  See IOI2003 Path Maintenance  Minimum Bottleneck Spanning Tree  The bottleneck of a spanning tree is the weight of its maximum weight edge  An algorithm that runs in O(V+E) exists

44 MST Extensions (NP-Hard)  Minimum Steiner Tree  No need to connect all vertices, but at least a given subset B  V  Degree-bounded MST  Every vertex of the spanning tree must have degree not greater than a given value K

45 Are you with me?  What is a Tree?  Disjoint Sets  Minimum Spanning Trees  Various Tree Topics

46 Various Tree Topics  Center and diameter  Tree isomorphism  Canonical representation  Prüfer code  Lowest common ancestor (LCA)

47 Supplementary Readings  Advanced:  Disjoint set forest (Lecture slides) Disjoint set forest  Prim’s algorithm Prim’s algorithm  Kruskal’s algorithm Kruskal’s algorithm  Center and diameter Center and diameter  Post-advanced (so-called Beginners):  Lowest common ancestor Lowest common ancestor  Maximum branching Maximum branching