How many edges does a tree of n nodes have? 1.2n-4 2.n(n-1)/2 3.n-1 4.n/2 5.Between n and 2n

Which of the following is true for all trees of n nodes 1.Connected 2.Has n/2 leaves 3.Every non-leaf has 2 children 4.At least one odd length cycle

Chapter 21 Chapter 23

Suppose you are laying the cables for a cable television company: Must hook every customer into the system Its more expensive to lay cables in some areas than others You need to minimize the total expense

We turn the problem into a graph: Nodes represent customers, routers and source Edges represent the cost of connecting two nodes (when possible) Objective: Pick the smallest total cost in edges that connect every node Notice: Solution will always be a tree

Given a weighted graph G: A spanning graph is a subgraph of G that uses all nodes of G A spanning tree is a subgraph of G that is also a tree. A minimum spanning tree is a spanning tree that has the minimum possible total weight

Prim’s MST 1.Start at any node in the graph –Mark the starting node as reached –Mark all the other nodes in the graph as unreached Right now, the Minimum cost Spanning Tree (MST) consists of the starting node New we expand the MST….. 2.Find an edge e with minimum cost in the graph that connects: A reached node x to an unreached node y 3.Add the edge e found in the previous step to the Minimum cost Spanning Tree Mark the unreached node y as reached 4.Repeat the steps 2 and 3 until all nodes in the graph have become reached

a d ghij e b f c adgh i ebfcj

Kruskal's MST Algorithm create a forest T (a set of trees), where each vertex in the graph is a separate tree create a set S containing all the edges in the graph while S is nonempty and T is not yet spanning –remove an edge with minimum weight from S –if that edge connects two different trees, then add it to the forest, combining two trees into a single tree, otherwise discard that edge.

a d ghij e b f c (a,d) (g,h) (c,f) (b,e) (i,j) (d,g) (d,h) (e,j) (h,i) (e,f) (b,c) (a,b) (a,h) (a,i) (e,i) (a,d)(g,h)(c,f)(b,e)(i,j)(d,g)(d,h)(e,j)(h,i) (e,f) (e,i) Edges sorted by length

a d ghij e b f c

a d ghij e b f c {a} {b} {c} {d} {e} {f} {g} {h} {i} {j} (a,d) (g,h) (c,f) (b,e) (i,j) (d,g) (d,h) (e,j) (h,i) (e,f) (b,c) (a,b) (a,h) (a,i) (e,i) {a,d} {g,h} {c,f} {b,e} {i,j} {a,d,g,h} {b,e,i,j} {a,b,d,e,g,h,i,j} {a,b,c,d,e,f,g,h,i,j}

Kruskal(Array EdgeList, int n) { sort(EdgeList) i, s  0 while (s < n-1) { if //EdgeList[i] is not contained within one // component TreeEdges[s] = EdgeList[i] // connect components s++ i++ } return TreeEdges; }

We have: –A set of elements X = {x 1,x 2,…,x n } –A set of sets S = {S 1,S 2,…,S k } where: S i  X (the sets are subsets of X) S i  S j =  if i  j (the sets are disjoint) S 1  S 2  …  S k = X (all the elements of X are in a set) In other words: Every element is in exactly one set We need: –To determine if two elements are in the same set –To be able to combine sets

We need the following operators: –AddElement(): Adds new element x n+1 and set S k+1 ={x n+1 } –Find( i ): Returns k such that x i  S k –Union( i,j ): Replaces sets Find(i) and Find(j) with a set Find(i)  Find(j) (if necessary)

Kruskal(Array EdgeList, int n) { sort(EdgeList) TreeEdges  () for i  1 to n U.AddElement(i) i, s  0 while (s < n-1) { if U.Find(EdgeList[i].s1) != U.Find(EdgeList[i].s2) TreeEdge[s] = EdgeList[i] U.Union(EdgeList[i].s1, EdgeList[i].s2) s++ i++ } return TreeEdges; }  m calls n-1 calls n calls to AddElement() n = number of nodes, m = number of edges Assumes: Edges are sorted (O(m log n))

Maintain an array A A[i] = k to show x i  S k S 1 ={x 1,x 3 } S 2 ={x 2,x 5,x 7 } S 3 ={x 4,x 6 } S 4 ={x 8 }

Using this implementation, what is the runtime for Union? 1.O(1) 2.O(log n) 3.O(n) 4.O(n log n) 5.O(n 2 )

AddElement(): Increase array size, set new element to k+1 –Single call:  (n) – n calls: O(n 2 ) Find(i): Return value of A[i] –Single call:  (1) – m calls:  (m) Union(i,j): Set A[l] = Find(i) for all l such that A[l] = Find(j) –Single call:  (n) – n calls:  (n 2 )

Kruskal(Array EdgeList, int n) { TreeEdges  () for i  1 to n U.AddElement(i) i, s  0 while (s < n-1) { if U.Find(EdgeList[i].s1) != U.Find(EdgeList[i].s2) TreeEdge[s] = EdgeList[i] U.Union(EdgeList[i].s1, EdgeList[i].s2) s++ i++ } return TreeEdges; }  2m calls =  (m) time n-1 calls =  (n 2 ) n calls to AddElement() =  (n 2 ) time n = number of nodes, m = number of edges Assumes: Edges are sorted (O(m log n)) Total:  (n 2 )

Kruskal using basic arrays: –AddElements(): n calls takes  (n) time –Find(i): 2m calls takes  (m) time –Union(i,j): n-1 calls takes  (n 2 ) time –Runtime:  (n + m + n 2 ) =  (n 2 )

To this point, we have identified a set by its index –Find(6) = 3 Alternative: Choose one representative element of the set –Arbitrarily chosen –Find(6) = 4 S 1 = {x 1,x 3 } S 2 = {x 2,x 5,x 7 } S 3 = {x 4,x 6 } S 4 = {x 8 } So Find(i) now returns j such that x j is the representative element of the set containing x i

Element Representation: A list node Set Representation: –A linked list of nodes –Maintain head and tail pointers to list –All nodes have a second reference back to the reference elements

x7x7 S 1 ={x 1,x 3 } S 2 ={x 2,x 5,x 7 } S 3 ={x 4,x 6 } S 4 ={x 8 } x1x1 x3x3 x2x2 x5x5 x4x4 x6x6 x8x8

With this implementation, what is the runtime for Find()? 1.O(1) 2.O(log n) 3.O(n) 4.O(n log n) 5.O(n 2 )

UnionFind(n): Create UnionFind object where |X| = n Find(j): Trace back pointer –1 call: O(1) time –m calls: O(m) time Union(i,j): Concatenate lists and update every node in the second list –1 call: O(n) –n calls: O(n 2 ) worst case Is this tight???

Kruskal(Array EdgeList, int n) { TreeEdges  () UnionFind U(n) i, s  0 while (s < n-1) { if U.Find(EdgeList[i].s1) != U.Find(EdgeList[i].s2) TreeEdge[s] = EdgeList[i] U.Union(EdgeList[i].s1, EdgeList[i].s2) s++ i++ } return TreeEdges; }  2m calls =  (m) time n-1 calls = O(n 2 ) (tight???) n = number of nodes, m = number of edges Assumes: Edges are sorted (O(m log n)) Total: O(n 2 ) (?)

Worse Case for n Unions for i <-- 1 to n AddElement(i) for i <-- n-1 to 1 Union(i, n) Number of redirects: … + (n-1) =  (n 2 ) Doesn’t help! x1x1 x n-1 xnxn x n-2 x1x1 x n-1 xnxn x n-2 x1x1 x n-1 xnxn x n-2

Simple augmentation of Union helps: –Keep track of set sizes (easy) –Always put the shorter list last Previous example: –Each Union involves a constant number of redirections –Previous sequence takes O(n) time Is there an example where it does not help as much?

Theorem: Any sequence of Union() calls over n elements will require O(n log n) time

Given any x i, how many times can its pointer be redirected? –Only redirected when it is in the smaller (or equal size) list –First time its redirected: It ends up in a list of at least two elements –Second time: Other list must have at least two elements New list must have at least four elements –Third time: New list must have at least eight elements

After the k th redirection of x i, it must be in a list of size at least 2 k –There are only n elements, so 2 k  n –If k  log 2 n then all sets are joined –Hence x i cannot be redirected more than log 2 n time If there are n elements: –Total number of redirections cannot be greater than n log 2 n

Worst Case Example x n-3 x n-1 xnxn x n-2 x1x1 x3x3 x4x4 x2x2 x n-3 x n-1 xnxn x n-2 x1x1 x3x3 x4x4 x2x2 x n-3 x n-1 xnxn x n-2 x1x1 x3x3 x4x4 x2x2

First “level” of merges: –n/2 merges, 1 redirect each –Total: n/2 redirects Second level: –n/4 merges, 2 redirects each –Total: n/2 i th level: n/2 i redirects Number of levels: log 2 n Total number of redirects: (n/2)log 2 n = O(n log n)

What is the runtime for UnionFind::Find for the array-based implementation? 1.O(1) 2.O(log n) 3.O(n) 4.O(n log n) 5.O(n 2 )

Do you understand why a given element can only be "repointed" O(log n) times? 1.1 = no = yes

Kruskal using basic arrays: Find(i): 2m calls takes  (m) time Union(i,j): n-1 calls takes  (n log n) time Runtime:  (m + n log m) =  (m + n log n)

Elements: Each element a tree node Sets: Each set a tree –Root is the representative element –Each other element points “up” to another node in the tree

Tree Representation S 1 ={x 1,x 3 } S 2 ={x 2,x 5,x 7,x 9 } S 3 ={x 4,x 6 } S 4 ={x 8 } x1x1 x3x3 x2x2 x5x5 x9x9 x7x7 x4x4 x6x6 x8x8

x2x2 x5x5 x9x9 x7x7 x4x4 x2x2 x5x5 x9x9 x7x7 x4x4 x2x2 x5x5 x9x9 x7x7 x4x4 S i = {x 2,x 4,x 5,x 7,x 9 } or There are no constraints on the shape of the tree.

AddElement(): Create a new node and make it the root of a (one element) tree –1 call: O(1) time –n calls: O(n) time Find(i): Follow pointers up to root –1 call: O(n) –m calls: Needs analysis Union(i,j): –Actual merge requires one reference change: O(1) –Finding the right pointer requires invoking Find(i)

x2x2 x5x5 x9x9 x7x7 x4x4 x3x3 x1x1 x8x8 x6x6 Union(9,1) x3x3 x1x1 x8x8 x6x6 x2x2 x5x5 x9x9 x7x7 x4x4

The runtime of Find(i) is determined by the depth of node i –Worst case runtime determined by largest height of all trees –Easy to create a tree of height n Exactly like we created the worst-case in the original List implementation Hence Find(i) is O(n) –Follows that Union(i,j) is O(n)

Easy to improve runtime as before: –Point root of less depth to root of greater depth –Bound: Maximum depth of tree is O(log n) Essentially the same proof as for lists –Runtime: O(log n) for both Find(i) and Union(i,j)

Worst-Case Depth x1x1 x2x2 x3x3 x4x4 x5x5 x7x7 x6x6 x8x8 x1x1 x2x2 x3x3 x4x4 x5x5 x7x7 x6x6 x8x8 x1x1 x2x2 x3x3 x4x4 x5x5 x7x7 x6x6 x8x8 x1x1 x2x2 x3x3 x4x4 x5x5 x7x7 x6x6 x8x8

Simple Representation Can represent with an array –If x i has parent x j : A[i] = j –If x i is a root: A[i] = -s s is the depth of the tree x1x1 x3x3 x5x5 x2x2 x9x9 x7x7 x4x4 x6x6 x8x

Path Compression Final trick: When executing a Find(i), update the tree –Point every node on the path to the root x1x1 x2x2 x3x3 x4x4 x5x5 x7x7 x6x6 x8x8 Find(8) x1x1 x2x2 x3x3 x4x4 x5x5 x7x7 x6x6 x8x8 Runtime: O(n) (an extra pass up the tree)

Ackerman’s Function “Worst” function Think of the fastest growing function you can This is worse

Sample values For n=0: –A(0,0) = 1 –A(1,0) = 2 –A(2,0) = 3 –A(3,0) = 5 –A(4,0) = 14 –A(5,0) = 65533

Sample values For n=2: –A(0,2) = 3 –A(1,2) = 4 –A(2,2) = 7 –A(3,2) = 29 –A(4,2) =

Sample Ackerman’s Function Values Knuth’s up-arrow notation

Fun Ackerman Facts Hard to calculate recursively –Simple Perl program bombs out on A(5,0) - - too much recursion No closed-form formula –Not “No known closed form formula” –It is impossible to create a closed form formula –The function is not Turing computable

Ackermen in Analysis Let  (n) = A(n,n) –For any “normal” function g(n), g(n)=O(  (n)) –If  (n) shows up in your runtime bound… …your are screwed

Inverse of  Consider  -1 (n): –  -1 (n) = k, where  (k) <= n <  (k+1)  (0) = 1  (1) = 3  (2) = 7  (3) = 61  (4) = 2^(2^(2^65536)) - 3  -1 (n) = 0, n  2  -1 (n) = 1, 3  n < 7  -1 (n) = 2, 7  n < 61  -1 (n) = 3, 61  n< … For reference: # atoms in the Universe  2 265

Union Find with Compression Union Find with rank ordering and path compression –For a Union Find set with n elements, a sequence of m Union and/or Find operations takes O(m  -1 (n)) time. –For all practical purposes, this is O(m)

Kruskal: Analysis Kruskal using basic arrays: –AddElements(): n calls takes O(n) time –Find(i) / Union(i,j): 2m+n-1 calls takes O((m+n)  -1 (n)) time –Runtime: O(n + m + (n+m)  -1 (n)) –Essentially O(n+m)