Minimum Spanning Trees Prim’s Algorithm Kruskal’s Algorithm Wed, July 9th

Outline For Today Graph Theory Part 1: Trees & Properties of Trees Minimum Spanning Trees Prim’s Algorithm And Run-time Analysis Graph Theory Part 2: Cuts & Properties of Cuts Prim’s Algorithm’s Correctness & “The Cut Property” Kruskal’s Algorithm

Outline For Today Graph Theory Part 1: Trees & Properties of Trees Minimum Spanning Trees Prim’s Algorithm And Run-time Analysis Graph Theory Part 2: Cuts & Properties of Cuts Prim’s Algorithm’s Correctness & “The Cut Property” Kruskal’s Algorithm

Outline For Today Graph Theory Part 1: Trees & Properties of Trees Minimum Spanning Trees Prim’s Algorithm And Run-time Analysis Graph Theory Part 2: Cuts & Properties of Cuts Prim’s Algorithm’s Correctness & “The Cut Property” Kruskal’s Algorithm

Tree Definition: An undirected graph G(V, E) is a tree iff G is connected G is acyclic

Example: Tree (1)

Example: Not a Tree (1)

Example: Not a Tree (2)

Example: Tree (2)

Example: Not a Tree (3)

Example: Tree (3)

Example: Not a Tree (3)

Removing any edge (u, v) from a tree T disconnects T! Breaking a Tree Lemma Removing any edge (u, v) from a tree T disconnects T! Why? No u⤳v path can exist! Assume it did… u x1 x2 v Contradicts acyclicity of T!

Then G is acyclic and hence a tree. Reverse is Also True Let G be a connected graph and assume removing any edge from G disconnects it. Then G is acyclic and hence a tree. Why? x1 xi xi+1 xk Claim: G cannot contain a cycle C

Proof That G Cannot Contain a Cycle C Breaking a Cycle Lemma: Removing any edge from a cycle cannot disconnect a connected graph! x1 xi xi+1 xk Take (xi xi+1), and any path P that was using it xi xi+1 t S

Proof That G Cannot Contain a Cycle C Breaking a Cycle Lemma: Removing any edge from a cycle cannot disconnect a connected graph! x1 xi xi+1 xk Take (xi xi+1), and any path P that was using it xi xi+1 t S

s and t are still connected! Proof That G Cannot Contain a Cycle C Breaking a Cycle Lemma: Removing any edge from a cycle cannot disconnect a connected graph! x1 xi xi+1 xk Take (xi xi+1), and any path P that was using it xi xi+1 t S s and t are still connected! xi-1 xk x1

Adding any edge (u, v) to a tree T creates a cycle! Cycle Creation Lemma Adding any edge (u, v) to a tree T creates a cycle!

Adding any edge (u, v) to a tree T creates a cycle! Cycle Creation Lemma Adding any edge (u, v) to a tree T creates a cycle!

Adding any edge (u, v) to a tree T creates a cycle! Cycle Creation Lemma Adding any edge (u, v) to a tree T creates a cycle! Proof: B/c T is connected, ∃path P from u to v. u v adding (u, v) closes the cycle.

Every tree of n vertices contains n-1 edges! Theorems Every tree of n vertices contains n-1 edges! Every n-1 acyclic set of edges is a tree => i.e., they connect V!

Outline For Today Graph Theory Part 1: Trees & Properties of Trees Minimum Spanning Trees Prim’s Algorithm And Run-time Analysis Graph Theory Part 2: Cuts & Properties of Cuts Prim’s Algorithm’s Correctness & “The Cut Property” Kruskal’s Algorithm

Minimum Spanning Tree Input: undirected G(V, E) and arbitrary weights on the edges Output: A tree T* of V such that w(T*) ≤ any other T of V w(T) = sum of the weights of all n-1 edges in T “spanning” tree of G(V, E) means T* has to connect all of V Assumptions: G is connected (minor) Edge weights are distinct (all of our algorithms work w/o this assumption)

Spanning Tree Example A D B C 1 2 4 2.5 3 5

Spanning Tree Example A D B C 1 2 4 2.5 3 5

Spanning Tree Example A D C 1 4 2.5 B C 1 4 2.5 W((A,C), (C,B), (B,D)) = 1 + 2.5 + 4 = 7.5

Spanning Tree Example A D B C 1 2 4 2.5 3 5

Spanning Tree Example A D C 1 2 4 B C 1 2 4 W((A,C), (C,D), (B,D)) = 1 + 2 + 4 = 7

MST Applications Designing all kinds of networks: datacenter networks road networks phone networks Circuit Design Clustering Image Segmentation …

Outline For Today Graph Theory Part 1: Trees & Properties of Trees Minimum Spanning Trees Prim’s Algorithm And Run-time Analysis Graph Theory Part 2: Cuts & Properties of Cuts Prim’s Algorithm’s Correctness & “The Cut Property” Kruskal’s Algorithm

Prim’s Algorithm Simulation (1) ?

Prim’s Algorithm Simulation (1) B C 1 4 6 D

Prim’s Algorithm Simulation (1) D E 1 2 6 5 4 C B

Prim’s Algorithm Simulation (1) D E 1 2 3 6 5 4 F C B

Prim’s Algorithm Simulation (1) D E 1 2 3 6 5 4 2.5 F C B 7.5 G

Prim’s Algorithm Simulation (1) D E 1 2 3 6 5 4 2.5 8 F C B 7 7.5 H G

Prim’s Algorithm Simulation (1) D E 1 2 3 6 5 4 2.5 8 F C B 7 7.5 H G

Prim’s Algorithm Simulation (1) D E 1 2 3 6 5 4 2.5 8 F C B 7 7.5 H G 9

Prim’s Algorithm Simulation (1) D E 1 2 3 6 5 4 2.5 8 F C B 7 7.5 H G 9

Prim’s Algorithm Simulation (1) D E 1 2 3 4 2.5 F C B 7 7.5 Final Tree! H G

Prim’s Algorithm procedure prim(G(V, E)): choose any s∈V let L={s}, R=V-{s} T = {} // final tree edges for i = 1 to n: 1. let (u, v) be the min cost edge crossing L-R 2. remove v from R and add to L 3. add (u, v) to T return T

Prim’s Algorithm Simulation (2) ?

Prim’s Algorithm Simulation (2) ∞ A B C 1 4 6 D

Prim’s Algorithm Simulation (2) 1 6 4 A B C D

Prim’s Algorithm Simulation (2) D E 1 2 ∞ 6 5 4 4 6 C B

Prim’s Algorithm Simulation (2) D E 1 2 2 6 5 4 4 5 C B

Prim’s Algorithm Simulation (2) D E 1 2 3 6 5 4 4 F 5 ∞ C B

Prim’s Algorithm Simulation (2) D E 1 2 3 6 5 4 4 F 5 3 C B

Prim’s Algorithm Simulation (2) D E 1 2 3 6 5 4 2.5 4 F 5 C B 7.5 ∞ G

Prim’s Algorithm Simulation (2) D E 1 2 3 6 5 4 2.5 4 F 2.5 C B 7.5 7.5 G

Prim’s Algorithm Simulation (2) D E 1 2 3 6 5 4 2.5 8 4 F C B 7 7.5 H ∞ 7.5 G

Prim’s Algorithm Simulation (2) D E 1 2 3 6 5 4 2.5 8 4 F C B 7 7.5 H 7 7.5 G

Prim’s Algorithm Simulation (2) D E 1 2 3 6 5 4 2.5 8 F C B 7 7.5 H 7 7.5 G

Prim’s Algorithm Simulation (2) D E 1 2 3 6 5 4 2.5 8 F C B 7 7.5 H 7.5 G 9

Prim’s Algorithm Simulation (2) D E 1 2 3 6 5 4 2.5 8 F C B 7 7.5 H G 9

Prim’s Algorithm Simulation (2) D E 1 2 3 4 2.5 F C B 7 7.5 Final Tree! H G

Prim’s Algorithm procedure prim(G(V, E)): choose any s∈V let L={s}, R=V-{s} let mWE store for each vertex in R, its min weight edge to L. T = {} // final tree edges for i = 1 to n: 1. let v be min mWE vertex let (u, v) be the edge minimizing v’s mWE 2. remove v from R & mWE;add to L 3. add (u, v) to T 4. update mWE for each neighbor of v in R return T

Exactly the same As Dijkstra’s Algorithm! Prim’s Algorithm Running Time Exactly the same As Dijkstra’s Algorithm! O((n+m)log(n)) = O(mlog(n)) when using a priority queue backed with a heap to store “mWE” values When a vertex y is moved from R to L, For each (x, y): Dijkstra Update: val[y]=min{val[y], shortDist[x] + w((x, y))} Prim Update: val[y]=min{val[y], w((x, y))}

Outline For Today Graph Theory Part 1: Trees & Properties of Trees Minimum Spanning Trees Prim’s Algorithm And Run-time Analysis Graph Theory Part 2: Cuts & Properties of Cuts Prim’s Algorithm’s Correctness & “The Cut Property” Kruskal’s Algorithm

Cuts A cut of G(V, E) is a slice of V into 2 non-empty sets A D B C 1 2 4 2.5 3 5

Cuts A cut of G(V, E) is a slice of V into 2 non-empty sets A D B C 1 2 4 2.5 3 5

Cuts A cut of G(V, E) is a slice of V into 2 non-empty sets A D B C 1 2 4 2.5 3 5

⇔ Empty Cut Lemma ∃ cut (X, Y) with no crossing edges A graph G(V, E) is not connected ⇔ [prove as exercise]

Claim: Then there is another e` ≠ e of C that also crosses (X, Y) Double Cut Crossing Lemma of Cycles Suppose a cycle C⊆ E has an edge e crossing a cut (X, Y) Claim: Then there is another e` ≠ e of C that also crosses (X, Y)

Double Cut Crossing Lemma of Cycles X Y e v u e` u` v`

Lonely Cut Corollary Suppose there is a cut (X, Y) which has only one edge e crossing it Claim: e cannot be part of any cycle C!

Lonely Cut Corollary Claim: e cannot be part of any cycle C! X Y e v u

Outline For Today Graph Theory Part 1: Trees & Properties of Trees Minimum Spanning Trees Prim’s Algorithm And Run-time Analysis Graph Theory Part 2: Cuts & Properties of Cuts Prim’s Algorithm’s Correctness & “The Cut Property” Kruskal’s Algorithm

For Correctness We Need To Prove 2 Things Outputs a Spanning Tree Tprim Tprim is a minimum spanning tree

1: Prim Outputs a Spanning Tree Proof: At each iteration i, Tprim,i spans Li AND is acyclic By induction: Holds for base case s Ind. Hyp. (IH).: Tprim,k spans L after kth iteration Prim picks the min edge (u, v) from Lk to Rk. By IH: u has a path to every vertex in Lk and vice versa => with addition of (u, v), v has a path to every vertex in Lk and vice versa => Tprim,k+1 is spanning Tprim,k+1 is acyclic b/c (u, v) is the first edge from Lk to Rk (by the lonely cut corollary) Q.E.D

Plan for 2: Tprim is a Minimum Spanning Tree “Cut Property” of Edges Argue that the Cut Property Implies Tprim is an MST

Claim: e belongs to every MST Cut Property of Edges Consider any cut (X, Y) of G, and suppose e is the minimum edge crossing (X, Y) Claim: e belongs to every MST

Cut Property of Edges X Y 1 8 3 4 5

Cut Property of Edges (u,v) belongs to every MST! X Y 1 u v 8 3 4 5

Cut Property => Tprim is a MST ?

Cut Property => Tprim is a MST B C 1 4 6 D

Cut Property => Tprim is a MST D E 1 2 6 5 4 C B

Cut Property => Tprim is a MST D E 1 2 3 6 5 4 F C B

Cut Property => Tprim is a MST D E 1 2 3 6 5 4 2.5 F C B 7.5 G

Cut Property => Tprim is a MST D E 1 2 3 6 5 4 2.5 8 F C B 7 7.5 H G

Cut Property => Tprim is a MST D E 1 2 3 6 5 4 2.5 8 F C B 7 7.5 H G

Cut Property => Tprim is a MST D E 1 2 3 6 5 4 2.5 8 F C B 7 7.5 H G 9

Cut Property => Tprim is a MST D E 1 2 3 6 5 4 2.5 8 F C B 7 7.5 H G 9

Cut Property => Tprim is a MST Prim looks at n-1 different cuts. Add n-1 edges that every MST includes. Since any spanning tree include n-1 edges => Tprim is “the” MST. (with distinct edge weights this proof also proves the uniqueness of MST)

Proof of Cut Property (Exchange Argument) e=min u v internal edges and vertices of X internal edges and vertices of Y

Proof of Cut Property (Exchange Argument) Let T be any MST that doesn’t contain e X Y e=min u v internal edges and vertices of X internal edges and vertices of Y

Proof of Cut Property (Exchange Argument) Let T be any MST that doesn’t contain e X Y u v internal edges of T in X internal edges of T in Y

Proof of Cut Property (Exchange Argument) add e to T X Y u v internal edges of T in X internal edges of T in Y By cycle creation lemma we created a cycle.

Proof of Cut Property (Exchange Argument) add e to T X Y u v internal edges of T in X internal edges of T in Y b a By Double Crossing Lemma ∃(a, b) that also crosses (X, Y)

Proof of Cut Property (Exchange Argument) v internal edges of T in X internal edges of T in Y b a Claim: Can safely remove (a, b) and still get a spanning tree. Why?

Proof of Cut Property (Exchange Argument) v internal edges of T in X internal edges of T in Y b a By Breaking a Cycle Lemma we can’t disconnect T ∪ (u, v)!

Proof of Cut Property (Exchange Argument) T` = T + (u, v) – (a, b) X Y u v internal edges of T in X internal edges of T in Y b T` is still a spanning tree and w(T`) < w(T) a

=> Every MST has to include e! Proof of Cut Property (Exchange Argument) Started with (X, Y) cut, and e=(u, v): min edge crossing (X, Y) Took any minimum spanning tree T that did not include e. Added e to T => created a cycle C (by cycle-creation-lemma) By DCL, there is another edge e` that crosses (X, Y) Removed e` => T` By Breaking A Cycle Lemma: T` is still connected Therefore w(T`) < w(T) => T was not an MST. => Every MST has to include e! Q.E.D.

Summary of Prim’s Algorithm’s Correctness First proved Tprim is a spanning tree By induction on Li Second proved Tprim is a an minimum spanning tree Note this version of Prim’s Correctness Proof contains: Exchange Argument: Cut Property Greedy Stays Ahead Argument: Prim adds n-1 edges and stay ahead at each addition.

Outline For Today Graph Theory Part 1: Trees & Properties of Trees Minimum Spanning Trees Prim’s Algorithm And Run-time Analysis Graph Theory Part 2: Cuts & Properties of Cuts Prim’s Algorithm’s Correctness & “The Cut Property” Kruskal’s Algorithm

Kruskal’s Algorithm Simulation D E 1 2 3 6 5 4 2.5 8 F C B 7 7.5 1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9 H G 9

Kruskal’s Algorithm Simulation D E 1 2 3 6 5 4 2.5 8 F C B 7 7.5 1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9 H G 9

Kruskal’s Algorithm Simulation D E 1 2 3 6 5 4 2.5 8 F C B 7 7.5 1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9 H G 9

Kruskal’s Algorithm Simulation D E 1 2 3 6 5 4 2.5 8 F C B 7 7.5 1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9 H G 9

Kruskal’s Algorithm Simulation D E 1 2 3 6 5 4 2.5 8 F C B 7 7.5 1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9 H G 9

Kruskal’s Algorithm Simulation D E 1 2 3 6 5 4 2.5 8 F C B 7 7.5 1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9 H G Creates a cycle 9

Kruskal’s Algorithm Simulation D E 1 2 3 6 5 4 2.5 8 F C B 7 7.5 1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9 H G Creates a cycle 9

Kruskal’s Algorithm Simulation D E 1 2 3 6 5 4 2.5 8 F C B 7 7.5 1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9 H G 9

Kruskal’s Algorithm Simulation D E 1 2 3 6 5 4 2.5 8 F C B 7 7.5 1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9 H G 9

Kruskal’s Algorithm Simulation D E 1 2 3 6 5 4 2.5 8 F C B 7 7.5 1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9 H G Creates a cycle 9

Kruskal’s Algorithm Simulation D E 1 2 3 6 5 4 2.5 8 F C B 7 7.5 1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9 H G Creates a cycle 9

Kruskal’s Algorithm Simulation D E 1 2 3 4 2.5 F C B 7 Final Tree! Same as Tprim 7.5 H G

Kruskal’s Algorithm Pseudocode procedure kruskal(G(V, E)): sort E in order of increasing weights rename E so w(e1) < w(e2) < … < w(em) T = {} // final tree edges for i = 1 to m: if T ∪ ei=(u,v) doesn’t create cycle add ei to T return T

For Correctness We Need To Prove 2 Things Outputs a Spanning Tree Tkrsk Tkrsk is a minimum spanning tree

1: Kruskal Outputs a Spanning Tree (1) Need to prove Tkrsk is spanning AND is acyclic Acyclic is by definition of the algorithm. Why is Tkrsk spanning (i.e., connected)? Recall Empty Cut Lemma: A graph is not connected iff ∃ cut (X, Y) with no crossing edges If all cuts have a crossing edge -> graph is connected!

1: Kruskal Outputs a Spanning Tree (2) Consider any cut (X, Y) Since input G is connected, ∃ edges crossing (X, Y) Let e* be the min-weight edge Kruskal inspects all edges Consider the time when Kruskal inspects e* Claim: No edge crossing (X, Y) is in Tkrsk And adding e* to Tkrsk cannot create a cycle => Why? (because of lonely cut corallary) Therefore Kruskal will add e*, so for each cut, there is an edge in Tkrsk crossing it => Tkrsk spans V.

Plan for 2: Tkrsk is a Minimum Spanning Tree Argue that the Cut Property Implies Tkrsk is an MST In particular we will argue that each edge (u, v) added to Tkrsk is a min-edge in a cut (X, Y) in G.

Cut Property => Tkrsk is a MST (1) Let (u, v) be any edge added by Kruskal’s Algorithm. u and v are in different comp. (b/c Kruskal checks for cycles) Claim: (u, v) is min-edge crossing this cut! v t z w u x y

Cut Property => Tkrsk is a MST (2) (u, v) is the min edge crossing the cut b/c Kruskal looks at edges in increasing weights. By the Cut Property => (u, v) is in every MST. Therefore all edges added to Tkrsk is in every MST => Tkrsk is “the” MST. Q.E.D.

Summary of Kruskal’s Correctness First proved Tkrsk is a spanning tree Acyclicity was by definition Connectedness followed from showing that there is an edge of Tkrsk crossing any cut (X, Y) of G, which by the empty cut lemma implied that Tkrsk was connected Second proved Tkrsk was a minimum spanning tree by arguing that any edge (u, v) if Tkrsk is a min-edge in some cut (X, Y) Note this version of Kruskal’s Correctness Proof contains: Exchange Argument: Cut Property Greedy Stays Ahead Argument: Krusk adds n-1 edges and all are justified.

Kruskal’s Algorithm Pseudocode procedure kruskal(G(V, E)): sort E in order of increasing weights rename E so w(e1) < w(e2) < … < w(em) T = {} // final tree edges for i = 1 to m: if T ∪ ei=(u,v) doesn’t create cycle add ei to T return T

Kruskal’s Algorithm Naïve Run-time O(mlog(n)) to sort the edges m iterations O(n) time to check for a cycle (BFS/DFS) Total: O(mn) total time On Monday will improve to O(mlog(n))