1. Minimum Spanning Trees
Prim’s Algorithm
Kruskal’s Algorithm
Wed, July 9th

2. Outline For Today Graph Theory Part 1: Trees & Properties of Trees
Minimum Spanning Trees
Prim’s Algorithm And Run-time Analysis
Graph Theory Part 2: Cuts & Properties of Cuts
Prim’s Algorithm’s Correctness & “The Cut Property”
Kruskal’s Algorithm

3. Outline For Today Graph Theory Part 1: Trees & Properties of Trees
Minimum Spanning Trees
Prim’s Algorithm And Run-time Analysis
Graph Theory Part 2: Cuts & Properties of Cuts
Prim’s Algorithm’s Correctness & “The Cut Property”
Kruskal’s Algorithm

4. Outline For Today Graph Theory Part 1: Trees & Properties of Trees
Minimum Spanning Trees
Prim’s Algorithm And Run-time Analysis
Graph Theory Part 2: Cuts & Properties of Cuts
Prim’s Algorithm’s Correctness & “The Cut Property”
Kruskal’s Algorithm

5. Tree Definition: An undirected graph G(V, E) is a tree iff
G is connected
G is acyclic

6. Example: Tree (1)

7. Example: Not a Tree (1)

8. Example: Not a Tree (2)

9. Example: Tree (2)

10. Example: Not a Tree (3)

11. Example: Tree (3)

12. Example: Not a Tree (3)

13. Removing any edge (u, v) from a tree T disconnects T!
Breaking a Tree Lemma
Removing any edge (u, v) from a tree T disconnects T!
Why?
No u⤳v path can exist! Assume it did… u x1 x2 v
Contradicts acyclicity of T!

14. Then G is acyclic and hence a tree.
Reverse is Also True
Let G be a connected graph and assume removing any edge from G disconnects it.
Then G is acyclic and hence a tree.
Why? x1 xi
xi+1 xk
Claim: G cannot contain a cycle C

15. Proof That G Cannot Contain a Cycle C
Breaking a Cycle Lemma: Removing any edge from a cycle cannot disconnect a connected graph! x1 xi
xi+1 xk
Take (xi xi+1), and any path P that was using it xi
xi+1 t S

16. Proof That G Cannot Contain a Cycle C
Breaking a Cycle Lemma: Removing any edge from a cycle cannot disconnect a connected graph! x1 xi
xi+1 xk
Take (xi xi+1), and any path P that was using it xi
xi+1 t S

17. s and t are still connected!
Proof That G Cannot Contain a Cycle C
Breaking a Cycle Lemma: Removing any edge from a cycle cannot disconnect a connected graph! x1 xi
xi+1 xk
Take (xi xi+1), and any path P that was using it xi
xi+1 t S
s and t are still connected!
xi-1 xk x1

18. Adding any edge (u, v) to a tree T creates a cycle!
Cycle Creation Lemma
Adding any edge (u, v) to a tree T creates a cycle!

19. Adding any edge (u, v) to a tree T creates a cycle!
Cycle Creation Lemma
Adding any edge (u, v) to a tree T creates a cycle!

20. Adding any edge (u, v) to a tree T creates a cycle!
Cycle Creation Lemma
Adding any edge (u, v) to a tree T creates a cycle!
Proof: B/c T is connected, ∃path P from u to v. u v
adding (u, v) closes the cycle.

21. Every tree of n vertices contains n-1 edges!
Theorems
Every tree of n vertices contains n-1 edges!
Every n-1 acyclic set of edges is a tree => i.e., they connect V!

22. Outline For Today Graph Theory Part 1: Trees & Properties of Trees
Minimum Spanning Trees
Prim’s Algorithm And Run-time Analysis
Graph Theory Part 2: Cuts & Properties of Cuts
Prim’s Algorithm’s Correctness & “The Cut Property”
Kruskal’s Algorithm

23. Minimum Spanning Tree
Input: undirected G(V, E) and arbitrary weights on the edges
Output: A tree T* of V such that w(T*) ≤ any other T of V
w(T) = sum of the weights of all n-1 edges in T
“spanning” tree of G(V, E) means T* has to connect all of V
Assumptions:
G is connected (minor)
Edge weights are distinct (all of our algorithms work w/o this assumption)

24. Spanning Tree Example A D B C 1 2 4
2.5 3 5

25. Spanning Tree Example A D B C 1 2 4
2.5 3 5

26. Spanning Tree Example A D C 1 4 2.5B C 1 4
2.5
W((A,C), (C,B), (B,D)) = = 7.5

27. Spanning Tree Example A D B C 1 2 4
2.5 3 5

28. Spanning Tree Example A D C 1 2 4B C 1 2 4
W((A,C), (C,D), (B,D)) = = 7

29. MST Applications Designing all kinds of networks: datacenter networks
road networks
phone networks
Circuit Design
Clustering
Image Segmentation …

30. Outline For Today Graph Theory Part 1: Trees & Properties of Trees
Minimum Spanning Trees
Prim’s Algorithm And Run-time Analysis
Graph Theory Part 2: Cuts & Properties of Cuts
Prim’s Algorithm’s Correctness & “The Cut Property”
Kruskal’s Algorithm

31. Prim’s Algorithm Simulation (1)?

32. Prim’s Algorithm Simulation (1)B C 1 4 6 D

33. Prim’s Algorithm Simulation (1)D E 1 2 6 5 4 C B

34. Prim’s Algorithm Simulation (1)D E 1 2 3 6 5 4 F C B

35. Prim’s Algorithm Simulation (1)D E 1 2 3 6 5 4
2.5 F C B
7.5 G

36. Prim’s Algorithm Simulation (1)D E 1 2 3 6 5 4
2.5 8 F C B 7
7.5 H G

37. Prim’s Algorithm Simulation (1)D E 1 2 3 6 5 4
2.5 8 F C B 7
7.5 H G

38. Prim’s Algorithm Simulation (1)D E 1 2 3 6 5 4
2.5 8 F C B 7
7.5 H G 9

39. Prim’s Algorithm Simulation (1)D E 1 2 3 6 5 4
2.5 8 F C B 7
7.5 H G 9

40. Prim’s Algorithm Simulation (1)D E 1 2 3 4
2.5 F C B 7
7.5
Final Tree! H G

41. Prim’s Algorithm procedure prim(G(V, E)): choose any s∈V
let L={s}, R=V-{s}
T = {} // final tree edges
for i = 1 to n:
1. let (u, v) be the min cost edge
crossing L-R
2. remove v from R and add to L
3. add (u, v) to T
return T

42. Prim’s Algorithm Simulation (2)?

43. Prim’s Algorithm Simulation (2)∞ A B C 1 4 6 D

44. Prim’s Algorithm Simulation (2)1 6 4 A B C D

45. Prim’s Algorithm Simulation (2)D E 1 2 ∞ 6 5 4 4 6 C B

46. Prim’s Algorithm Simulation (2)D E 1 2 2 6 5 4 4 5 C B

47. Prim’s Algorithm Simulation (2)D E 1 2 3 6 5 4 4 F 5 ∞ C B

48. Prim’s Algorithm Simulation (2)D E 1 2 3 6 5 4 4 F 5 3 C B

49. Prim’s Algorithm Simulation (2)D E 1 2 3 6 5 4
2.5 4 F 5 C B
7.5 ∞ G

50. Prim’s Algorithm Simulation (2)D E 1 2 3 6 5 4
2.5 4 F
2.5 C B
7.5
7.5 G

51. Prim’s Algorithm Simulation (2)D E 1 2 3 6 5 4
2.5 8 4 F C B 7
7.5 H ∞
7.5 G

52. Prim’s Algorithm Simulation (2)D E 1 2 3 6 5 4
2.5 8 4 F C B 7
7.5 H 7
7.5 G

53. Prim’s Algorithm Simulation (2)D E 1 2 3 6 5 4
2.5 8 F C B 7
7.5 H 7
7.5 G

54. Prim’s Algorithm Simulation (2)D E 1 2 3 6 5 4
2.5 8 F C B 7
7.5 H
7.5 G 9

55. Prim’s Algorithm Simulation (2)D E 1 2 3 6 5 4
2.5 8 F C B 7
7.5 H G 9

56. Prim’s Algorithm Simulation (2)D E 1 2 3 4
2.5 F C B 7
7.5
Final Tree! H G

57. Prim’s Algorithm procedure prim(G(V, E)): choose any s∈V
let L={s}, R=V-{s}
let mWE store for each vertex
in R, its min weight edge to L.
T = {} // final tree edges
for i = 1 to n:
1. let v be min mWE vertex
let (u, v) be the edge
minimizing v’s mWE
2. remove v from R & mWE;add to L
3. add (u, v) to T
4. update mWE for each neighbor of v in R
return T

58. Exactly the same As Dijkstra’s Algorithm!
Prim’s Algorithm Running Time
Exactly the same As Dijkstra’s Algorithm!
O((n+m)log(n)) = O(mlog(n)) when using a priority queue backed with a heap to store “mWE” values
When a vertex y is moved from R to L, For each (x, y):
Dijkstra Update: val[y]=min{val[y],
shortDist[x] + w((x, y))}
Prim Update: val[y]=min{val[y], w((x, y))}

59. Outline For Today Graph Theory Part 1: Trees & Properties of Trees
Minimum Spanning Trees
Prim’s Algorithm And Run-time Analysis
Graph Theory Part 2: Cuts & Properties of Cuts
Prim’s Algorithm’s Correctness & “The Cut Property”
Kruskal’s Algorithm

60. Cuts
A cut of G(V, E) is a slice of V into 2 non-empty sets A D B C 1 2 4
2.5 3 5

61. Cuts
A cut of G(V, E) is a slice of V into 2 non-empty sets A D B C 1 2 4
2.5 3 5

62. Cuts
A cut of G(V, E) is a slice of V into 2 non-empty sets A D B C 1 2 4
2.5 3 5

63. ⇔ Empty Cut Lemma ∃ cut (X, Y) with no crossing edges
A graph G(V, E) is not connected ⇔
[prove as exercise]

64. Claim: Then there is another e` ≠ e of C that also crosses (X, Y)
Double Cut Crossing Lemma of Cycles
Suppose a cycle C⊆ E has an edge e crossing a cut (X, Y)
Claim: Then there is another e` ≠ e of C that also crosses (X, Y)

65. Double Cut Crossing Lemma of CyclesX Y e v u e` u` v`

66. Lonely Cut Corollary
Suppose there is a cut (X, Y) which has only one edge e crossing it
Claim: e cannot be part of any cycle C!

67. Lonely Cut Corollary
Claim: e cannot be part of any cycle C! X Y e v u

68. Outline For Today Graph Theory Part 1: Trees & Properties of Trees
Minimum Spanning Trees
Prim’s Algorithm And Run-time Analysis
Graph Theory Part 2: Cuts & Properties of Cuts
Prim’s Algorithm’s Correctness & “The Cut Property”
Kruskal’s Algorithm

69. For Correctness We Need To Prove 2 Things
Outputs a Spanning Tree Tprim
Tprim is a minimum spanning tree

70. 1: Prim Outputs a Spanning Tree
Proof: At each iteration i, Tprim,i spans Li AND is acyclic
By induction: Holds for base case s
Ind. Hyp. (IH).: Tprim,k spans L after kth iteration
Prim picks the min edge (u, v) from Lk to Rk.
By IH: u has a path to every vertex in Lk and vice versa
=> with addition of (u, v), v has a path to every
vertex in Lk and vice versa => Tprim,k+1 is spanning
Tprim,k+1 is acyclic b/c (u, v) is the first edge from Lk to Rk (by the lonely cut corollary)
Q.E.D

71. Plan for 2: Tprim is a Minimum Spanning Tree
“Cut Property” of Edges
Argue that the Cut Property Implies Tprim is an MST

72. Claim: e belongs to every MST
Cut Property of Edges
Consider any cut (X, Y) of G, and suppose e is the minimum edge crossing (X, Y)
Claim: e belongs to every MST

73. Cut Property of Edges X Y 1 8 3 4 5

74. Cut Property of Edges
(u,v) belongs to every MST! X Y 1 u v 8 3 4 5

75. Cut Property => Tprim is a MST?

76. Cut Property => Tprim is a MSTB C 1 4 6 D

77. Cut Property => Tprim is a MSTD E 1 2 6 5 4 C B

78. Cut Property => Tprim is a MSTD E 1 2 3 6 5 4 F C B

79. Cut Property => Tprim is a MSTD E 1 2 3 6 5 4
2.5 F C B
7.5 G

80. Cut Property => Tprim is a MSTD E 1 2 3 6 5 4
2.5 8 F C B 7
7.5 H G

81. Cut Property => Tprim is a MSTD E 1 2 3 6 5 4
2.5 8 F C B 7
7.5 H G

82. Cut Property => Tprim is a MSTD E 1 2 3 6 5 4
2.5 8 F C B 7
7.5 H G 9

83. Cut Property => Tprim is a MSTD E 1 2 3 6 5 4
2.5 8 F C B 7
7.5 H G 9

84. Cut Property => Tprim is a MST
Prim looks at n-1 different cuts.
Add n-1 edges that every MST includes.
Since any spanning tree include n-1 edges
=> Tprim is “the” MST.
(with distinct edge weights this proof also proves the uniqueness of MST)

85. Proof of Cut Property (Exchange Argument)
e=min u v
internal edges and vertices of X
internal edges and vertices of Y

86. Proof of Cut Property (Exchange Argument)
Let T be any MST that doesn’t contain e X Y
e=min u v
internal edges and vertices of X
internal edges and vertices of Y

87. Proof of Cut Property (Exchange Argument)
Let T be any MST that doesn’t contain e X Y u v
internal edges of T in X
internal edges of T in Y

88. Proof of Cut Property (Exchange Argument)
add e to T X Y u v
internal edges of T in X
internal edges of T in Y
By cycle creation lemma
we created a cycle.

89. Proof of Cut Property (Exchange Argument)
add e to T X Y u v
internal edges of T in X
internal edges of T in Y b a
By Double Crossing Lemma ∃(a, b) that also crosses (X, Y)

90. Proof of Cut Property (Exchange Argument)v
internal edges of T in X
internal edges of T in Y b a
Claim: Can safely
remove (a, b) and still
get a spanning tree. Why?

91. Proof of Cut Property (Exchange Argument)v
internal edges of T in X
internal edges of T in Y b a
By Breaking a Cycle Lemma we can’t disconnect T ∪ (u, v)!

92. Proof of Cut Property (Exchange Argument)
T` = T + (u, v) – (a, b) X Y u v
internal edges of T in X
internal edges of T in Y b
T` is still a spanning
tree and w(T`) < w(T) a

93. => Every MST has to include e!
Proof of Cut Property (Exchange Argument)
Started with (X, Y) cut, and e=(u, v): min edge crossing (X, Y)
Took any minimum spanning tree T that did not include e.
Added e to T => created a cycle C (by cycle-creation-lemma)
By DCL, there is another edge e` that crosses (X, Y)
Removed e` => T`
By Breaking A Cycle Lemma: T` is still connected
Therefore w(T`) < w(T) => T was not an MST.
=> Every MST has to include e!
Q.E.D.

94. Summary of Prim’s Algorithm’s Correctness
First proved Tprim is a spanning tree
By induction on Li
Second proved Tprim is a an minimum spanning tree
Note this version of Prim’s Correctness Proof contains:
Exchange Argument: Cut Property
Greedy Stays Ahead Argument: Prim adds n-1 edges and stay ahead at each addition.

95. Outline For Today Graph Theory Part 1: Trees & Properties of Trees
Minimum Spanning Trees
Prim’s Algorithm And Run-time Analysis
Graph Theory Part 2: Cuts & Properties of Cuts
Prim’s Algorithm’s Correctness & “The Cut Property”
Kruskal’s Algorithm

96. Kruskal’s Algorithm SimulationD E 1 2 3 6 5 4
2.5 8 F C B 7
7.5
1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9 H G 9

97. Kruskal’s Algorithm SimulationD E 1 2 3 6 5 4
2.5 8 F C B 7
7.5
1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9 H G 9

98. Kruskal’s Algorithm SimulationD E 1 2 3 6 5 4
2.5 8 F C B 7
7.5
1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9 H G 9

99. Kruskal’s Algorithm SimulationD E 1 2 3 6 5 4
2.5 8 F C B 7
7.5
1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9 H G 9

100. Kruskal’s Algorithm SimulationD E 1 2 3 6 5 4
2.5 8 F C B 7
7.5
1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9 H G 9

101. Kruskal’s Algorithm SimulationD E 1 2 3 6 5 4
2.5 8 F C B 7
7.5
1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9 H G
Creates a cycle 9

102. Kruskal’s Algorithm SimulationD E 1 2 3 6 5 4
2.5 8 F C B 7
7.5
1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9 H G
Creates a cycle 9

103. Kruskal’s Algorithm SimulationD E 1 2 3 6 5 4
2.5 8 F C B 7
7.5
1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9 H G 9

104. Kruskal’s Algorithm SimulationD E 1 2 3 6 5 4
2.5 8 F C B 7
7.5
1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9 H G 9

105. Kruskal’s Algorithm SimulationD E 1 2 3 6 5 4
2.5 8 F C B 7
7.5
1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9 H G
Creates a cycle 9

106. Kruskal’s Algorithm SimulationD E 1 2 3 6 5 4
2.5 8 F C B 7
7.5
1, 2, 2.5, 3, 4, 5, 6, 7, 7.5, 8, 9 H G
Creates a cycle 9

107. Kruskal’s Algorithm SimulationD E 1 2 3 4
2.5 F C B 7
Final Tree!
Same as Tprim
7.5 H G

108. Kruskal’s Algorithm Pseudocode
procedure kruskal(G(V, E)):
sort E in order of increasing weights
rename E so w(e1) < w(e2) < … < w(em)
T = {} // final tree edges
for i = 1 to m:
if T ∪ ei=(u,v) doesn’t create cycle
add ei to T
return T

109. For Correctness We Need To Prove 2 Things
Outputs a Spanning Tree Tkrsk
Tkrsk is a minimum spanning tree

110. 1: Kruskal Outputs a Spanning Tree (1)
Need to prove Tkrsk is spanning AND is acyclic
Acyclic is by definition of the algorithm.
Why is Tkrsk spanning (i.e., connected)?
Recall Empty Cut Lemma:
A graph is not connected iff ∃ cut (X, Y) with no crossing edges
If all cuts have a crossing edge -> graph is connected!

111. 1: Kruskal Outputs a Spanning Tree (2)
Consider any cut (X, Y)
Since input G is connected, ∃ edges crossing (X, Y)
Let e* be the min-weight edge
Kruskal inspects all edges
Consider the time when Kruskal inspects e*
Claim: No edge crossing (X, Y) is in Tkrsk
And adding e* to Tkrsk cannot create a cycle => Why?
(because of lonely cut corallary)
Therefore Kruskal will add e*, so for each cut, there is an edge in Tkrsk crossing it => Tkrsk spans V.

112. Plan for 2: Tkrsk is a Minimum Spanning Tree
Argue that the Cut Property Implies Tkrsk is an MST
In particular we will argue that each edge (u, v) added to Tkrsk is a min-edge in a cut (X, Y) in G.

113. Cut Property => Tkrsk is a MST (1)
Let (u, v) be any edge added by Kruskal’s Algorithm.
u and v are in different comp. (b/c Kruskal checks for cycles)
Claim: (u, v) is min-edge crossing this cut! v t z w u x y

114. Cut Property => Tkrsk is a MST (2)
(u, v) is the min edge crossing the cut b/c Kruskal looks at edges in increasing weights.
By the Cut Property => (u, v) is in every MST.
Therefore all edges added to Tkrsk is in every MST
=> Tkrsk is “the” MST.
Q.E.D.

115. Summary of Kruskal’s Correctness
First proved Tkrsk is a spanning tree
Acyclicity was by definition
Connectedness followed from showing that there is an edge of Tkrsk crossing any cut (X, Y) of G, which by the empty cut lemma implied that Tkrsk was connected
Second proved Tkrsk was a minimum spanning tree by arguing that any edge (u, v) if Tkrsk is a min-edge in some cut (X, Y)
Note this version of Kruskal’s Correctness Proof contains:
Exchange Argument: Cut Property
Greedy Stays Ahead Argument: Krusk adds n-1 edges and all are justified.

116. Kruskal’s Algorithm Pseudocode
procedure kruskal(G(V, E)):
sort E in order of increasing weights
rename E so w(e1) < w(e2) < … < w(em)
T = {} // final tree edges
for i = 1 to m:
if T ∪ ei=(u,v) doesn’t create cycle
add ei to T
return T

117. Kruskal’s Algorithm Naïve Run-time
O(mlog(n)) to sort the edges
m iterations
O(n) time to check for a cycle (BFS/DFS)
Total: O(mn) total time
On Monday will improve to O(mlog(n))