1. CS 3343: Analysis of Algorithms
Lecture 22: Minimum Spanning Tree

2. Outline Review of last lecture
Prim’s algorithm and Kruskal’s algorithm for MST in detail

3. Graph Review G = (V, E) Types of graphs V: a set of vertices
E: a set of edges
Degree: number of edges
In-degree vs out-degree
Types of graphs
Directed / undirected
Weighted / un-weighted
Dense / sparse
Connected / disconnected

4. Graph representations
Adjacency matrix A 1 2 3 4 1 2 4 3
How much storage does the adjacency matrix require?
A: O(V2)

5. Graph representations
Adjacency list 1 2 3 3 2 4 3 3
How much storage does the adjacency list require?
A: O(V+E)

6. Graph representations
Undirected graph 4 3 2 1 A 1 2 4 3 2 3 1 3 1 2 4 3

7. Graph representations
Weighted graph 4 3 2 9 6 5 1 A 1 5 2 6 4 9 4 3
2,5
3,6
1,5
3,9
1,6
2,9
4,4
3,4

8. Analysis of time complexity
Convention:
Number of vertices: n, |V|, V
Number of edges: m, |E|, E
O(V+E) is the same as O(n+m) or O(|V|+|E|)

9. Tradeoffs between the two representations
|V| = n, |E| = m
Adj Matrix
Adj List
test (u, v)  E
Θ(1)
O(n)
Degree(u)
Θ(n)
Memory
Θ(n2)
Θ(n+m)
Edge insertion
Edge deletion
Graph traversal
Both representations are very useful and have different properties, although adjacency lists are probably better for most problems

10. Minimum Spanning Tree
Problem: given a connected, undirected, weighted graph: 14 10 3 6 4 5 2 9 15 8

11. Minimum Spanning Tree
Problem: given a connected, undirected, weighted graph, find a spanning tree using edges that minimize the total weight 6 4 5 9
A spanning tree is a tree that connects all vertices
Number of edges = ?
A spanning tree has no designated root. 14 2 10 15 3 8

12. Minimum Spanning Tree
MSTs satisfy the optimal substructure property: an optimal tree is composed of optimal subtrees
Let T be an MST of G with an edge (u,v) in the middle
Removing (u,v) partitions T into two trees T1 and T2
w(T) = w(u,v) + w(T1) + w(T2)
Claim 1: T1 is an MST of G1 = (V1, E1), and T2 is an MST of G2 = (V2, E2)
Proof by contradiction:
if T1 is not optimal, we can replace T1 with a better spanning tree, T1’
T1’, T2 and (u, v) form a new spanning tree T’
W(T’) < W(T). Contradiction. T1 T2 u v
T1’

13. Minimum Spanning Tree
MSTs satisfy the optimal substructure property: an optimal tree is composed of optimal subtrees
Let T be an MST of G with an edge (u,v) in the middle
Removing (u,v) partitions T into two trees T1 and T2
w(T) = w(u,v) + w(T1) + w(T2)
Claim 2: (u, v) is the lightest edge connecting G1 = (V1, E1) and G2 = (V2, E2)
Proof by contradiction:
if (u, v) is not the lightest edge, we can remove it, and reconnect T1 and T2 with a lighter edge (x, y)
T1, T2 and (x, y) form a new spanning tree T’
W(T’) < W(T). Contradiction. T1 T2 x y u v

14. Algorithms Generic idea: Two of the most well-known algorithms
Compute MSTs for sub-graphs
Connect two MSTs for sub-graphs with the lightest edge
Two of the most well-known algorithms
Prim’s algorithm
Kruskal’s algorithm
Let’s first talk about the ideas behind the algorithms without worrying about the implementation and analysis

15. Prim’s algorithm Basic idea: Start from an arbitrary single node
A MST for a single node has no edge
Gradually build up a single larger and larger MST 6 5
Not yet discovered 7
Fully explored nodes
Discovered but not fully explored nodes

16. Prim’s algorithm Basic idea: Start from an arbitrary single node
A MST for a single node has no edge
Gradually build up a single larger and larger MST 6 2 5 9
Not yet discovered 4 7
Fully explored nodes
Discovered but not fully explored nodes

17. Prim’s algorithm Basic idea: Start from an arbitrary single node
A MST for a single node has no edge
Gradually build up a single larger and larger MST 2 6 5 9 4 7

18. Prim’s algorithm in words
Randomly pick a vertex as the initial tree T
Gradually expand into a MST:
For each vertex that is not in T but directly connected to some nodes in T
Compute its minimum distance to any vertex in T
Select the vertex that is closest to T
Add it to T

19. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d

20. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d

21. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d

22. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d

23. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d

24. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d

25. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d

26. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d

27. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d

28. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d

29. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d

30. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d

31. Pseudocode for Prim’s algorithm
Given G = (V, E). Output: a MST T.
Randomly select a vertex v
S = {v}; A = V \ S; T = {}.
While (A is not empty)
find a vertex u  A that connects to vertex v  S such that w(u, v) ≤ w(x, y), for any x  A and y  S
S = S U {u}; A = A \ {u}; T = T U (u, v).
End
Return T

32. Time complexity Given G = (V, E). Output: a MST T.
Randomly select a vertex v
S = {v}; A = V \ S; T = {}.
While (A is not empty)
find a vertex u  A that connects to vertex v  S such that w(u, v) ≤ w(x, y), for any x  A and y  S
S = S U {u}; A = A \ {u}; T = T U (u, v).
End
Return T
n vertices
Time complexity: n * (time spent on purple line per vertex)
Naïve: test all edges => Θ(n * m)
Improve: keep the list of candidates in an array => Θ(n2)
Better: with priority queue => Θ(m log n)

33. Idea 1: naive
find a vertex u  A that connects to vertex v  S such that w(u, v) ≤ w(x, y), for any x  A and y  S
min_weight = infinity.
For each edge (x, y)  E
if x  A, y  S, and w(x, y) < min_weight
u = x; v = y; min_weight = w(x, y);
time spent per vertex: Θ(m)
Total time complexity: Θ(n*m)

34. Idea 2: distance array For each v  V d[v] = infinity; p[v] = null;
// For each vertex v, d[v] is the min distance from v to any node already in S
// p[v] is the parent node of v in the spanning tree
For each v  V
d[v] = infinity; p[v] = null;
Randomly select a v, d[v] = 1; // d[v]=1 just to ensure proper start
S = {}; A = V; T = {}.
While (A is not empty)
Search d to find the smallest d[u] > 0.
S = S U {u}; A = A \ {u}; T = T U (u, p[u]);
d[u] = 0.
For each v in adj[u]
if d[v] > w(u, v)
d[v] = w(u, v);
p[v] = u;
End

35. a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d a b c d e f g h ∞ ∞ 1 p / /

36. a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d a b c d e f g h ∞ ∞ p / /

37. a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d a b c d e f g h ∞ 14 3 ∞ p /
Discovered 14 7 15 8 c e h 3 10
Explored d
Not discovered a b c d e f g h ∞ 14 3 ∞ p / /

38. a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d a b c d e f g h ∞ 14 ∞ p / /

39. a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d a b c d e f g h ∞ 8 10 ∞ p /10 ∞ p / /

40. a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d a b c d e f g h ∞ 10 ∞ p / /

41. a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d a b c d e f g h 6 10 5 ∞ p b10 5 ∞ p b /

42. a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d a b c d e f g h 6 10 ∞ p b /

43. a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d a b c d e f g h 6 7 9 ∞ p b7 9 ∞ p b /

44. a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d a b c d e f g h 0 7 9 ∞ p b7 9 ∞ p b /

45. a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d a b c d e f g h 0 9 ∞ p b /

46. a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d a b c d e f g h 0 9 15 p b /

47. a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d a b c d e f g h 0 15 p b /

48. a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d a b c d e f g h 0 p b /

49. Time complexity? n vertices Θ(n) per vertex Overall complexity: Θ (n2)
// For each vertex v, d[v] is the min distance from v to any node already in S
// p[v] is the parent node of v in the spanning tree
For each v  V
d[v] = infinity; p[v] = null;
Randomly select a v, d[v] = 1; // d[v]=1 just to ensure proper start
S = {}. T = {}. A = V.
While (A is not empty)
Search d to find the smallest d[u] > 0.
S = S U {u}; A = A \ {u}; T = T U (u, p[u]);
d[u] = 0.
For each v in adj[u]
if d[v] > w(u, v)
d[v] = w(u, v);
p[v] = u;
End
n vertices
Θ(n) per vertex
O(n) per vertex if using adj list
(n) per vertex if using adj matrix
Overall complexity: Θ (n2)

50. Idea 3: priority queue (min-heap)
Observation
In idea 2, we need to search the distance array n times, each time we only look for the minimum element.
Distance array size = n
n x n = n2
Can we do better?
Priority queue (heap) enables fast retrieval of min (or max) elements
log (n) for most operations

51. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d a b c d e f g h ∞ ∞

52. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d ∞ c b a d e f g h 0
ChangeKey c b a d e f g h 0 ∞

53. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d ∞ h b a d e f g
ExctractMin h b a d e f g ∞

54. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d 3 14 ∞ d b a h e f g
ChangeKey d b a h e f g 3 14 ∞

55. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d ∞ b g a h e f 14
ExctractMin b g a h e f
14 ∞

56. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d 10 ∞ b e a h g f 8
Changekey b e a h g f 8 10 ∞

57. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d 10 ∞ e f a h g
ExtractMin e f a h g 10 ∞

58. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d 5 10 6 ∞ f e a h g
Changekey f e a h g 5 10 6 ∞

59. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d 6 10 ∞ a e g h
ExtractMin a e g h 6 10 ∞

60. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d 6 7 9 ∞ a e g h
Changekey a e g h 6 7 9 ∞

61. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d
ExtractMin e h g 7 ∞ 9

62. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d
ExtractMin g h 9 ∞

63. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d
Changekey g h 9 15

64. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d
ExtractMin h 15

65. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d

66. Complete Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u  Q
key[u] = ;
p[u] = null;
key[r] = 0;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
ChangeKey(v, w(u,v));
Overall running time: Θ(m log n)
Cost per ChangeKey
n vertices
Θ(n) times
Θ(n2) times?
Θ(m) times (with adj list)
How often is ExtractMin() called?
How often is ChangeKey() called?

67. Kruskal’s algorithm in words
Procedure:
Sort all edges into non-decreasing order
Initially each node is in its own tree
For each edge in the sorted list
If the edge connects two separate trees, then
join the two trees together with that edge

68. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d c-d: 3 b-f: 5 b-a: 6
f-e: 7
b-d: 8
f-g: 9
d-e: 10
a-f: 12
b-c: 14
e-h: 15 a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d

69. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d c-d: 3 b-f: 5 b-a: 6
f-e: 7
b-d: 8
f-g: 9
d-e: 10
a-f: 12
b-c: 14
e-h: 15 a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d

70. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d c-d: 3 b-f: 5 b-a: 6
f-e: 7
b-d: 8
f-g: 9
d-e: 10
a-f: 12
b-c: 14
e-h: 15 a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d

71. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d c-d: 3 b-f: 5 b-a: 6
f-e: 7
b-d: 8
f-g: 9
d-e: 10
a-f: 12
b-c: 14
e-h: 15 a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d

72. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d c-d: 3 b-f: 5 b-a: 6
f-e: 7
b-d: 8
f-g: 9
d-e: 10
a-f: 12
b-c: 14
e-h: 15 a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d

73. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d c-d: 3 b-f: 5 b-a: 6
f-e: 7
b-d: 8
f-g: 9
d-e: 10
a-f: 12
b-c: 14
e-h: 15 a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d

74. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d c-d: 3 b-f: 5 b-a: 6
f-e: 7
b-d: 8
f-g: 9
d-e: 10
a-f: 12
b-c: 14
e-h: 15 a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d

75. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d c-d: 3 b-f: 5 b-a: 6
f-e: 7
b-d: 8
f-g: 9
d-e: 10
a-f: 12
b-c: 14
e-h: 15 a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d

76. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d c-d: 3 b-f: 5 b-a: 6
f-e: 7
b-d: 8
f-g: 9
d-e: 10
a-f: 12
b-c: 14
e-h: 15 a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d

77. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d c-d: 3 b-f: 5 b-a: 6
f-e: 7
b-d: 8
f-g: 9
d-e: 10
a-f: 12
b-c: 14
e-h: 15 a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d

78. Example a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d c-d: 3 b-f: 5 b-a: 6
f-e: 7
b-d: 8
f-g: 9
d-e: 10
a-f: 12
b-c: 14
e-h: 15 a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d

79. Time complexity Depend on implementation Pseudocode
sort all edges according to weights
T = {}. tree(v) = v for all v.
for each edge (u, v)
if tree(u) != tree(v)
T = T U (u, v);
union (tree(u), tree(v))
Θ(m log m)
= Θ(m log n)
m edges
m times
(n-1) times.
Overall time complexity: m log n + m t + n u
Naïve: Θ(nm)
Better implementation: Θ(m log n)
tree union
tree finding

80. Disjoint Set Union
Use a linked list to represent tree elements, with pointers back to root:
tree[u] != tree[v]: O(1)
Union(tree[u],tree[v]): “Copy” elements of A into set B by adjusting elements of A to point to B
Each union may take O(n) time
Precisely n-1 unions
O(n2)?

81. Disjoint Set Union Better strategy and analysis
Always copy smaller list into larger list
Size of combined list is at least twice the size of smaller list
Each vertex copied at most logn times before a single tree emerges
Total number of copy operations for n vertex is therefore O(n log n)
Overall time complexity: m log n + m t + n u
t: time for finding tree root is constant
u: time for n union operations is at most nlog (n)
m log n + m t + n u = Θ (m log n + m + n log n) = Θ (m log n)
Conclusion
Kruskal’s algorithm runs in Θ(m log n) for both dense and sparse graph

82. Example with disjoint set union
c-d: 3
b-f: 5
b-a: 6
f-e: 7
b-d: 8
f-g: 9
d-e: 10
a-f: 12
b-c: 14
e-h: 15 a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d a b c d e f g h

83. Example with disjoint set union
c-d: 3
b-f: 5
b-a: 6
f-e: 7
b-d: 8
f-g: 9
d-e: 10
a-f: 12
b-c: 14
e-h: 15 a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d a b c d e f g h

84. Example with disjoint set union
c-d: 3
b-f: 5
b-a: 6
f-e: 7
b-d: 8
f-g: 9
d-e: 10
a-f: 12
b-c: 14
e-h: 15 a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d a b f c d e g h

85. Example with disjoint set union
c-d: 3
b-f: 5
b-a: 6
f-e: 7
b-d: 8
f-g: 9
d-e: 10
a-f: 12
b-c: 14
e-h: 15 a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d b f a c d e g h

86. Example with disjoint set union
c-d: 3
b-f: 5
b-a: 6
f-e: 7
b-d: 8
f-g: 9
d-e: 10
a-f: 12
b-c: 14
e-h: 15 a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d b f e a c d g h

87. Example with disjoint set union
c-d: 3
b-f: 5
b-a: 6
f-e: 7
b-d: 8
f-g: 9
d-e: 10
a-f: 12
b-c: 14
e-h: 15 a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d b c d f e a g h

88. Example with disjoint set union
c-d: 3
b-f: 5
b-a: 6
f-e: 7
b-d: 8
f-g: 9
d-e: 10
a-f: 12
b-c: 14
e-h: 15 a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d b c d f g e a h

89. Example with disjoint set union
c-d: 3
b-f: 5
b-a: 6
f-e: 7
b-d: 8
f-g: 9
d-e: 10
a-f: 12
b-c: 14
e-h: 15 a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d b c d f g e a h

90. Example with disjoint set union
c-d: 3
b-f: 5
b-a: 6
f-e: 7
b-d: 8
f-g: 9
d-e: 10
a-f: 12
b-c: 14
e-h: 15 a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d b c d f g e a h

91. Example with disjoint set union
c-d: 3
b-f: 5
b-a: 6
f-e: 7
b-d: 8
f-g: 9
d-e: 10
a-f: 12
b-c: 14
e-h: 15 a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d b c d f g e a h

92. Example with disjoint set union
c-d: 3
b-f: 5
b-a: 6
f-e: 7
b-d: 8
f-g: 9
d-e: 10
a-f: 12
b-c: 14
e-h: 15 a 6 12 5 9 b f g 14 7 15 8 c e h 3 10 d b c d f g e h a

93. Disjoint Set Union Better strategy and analysis
Always copy smaller list into larger list
Size of combined list is at least twice the size of smaller list
Each vertex copied at most logn times before a single tree emerges
Total number of copy operations for n vertex is therefore O(n log n)
Overall time complexity: m log n + m t + n u
t: time for finding tree root is constant
u: time for n union operations is at most nlog (n)
m log n + m t + n u = Θ (m log n + m + n log n) = Θ (m log n)
Conclusion
Kruskal’s algorithm runs in Θ(m log n) for both dense and sparse graph
How about using counting sort?
Θ(m + n log n)

94. Summary Kruskal’s algorithm Prim’s algorithm Θ(m log n)
Possibly Θ(m + n log n) with counting sort
Prim’s algorithm
With priority queue : Θ(m log n)
Assume graph represented by adj list
With distance array : Θ(n^2)
Adj list or adj matrix
For sparse graphs priority queue wins
For dense graphs distance array may be better