1. Algorithm Analysis Fall 2017 CS 4306/03
Graph Algorithms (Ch 22.1, Ch 23.1)
Graph representation
Minimum spanning trees
Greedy Algorithms (Ch 23.1)
Optimal substructure
Greedy choice
MST Algorithms (Ch 23.2)
Prim’s MST algorithm
Kruskal’s MST algorithm
Dr. Donghyun Kim
Department of Computer Science
College of Computing and Software Engineering
Kennesaw State University
Marietta, GA, USA
This slide is courtesy of Prof. Charles E. Leiserson at Massachusetts Institute of Technology.

2. Graphs (review) Definition. A directed graph (digraph)
G = (V, E) is an ordered pair consisting of
a set V of vertices (singular: vertex),
a set E  V  V of edges.
In an undirected graph G = (V, E), the edge set E consists of unordered pairs of vertices.
In either case, we have | E | = O(V 2). Moreover, if G is connected, then | E |  | V | – 1, which implies that lg | E | = (lg V).
(Review CLRS, Appendix B.)
Fall 2016
CS 4306 Algorithm Analysis

3. Adjacency-matrix representation
The adjacency matrix of a graph G = (V, E), where V = {1, 2, …, n}, is the matrix A[1 . . n, n] given by
1 if (i, j)  E,
0 if (i, j)  E.
A[i, j] =
Fall 2016
CS 4306 Algorithm Analysis

4. Adjacency-matrix representation
The adjacency matrix of a graph G = (V, E), where V = {1, 2, …, n}, is the matrix A[1 . . n, n] given by
1 if (i, j)  E,
0 if (i, j)  E.
A[i, j] = A 1 2 3 4
(V 2) storage
 dense
representation.
Fall 2016
CS 4306 Algorithm Analysis

5. Adjacency-list representation
An adjacency list of a vertex v  V is the list Adj[v] of vertices adjacent to v.
Adj[1] = {2, 3}
Adj[2] = {3}
Adj[3] = {}
Adj[4] = {3}
Fall 2016
CS 4306 Algorithm Analysis

6. Adjacency-list representation
An adjacency list of a vertex v  V is the list Adj[v] of vertices adjacent to v.
Adj[1] = {2, 3}
Adj[2] = {3}
Adj[3] = {}
Adj[4] = {3}
For undirected graphs, | Adj[v] | = degree(v). For digraphs, | Adj[v] | = out-degree(v).
Fall 2016
CS 4306 Algorithm Analysis

7. Adjacency-list representation
An adjacency list of a vertex v  V is the list Adj[v] of vertices adjacent to v.
Adj[1] = {2, 3}
Adj[2] = {3}
Adj[3] = {}
Adj[4] = {3}
For undirected graphs, | Adj[v] | = degree(v). For digraphs, | Adj[v] | = out-degree(v).
Handshaking Lemma: vV = 2 |E| for undirected graphs  adjacency lists use (V + E) storage — a sparse representation (for either type of graph).
Fall 2016
CS 4306 Algorithm Analysis

8. Minimum spanning trees
Input: A connected, undirected graph G = (V, E) with weight function w : E  R.
For simplicity, assume that all edge weights are distinct. (CLRS covers the general case.)
Fall 2016
CS 4306 Algorithm Analysis

9. Minimum spanning trees
Input: A connected, undirected graph G = (V, E) with weight function w : E  R.
For simplicity, assume that all edge weights are distinct. (CLRS covers the general case.)
Output: A spanning tree T — a tree that connects all vertices — of minimum weight:
 w(u, v) .
(u,v)T
w(T ) 
Fall 2016
CS 4306 Algorithm Analysis

10. Example of MST
6 12 5 9 14 7 15 8 3 10
Fall 2016
CS 4306 Algorithm Analysis

11. Example of MST 6 12 5 9 14 7 15 8 3 10
Fall 2016
CS 4306 Algorithm Analysis

12. Optimal substructure MST T: (Other edges of G are not shown.)
Fall 2016
CS 4306 Algorithm Analysis

13. Optimal substructure MST T: u v Remove any edge (u, v)  T.
(Other edges of G
are not shown.) u v
Remove any edge (u, v)  T.
Fall 2016
CS 4306 Algorithm Analysis

14. Optimal substructure MST T: u v Remove any edge (u, v)  T.
(Other edges of G
are not shown.) u v
Remove any edge (u, v)  T.
Fall 2016
CS 4306 Algorithm Analysis

15. Optimal substructure MST T: u T T1 v
(Other edges of G
are not shown.) u T 2 T1 v
Remove any edge (u, v)  T. Then, T is partitioned into two subtrees T1 and T2.
Fall 2016
CS 4306 Algorithm Analysis

16. Optimal substructure MST T: u T T1 v
(Other edges of G
are not shown.) u T 2 T1 v
Remove any edge (u, v)  T. Then, T is partitioned into two subtrees T1 and T2.
Theorem. The subtree T1 is an MST of G1 = (V1, E1), the subgraph of G induced by the vertices of T1:
V1 = vertices of T1,
E1 = { (x, y)  E : x, y  V1 }.
Similarly for T2.
Fall 2016
CS 4306 Algorithm Analysis

17. Proof of optimal substructure
Proof. Cut and paste:
w(T) = w(u, v) + w(T1) + w(T2).
If T1 were a lower-weight spanning tree than T1 for
G1, then T  = {(u, v)}  T1  T2 would be a
lower-weight spanning tree than T for G.
Fall 2016
CS 4306 Algorithm Analysis

18. Proof of optimal substructure
Proof. Cut and paste:
w(T) = w(u, v) + w(T1) + w(T2).
If T1 were a lower-weight spanning tree than T1 for
G1, then T  = {(u, v)}  T1  T2 would be a
lower-weight spanning tree than T for G.
Do we also have overlapping subproblems?
•Yes.
Fall 2016
CS 4306 Algorithm Analysis

19. Proof of optimal substructure
Proof. Cut and paste:
w(T) = w(u, v) + w(T1) + w(T2).
If T1 were a lower-weight spanning tree than T1 for
G1, then T  = {(u, v)}  T1  T2 would be a
lower-weight spanning tree than T for G.
Do we also have overlapping subproblems?
•Yes.
Great, then dynamic programming may work!
•Yes, but MST exhibits another powerful property which leads to an even more efficient algorithm.
Fall 2016
CS 4306 Algorithm Analysis

20. Greedy-choice property A locally optimal choice is globally optimal.
Hallmark for “greedy” algorithms
Greedy-choice property A locally optimal choice is globally optimal.
Fall 2016
CS 4306 Algorithm Analysis

21. Hallmark for “greedy” algorithms
Greedy-choice property A locally optimal choice is globally optimal.
Theorem. Let T be the MST of G = (V, E), and let A  V. Suppose that (u, v)  E is the least-weight edge connecting A to V – A. Then, (u, v)  T.
Fall 2016
CS 4306 Algorithm Analysis

22. Proof of theorem Proof. Suppose (u, v)  T. Cut and paste. T: v u  A
 V – A
(u, v) = least-weight edge connecting A to V – A
Fall 2016
CS 4306 Algorithm Analysis

23. Proof of theorem Proof. Suppose (u, v)  T. Cut and paste. T: v u  A
 V – A
(u, v) = least-weight edge connecting A to V – A
Consider the unique simple path from u to v in T.
Fall 2016
CS 4306 Algorithm Analysis

24. Proof of theorem Proof. Suppose (u, v)  T. Cut and paste. T: v u  A
 V – A
(u, v) = least-weight edge connecting A to V – A
Consider the unique simple path from u to v in T.
Swap (u, v) with the first edge on this path that connects a vertex in A to a vertex in V – A.
Fall 2016
CS 4306 Algorithm Analysis

25. Proof of theorem Proof. Suppose (u, v)  T. Cut and paste. T : v u
 V – A
(u, v) = least-weight edge connecting A to V – A
Consider the unique simple path from u to v in T.
Swap (u, v) with the first edge on this path that connects a vertex in A to a vertex in V – A.
A lighter-weight spanning tree than T results.
Fall 2016
CS 4306 Algorithm Analysis

26. Example of Prim’s algorithm
 A
 V – A 
6 12 5  9 
7 15 10   14  8   3 
Fall 2016
CS 4306 Algorithm Analysis

27. Example of Prim’s algorithm
 A
 V – A 
6 12 5  9 
7 15 10   14  8   3 
Fall 2016
CS 4306 Algorithm Analysis

28. Example of Prim’s algorithm
 A
 V – A 
6 12 5  9    14  7 10 15 15 8  3 
Fall 2016
CS 4306 Algorithm Analysis

29. Example of Prim’s algorithm
 A
 V – A 
6 12
5 9
    14  7 10 15 15 8  3 
Fall 2016
CS 4306 Algorithm Analysis

30. Example of Prim’s algorithm
 V – A  5 6 12 9  5  9  14  7 10 15 15 8  3  
Fall 2016
CS 4306 Algorithm Analysis

31. Example of Prim’s algorithm
 V – A  5 6 12 9   9  14  7 10 15 15 8  3  
Fall 2016
CS 4306 Algorithm Analysis

32. Example of Prim’s algorithm
 V – A  5 6 12 9   9  14  7 15 15 8  3 10  
Fall 2016
CS 4306 Algorithm Analysis

33. Example of Prim’s algorithm
 A
 V – A 6 12 5 9   9  14  7 15 15 8  3 10 
Fall 2016
CS 4306 Algorithm Analysis

34. Example of Prim’s algorithm
 A
 V – A 6 12 5 9   9  14  7 15 15 8  3 10 
Fall 2016
CS 4306 Algorithm Analysis

35. Example of Prim’s algorithm
 A
 V – A 6 12 5 9   9  14  7 10 15 15 8  3 
Fall 2016
CS 4306 Algorithm Analysis

36. Example of Prim’s algorithm
 A
 V – A 6 12 5 9   9  14  7 10 15 15 8  3 
Fall 2016
CS 4306 Algorithm Analysis

37. Example of Prim’s algorithm
 A
 V – A 6 12 5 9    14  7 10 15 15 8  3 
Fall 2016
CS 4306 Algorithm Analysis

38. Example of Prim’s algorithm
 A
 V – A 6 12 5 9    14  7 10 15 8  3 
Fall 2016
CS 4306 Algorithm Analysis

39. Prim’s algorithm
IDEA: Maintain V – A as a priority queue Q. Key each vertex in Q with the weight of the least-weight edge connecting it to a vertex in A={v | vV-{s}-Q}.
Q  V
key[v]   for all v  V
key[s]  0 for some arbitrary s  V.
while Q  
do u  EXTRACT-MIN(Q)
for each v  Adj[u]
do if v  Q and w(u, v) < key[v]
// item in Q with minimum key, initially s
then key[v]  w(u, v)
[v]  u
⊳ DECREASE-KEY
At the end, {(v, [v])} forms the MST.
Fall 2016
CS 4306 Algorithm Analysis

40. Analysis of Prim Q  V key[v]   for all v  V
key[s]  0 for some arbitrary s  V
while Q  
do u  EXTRACT-MIN(Q)
for each v  Adj[u]
do if v  Q and w(u, v) < key[v]
then key[v]  w(u, v)
[v]  u
Fall 2016
CS 4306 Algorithm Analysis

41. Analysis of Prim Q  V key[v]   for all v  V (V)
key[s]  0 for some arbitrary s  V
while Q  
do u  EXTRACT-MIN(Q)
for each v  Adj[u]
do if v  Q and w(u, v) < key[v]
then key[v]  w(u, v)
[v]  u
Q  V
(V)
total
Fall 2016
CS 4306 Algorithm Analysis

42. Analysis of Prim Q  V key[v]   for all v  V (V)
key[s]  0 for some arbitrary s  V
while Q  
do u  EXTRACT-MIN(Q)
for each v  Adj[u]
do if v  Q and w(u, v) < key[v]
then key[v]  w(u, v)
[v]  u
Q  V
(V)
total
|V |
times
Fall 2016
CS 4306 Algorithm Analysis

43. Analysis of Prim key[v]   for all v  V
key[s]  0 for some arbitrary s  V
while Q  
do u  EXTRACT-MIN(Q)
for each v  Adj[u]
Q  V
(V)
total
|V |
times
degree(u) times
do if v  Q and w(u, v) < key[v]
then key[v]  w(u, v)
[v]  u
Fall 2016
CS 4306 Algorithm Analysis

44. Analysis of Prim key[v]   for all v  V
key[s]  0 for some arbitrary s  V
while Q  
do u  EXTRACT-MIN(Q)
for each v  Adj[u]
Q  V
(V)
total
|V |
times
degree(u) times
do if v  Q and w(u, v) < key[v]
then key[v]  w(u, v)
[v]  u
Handshaking Lemma  (E) implicit DECREASE-KEY’s.
Fall 2016
CS 4306 Algorithm Analysis

45. Time = (V)·TEXTRACT-MIN + (E)·TDECREASE-KEY
Analysis of Prim
key[v]   for all v  V
key[s]  0 for some arbitrary s  V
while Q  
do u  EXTRACT-MIN(Q)
for each v  Adj[u]
Q  V
(V)
total
|V |
times
degree(u) times
do if v  Q and w(u, v) < key[v]
then key[v]  w(u, v)
[v]  u
Handshaking Lemma  (E) implicit DECREASE-KEY’s.
Time = (V)·TEXTRACT-MIN + (E)·TDECREASE-KEY
Fall 2016
CS 4306 Algorithm Analysis

46. Analysis of Prim (continued)
Time = (V)·TEXTRACT-MIN + (E)·TDECREASE-KEY
Fall 2016
CS 4306 Algorithm Analysis

47. Analysis of Prim (continued)
Time = (V)·TEXTRACT-MIN + (E)·TDECREASE-KEY Q
Total
TEXTRACT-MIN TDECREASE-KEY
Fall 2016
CS 4306 Algorithm Analysis

48. Analysis of Prim (continued)
Time = (V)·TEXTRACT-MIN + (E)·TDECREASE-KEY
TEXTRACT-MIN TDECREASE-KEY O(V) O(1) Q
Total
array
O(V2)
Fall 2016
CS 4306 Algorithm Analysis

49. Analysis of Prim (continued)
Time = (V)·TEXTRACT-MIN + (E)·TDECREASE-KEY
TEXTRACT-MIN TDECREASE-KEY O(V) O(1)
O(lg V) O(lg V) Q
Total
array
binary heap
O(V2)
O(E lg V)
Fall 2016
CS 4306 Algorithm Analysis

50. Analysis of Prim (continued)
Time = (V)·TEXTRACT-MIN + (E)·TDECREASE-KEY Q
Total
TEXTRACT-MIN TDECREASE-KEY
array
binary heap
Fibonacci heap
O(V) O(1)
O(V2)
O(E lg V) O(E + V lg V)
worst case
O(lg V)
O(lg V) amortized
O(lg V)
O(1)
amortized
Fall 2016
CS 4306 Algorithm Analysis

51. Kruskal’s algorithm Kruskal’s algorithm (see CLRS):
Uses the disjoint-set data structure (Lecture 10).
Running time = O(E lg V).
Best to date:
Karger, Klein, and Tarjan [1993].
Randomized algorithm.
O(V + E) expected time.
Fall 2016
CS 4306 Algorithm Analysis

52. Kruskal’s algorithm Main Idea
Use a priority queue using the edge weights as keys
Each node starts off as it’s own tree
While the queue is not empty, if the edge retrieved connects two trees, connect them, if not, discard it
Once the queue is empty, you are left with the minimum spanning tree
Fall 2016
CS 4306 Algorithm Analysis

53. Kruskal's Minimum Spanning Tree Algorithm2 23 3 9 1 18 14 2 6 6 4 30 19 11 14 5 5 6 20 16 8 7 44

54. Kruskal's Minimum Spanning Tree Algorithm
PQ = ( 2, 5, 6, 6, 9, 11, 14, 14, 16, 18, 19, 20, 23, 30, 44 ) 2 23 3 9 1 18 14 2 6 6 4 30 19 11 14 5 5 6 20 16 8 7 44

55. Kruskal's Minimum Spanning Tree Algorithm
PQ = ( 2, 5, 6, 6, 9, 11, 14, 14, 16, 18, 19, 20, 23, 30, 44 ) 2 23 3 9 1
MIN 18 14 2 6 6 4 30 19 11 14 5 5 6 20 16 8 7 44

56. Kruskal's Minimum Spanning Tree Algorithm
PQ = ( 5, 6, 6, 9, 11, 14, 14, 16, 18, 19, 20, 23, 30, 44 ) 2 23 3 9 1 18 14 2 6 6 4 30 19 11 14 5 5 6 20 16 8 7 44

57. Kruskal's Minimum Spanning Tree Algorithm
PQ = ( 5, 6, 6, 9, 11, 14, 14, 16, 18, 19, 20, 23, 30, 44 ) 2 23 3 9 1 18 14 2 6 6 4 30 19
MIN 11 14 5 5 6 20 16 8 7 44

58. Kruskal's Minimum Spanning Tree Algorithm
PQ = ( 6, 6, 9, 11, 14, 14, 16, 18, 19, 20, 23, 30, 44 ) 2 23 3 9 1 18 14 2 6 6 4 30 19 11 14 5 5 6 20 16 8 7 44

59. Kruskal's Minimum Spanning Tree Algorithm
PQ = ( 6, 6, 9, 11, 14, 14, 16, 18, 19, 20, 23, 30, 44 ) 2 23 3 9 1 18 14 2 6 6 4 30
MIN 19 11 14 5 5 6 20 16 8 7 44

60. Kruskal's Minimum Spanning Tree Algorithm
PQ = ( 6, 9, 11, 14, 14, 16, 18, 19, 20, 23, 30, 44 ) 2 23 3 9 1 18 14 2 6 6 4 30 19 11 14 5 5 6 20 16 8 7 44

61. Kruskal's Minimum Spanning Tree Algorithm
PQ = ( 6, 9, 11, 14, 14, 16, 18, 19, 20, 23, 30, 44 ) 2 23 3 9 1 18 14 2 6 6 4 30 19 11 14 5 5 6 20 16
MIN 8 7 44

62. Kruskal's Minimum Spanning Tree Algorithm
PQ = ( 9, 11, 14, 14, 16, 18, 19, 20, 23, 30, 44 ) 2 23 3 9 1 18 14 2 6 6 4 30 19 11 14 5 5 6 20 16 8 7 44

63. Kruskal's Minimum Spanning Tree Algorithm
PQ = ( 9, 11, 14, 14, 16, 18, 19, 20, 23, 30, 44 )
MIN 2 23 3 9 1 18 14 2 6 6 4 30 19 11 14 5 5 6 20 16 8 7 44

64. Kruskal's Minimum Spanning Tree Algorithm
PQ = ( 11, 14, 14, 16, 18, 19, 20, 23, 30, 44 ) 2 23 3 9 1 18 14 2 6 6 4 30 19 11 14 5 5 6 20 16 8 7 44

65. Kruskal's Minimum Spanning Tree Algorithm
PQ = ( 11, 14, 14, 16, 18, 19, 20, 23, 30, 44 ) 2 23 3 9 1 18 14 2 6 6 4 30 19 11 14 5 5
MIN 6 20 16 8 7 44

66. Kruskal's Minimum Spanning Tree Algorithm
PQ = ( 14, 14, 16, 18, 19, 20, 23, 30, 44 ) 2 23 3 9 1 18 14 2 6 6 4 30 19 11 14 5 5 6 20 16 8 7 44

67. Kruskal's Minimum Spanning Tree Algorithm
PQ = ( 14, 14, 16, 18, 19, 20, 23, 30, 44 ) 2 23 3 9 1 18 14 2 6 6 4 30 19 11 14 5 5 6 20 16 8 7 44

68. Kruskal's Minimum Spanning Tree Algorithm
PQ = ( 14, 14, 16, 18, 19, 20, 23, 30, 44 ) 2 23 3 9
MIN 1 18 14 2 6 6 4 30 19 11 14 5 5 6 20 16 8 7 44

69. Kruskal's Minimum Spanning Tree Algorithm
PQ = ( 14, 16, 18, 19, 20, 23, 30, 44 ) 2 23 3 9 1 18 14 2 6 6 4 30 19 11 14 5 5 6 20 16 8 7 44

70. Kruskal's Minimum Spanning Tree Algorithm
PQ = ( 14, 16, 18, 19, 20, 23, 30, 44 ) 2 23 3 9 1 18 14 2 6 6 4 30 19 11 14 5 5 6
MIN 20 16 8 7 44

71. Kruskal's Minimum Spanning Tree Algorithm
PQ = ( 16, 18, 19, 20, 23, 30, 44 ) 2 23 3 9 1 18 14 2 6 6 4 30 19 11 14 5 5 6 20 16 8 7 44

72. Kruskal's Minimum Spanning Tree Algorithm
PQ = ( 16, 18, 19, 20, 23, 30, 44 ) 2 23 3 9 1 18 14 2 6 6 4 30 19 11 14 5 5 6 20 16 8 7 44

73. Kruskal's Minimum Spanning Tree Algorithm
PQ = ( 16, 18, 19, 20, 23, 30, 44 ) 2 23 3 9 1 18 14 2 6 6 4 30 19 11 14 5 5 6 20 16
MIN 8 7 44

74. Kruskal's Minimum Spanning Tree Algorithm
PQ = ( 18, 19, 20, 23, 30, 44 ) 2 23 3 9 1 18 14 2 6 6 4 30 19 11 14 5 5 6 20 16 8 7 44

75. Kruskal's Minimum Spanning Tree Algorithm
PQ = ( 18, 19, 20, 23, 30, 44 ) 2 23 3 9 1 18 14 2 6 6 4 30 19 11 14 5 5 6 20 16 8 7 44

76. Kruskal's Minimum Spanning Tree Algorithm
PQ = ( 18, 19, 20, 23, 30, 44 ) 2 23 3
MIN 9 1 18 14 2 6 6 4 30 19 11 14 5 5 6 20 16 8 7 44

77. Kruskal's Minimum Spanning Tree Algorithm
PQ = ( 19, 20, 23, 30, 44 ) 2 23 3 9 1 18 14 2 6 6 4 30 19 11 14 5 5 6 20 16 8 7 44

78. Problem Set (for Exam 2) Exercises 22.1-1, 22.1-2
* Your answer must be well-organized; starts from given facts and assumptions, follow your logics line by line, and should be connected to the conclusion to support your answer.
Fall 2017
CS 4306 Algorithm Analysis

79. Problem Set (for Exam 3)
You should be able to run each MST algorithm for a given input graph.
Exercises , , ,
Problem 23-1
* Your answer must be well-organized; starts from given facts and assumptions, follow your logics line by line, and should be connected to the conclusion to support your answer.
Fall 2017
CS 4306 Algorithm Analysis