1. CS6045: Advanced Algorithms
Graph Algorithms

2. Graph Representation A graph G = (V, E) V = set of vertices
E = set of edges = subset of V  V
Thus |E| = O(|V|2)

3. Graph Variations Variations:
A connected graph has a path from every vertex to every other
In an undirected graph:
Edge (u,v) = edge (v,u)
No self-loops
In a directed graph:
Edge (u,v) goes from vertex u to vertex v, notated uv

4. Graph Variations More variations:
A weighted graph associates weights with either the edges or the vertices
e.g., a road map: edges might be weighted with distance
A hypergraph allows multiple edges between the same vertices
e.g., the call graph in a program (a function can get called from multiple points in another function)

5. Graphs We will typically express running times in terms of |E| and |V|
If |E|  |V|2 the graph is dense
If |E|  |V| the graph is sparse
If you know you are dealing with dense or sparse graphs, different data structures may make sense

6. Representing Graphs Assume V = {1, 2, …, n}
An adjacency matrix represents the graph as a n x n matrix A:
A[i, j] = 1, if edge (i, j)  E = 0, if edge (i, j)  E
A[i, j] = weight on the edge, if edge (i, j)  E = 0, if edge (i, j)  E

7. Graphs: Adjacency Matrix
Example: A 1 2 3 4 ?? 1 a 2 d 4 b c 3

8. Graphs: Adjacency Matrix
Example: A 1 2 3 4 1 a 2 d 4 b c 3
How much storage does the adjacency matrix require?
A: O(V2)

9. Graphs: Adjacency Matrix
The adjacency matrix is a dense representation
Usually too much storage for large graphs
But can be very efficient for small graphs
Most large interesting graphs are sparse
e.g., planar graphs, in which no edges cross, have |E| = O(|V|) by Euler’s formula
For this reason the adjacency list is often a more appropriate representation

10. Graphs: Adjacency List
Adjacency list: for each vertex v  V, store a list of vertices adjacent to v
Example: For an undirected graph

11. Graphs: Adjacency List
Adjacency list: for each vertex v  V, store a list of vertices adjacent to v
Example: For a directed graph

12. Graphs: Adjacency List
How much storage is required?
The degree of a vertex v = # of edges
Directed graphs have in-degree, out-degree
For directed graphs, # of items in adjacency lists is  out-degree(v) = |E| takes O(V + E) storage
For undirected graphs, # items in adjacency lists is  degree(v) = 2 |E| (handshaking lemma) also O(V + E) storage
So: Adjacency lists take O(V+E) storage

13. Graph Searching Given: a graph G = (V, E), directed or undirected
Goal: methodically explore every vertex and every edge
Ultimately: build a tree on the graph
Pick a vertex as the root
Choose certain edges to produce a tree
Note: might also build a forest if graph is not connected

14. Breadth-First Search “Explore” a graph, turning it into a tree
One vertex at a time
Expand frontier of explored vertices across the breadth of the frontier
Builds a tree over the graph
Pick a source vertex to be the root
Find (“discover”) its children, then their children, etc.

15. Breadth-First Search
Again will associate vertex “colors” to guide the algorithm
White vertices have not been discovered
All vertices start out white
Grey vertices are discovered but not fully explored
They may be adjacent to white vertices
Black vertices are discovered and fully explored
They are adjacent only to black and gray vertices
Explore vertices by scanning adjacency list of grey vertices

16. Breadth-First Search What does v->d represent?
BFS(G, s) {
initialize vertices V.d = ;
s.d = 0;
Q = {s}; // Q is a queue; initialize to s
while (Q not empty) {
u = Dequeue(Q);
for each v  u->adj {
if (v->color == WHITE)
v->color = GREY;
v->d = u->d + 1;
v->p = u;
Enqueue(Q, v); }
u->color = BLACK;
What does v->d represent?
What does v->p represent?

17. Breadth-First Search: Exampleu         v w x y

18. Breadth-First Search: Exampleu        v w x y Q: s

19. Breadth-First Search: Exampleu 1    1   v w x y Q: w r

20. Breadth-First Search: Exampleu 1 2   1 2  v w x y Q: r t x

21. Breadth-First Search: Exampleu 1 2  2 1 2  v w x y Q: t x v

22. Breadth-First Search: Exampleu 1 2 3 2 1 2  v w x y Q: x v u

23. Breadth-First Search: Exampleu 1 2 3 2 1 2 3 v w x y Q: v u y

24. Breadth-First Search: Exampleu 1 2 3 2 1 2 3 v w x y Q: u y

25. Breadth-First Search: Exampleu 1 2 3 2 1 2 3 v w x y Q: y

26. Breadth-First Search: Exampleu 1 2 3 2 1 2 3 v w x y Q: Ø

27. BFS: The Code Again Touch every vertex: O(V)
BFS(G, s) {
initialize vertices V.d = ;
s.d = 0;
Q = {s};
while (Q not empty) {
u = Dequeue(Q);
for each v  u->adj {
if (v->color == WHITE)
v->color = GREY;
v->d = u->d + 1;
v->p = u;
Enqueue(Q, v); }
u->color = BLACK;
u = every vertex, but only once
v = every vertex that appears in some other vert’s adjacency list
What will be the running time?
Total running time: O(V+E)

28. BFS: The Code Again What will be the storage cost
BFS(G, s) {
initialize vertices V.d = ;
s.d = 0;
Q = {s};
while (Q not empty) {
u = Dequeue(Q);
for each v  u->adj {
if (v->color == WHITE)
v->color = GREY;
v->d = u->d + 1;
v->p = u;
Enqueue(Q, v); }
u->color = BLACK;
What will be the storage cost
in addition to storing the tree?
Total space used: O(max(degree(v))) = O(E)

29. Breadth-First Search: Properties
BFS calculates the shortest-path distance to the source node
Shortest-path distance (s,v) = minimum number of edges from s to v, or  if v not reachable from s
BFS builds breadth-first tree, in which paths to root represent shortest paths in G
Thus can use BFS to calculate shortest path from one vertex to another in O(V+E) time

30. Depth-First Search
Depth-first search is another strategy for exploring a graph
Explore “deeper” in the graph whenever possible
Edges are explored out of the most recently discovered vertex v that still has unexplored edges
When all of v’s edges have been explored, backtrack to the vertex from which v was discovered

31. Depth-First Search Vertices initially colored white
Then colored gray when discovered
Then black when finished

32. Depth-First Search: The Code
DFS(G) {
for each vertex u  G->V
u->color = WHITE; }
time = 0;
if (u->color == WHITE)
DFS_Visit(u);
DFS_Visit(u) {
u->color = GREY;
time = time+1;
u->d = time;
for each v  u->Adj[]
if (v->color == WHITE)
DFS_Visit(v); }
u->color = BLACK;
u->f = time;
What does u->d represent?
What does u->f represent?
Will all vertices eventually be colored black?
What will be the running time?

33. Depth-First Search: The Code
DFS(G) {
for each vertex u  G->V
u->color = WHITE; }
time = 0;
if (u->color == WHITE)
DFS_Visit(u);
DFS_Visit(u) {
u->color = GREY;
time = time+1;
u->d = time;
for each v  u->Adj[]
if (v->color == WHITE)
DFS_Visit(v); }
u->color = BLACK;
u->f = time;
Running time: O(n2) because call DFS_Visit on each vertex, and the loop over Adj[] can run as many as |V| times

34. Depth-First Search: The Code
DFS(G) {
for each vertex u  G->V
u->color = WHITE; }
time = 0;
if (u->color == WHITE)
DFS_Visit(u);
DFS_Visit(u) {
u->color = GREY;
time = time+1;
u->d = time;
for each v  u->Adj[]
if (v->color == WHITE)
DFS_Visit(v); }
u->color = BLACK;
u->f = time;
BUT, there is actually a tighter bound. Running time of DFS = O(V+E)

35. DFS Example
source vertex 1 2 3 6 7 8 4 5

36. DFS Example
source vertex
d | f 1 2 3 6 7 8 4 5 1| | | | | | | |

37. DFS Example
source vertex
d | f 1 2 3 1| | | 4 5 2| | | | | 6 7 8

38. DFS Example
source vertex
d | f 1 2 3 6 7 8 4 5 1| | | 2| | 3| | |

39. DFS Example
source vertex
d | f 1 2 3 6 7 8 4 5 1| | | 2| |
3|4 | |

40. DFS Example
source vertex
d | f 1 2 3 6 7 8 4 5 1| | | 2| |
3|4 5| |

41. DFS Example
source vertex
d | f 1 2 3 6 7 8 4 5 1| | | 2| |
3|4
5|6 |

42. DFS Example
source vertex
d | f 1 2 3 6 7 8 4 5 1| | |
2|7 |
3|4
5|6 |

43. DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1| 8| | 2|7 | 3|4 5|6

44. What do the grey vertices represent?
DFS Example
source vertex
d | f 1 2 3 6 7 8 4 5 1| 8| |
2|7 9|
3|4
5|6 |
What do the grey vertices represent?

45. DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1| 8| | 2|7 9|10 3|4
5|6 |

46. DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1| 8|11 | 2|7 9|10 3|4
5|6 |

47. DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1|12 8|11 | 2|7 9|10
3|4
5|6 |

48. DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1|12 8|11 13| 2|7 9|10
3|4
5|6 |

49. DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1|12 8|11 13| 2|7 9|10
3|4
5|6
14|

50. DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1|12 8|11 13| 2|7 9|10
3|4
5|6
14|15

51. DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1|12 8|11 13|16 2|7
9|10
3|4
5|6
14|15

52. DFS Properties Parenthesis Theorem Corollary
For all u, v, exactly one of the following holds:
The intervals [u.d, u.f] and [v.d, v.f] are disjoint, neither of u and v is a descendant of the other
The interval [u.d, u.f] is contained entirely within the interval [v.d, v.f], u is a descendant of v in a depth-first tree
The interval [u.d, u.f] is contained entirely within the interval [v.d, v.f], v is a ancestor of u in a depth-first tree
Corollary
v is a proper descendant of u if and only if u.d<v.d<v.f<u.f

53. DFS: Kinds of edges
DFS introduces an important distinction among edges in the original graph:
Tree edge: encounter new (white) vertex
The tree edges form a spanning forest
Can tree edges form cycles? Why or why not?

54. DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1|12 8|11 13|16 2|7
9|10
3|4
5|6
14|15
Tree edges

55. DFS: Kinds of edges
DFS introduces an important distinction among edges in the original graph:
Tree edge: encounter new (white) vertex
Back edge: from descendent to ancestor
Encounter a grey vertex (grey to grey)

56. DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1|12 8|11 13|16 2|7
9|10
3|4
5|6
14|15
Tree edges
Back edges

57. DFS: Kinds of edges
DFS introduces an important distinction among edges in the original graph:
Tree edge: encounter new (white) vertex
Back edge: from descendent to ancestor
Forward edge: from ancestor to descendent
Not a tree edge
From grey node to black node

58. DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1|12 8|11 13|16 2|7
9|10
3|4
5|6
14|15
Tree edges
Back edges
Forward edges

59. DFS: Kinds of edges
DFS introduces an important distinction among edges in the original graph:
Tree edge: encounter new (white) vertex
Back edge: from descendent to ancestor
Forward edge: from ancestor to descendent
Cross edge: between a tree or subtrees
Any other edges
From a grey node to a black node

60. DFS Example source vertex d | f 1 2 3 6 7 8 4 5 1|12 8|11 13|16 2|7
9|10
3|4
5|6
14|15
Tree edges
Back edges
Forward edges
Cross edges

61. DFS: Kinds of edges
DFS introduces an important distinction among edges in the original graph:
Tree edge: encounter new (white) vertex
Back edge: from descendent to ancestor
Forward edge: from ancestor to descendent
Cross edge: between a tree or subtrees
Note: If G is undirected, a DFS produces only tree and back edges. No forward or cross edges.

62. DFS And Graph Cycles
Theorem: An undirected graph is acyclic iff a DFS yields no back edges
If acyclic, no back edges (because a back edge implies a cycle
If no back edges, acyclic
No back edges implies only tree edges
Only tree edges implies we have a tree or a forest
Which by definition is acyclic
Thus, can run DFS to find whether a graph has a cycle

63. Directed Acyclic Graphs
A directed acyclic graph or DAG is a directed graph with no directed cycles:

64. Topological Sort Topological sort of a DAG:
Linear ordering of all vertices in graph G such that vertex u appears somewhere before vertex v if edge (u, v)  G
Real-world example: getting dressed

65. Getting Dressed Underwear Socks Watch Pants Shoes Shirt Belt Tie
Jacket

66. Getting Dressed Underwear Socks Watch Pants Shoes Shirt Belt Tie
Jacket
Socks
Underwear
Pants
Shoes
Watch
Shirt
Belt
Tie
Jacket

67. Topological Sort Algorithm{
Run DFS to compute finish time v.f for all v V
Output vertices in order of decreasing finishing times }
Time: O(V+E)

68. Getting Dressed 11/16 Underwear Socks 17/18 Watch 9/10 12/15 Pants
Shoes
13/14
Shirt
1/8
6/7
Belt
Tie
2/5
Jacket
3/4
Socks
Underwear
Pants
Shoes
Watch
Shirt
Belt
Tie
Jacket
17/18
11/16
12/15
13/14
9/10
1/8
6/7
2/5
3/4

69. Minimum Spanning Tree
Problem: given a connected, undirected, weighted graph, find a spanning tree (T) using edges that minimize the total weight, i.e., minimize (𝑢,𝑣)∈𝑇 𝑤(𝑢,𝑣)

70. Minimum Spanning Tree
Which edges form the minimum spanning tree (MST) of the below graph? A 6 4 5 9 H B C 14 2 10 15 G E D 3 8 F

71. Minimum Spanning Tree
Answer: A 6 4 5 9 H B C 14 2 10 15 G E D 3 8 F

72. Minimum Spanning Tree
MSTs satisfy the optimal substructure property: an optimal tree is composed of optimal subtrees
Let T be an MST of G with an edge (u,v) in the middle
Removing (u,v) partitions T into two trees T1 and T2
Claim: T1 is an MST of G1 = (V1,E1), and T2 is an MST of G2 = (V2,E2)
Proof: w(T) = w(u,v) + w(T1) + w(T2)

73. Minimum Spanning Tree MSTs satisfy the greedy-choice property:
Let T be MST of G, and let A  T be subtree of T
Let (u,v) be min-weight edge connecting A to the rest of the Graph (V - A)
Then (u,v)  T

74. Prim’s Algorithm Run on example graph MST-Prim(G, w, r) Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v); 6 4 9 5 14 2 10 15 3 8
Run on example graph

75. Prim’s Algorithm         MST-Prim(G, w, r) Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);  6 4 9 5    14 2 10 15    3 8 

76. Prim’s Algorithm     r    Pick a start vertex r
MST-Prim(G, w, r)
Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);  6 4 9 5    14 2 10 15 r   3 8 
Pick a start vertex r

77. Red vertices have been removed from Q
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);  6 4 9 5    14 2 10 15 u   3 8 
Red vertices have been removed from Q

78. Red arrows indicate parent pointers
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);  6 4 9 5    14 2 10 15 u   3 8 3
Red arrows indicate parent pointers

79. Prim’s Algorithm  14   u   3 MST-Prim(G, w, r) Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);  6 4 9 5 14   14 2 10 15 u   3 8 3

80. Prim’s Algorithm  14     3 u MST-Prim(G, w, r) Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);  6 4 9 5 14   14 2 10 15   3 8 3 u

81. Prim’s Algorithm  14   8  3 u MST-Prim(G, w, r) Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);  6 4 9 5 14   14 2 10 15 8  3 8 3 u

82. Prim’s Algorithm  10   8  3 u MST-Prim(G, w, r) Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);  6 4 9 5 10   14 2 10 15 8  3 8 3 u

83. Prim’s Algorithm  10   8  3 u MST-Prim(G, w, r) Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);  6 4 9 5 10   14 2 10 15 8  3 8 3 u

84. Prim’s Algorithm  10 2  8  3 u MST-Prim(G, w, r) Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);  6 4 9 5 10 2  14 2 10 15 8  3 8 3 u

85. Prim’s Algorithm  10 2  8 15 3 u MST-Prim(G, w, r) Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);  6 4 9 5 10 2  14 2 10 15 8 15 3 8 3 u

86. Prim’s Algorithm  u 10 2  8 15 3 MST-Prim(G, w, r) Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);  6 4 u 9 5 10 2  14 2 10 15 8 15 3 8 3

87. Prim’s Algorithm  u 10 2 9 8 15 3 MST-Prim(G, w, r) Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);  6 4 u 9 5 10 2 9 14 2 10 15 8 15 3 8 3

88. Prim’s Algorithm 4 u 10 2 9 8 15 3 MST-Prim(G, w, r) Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v); 4 6 4 u 9 5 10 2 9 14 2 10 15 8 15 3 8 3

89. Prim’s Algorithm 4 u 5 2 9 8 15 3 MST-Prim(G, w, r) Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v); 4 6 4 u 9 5 5 2 9 14 2 10 15 8 15 3 8 3

90. Prim’s Algorithm u 4 5 2 9 8 15 3 MST-Prim(G, w, r) Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v); 4 6 4 9 5 5 2 9 14 2 10 15 8 15 3 8 3

91. Prim’s Algorithm 4 u 5 2 9 8 15 3 MST-Prim(G, w, r) Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v); 4 6 4 9 5 u 5 2 9 14 2 10 15 8 15 3 8 3

92. Prim’s Algorithm u 4 5 2 9 8 15 3 MST-Prim(G, w, r) Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v); 4 6 4 9 5 5 2 9 14 2 10 15 8 15 3 8 3

93. Prim’s Algorithm 4 5 2 9 8 15 3 u MST-Prim(G, w, r) Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v); 4 6 4 9 5 5 2 9 14 2 10 15 8 15 3 8 3 u

94. Prim’s Algorithm What is the hidden cost in this code?
MST-Prim(G, w, r)
Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
What is the hidden cost in this code?
A: DECREASE-KEY operation
on the min-heap
What will be the running time? A: Depends on queue binary heap: O(E lg V)

95. Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ;
for each v  V
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); } 2 19 9 14 17 8 25 5 21 13 1

96. Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v  V MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1

97. Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v  V MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1

98. Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v  V MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1?

99. Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v  V MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1

100. Kruskal’s Algorithm 2? 19 Kruskal() { T = ; for each v  V
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1

101. Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v  V MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1

102. Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v  V MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5? 21 13 1

103. Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v  V MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1

104. Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v  V MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); } 9 14 17 8? 25 5 21 13 1

105. Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v  V MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1

106. Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v  V MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); } 9? 14 17 8 25 5 21 13 1

107. Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v  V MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1

108. Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v  V MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21
13? 1

109. Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v  V MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1

110. Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v  V MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); } 9
14? 17 8 25 5 21 13 1

111. Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v  V MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1

112. Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v  V MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); } 9 14
17? 8 25 5 21 13 1

113. Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v  V MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1

114. Kruskal’s Algorithm 2 19? Kruskal() { T = ; for each v  V
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1

115. Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v  V MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1

116. Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v  V MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5
21? 13 1

117. Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v  V MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1

118. Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v  V MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); } 9 14 17 8
25? 5 21 13 1

119. Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v  V MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1

120. Kruskal’s Algorithm 2 19 Kruskal() { T = ; for each v  V MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); } 9 14 17 8 25 5 21 13 1

121. Correctness Of Kruskal’s Algorithm
Sketch of a proof that this algorithm produces an MST for T:
Assume algorithm is wrong: result is not an MST
Then algorithm adds a wrong edge at some point
If it adds a wrong edge, there must be a lower weight edge
But algorithm chooses lowest weight edge at each step. Contradiction

122. Kruskal’s Algorithm What will affect the running time? Kruskal() Sort
O(V) MakeSet() calls
O(E) FindSet() calls O(V) Union() calls
Kruskal() {
T = ;
for each v  V
MakeSet(v);
sort E by increasing edge weight w
for each (u,v)  E (in sorted order)
if FindSet(u)  FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v)); }

123. Kruskal’s Algorithm: Running Time
To summarize:
Sort edges: O(E lg E)
O(V) MakeSet()’s
O(E) FindSet()’s
O(V) Union()’s
Overall O(E lg E)

124. Single-Source Shortest Path
Problem: given a weighted directed graph G, find the minimum-weight path from a given source vertex s to another vertex v
“Shortest-path” = minimum weight
Weight of path is sum of edges
Might not be unique

125. Shortest Path Properties
Again, we have optimal substructure: the shortest path consists of shortest subpaths:
Proof: suppose some subpath is not a shortest path
There must then exist a shorter subpath
Could substitute the shorter subpath for a shorter path
But then overall path is not shortest path. Contradiction

126. Shortest Path Properties
Define (u,v) to be the weight of the shortest path from u to v
Shortest paths satisfy the triangle inequality: (u,v)  (u,x) + (x,v) x u v
This path is no longer than any other path

127. Shortest Path Properties
In graphs with negative weight cycles, some shortest paths will not exist
Shortest path can’t contain cycles
< 0

128. Relaxation A key technique in shortest path algorithms is relaxation
Idea: for all v, maintain upper bound d[v] on (s,v)
Relax(u,v,w) {
if (d[v] > d[u]+w) then d[v]=d[u]+w; } 9 5 2 7
Relax 6 5 2
Relax

129. Bellman-Ford Algorithm
for each v  V
d[v] = ;
d[s] = 0;
for i=1 to |V|-1
for each edge (u,v)  E
Relax(u,v, w(u,v));
if (d[v] > d[u] + w(u,v))
return “no solution”;
Relax(u,v,w): if (d[v] > d[u]+) then d[v]=d[u]+w
Initialize d[], which
will converge to shortest-path value 
Relaxation: Make |V|-1 passes,
relaxing each edge
Test for solution:
have we converged yet?
i.e.,  negative cycle?

130. Bellman-Ford Algorithm
for each v  V
d[v] = ;
d[s] = 0;
for i=1 to |V|-1
for each edge (u,v)  E
Relax(u,v, w(u,v));
if (d[v] > d[u] + w(u,v))
return “no solution”;
What will be the running time?
A: O(VE)

131. Bellman-Ford Algorithm
for each v  V
d[v] = ;
d[s] = 0;
for i=1 to |V|-1
for each edge (u,v)  E
Relax(u,v, w(u,v));
if (d[v] > d[u] + w(u,v))
return “no solution”;
Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w B  s -1 2 A E  2 3 1 -3 4   5 C D

132. Bellman-Ford Algorithm
for each v  V
d[v] = ;
d[s] = 0;
for i=1 to |V|-1
for each edge (u,v)  E
Relax(u,v, w(u,v));
if (d[v] > d[u] + w(u,v))
return “no solution”;
Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w B -1 s -1 2 A E  2 3 1 -3 4 4  5 C D

133. Bellman-Ford Algorithm
for each v  V
d[v] = ;
d[s] = 0;
for i=1 to |V|-1
for each edge (u,v)  E
Relax(u,v, w(u,v));
if (d[v] > d[u] + w(u,v))
return “no solution”;
Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w B -1 s -1 2 A E 1 2 3 1 -3 4 2 1 5 C D

134. Bellman-Ford Algorithm
for each v  V
d[v] = ;
d[s] = 0;
for i=1 to |V|-1
for each edge (u,v)  E
Relax(u,v, w(u,v));
if (d[v] > d[u] + w(u,v))
return “no solution”;
Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w B -1 s -1 2 A E 1 2 3 1 -3 4 2 -2 5 C D

135. Bellman-Ford Algorithm
for each v  V
d[v] = ;
d[s] = 0;
for i=1 to |V|-1
for each edge (u,v)  E
Relax(u,v, w(u,v));
if (d[v] > d[u] + w(u,v))
return “no solution”;
Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w B -1 s -1 2 A E 1 2 3 1 -3 4 2 -2 5 C D

136. Bellman-Ford Algorithm
for each v  V
d[v] = ;
d[s] = 0;
for i=1 to |V|-1
for each edge (u,v)  E
Relax(u,v, w(u,v));
if (d[v] > d[u] + w(u,v))
return “no solution”;
Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w B -1 s -1 2 A E 1 2 3 1 -3 4 2 -2 5 C D

137. Bellman-Ford Algorithm
Note that order in which edges are processed affects how quickly it converges
Correctness: show d[v] = (s,v) after |V|-1 passes

138. DAG Shortest Paths Problem: finding shortest paths in DAG
Bellman-Ford takes O(VE) time.
How can we do better?
Idea: use topological sort
DAG-Shortest-Paths(G,w,s)
topologically sort the vertices
for each v  V
d[v] = ;
d[s] = 0;
for each vertex u, taken in topologically sorted order
for each vertex v  Adj[u]
Relax(u,v,w)

139. DAG Shortest Paths  A B E C D 2 7 6 -1 -2 1  6 2 A B E C D 7 -1 -2 1A B E C D 2 7 6 -1 -2 1  6 2 A B E C D 7 -1 -2 1 5 7 6 2 A B E C D -1 -2 1 5 3 6 2 A B E C D 7 -1 -2 1

140. DAG Shortest Paths Problem: finding shortest paths in DAG
Idea: use topological sort
If were lucky and processes vertices on each shortest path from left to right, would be done in one pass
Every path in a DAG is subsequence of topologically sorted vertex order, so processing verts in that order, we will do each path in forward order (will never relax edges out of vertices before doing all edges into vertices).
Thus: just one pass.
Time: O(V + E)

141. Dijkstra’s Algorithm
If no negative edge weights, Dijkstra’s essentially a weighted version of breadth-first search
Similar to breadth-first search
Grow a tree gradually, advancing from vertices taken from a queue
Also similar to Prim’s algorithm for MST
Use a priority queue keyed on d[v]

142. Dijkstra’s Algorithm 10 4 3 2 1 5 A B C D Dijkstra(G) for each v  V
d[v] = ;
d[s] = 0; S = ; Q = V;
while (Q  )
u = ExtractMin(Q);
S = S U {u};
for each v  u->Adj[]
if (d[v] > d[u]+w(u,v))
d[v] = d[u]+w(u,v);
Relaxation Step

143. Dijkstra’s Algorithm B Dijkstra(G) for each v  V d[v] = ;
d[s] = 0; S = ; Q = V;
while (Q  )
u = ExtractMin(Q);
S = S U {u};
for each v  u->Adj[]
if (d[v] > d[u]+w(u,v))
d[v] = d[u]+w(u,v);  2 s A 10 D 4 3  5 1  C

144. Dijkstra’s Algorithm B Dijkstra(G) for each v  V d[v] = ;
d[s] = 0; S = ; Q = V;
while (Q  )
u = ExtractMin(Q);
S = S U {u};
for each v  u->Adj[]
if (d[v] > d[u]+w(u,v))
d[v] = d[u]+w(u,v); 10 2 s A 10 D 4 3  5 1 5 C
S = {A}

145. Dijkstra’s Algorithm B Dijkstra(G) for each v  V d[v] = ;
d[s] = 0; S = ; Q = V;
while (Q  )
u = ExtractMin(Q);
S = S U {u};
for each v  u->Adj[]
if (d[v] > d[u]+w(u,v))
d[v] = d[u]+w(u,v); 9 2 s A 10 D 4 3 6 5 1 5 C
S = {A, C}

146. Dijkstra’s Algorithm B Dijkstra(G) for each v  V d[v] = ;
d[s] = 0; S = ; Q = V;
while (Q  )
u = ExtractMin(Q);
S = S U {u};
for each v  u->Adj[]
if (d[v] > d[u]+w(u,v))
d[v] = d[u]+w(u,v); 8 2 s A 10 D 4 3 6 5 1 5 C
S = {A, C, D}

147. Dijkstra’s Algorithm B Dijkstra(G) for each v  V d[v] = ;
d[s] = 0; S = ; Q = V;
while (Q  )
u = ExtractMin(Q);
S = S U {u};
for each v  u->Adj[]
if (d[v] > d[u]+w(u,v))
d[v] = d[u]+w(u,v); 8 2 s A 10 D 4 3 6 5 1 5 C
S = {A, C, D, B}

148. Dijkstra’s Algorithm B Dijkstra(G) for each v  V d[v] = ;
d[s] = 0; S = ; Q = V;
while (Q  )
u = ExtractMin(Q);
S = S U {u};
for each v  u->Adj[]
if (d[v] > d[u]+w(u,v))
d[v] = d[u]+w(u,v); 8 2 s A 10 D 4 3 6 5 1 5 C

149. Dijkstra’s Algorithm Dijkstra(G) for each v  V d[v] = ;
d[s] = 0; S = ; Q = V;
while (Q  )
u = ExtractMin(Q);
S = S U {u};
for each v  u->Adj[]
if (d[v] > d[u]+w(u,v))
d[v] = d[u]+w(u,v);
How many times is ExtractMin() called?
How many times is Relax() called?
What will be the total running time?
A: O(E lg V) using binary heap for Q