1. Minimum Spanning Tree Shortest Paths
CS 2133: Algorithms
Minimum Spanning Tree
Shortest Paths

2. Review: Getting Dressed
Underwear
Socks
Watch
Pants
Shoes
Shirt
Belt
Tie
Jacket
Socks
Underwear
Pants
Shoes
Watch
Shirt
Belt
Tie
Jacket

3. Review: Topological Sort Algorithm{
Run DFS
When a vertex is finished, output it
Vertices are output in reverse topological order }
Time: O(V+E)

4. Review: Minimum Spanning Tree
Problem: given a connected, undirected, weighted graph, find a spanning tree using edges that minimize the total weight 6 4 5 9 14 2 10 15 3 8

5. Review: Minimum Spanning Tree
Problem: given a connected, undirected, weighted graph, find a spanning tree using edges that minimize the total weight 6 4 5 9 14 2 10 15 3 8

6. Review: Minimum Spanning Tree
MSTs satisfy the optimal substructure property: an optimal tree is composed of optimal subtrees
If T is MST of G, and A  T is a subtree of T, and (u,v) is the min-weight edge connecting A to V-A, then (u,v)  T

7. Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v); 6 4 9 5 14 2 10 15 3 8
Run on example graph

8. Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);  6 4 9 5    14 2 10 15    3 8 
Run on example graph

9. Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);  6 4 9 5    14 2 10 15 r   3 8 
Pick a start vertex r

10. Red vertices have been removed from Q
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);  6 4 9 5    14 2 10 15 u   3 8 
Red vertices have been removed from Q

11. Red arrows indicate parent pointers
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);  6 4 9 5    14 2 10 15 u   3 8 3
Red arrows indicate parent pointers

12. Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);  6 4 9 5 14   14 2 10 15 u   3 8 3

13. Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);  6 4 9 5 14   14 2 10 15   3 8 3 u

14. Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);  6 4 9 5 14   14 2 10 15 8  3 8 3 u

15. Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);  6 4 9 5 10   14 2 10 15 8  3 8 3 u

16. Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);  6 4 9 5 10   14 2 10 15 8  3 8 3 u

17. Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);  6 4 9 5 10 2  14 2 10 15 8  3 8 3 u

18. Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);  6 4 9 5 10 2  14 2 10 15 8 15 3 8 3 u

19. Prim’s Algorithm u MST-Prim(G, w, r) Q = V[G]; for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);  6 4 9 5 10 2  14 2 10 15 8 15 3 8 3

20. Prim’s Algorithm u MST-Prim(G, w, r) Q = V[G]; for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);  6 4 9 5 10 2 9 14 2 10 15 8 15 3 8 3

21. Prim’s Algorithm u MST-Prim(G, w, r) Q = V[G]; for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v); 4 6 4 9 5 10 2 9 14 2 10 15 8 15 3 8 3

22. Prim’s Algorithm u MST-Prim(G, w, r) Q = V[G]; for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v); 4 6 4 9 5 5 2 9 14 2 10 15 8 15 3 8 3

23. Prim’s Algorithm u MST-Prim(G, w, r) Q = V[G]; for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v); 4 6 4 9 5 5 2 9 14 2 10 15 8 15 3 8 3

24. Prim’s Algorithm u MST-Prim(G, w, r) Q = V[G]; for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v); 4 6 4 9 5 5 2 9 14 2 10 15 8 15 3 8 3

25. Prim’s Algorithm u MST-Prim(G, w, r) Q = V[G]; for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v); 4 6 4 9 5 5 2 9 14 2 10 15 8 15 3 8 3

26. Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v); 4 6 4 9 5 5 2 9 14 2 u 10 15 8 15 3 8 3

27. Review: Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
What is the hidden cost in this code?

28. Review: Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
DecreaseKey(v, w(u,v));

29. Review: Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
DecreaseKey(v, w(u,v));
How often is ExtractMin() called?
How often is DecreaseKey() called?

30. Review: Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);// log V
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v); // decrease // key=O(log V)
What will be the running time? A: Depends on queue binary heap: O(E lg V)

31. Single-Source Shortest Path
Problem: given a weighted directed graph G, find the minimum-weight path from a given source vertex s to another vertex v
“Shortest-path” = minimum weight
Weight of path is sum of edges
E.g., a road map: what is the shortest path from Chapel Hill to Charlottesville?

32. Shortest Path Properties
Again, we have optimal substructure: the shortest path consists of shortest subpaths:
Proof: suppose some subpath is not a shortest path
There must then exist a shorter subpath
Could substitute the shorter subpath for a shorter path
But then overall path is not shortest path. Contradiction

33. Shortest Path Properties
Define (u,v) to be the weight of the shortest path from u to v
Shortest paths satisfy the triangle inequality: (u,v)  (u,x) + (x,v)
“Proof”: x u v
This path is no longer than any other path

34. Shortest Path Properties
In graphs with negative weight cycles, some shortest paths will not exist (Why?):
< 0

35. Relaxation A key technique in shortest path algorithms is relaxation
Idea: for all v, maintain upper bound d[v] on (s,v)
Relax(u,v,w) {
if (d[v] > d[u]+w) then d[v]=d[u]+w; } 9 5 2 7
Relax 6 5 2
Relax

36. Dijkstra’s Algorithm Here we assume no negative edge weights
Similar to breadth-first search
Grow a tree gradually, advancing from vertices taken from a queue
Also similar to Prim’s algorithm for MST
Use a priority queue keyed on d[v]

37. Dijkstra’s Algorithm Dijkstra(G) for each v  V d[v] = ;
d[s] = 0; S = ; Q = V;
while (Q  )
u = ExtractMin(Q);
S = S  {u};
for each v  u->Adj[]
if (d[v] > d[u]+w(u,v))
d[v] = d[u]+w(u,v); B C D A 10 4 3 2 1 5
Ex: run the algorithm
Relaxation Step
Note: this is really a call to Q->DecreaseKey()

38. Dijkstra’s Algorithm Dijkstra(G) for each v  V
d[v] = ;
d[s] = 0; S = ; Q = V;
while (Q  )
u = ExtractMin(Q);
S = S  {u};
for each v  u->Adj[]
if (d[v] > d[u]+w(u,v))
d[v] = d[u]+w(u,v);
How many times is ExtractMin() called?
How many times is DecreaseKey() called?
What will be the total running time?

39. Dijkstra’s Algorithm Dijkstra(G) for each v  V
d[v] = ;
d[s] = 0; S = ; Q = V;
while (Q  )
u = ExtractMin(Q);
S = S  {u};
for each v  u->Adj[]
if (d[v] > d[u]+w(u,v))
d[v] = d[u]+w(u,v);
How many times is ExtractMin() called?
How many times is DecraseKey() called?
A: O(E lg V) using binary heap for Q
Can acheive O(V lg V + E) with Fibonacci heaps

40. Dijkstra’s Algorithm Dijkstra(G) for each v  V d[v] = ;
d[s] = 0; S = ; Q = V;
while (Q  )
u = ExtractMin(Q);
S = S  {u};
for each v  u->Adj[]
if (d[v] > d[u]+w(u,v))
d[v] = d[u]+w(u,v);
Correctness: we must show that when u is removed from Q, it has already converged

41. Correctness Of Dijkstra's Algorithmp2 u s y x p2
Note that d[v]  (s,v) v
Let u be first vertex picked s.t.  shorter path than d[u] d[u] > (s,u)
Let y be first vertex V-S on actual shortest path from su  d[y] = (s,y)
Because d[x] is set correctly for y's predecessor x  S on the shortest path, and
When we put x into S, we relaxed (x,y), giving d[y] the correct value

42. Correctness Of Dijkstra's Algorithmp2 u s y x p2
Note that d[v]  (s,v) v
Let u be first vertex picked s.t.  shorter path than d[u] d[u] > (s,u)
Let y be first vertex V-S on actual shortest path from su  d[y] = (s,y)
d[u] > (s,u) = (s,y) + (y,u) (Why?) = d[y] + (y,u)  d[y] But if d[u] > d[y], wouldn't have chosen u. Contradiction.

43. The End