1. Shortest paths Given: A single source vertex (given s) in a weighted, directed graph. Want to compute a shortest path for each possible destination. (Single Source Shortest Paths) SSSP. –Similar to BFS. We will assume either –no negative-weight edges, or –no reachable negative-weight cycles. Algorithms –Dijkstra’s (in comparison to MST prim) –Bellman-Ford –DAG Shortest Paths The shortest paths between all pairs of vertices (All Pairs Shortest Paths). Floyd Warshall Algorithm (DP), Johnson’s. where the length of a path is the sum of its edge weights.

2. props Triangle inequality (Lemma 24.10) For any edge (u, v) ² E, we have δ(s, v) ≤ δ(s, u) + w(u, v). Upper-bound property (Lemma 24.11) We always have d[v] ≥ δ(s, v) for all vertices v ² V, and once d[v] achieves the value δ(s, v), it never changes. No-path property (Corollary 24.12) If there is no path from s to v, then we always have d[v] = δ(s, v)=∞. Convergence property (Lemma 24.14) If s ❀ u → v is a shortest path in G for some u, v ∈ V, and if d[u] = δ(s, u) at any time prior to relaxing edge (u, v), then d[v] = δ(s, v) at all times afterward. Path-relaxation property (Lemma 24.15) If p = v0, v1,..., vk is a shortest path from s = v0 to vk, and the edges of p are relaxed in the order (v0, v1), (v1, v2),..., (vk−1, vk ), then d[vk ] = δ(s, vk ). This property holds regardless of any other relaxation steps that occur, even if they are intermixed with relaxations of the edges of p. Predecessor-subgraph property (Lemma 24.17) Once d[v] = δ(s, v) for all v ∈ V, the predecessor subgraph,  [u] is a shortest-paths.

3. A Fact About Shortest Paths Theorem: If p is a shortest path from u to v, then any subpath of p is also a shortest path. Proof: Consider a subpath of p from x to y. If there were a shorter path from x to y, then there would be a shorter path from u to v. u xyv shorter?

4. Single-Source Shortest Paths Given a directed graph with weighted edges, what are the shortest paths from some source vertex s to all other vertices? Note: shortest path to single destination cannot be done asymptotically faster, as far as we know. 3 11 9 5 0 3 6 5 4 3 6 2 1 2 7 s

5. Path Recovery We would like to find the path itself, not just its length. We’ll construct a shortest-paths tree: 3 11 9 5 0 3 6 5 4 3 6 2 7 2 1 s uv xy 3 9 5 0 3 6 5 3 6 2 7 4 2 1 s u v x y

6. Shortest-Paths Idea  (u,v)  length of the shortest path from u to v. All SSSP algorithms maintain a field d[u] for every vertex u. d[u] will be an estimate of  (s,u). As the algorithm progresses, we will refine d[u] until, at termination, d[u] =  (s,u). Whenever we discover a new shortest path to u, we update d[u]. In fact, d[u] will always be an overestimate of  (s,u): d[u]  (s,u) predecessor We’ll use  [u] to point to the parent (or predecessor) of u on the shortest path from s to u. We update  [u] when we update d[u].

7. SSSP Subroutine RELAX(u, v, w)  (Maybe) improve our estimate of the distance to v  by considering a path along the edge (u, v). if d[u] + w(u, v) < d[v] then d[v]  d[u] + w(u, v)  actually, DECREASE-KEY  [v]  u  remember predecessor on path u v w(u,v) d[v]d[u]

8. Shortest Path Properties Again, we have optimal substructure: the shortest path consists of shortest subpaths: –Proof: suppose some subpath is not a shortest path There must then exist a shorter subpath Could substitute the shorter subpath for a shorter path But then overall path is not shortest path. Contradiction

9. Shortest Path Properties Define  (u,v) to be the weight of the shortest path from u to v triangle inequality Shortest paths satisfy the triangle inequality:  (u,v)   (u,x) +  (x,v) “Proof”: x u v This path is no longer than any other path

10. Shortest Path Properties In graphs with negative weight cycles, some shortest paths will not exist (Why?): < 0

11. Relaxation A key technique in shortest path algorithms is relaxation –Idea: for all v, maintain upper bound d[v] on  (s,v) Relax(u,v,w) { if (d[v] > d[u]+w) then d[v]=d[u]+w; } 9 5 2 7 5 2 Relax 6 5 2 6 5 2

12. Dijkstra’s Algorithm (pronounced “DIKE-stra”) Assume that all edge weights are  0. Idea: say we have a set K containing all vertices whose shortest paths from s are known (i.e. d[u] =  (s,u) for all u in K). Now look at the “frontier” of K—all vertices adjacent to a vertex in K. the rest of the graph s K

13. Dijkstra’s: Theorem At each frontier vertex u, update d[u] to be the minimum from all edges from K. Now pick the frontier vertex u with the smallest value of d[u]. Claim: d[u] =  (s,u) s 4 9 6 6 2 1 3 8 min(4+2, 6+1) = 6 min(4+8, 6+3) = 9

14. Dijkstra’s: Proof By construction, d[u] is the length of the shortest path to u going through only vertices in K. Another path to u must leave K and go to v on the frontier. But the length of this path is at least d[v], (assuming non-negative edge weights), which is  d[u].

15. Proof Explained Why is the path through v at least d[v] in length? We know the shortest paths to every vertex in K. We’ve set d[v] to the shortest distance from s to v via K. The additional edges from v to u cannot decrease the path length. s u v d[u]  d[v] K shortest path to u another path to u, via v

16. Dijkstra’s Algorithm, Rough Draft u K K u

17. A Refinement Note: we don’t really need to keep track of the frontier. When we add a new vertex u to K, just update vertices adjacent to u.

18. Dijkstra Running Time (cont.) 1. Priority queue is an array. EXTRACT-MIN in  (n) time, DECREASE-KEY in  (1) Total time:  (V + VV + E) =  (V 2 ) 2. (“Modified Dijkstra”) Priority queue is a binary (standard) heap. EXTRACT-MIN in  (lgn) time, also DECREASE-KEY Total time:  (VlgV + ElgV) 3. Priority queue is Fibonacci heap. (Of theoretical interest only.) EXTRACT-MIN in  (lgn), DECREASE-KEY in  (1) (amortized) Total time:  (VlgV+E)

19. The Bellman-Ford Algorithm Handles negative edge weights Detects negative cycles Is slower than Dijkstra 4 5 -10 a negative cycle

20. Bellman-Ford: Idea Repeatedly update d for all pairs of vertices connected by an edge. Theorem: If u and v are two vertices with an edge from u to v, and s  u  v is a shortest path, and d[u] =  (s,u), then d[u]+w(u,v) is the length of a shortest path to v. Proof: Since s  u  v is a shortest path, its length is  (s,u) + w(u,v) = d[u] + w(u,v).

21. Why Bellman-Ford Works On the first pass, we find  (s,u) for all vertices whose shortest paths have one edge. On the second pass, the d[u] values computed for the one-edge-away vertices are correct (=  (s,u)), so they are used to compute the correct d values for vertices whose shortest paths have two edges. Since no shortest path can have more than |V[G]|-1 edges, after that many passes all d values are correct. Note: all vertices not reachable from s will have their original values of infinity. (Same, by the way, for Dijkstra).

22. Bellman-Ford: Algorithm BELLMAN-FORD(G, w, s) 1 foreach vertex v  V[G] do //INIT_SINGLE_SOURCE 2 d[v]  3  [v]  NIL 4 d[s]  0 5 for i  1 to |V[G]|-1 do  each iteration is a “pass” 6 for each edge (u,v) in E[G] do 7 RELAX(u, v, w) 8  check for negative cycles 9 for each edge (u,v) in E[G] do 10 if d[v] > d[u] + w(u,v) then 11 return FALSE 12 return TRUE Running time  VE)

23. Negative Cycle Detection What if there is a negative-weight cycle reachable from s? Assume:d[u]  d[x]+4 d[v]  d[u]+5 d[x]  d[v]-10 Adding: d[u]+d[v]+d[x]  d[x]+d[u]+d[v]-1 Because it’s a cycle, vertices on left are same as those on right. Thus we get 0  -1; a contradiction. So for at least one edge (u,v), d[v] > d[u] + w(u,v) This is exactly what Bellman-Ford checks for. u x v 4 5 -10

24. Example 0    z uv x y 6 5 –3 9 7 7 8 –2 –4 2

25. Example 0  7  6 z uv x y 6 5 –3 9 7 7 8 –2 –4 2

26. Example 0 2 7 46 z uv x y 6 5 –3 9 7 7 8 –2 –4 2

27. Example 0 2 7 42 z uv x y 6 5 –3 9 7 7 8 –2 –4 2

28. Example 0 -2 7 42 z uv x y 6 5 –3 9 7 7 8 –2 –4 2

29. BF Dynamic aspect for the same graph Note: This is essentially dynamic programming. Let d(i, j) = cost of the shortest path from s to i that is at most j hops. d(i, j) = 0 if i = s  j = 0  if i  s  j = 0 min({d(k, j–1) + w(k, i): i  Adj(k)}  {d(i, j–1)}) if j > 0 zuvxy 1 2 3 4 5 00  106  7  206472 302472 40247 –2 j i

30. A C B ED 2 4 -3 23 53 4 π: nil d: 0 π: nil d: ∞ π: nil d: ∞ π: nil d: ∞ π: nil d: ∞ Bellman-Ford where G contains no negative weight cycles (iterations from E to A)

31. A C B ED 2 4 -3 23 53 4 π: nil d: 0 π: nil A d: ∞ -1 π: nil C d: ∞ 2 π: nil A d: ∞ 4 π: nil d: ∞

32. A C B ED 2 4 -3 23 53 4 π: nil d: 0 π: nil A d: ∞ -1 π: nil C d: ∞ 2 π: nil A d: ∞ 4 π: nil C d: ∞ 3

33. A C B ED 2 4 -3 23 53 4 π: nil d: 0 π: nil A d: ∞ -1 π: nil C d: ∞ 2 π: nil A d: ∞ 4 π: nil C d: ∞ 3

34. A C B ED 2 4 -3 23 53 4 π: nil d: 0 π: nil A d: ∞ -1 π: nil C d: ∞ 2 π: nil A d: ∞ 4 π: nil C d: ∞ 3

35. A C B ED 2 4 -3 21 51 4 π: nil d: 0 π: nil d: ∞ π: nil d: ∞ π: nil d: ∞ π: nil d: ∞ Bellman-Ford where G contains no negative weight cycles (iterations from A to E)

36. A C B ED 2 4 -3 21 51 4 π: nil d: 0 π: nil A d: ∞ -1 π: nil C d: ∞ 2 π: nil A d: ∞ 4 π: nil d: ∞

37. A C B ED 2 4 -3 21 51 4 π: nil d: 0 π: nil A d: ∞ -1 π: nil C B d: ∞ 2 0 π: nil A d: ∞ 4 π: nil C d: ∞ 3

38. A C B ED 2 4 -3 21 51 4 π: nil d: 0 π: nil A d: ∞ -1 π: nil C B d: ∞ 2 0 π: nil A d: ∞ 4 π: nil C C d: ∞ 3 1

39. A C B ED 2 4 -3 21 51 4 π: nil d: 0 π: nil A E d: ∞ -1 -2 π: nil C B B d: ∞ 2 0 -1 π: nil A d: ∞ 4 π: nil C C d: ∞ 3 1

40. Comp 122, Fall 2003 Single-source SPs - 40 Lemma 24.2 Lemma 24.2: Assuming no negative-weight cycles reachable from s, d[v] =  (s, v) holds upon termination for all vertices v reachable from s. Lemma 24.2: Assuming no negative-weight cycles reachable from s, d[v] =  (s, v) holds upon termination for all vertices v reachable from s. Proof: Consider a SP p, where p = ‹v 0, v 1, …, v k ›, where v 0 = s and v k = v. Assume k  |V| – 1, otherwise p has a cycle. Claim: d[v i ] =  (s, v i ) holds after the i th pass over edges. Proof follows by induction on i. By Lemma 24.11, once d[v i ] =  (s, v i ) holds, it continues to hold.

41. Comp 122, Fall 2003 Single-source SPs - 41 Correctness Claim: Algorithm returns the correct value. (Part of Theorem 24.4. Other parts of the theorem follow easily from earlier results.) Case 1: There is no reachable negative-weight cycle. Upon termination, we have for all (u, v): d[v] =  (s, v), by lemma 24.2 (last slide) if v is reachable; d[v] =  (s, v) =  otherwise.   (s, u) + w(u, v), by Lemma 24.10. = d[u] + w(u, v) So, algorithm returns true.

42. Comp 122, Fall 2003 Single-source SPs - 42 Case 2 Case 2: There exists a reachable negative-weight cycle c = ‹v 0, v 1, …, v k ›, where v 0 = v k. We have  i = 1, …, k w(v i-1, v i ) < 0. (*) Suppose algorithm returns true. Then, d[v i ]  d[v i-1 ] + w(v i-1, v i ) for i = 1, …, k. (because Relax didn’t change any d[v i ] ). Thus,  i = 1, …, k d[v i ]   i = 1, …, k d[v i-1 ] +  i = 1, …, k w(v i-1, v i ) But,  i = 1, …, k d[v i ] =  i = 1, …, k d[v i-1 ]. Can show no d[v i ] is infinite. Hence, 0   i = 1, …, k w(v i-1, v i ). Contradicts (*). Thus, algorithm returns false.

43. Review: Getting Dressed UnderwearSocks ShoesPants Belt Shirt Watch Tie Jacket SocksUnderwearPantsShoesWatchShirtBeltTieJacket Topological-Sort() { 1- Run DFS 2- When a vertex is finished, output it 3- Vertices are output in reverse topological order } Time: O(V+E)

44. SSSP in a DAG Recall: a dag is a directed acyclic graph. If we update the edges in topologically sorted order, we correctly compute the shortest paths. Reason: the only paths to a vertex come from vertices before it in the topological sort. 0146 132 9 s

45. SSSP in a DAG Theorem Theorem: For any vertex u in a dag, if all the vertices before u in a topological sort of the dag have been updated, then d[u] =  (s,u). Proof: By induction on the position of a vertex in the topological sort. Base case: d[s] is initialized to 0. Inductive case: Assume all vertices before u have been updated, and for all such vertices v, d[v]=  (s,v). (continued)

46. Proof, Continued Some edge (v,u) where v is before u, must be on the shortest path to u, since there are no other paths to u. When v was updated, we set d[u] to d[v]+w(v,u) =  (s,v) + w(v,u) =  (s,u)

47. SSSP-DAG Algorithm DAG-SHORTEST-PATHS(G,w,s) 1topologically sort the vertices of G 2initialize d and  as in previous algorithms 3for each vertex u in topological sort order do 4for each vertex v in Adj[u] do 5 RELAX(u, v, w) Running time:  (V+E), same as topological sort

48. Comp 122, Fall 2003 Single-source SPs - 48 Shortest Paths in DAGs Topologically sort vertices in G; Initialize(G, s); for each u in V[G] (in order) do for each v in Adj[u] do Relax(u, v, w) od Topologically sort vertices in G; Initialize(G, s); for each u in V[G] (in order) do for each v in Adj[u] do Relax(u, v, w) od

49. Comp 122, Fall 2003 Single-source SPs - 49 Example  0  r s t u v w 5 27 –1 –2 61 3 2 4

50. Comp 122, Fall 2003 Single-source SPs - 50 Example  0  r s t u v w 5 27 –1 –2 61 3 2 4

51. Comp 122, Fall 2003 Single-source SPs - 51 Example  026  r s t u v w 5 27 –1 –2 61 3 2 4

52. Comp 122, Fall 2003 Single-source SPs - 52 Example  02664 r s t u v w 5 27 –1 –2 61 3 2 4

53. Comp 122, Fall 2003 Single-source SPs - 53 Example  02654 r s t u v w 5 27 –1 –2 61 3 2 4

54. Comp 122, Fall 2003 Single-source SPs - 54 Example  02653 r s t u v w 5 27 –1 –2 61 3 2 4

55. Comp 122, Fall 2003 Single-source SPs - 55 Example  02653 r s t u v w 5 27 –1 –2 61 3 2 4

56. Bellman-Ford Algorithm Dynamic Now consider parallel computations for all destinations d Initialization –Each node has 1 row for each destination d –Distance of node d to itself is zero: D d (d)=0 –Distance of other node j to d is infinite: D j (d)= , for j  d –Next node n j = -1 since not yet defined Send Step –Send new distance vector to immediate neighbors across local link Receive Step –For each destination d, find the next hop that gives the minimum distance to d, Min j { C ij + D j (d) }Min j { C ij + D j (d) } Replace old (n j, D i (d)) by new (n j *, D j *(d)) if new next node or distance foundReplace old (n j, D i (d)) by new (n j *, D j *(d)) if new next node or distance found –Go to send step

57. IterationNode 1Node 2Node 3Node 4Node 5 Initial (-1,  ) 1 2 3 3 1 5 4 6 2 2 3 4 2 1 1 2 3 5 San Jose Table entry @ node 1 for dest SJ Table entry @ node 3 for dest SJ

58. IterationNode 1Node 2Node 3Node 4Node 5 Initial (-1,  ) 1 (6,1) (-1,  ) (6,2) 2 3 San Jose D 6 =0 D 3 =D 6 +1 n 3 =6 3 1 5 4 6 2 2 3 4 2 1 1 2 3 5 D 6 =0 D 5 =D 6 +2 n 5 =6 0 2 1

59. IterationNode 1Node 2Node 3Node 4Node 5 Initial (-1,  ) 1 (6, 1) (-1,  ) (6,2) 2(3,3)(5,6)(6, 1)(3,3)(6,2) 3 San Jose 3 1 5 4 6 2 2 3 4 2 1 1 2 3 5 0 1 2 3 3 6

60. IterationNode 1Node 2Node 3Node 4Node 5 Initial (-1,  ) 1 (6, 1) (-1,  ) (6,2) 2(3,3)(5,6)(6, 1)(3,3)(6,2) 3(3,3)(4,4)(6, 1)(3,3)(6,2) San Jose 3 1 5 4 6 2 2 3 4 2 1 1 2 3 5 0 1 2 6 3 3 4

61. IterationNode 1Node 2Node 3Node 4Node 5 Initial(3,3)(4,4)(6, 1)(3,3)(6,2) 1(3,3)(4,4)(4, 5)(3,3)(6,2) 2 3 San Jose 3 1 5 4 6 2 2 3 4 2 1 1 2 3 5 0 1 2 3 3 4 Network disconnected; Loop created between nodes 3 and 4 5

62. IterationNode 1Node 2Node 3Node 4Node 5 Initial(3,3)(4,4)(6, 1)(3,3)(6,2) 1(3,3)(4,4)(4, 5)(3,3)(6,2) 2(3,7)(4,4)(4, 5)(5,5)(6,2) 3 San Jose 3 1 5 4 6 2 2 3 4 2 1 1 2 3 5 0 2 5 3 3 4 7 5 Node 4 could have chosen 2 as next node because of tie

63. IterationNode 1Node 2Node 3Node 4Node 5 Initial(3,3)(4,4)(6, 1)(3,3)(6,2) 1(3,3)(4,4)(4, 5)(3,3)(6,2) 2(3,7)(4,4)(4, 5)(5,5)(6,2) 3(3,7)(4,6)(4, 7)(5,5)(6,2) San Jose 3 1 5 4 6 2 2 3 4 2 1 1 2 3 5 0 2 5 5 7 4 7 6 Node 2 could have chosen 5 as next node because of tie

64. 3 5 4 6 2 2 3 4 2 1 1 2 3 5 1 IterationNode 1Node 2Node 3Node 4Node 5 1(3,3)(4,4)(4, 5)(3,3)(6,2) 2(3,7)(4,4)(4, 5)(2,5)(6,2) 3(3,7)(4,6)(4, 7)(5,5)(6,2) 4(2,9)(4,6)(4, 7)(5,5)(6,2) San Jose 0 7 7 5 6 9 2 Node 1 could have chose 3 as next node because of tie

65. 3124 111 3124 11 X (a) (b) UpdateNode 1Node 2Node 3 Before break (2,3)(3,2)(4, 1) After break (2,3)(3,2)(2,3) 1 (3,4)(2,3) 2(2,5)(3,4)(2,5) 3 (3,6)(2,5) 4(2,7)(3,6)(2,7) 5 (3,8)(2,7) ………… Counting to Infinity Problem Nodes believe best path is through each other (Destination is node 4)