1. 1 Weighted Graphs

2. 2 Outline (single-source) shortest path –Dijkstra (Section 4.4) –Bellman-Ford (Section 4.6) (all-pairs) shortest path –Floyd-Warshall (Section 6.6) minimum spanning tree –Kruskal (Section 5.1.3) –Prim (Section 5.1.5)

3. 3 Shortest Path Problems How can we find the shortest route between two points on a road map? Model the problem as a graph problem: –Road map is a weighted graph: vertices = cities edges = road segments between cities edge weights = road distances –Goal: find a shortest path between two vertices (cities)

4. 4 Shortest Path Problem Input: –Directed graph G = (V, E) –Weight function w : E → R Weight of path p =  v 0, v 1,..., v k  Shortest-path weight from u to v : δ (u, v) = min w(p) : u v if there exists a path from u to v ∞ otherwise Note: there might be multiple shortest paths from u to v p 0 39 511 3 6 5 7 6 s tx yz 2 2 1 4 3

5. 5 Variants of Shortest Path Single-source shortest paths –G = (V, E)  find a shortest path from a given source vertex s to each vertex v  V Single-destination shortest paths –Find a shortest path to a given destination vertex t from each vertex v –Question: How to solve it ?

6. 6 Variants of Shortest Path Single-source shortest paths –G = (V, E)  find a shortest path from a given source vertex s to each vertex v  V Single-destination shortest paths –Find a shortest path to a given destination vertex t from each vertex v –Reversing the direction of each edge  single-source

7. 7 Variants of Shortest Paths (cont’d) Single-pair shortest path –Find a shortest path from u to v for given vertices u and v All-pairs shortest-paths –Find a shortest path from u to v for every pair of vertices u and v

8. 8 Shortest-Path Problems Single-source (single-destination). Find a shortest path from a given source (vertex s) to all the other vertices positive weights → greedy algorithm pos. & neg. weights → dynamic programming All-pairs. Find shortest-paths for every pair of vertices pos. & neg. weights → dynamic programming

9. 9 Optimal Substructure Theorem Given: –A weighted, directed graph G = (V, E) –A weight function w: E  R, –A shortest path p =  v 1, v 2,..., v k  from v 1 to v k –A subpath of p : p ij =  v i, v i+1,..., v j , with 1  i  j  k Then: p ij is a shortest path from v i to v j Proof: p = v 1 v i v j v k w(p) = w(p 1i ) + w(p ij ) + w(p jk ) Assume  p ij ’ from v i to v j with w(p ij ’) < w(p ij )  w(p’) = w(p 1i ) + w(p ij ’) + w(p jk ) < w(p) contradiction! p 1i p ij p jk v1v1 vivi vjvj vkvk p 1i p ij p ij ’ p jk

10. 10 Triangle Inequality for positive weights For all ( u, v )  E, we have: δ (s, v) ≤ δ (s, u) + δ (u, v) - If u is on the shortest path to v we have the equality sign uv s v s u

11. 11 Algorithms Operations common in both algorithms: –Initialization –Relaxation Dijkstra’s algorithm –Negative weights are not allowed Bellman-Ford algorithm –Negative weights are allowed –Negative cycles reachable from the source are not allowed.

12. 12 Shortest-Paths Notation For each vertex v  V: δ(s, v): shortest-path weight d[v]: shortest-path weight estimate –Initially, d[v]=∞ –d[v]  δ(s,v) as algorithm progresses  [v] = predecessor of v on a shortest path from s –If no predecessor,  [v] = NIL –  induces a tree—shortest-path tree 0 39 511 3 6 5 7 6 s tx yz 2 2 1 4 3

13. 13 Initialization Alg.: INITIALIZE-SINGLE-SOURCE(V, s) 1. for each v  V 2. do d[v] ←  3.  [v] ← NIL 4.d[s] ← 0 All the shortest-paths algorithms start with INITIALIZE-SINGLE-SOURCE

14. 14 Relaxation Step Relaxing an edge (u, v) = testing whether we can improve the shortest path to v found so far by going through u If d[v] > d[u] + w(u, v) we can improve the shortest path to v  d[v]=d[u]+w(u,v)   [v] ← u 59 2 uv 57 2 uv RELAX(u, v, w) 56 2 uv 56 2 uv After relaxation: d[v]  d[u] + w(u, v) ss no change

15. 15 Dijkstra’s algorithm

16. 16 Dijkstra’s Algorithm Single-source shortest path problem: –positive-weight edges: w(u, v) > 0,  (u, v)  E Each edge is relaxed only once! Maintains two sets of vertices: d[v]=δ (s, v) d[v]>δ (s, v)

17. 17 Dijkstra’s Algorithm (cont.) Vertices in V – S reside in a min-priority queue –Keys in Q are estimates of shortest-path weights d[u] Repeatedly select a vertex u  V – S, with the minimum shortest-path estimate d[u] Relax all edges leaving u

18. 18 Dijkstra (G, w, s) 0   10 1 5 2 s tx yz 2 3 9 7 4 6 0   1 5 2 s tx yz 2 3 9 7 4 6 5 S=<> Q= S= Q=

19. 19 Example (cont.) 0 10  5  1 5 2 s tx yz 2 3 9 7 4 6 814 7 0 8 5 7 10 1 5 2 s tx yz 2 3 9 7 4 6 13 S= Q=

20. 20 Example (cont.) 0 8 13 57 10 1 5 2 s tx yz 2 3 9 7 4 6 9 0 8 9 57 1 5 2 s tx yz 2 3 9 7 4 6 S= Q= S= Q=<>

21. 21 Dijkstra (G, w, s) 1. INITIALIZE-SINGLE-SOURCE( V, s ) 2. S ←  3.Q ← V[G] 4. while Q   5. do u ← EXTRACT-MIN(Q) 6. S ← S  { u } 7. for each vertex v  Adj[u] 8. do RELAX( u, v, w ) 9. Update Q (DECREASE_KEY) Running time: O(VlgV + ElgV) = O(ElgV)  (V) O(V) build min-heap Executed O(V) times O(lgV) O(E) times (total) O(lgV) O(VlgV) O(ElgV)

22. 22 Binary Heap vs Fibonacci Heap Running time depends on the implementation of the heap

23. 23 Correctness Dijkstra’s algorithm is a greedy algorithm – make choices that currently seem the best – locally optimal does not always mean globally optimal Correct because maintains following two properties: – for every known vertex, recorded distance is shortest distance to that vertex from source vertex – for every unknown vertex v, its recorded distance is shortest path distance to v from source vertex, considering only currently known vertices and v – Let’s prove the Correctness

24. 24 Correctness of Dijskstra’s Algorithm For each vertex u  V, we have d[u] = δ(s, u) at the time when u is added to S. Proof: Let u be the first vertex for which d[u]  δ(s, u) when added to S Let’s look at a true shortest path p from s to u:

25. 25 Correctness of Dijskstra’s Algorithm Since u’ is in the shortest path of u: d[u’]<δ(s,u) What is the value of d[u’]? d[u’]≤d[v’]+w(v’,u’)= δ(s,v’)+w(v’,u’) What is the value of d[u]? d[u]≤d[v]+w(v,u)= δ(s,v)+w(v,u) Using the upper bound property: d[u]>δ(s,u) d[u’]<d[u] Priority Queue Q: (i.e., d[u]<…<d[u’]<… ) Contradiction!

26. Example 2 Consider the graph: –the distances are appropriately initialized –all vertices are marked as being unvisited

27. Example 2 Visit vertex 1 and update its neighbours, marking it as visited –the shortest paths to 2, 4, and 5 are updated

28. Example 2 The next vertex we visit is vertex 4 –vertex 51 + 11 ≥ 8don’t update –vertex 71 + 9 < ∞update –vertex 81 + 8 < ∞update

29. Example 2 Next, visit vertex 2 –vertex 34 + 1 < ∞ update –vertex 4already visited –vertex 54 + 6 ≥ 8don’t update –vertex 64 + 1 < ∞ update

30. Example 2 Next, we have a choice of either 3 or 6 We will choose to visit 3 –vertex 55 + 2 < 8 update –vertex 65 + 5 ≥ 5don’t update

31. Example 2 We then visit 6 –vertex 85 + 7 ≥ 9don’t update –vertex 95 + 8 < ∞ update

32. Example 2 Next, we finally visit vertex 5: –vertices 4 and 6 have already been visited –vertex 77 + 1 < 10 update –vertex 87 + 1 < 9 update –vertex 97 + 8 ≥ 13don’t update

33. Example 2 Given a choice between vertices 7 and 8, we choose vertex 7 –vertices 5 has already been visited –vertex 88 + 2 ≥ 8don’t update

34. Example 2 Next, we visit vertex 8: –vertex 98 + 3 < 13update

35. Example 2 Finally, we visit the end vertex Therefore, the shortest path from 1 to 9 has length 11

36. Example 2 We can find the shortest path by working back from the final vertex: –9, 8, 5, 3, 2, 1 Thus, the shortest path is (1, 2, 3, 5, 8, 9)

37. Example 3 37

38. Example 3 38

39. Example 3 39

40. Example 3 40

41. Example 3 41

42. Example 3 42

43. Example 3 43

44. Example 3 44

45. 45 Example 4: Initialization A GF B ECD 4 1 2 103 64 22 85 1 0     Pick vertex in List with minimum distance.  Distance(source) = 0 Distance (all vertices but source) = 

46. 46 Example 4: Update neighbors' distance A GF B ECD 4 1 2 103 64 22 85 1 0 2   1  Distance(B) = 2 Distance(D) = 1

47. 47 Example 4: Remove vertex with minimum distance Pick vertex in List with minimum distance, i.e., D A GF B EC D 4 1 2 103 64 22 85 1 0 2   1 

48. 48 Example 4: Update neighbors A GF B EC D 4 1 2 103 64 22 85 1 0 2 3 3 1 95 Distance(C) = 1 + 2 = 3 Distance(E) = 1 + 2 = 3 Distance(F) = 1 + 8 = 9 Distance(G) = 1 + 4 = 5

49. 49 Example 4: Continued... A GF B ECD 4 1 2 103 64 22 85 1 0 2 3 3 1 Pick vertex in List with minimum distance (B) and update neighbors 95 Note : distance(D) not updated since D is already known and distance(E) not updated since it is larger than previously computed

50. 50 Example 4: Continued... A GF B E CD 4 1 2 103 64 22 85 1 0 2 3 3 1 95 No updating Pick vertex List with minimum distance (E) and update neighbors

51. 51 Example 4: Continued... A GF B E C D 4 1 2 103 64 22 85 1 0 2 3 3 1 8 5 Pick vertex List with minimum distance (C) and update neighbors Distance(F) = 3 + 5 = 8

52. 52 Example 4: Continued... A G F B ECD 4 1 2 103 64 22 85 1 0 2 3 3 1 6 5 Distance(F) = min (8, 5+1) = 6 Previous distance Pick vertex List with minimum distance (G) and update neighbors

53. 53 Example 4 (end) A G F B ECD 4 1 2 103 64 22 85 1 0 2 3 3 1 Pick vertex not in S with lowest cost (F) and update neighbors 65

54. 54 Example 5: s=BWI

55. 55 Example 5

56. 56 Example 5

57. 57 Example 5

58. 58 Example 5

59. 59 Negative weights Dijkstra fails on graphs with negative edges Example: Bringing z into S and performing edge relaxation invalidates the previously computed shorted path distance (124) to x S

60. Quiz 3 November 19 Everything up to today including today No recurrence solving Emphasis on graphs Need to know greedy, dynamic, divide and concuer 60

61. 61 Bellman-Ford’s algorithm

62. 62 Bellman-Ford’s algorithm Dijkstra’s algorithm does not work when the weighted graph contains negative edges –we cannot be greedy anymore on the assumption that the lengths of paths will not decrease in the future Bellman-Ford’s algorithm detects negative cycles (returns false) or returns the shortest path-tree

63. 63 Bellman-Ford’s algorithm Use d[v] labels (like in Dijkstra’s and Prim’s) Initialize d[s]=0, d[v]= ∞  otherwise Perform |V|-1 rounds In each round, we attempt an edge relaxation for all the edges in the graph (arbitrary order) An extra round of edge relaxation can tell the presence of a negative cycle

64. 64

65. 65 Bellman-Ford’s algorithm Algorithm Bellman-Ford(G(V,E),s) for each u in  V d[u]  ∞ d[s]  0 for i  1 to |V|-1 do for each (u,v)  in  E do if d[v] > d[u] + w(u,v) then d[v]  d[u] + w(u,v) for each (u,v) in E do if d[v] > d[u] + w(u,v) then return F ALSE return d[],T RUE

66. 66 0 ∞ ∞ ∞∞ z uv x y 6 5 –3 9 7 7 8 –2 –4 2 Iteration 0

67. 67 0 ∞ 7 ∞6 z uv x y 6 5 –3 9 7 7 8 –2 –4 2 Iteration 1

68. 68 0 2 7 46 z uv x y 6 5 –3 9 7 7 8 –2 –4 2 Iteration 2

69. 69 0 2 7 42 z uv x y 6 5 –3 9 7 7 8 –2 –4 2 Iteration 3

70. 70 0 -2 7 42 z uv x y 6 5 –3 9 7 7 8 –2 –4 2 Iteration 4

71. 71 Observe that BF is essentially dynamic programming. Let d(i, j) = “cost of the shortest path from s to i that uses at most j edges” d(i, j) = 0 if i = s & j = 0 ∞ if i ≠ s & j = 0 min (k,i) in E {d(k, j–1) + w(k,i), d(i, j–1)} if j > 0 zuvxy 1 2 3 4 5 ∞∞∞∞ 00∞∞∞∞ ∞∞ 106∞7∞ 206472 302472 40247 –2 j i

72. 72 Bellman-Ford’s correctness Works for negative-weight edges Can detect the presence of negative-weight cycles Running time is O(nm)

73. 73 Shortest path on a DAG

74. 74 Shortest path on DAGs When the graph is acyclic (DAG), we can solve the single-source shortest path faster than using Dijkstra’s or Bellman-Ford’s Steps: –Run topological sorting –Relax edges in the sorted order

75. 75 Shortest path on DAGs Topological order: ?

76. 76 Shortest path on DAGs Topological order: 1, 2, 3, 4, 5, 6

77. 77 Shortest path on DAGs

78. 78 Shortest path on DAGs

79. 79 Shortest path on DAGs

80. 80 Shortest path on DAGs Topological sorting runs in O(n+m) The algorithms check each node once, and each outgoing edge once Total time is O(n+m)

81. 81 Floyd-Warshall’s algorithm

82. 82 All-pairs shortest path We want to compute the shortest path distance between every pair of vertices in a directed graph G (n vertices, m edges) We want to know D[i,j] for all i,j, where D[i,j]=shortest distance from v i to v j

83. 83 All-pairs shortest path If G has no negative-weight edges, we could use Dijkstra’s algorithm repeatedly from each vertex It would take O(n (m+n) log n) time, that is O(n 2 log n + nm log n) time, which could be as large as O(n 3 log n)

84. 84 All-pairs shortest path If G has negative-weight edges (but no negative-weight cycles) we could use Bellman-Ford’s algorithm repeatedly from each vertex Recall that Bellman-Ford’s algorithm runs in O(nm) It would take O(n 2 m) time, which could be as large O(n 4 ) time

85. 85 All-pairs shortest path We will see an algorithm to solve the all- pairs shortest path in O(n 3 ) time The graph can contain negative-weight edges (but no negative-weight cycles)

86. 86 All-pairs shortest path Let G=(V,E) a weighted directed graph Let V=(v 1,v 2,…,v n )

87. 87 A dynamic programming shortest-path

88. 88 A dynamic programming shortest-path The cost of going from v i to v j using vertices 1,…,k is the shortest between –(do not use v k ) The shortest path from v i to v j using vertices 1,…,k-1 –(use v k ) The shortest path from v i to v k using 1,…,k-1 plus the cost of the shortest path from v k to v j using 1,…,k-1

89. 89 A dynamic programming shortest-path There is no shorter path from v i to v j using vertices 1,…,k that the previous two because –If there was a shorter path not using v k then it would violate the definition of D i,j k-1 –If there was a shorter path using v k then it would violate the definition of D i,k k-1 or D k,j k-1

90. 90 All-pairs shortest path

91. 91 All-pairs shortest path Floyd-Warshall’s algorithm computes the shortest path distance between each pair of vertices of G in O(n 3 ) time

92. Graph Algorithms All-Pairs Shortest Paths Given graph (directed or undirected) G = (V,E) with weight function w: E  R find for all pairs of vertices u,v  V the minimum possible weight for path from u to v. [Adapted from M.Jacome]

93. Graph Algorithms Floyd-Warshall Algorithm - Idea [Adapted from M.Jacome]

94. Graph Algorithms Floyd-Warshall Algorithm - Idea [Adapted from M.Jacome]

95. Graph Algorithms Floyd-Warshall Algorithm - Idea d s,t (i) – the shortest path from s to t containing only vertices v 1,..., v i d s,t (0) = w(s,t) d s,t (k) = w(s,t)if k = 0 min{d s,t (k-1), d s,k (k-1) + d k,t (k-1) }if k > 0

96. Graph Algorithms Floyd-Warshall Algorithm - Algorithm FloydWarshall(matrix W, integer n) for k  1 to n do for i  1 to n do for j  1 to n do d ij (k)  min(d ij (k-1), d ik (k-1) + d kj (k-1) ) return D (n)

97. Graph Algorithms Floyd-Warshall Algorithm - Example 2 45 13 3 4 -4 -5 6 7 1 8 2 038  -4  0  17  40  2  -50   60 W

98. Graph Algorithms Floyd-Warshall Algorithm - Example 038  -4  0  17  40  2  -50   60 0000 000 00 000 00 D (0)  (0)

99. Graph Algorithms Floyd-Warshall Algorithm - Example 038  -4  0  17  40  25-50-2  60 0000 000 00 01001 00 D (1)  (1)

100. Graph Algorithms Floyd-Warshall Algorithm - Example 0384-4  0  17  40511 25-50-2  60 00020 000 0022 01001 00 D (2)  (2)

101. Graph Algorithms Floyd-Warshall Algorithm - Example 0384-4  0  17  40511 2-50-2  60 00020 000 0022 03001 00 D (3)  (3)

102. Graph Algorithms Floyd-Warshall Algorithm - Example 034-4 30 1 74053 2 -50-2 85160 00420 40401 40021 03001 43400 D (4)  (4)

103. Graph Algorithms Floyd-Warshall Algorithm - Example 032-4 30 1 74053 2 -50-2 85160 00450 40401 40021 03001 43400 D (5)  (5)

104. Graph Algorithms Floyd-Warshall Algorithm - Extracting the shortest paths [Adapted from S.Cheng]

105. Graph Algorithms Floyd-Warshall Algorithm - Complexity T(V,E) =  (n 3 ) =  (V 3 ) FloydWarshall(matrix W, integer n) for k  1 to n do for i  1 to n do for j  1 to n do d ij (k)  min(d ij (k-1), d ik (k-1) + d kj (k-1) ) return D (n) 3 for cycles, each executed exactly n times

106. 106 Reading Assignment Dasgupta –single-source shortest path (4.4, 4.6 and 4.7) –all-pairs shortest path (6.6) –minimum spanning tree (5.1.3, 5.1.5)