1 Weighted Graphs

2 Outline (single-source) shortest path –Dijkstra (Section 4.4) –Bellman-Ford (Section 4.6) (all-pairs) shortest path –Floyd-Warshall (Section 6.6) minimum spanning tree –Kruskal (Section 5.1.3) –Prim (Section 5.1.5)

3 Shortest Path Problems How can we find the shortest route between two points on a road map? Model the problem as a graph problem: –Road map is a weighted graph: vertices = cities edges = road segments between cities edge weights = road distances –Goal: find a shortest path between two vertices (cities)

4 Shortest Path Problem Input: –Directed graph G = (V, E) –Weight function w : E → R Weight of path p =  v 0, v 1,..., v k  Shortest-path weight from u to v : δ (u, v) = min w(p) : u v if there exists a path from u to v ∞ otherwise Note: there might be multiple shortest paths from u to v p s tx yz

5 Variants of Shortest Path Single-source shortest paths –G = (V, E)  find a shortest path from a given source vertex s to each vertex v  V Single-destination shortest paths –Find a shortest path to a given destination vertex t from each vertex v –Question: How to solve it ?

6 Variants of Shortest Path Single-source shortest paths –G = (V, E)  find a shortest path from a given source vertex s to each vertex v  V Single-destination shortest paths –Find a shortest path to a given destination vertex t from each vertex v –Reversing the direction of each edge  single-source

7 Variants of Shortest Paths (cont’d) Single-pair shortest path –Find a shortest path from u to v for given vertices u and v All-pairs shortest-paths –Find a shortest path from u to v for every pair of vertices u and v

8 Shortest-Path Problems Single-source (single-destination). Find a shortest path from a given source (vertex s) to all the other vertices positive weights → greedy algorithm pos. & neg. weights → dynamic programming All-pairs. Find shortest-paths for every pair of vertices pos. & neg. weights → dynamic programming

9 Optimal Substructure Theorem Given: –A weighted, directed graph G = (V, E) –A weight function w: E  R, –A shortest path p =  v 1, v 2,..., v k  from v 1 to v k –A subpath of p : p ij =  v i, v i+1,..., v j , with 1  i  j  k Then: p ij is a shortest path from v i to v j Proof: p = v 1 v i v j v k w(p) = w(p 1i ) + w(p ij ) + w(p jk ) Assume  p ij ’ from v i to v j with w(p ij ’) < w(p ij )  w(p’) = w(p 1i ) + w(p ij ’) + w(p jk ) < w(p) contradiction! p 1i p ij p jk v1v1 vivi vjvj vkvk p 1i p ij p ij ’ p jk

10 Triangle Inequality for positive weights For all ( u, v )  E, we have: δ (s, v) ≤ δ (s, u) + δ (u, v) - If u is on the shortest path to v we have the equality sign uv s v s u

11 Algorithms Operations common in both algorithms: –Initialization –Relaxation Dijkstra’s algorithm –Negative weights are not allowed Bellman-Ford algorithm –Negative weights are allowed –Negative cycles reachable from the source are not allowed.

12 Shortest-Paths Notation For each vertex v  V: δ(s, v): shortest-path weight d[v]: shortest-path weight estimate –Initially, d[v]=∞ –d[v]  δ(s,v) as algorithm progresses  [v] = predecessor of v on a shortest path from s –If no predecessor,  [v] = NIL –  induces a tree—shortest-path tree s tx yz

13 Initialization Alg.: INITIALIZE-SINGLE-SOURCE(V, s) 1. for each v  V 2. do d[v] ←  3.  [v] ← NIL 4.d[s] ← 0 All the shortest-paths algorithms start with INITIALIZE-SINGLE-SOURCE

14 Relaxation Step Relaxing an edge (u, v) = testing whether we can improve the shortest path to v found so far by going through u If d[v] > d[u] + w(u, v) we can improve the shortest path to v  d[v]=d[u]+w(u,v)   [v] ← u 59 2 uv 57 2 uv RELAX(u, v, w) 56 2 uv 56 2 uv After relaxation: d[v]  d[u] + w(u, v) ss no change

15 Dijkstra’s algorithm

16 Dijkstra’s Algorithm Single-source shortest path problem: –positive-weight edges: w(u, v) > 0,  (u, v)  E Each edge is relaxed only once! Maintains two sets of vertices: d[v]=δ (s, v) d[v]>δ (s, v)

17 Dijkstra’s Algorithm (cont.) Vertices in V – S reside in a min-priority queue –Keys in Q are estimates of shortest-path weights d[u] Repeatedly select a vertex u  V – S, with the minimum shortest-path estimate d[u] Relax all edges leaving u

18 Dijkstra (G, w, s) 0   s tx yz   s tx yz S=<> Q= S= Q=

19 Example (cont.) 0 10  5  s tx yz s tx yz S= Q=

20 Example (cont.) s tx yz s tx yz S= Q= S= Q=<>

21 Dijkstra (G, w, s) 1. INITIALIZE-SINGLE-SOURCE( V, s ) 2. S ←  3.Q ← V[G] 4. while Q   5. do u ← EXTRACT-MIN(Q) 6. S ← S  { u } 7. for each vertex v  Adj[u] 8. do RELAX( u, v, w ) 9. Update Q (DECREASE_KEY) Running time: O(VlgV + ElgV) = O(ElgV)  (V) O(V) build min-heap Executed O(V) times O(lgV) O(E) times (total) O(lgV) O(VlgV) O(ElgV)

22 Binary Heap vs Fibonacci Heap Running time depends on the implementation of the heap

23 Correctness Dijkstra’s algorithm is a greedy algorithm – make choices that currently seem the best – locally optimal does not always mean globally optimal Correct because maintains following two properties: – for every known vertex, recorded distance is shortest distance to that vertex from source vertex – for every unknown vertex v, its recorded distance is shortest path distance to v from source vertex, considering only currently known vertices and v – Let’s prove the Correctness

24 Correctness of Dijskstra’s Algorithm For each vertex u  V, we have d[u] = δ(s, u) at the time when u is added to S. Proof: Let u be the first vertex for which d[u]  δ(s, u) when added to S Let’s look at a true shortest path p from s to u:

25 Correctness of Dijskstra’s Algorithm Since u’ is in the shortest path of u: d[u’]<δ(s,u) What is the value of d[u’]? d[u’]≤d[v’]+w(v’,u’)= δ(s,v’)+w(v’,u’) What is the value of d[u]? d[u]≤d[v]+w(v,u)= δ(s,v)+w(v,u) Using the upper bound property: d[u]>δ(s,u) d[u’]<d[u] Priority Queue Q: (i.e., d[u]<…<d[u’]<… ) Contradiction!

Example 2 Consider the graph: –the distances are appropriately initialized –all vertices are marked as being unvisited

Example 2 Visit vertex 1 and update its neighbours, marking it as visited –the shortest paths to 2, 4, and 5 are updated

Example 2 The next vertex we visit is vertex 4 –vertex ≥ 8don’t update –vertex < ∞update –vertex < ∞update

Example 2 Next, visit vertex 2 –vertex < ∞ update –vertex 4already visited –vertex ≥ 8don’t update –vertex < ∞ update

Example 2 Next, we have a choice of either 3 or 6 We will choose to visit 3 –vertex < 8 update –vertex ≥ 5don’t update

Example 2 We then visit 6 –vertex ≥ 9don’t update –vertex < ∞ update

Example 2 Next, we finally visit vertex 5: –vertices 4 and 6 have already been visited –vertex < 10 update –vertex < 9 update –vertex ≥ 13don’t update

Example 2 Given a choice between vertices 7 and 8, we choose vertex 7 –vertices 5 has already been visited –vertex ≥ 8don’t update

Example 2 Next, we visit vertex 8: –vertex < 13update

Example 2 Finally, we visit the end vertex Therefore, the shortest path from 1 to 9 has length 11

Example 2 We can find the shortest path by working back from the final vertex: –9, 8, 5, 3, 2, 1 Thus, the shortest path is (1, 2, 3, 5, 8, 9)

Example 3 37

Example 3 38

Example 3 39

Example 3 40

Example 3 41

Example 3 42

Example 3 43

Example 3 44

45 Example 4: Initialization A GF B ECD     Pick vertex in List with minimum distance.  Distance(source) = 0 Distance (all vertices but source) = 

46 Example 4: Update neighbors' distance A GF B ECD   1  Distance(B) = 2 Distance(D) = 1

47 Example 4: Remove vertex with minimum distance Pick vertex in List with minimum distance, i.e., D A GF B EC D   1 

48 Example 4: Update neighbors A GF B EC D Distance(C) = = 3 Distance(E) = = 3 Distance(F) = = 9 Distance(G) = = 5

49 Example 4: Continued... A GF B ECD Pick vertex in List with minimum distance (B) and update neighbors 95 Note : distance(D) not updated since D is already known and distance(E) not updated since it is larger than previously computed

50 Example 4: Continued... A GF B E CD No updating Pick vertex List with minimum distance (E) and update neighbors

51 Example 4: Continued... A GF B E C D Pick vertex List with minimum distance (C) and update neighbors Distance(F) = = 8

52 Example 4: Continued... A G F B ECD Distance(F) = min (8, 5+1) = 6 Previous distance Pick vertex List with minimum distance (G) and update neighbors

53 Example 4 (end) A G F B ECD Pick vertex not in S with lowest cost (F) and update neighbors 65

54 Example 5: s=BWI

55 Example 5

56 Example 5

57 Example 5

58 Example 5

59 Negative weights Dijkstra fails on graphs with negative edges Example: Bringing z into S and performing edge relaxation invalidates the previously computed shorted path distance (124) to x S

Quiz 3 November 19 Everything up to today including today No recurrence solving Emphasis on graphs Need to know greedy, dynamic, divide and concuer 60

61 Bellman-Ford’s algorithm

62 Bellman-Ford’s algorithm Dijkstra’s algorithm does not work when the weighted graph contains negative edges –we cannot be greedy anymore on the assumption that the lengths of paths will not decrease in the future Bellman-Ford’s algorithm detects negative cycles (returns false) or returns the shortest path-tree

63 Bellman-Ford’s algorithm Use d[v] labels (like in Dijkstra’s and Prim’s) Initialize d[s]=0, d[v]= ∞  otherwise Perform |V|-1 rounds In each round, we attempt an edge relaxation for all the edges in the graph (arbitrary order) An extra round of edge relaxation can tell the presence of a negative cycle

64

65 Bellman-Ford’s algorithm Algorithm Bellman-Ford(G(V,E),s) for each u in  V d[u]  ∞ d[s]  0 for i  1 to |V|-1 do for each (u,v)  in  E do if d[v] > d[u] + w(u,v) then d[v]  d[u] + w(u,v) for each (u,v) in E do if d[v] > d[u] + w(u,v) then return F ALSE return d[],T RUE

66 0 ∞ ∞ ∞∞ z uv x y 6 5 – –2 –4 2 Iteration 0

67 0 ∞ 7 ∞6 z uv x y 6 5 – –2 –4 2 Iteration 1

z uv x y 6 5 – –2 –4 2 Iteration 2

z uv x y 6 5 – –2 –4 2 Iteration 3

z uv x y 6 5 – –2 –4 2 Iteration 4

71 Observe that BF is essentially dynamic programming. Let d(i, j) = “cost of the shortest path from s to i that uses at most j edges” d(i, j) = 0 if i = s & j = 0 ∞ if i ≠ s & j = 0 min (k,i) in E {d(k, j–1) + w(k,i), d(i, j–1)} if j > 0 zuvxy ∞∞∞∞ 00∞∞∞∞ ∞∞ 106∞7∞ –2 j i

72 Bellman-Ford’s correctness Works for negative-weight edges Can detect the presence of negative-weight cycles Running time is O(nm)

73 Shortest path on a DAG

74 Shortest path on DAGs When the graph is acyclic (DAG), we can solve the single-source shortest path faster than using Dijkstra’s or Bellman-Ford’s Steps: –Run topological sorting –Relax edges in the sorted order

75 Shortest path on DAGs Topological order: ?

76 Shortest path on DAGs Topological order: 1, 2, 3, 4, 5, 6

77 Shortest path on DAGs

78 Shortest path on DAGs

79 Shortest path on DAGs

80 Shortest path on DAGs Topological sorting runs in O(n+m) The algorithms check each node once, and each outgoing edge once Total time is O(n+m)

81 Floyd-Warshall’s algorithm

82 All-pairs shortest path We want to compute the shortest path distance between every pair of vertices in a directed graph G (n vertices, m edges) We want to know D[i,j] for all i,j, where D[i,j]=shortest distance from v i to v j

83 All-pairs shortest path If G has no negative-weight edges, we could use Dijkstra’s algorithm repeatedly from each vertex It would take O(n (m+n) log n) time, that is O(n 2 log n + nm log n) time, which could be as large as O(n 3 log n)

84 All-pairs shortest path If G has negative-weight edges (but no negative-weight cycles) we could use Bellman-Ford’s algorithm repeatedly from each vertex Recall that Bellman-Ford’s algorithm runs in O(nm) It would take O(n 2 m) time, which could be as large O(n 4 ) time

85 All-pairs shortest path We will see an algorithm to solve the all- pairs shortest path in O(n 3 ) time The graph can contain negative-weight edges (but no negative-weight cycles)

86 All-pairs shortest path Let G=(V,E) a weighted directed graph Let V=(v 1,v 2,…,v n )

87 A dynamic programming shortest-path

88 A dynamic programming shortest-path The cost of going from v i to v j using vertices 1,…,k is the shortest between –(do not use v k ) The shortest path from v i to v j using vertices 1,…,k-1 –(use v k ) The shortest path from v i to v k using 1,…,k-1 plus the cost of the shortest path from v k to v j using 1,…,k-1

89 A dynamic programming shortest-path There is no shorter path from v i to v j using vertices 1,…,k that the previous two because –If there was a shorter path not using v k then it would violate the definition of D i,j k-1 –If there was a shorter path using v k then it would violate the definition of D i,k k-1 or D k,j k-1

90 All-pairs shortest path

91 All-pairs shortest path Floyd-Warshall’s algorithm computes the shortest path distance between each pair of vertices of G in O(n 3 ) time

Graph Algorithms All-Pairs Shortest Paths Given graph (directed or undirected) G = (V,E) with weight function w: E  R find for all pairs of vertices u,v  V the minimum possible weight for path from u to v. [Adapted from M.Jacome]

Graph Algorithms Floyd-Warshall Algorithm - Idea [Adapted from M.Jacome]

Graph Algorithms Floyd-Warshall Algorithm - Idea [Adapted from M.Jacome]

Graph Algorithms Floyd-Warshall Algorithm - Idea d s,t (i) – the shortest path from s to t containing only vertices v 1,..., v i d s,t (0) = w(s,t) d s,t (k) = w(s,t)if k = 0 min{d s,t (k-1), d s,k (k-1) + d k,t (k-1) }if k > 0

Graph Algorithms Floyd-Warshall Algorithm - Algorithm FloydWarshall(matrix W, integer n) for k  1 to n do for i  1 to n do for j  1 to n do d ij (k)  min(d ij (k-1), d ik (k-1) + d kj (k-1) ) return D (n)

Graph Algorithms Floyd-Warshall Algorithm - Example  -4  0  17  40  2  -50   60 W

Graph Algorithms Floyd-Warshall Algorithm - Example 038  -4  0  17  40  2  -50   D (0)  (0)

Graph Algorithms Floyd-Warshall Algorithm - Example 038  -4  0  17  40   D (1)  (1)

Graph Algorithms Floyd-Warshall Algorithm - Example  0  17   D (2)  (2)

Graph Algorithms Floyd-Warshall Algorithm - Example  0  17   D (3)  (3)

Graph Algorithms Floyd-Warshall Algorithm - Example D (4)  (4)

Graph Algorithms Floyd-Warshall Algorithm - Example D (5)  (5)

Graph Algorithms Floyd-Warshall Algorithm - Extracting the shortest paths [Adapted from S.Cheng]

Graph Algorithms Floyd-Warshall Algorithm - Complexity T(V,E) =  (n 3 ) =  (V 3 ) FloydWarshall(matrix W, integer n) for k  1 to n do for i  1 to n do for j  1 to n do d ij (k)  min(d ij (k-1), d ik (k-1) + d kj (k-1) ) return D (n) 3 for cycles, each executed exactly n times

106 Reading Assignment Dasgupta –single-source shortest path (4.4, 4.6 and 4.7) –all-pairs shortest path (6.6) –minimum spanning tree (5.1.3, 5.1.5)