Copyright © 2011-2016 Aiman Hanna All rights reserved Shortest Paths Dr. Aiman Hanna Department of Computer Science & Software Engineering Concordia University, Montreal, Canada These slides has been extracted, modified and updated from original slides of : Data Structures and Algorithms in Java, 5th edition. John Wiley& Sons, 2010. ISBN 978-0-470-38326-1. Data Structures and the Java Collections Framework by William J. Collins, 3rdedition, ISBN 978-0-470-48267-4. Both books are published by Wiley. Copyright © 2010-2011 Wiley Copyright © 2010 Michael T. Goodrich, Roberto Tamassia Copyright © 2011 William J. Collins Copyright © 2011-2016 Aiman Hanna All rights reserved

Coverage Shortest Paths C B A E D F 3 2 8 5 4 7 1 9 Shortest Paths

Weighted Graphs PVD ORD SFO LGA HNL LAX DFW MIA Breadth-first search can be used to find the shortest path from one starting vertex to all other ones assuming that all edges in the graph have the same weight (same length, cost, distance, etc.). However, in many situations this may not be the case, and hence BFS is inappropriate. Note: If you are wondering whether these distance are actually true, here are these airports; you can check these ditances out! ORD - Chicago O'Hare International Airport PVD - T. F. Green Airport in Providence, in Kent County, Rhode Island LGA - LaGuardia Airport - New Jersey DFW - Dallas/Fort Worth International Airport LAX - Los Angeles International Airport SFO - San Francisco International Airport HNL - Honolulu International Airport MIA - Miami International Airport 849 PVD 1843 ORD 142 SFO 802 LGA 1743 1205 337 1387 HNL 2555 1099 LAX 1233 DFW 1120 MIA Shortest Paths

Weighted Graphs PVD ORD SFO LGA HNL LAX DFW MIA 849 1843 142 802 1743 Graphs where edges have different weight, are called weighted graphs. In a weighted graph, each edge has an associated numerical value, called the weight of the edge, which can be different from one edge to another. Edge weight may represent, distances, costs, etc. For instance, in a flight route graph, the weight of an edge represents the distance in miles between the endpoint airports ORD PVD MIA DFW SFO LAX LGA HNL 849 802 1387 1743 1843 1099 1120 1233 337 2555 142 1205 Shortest Paths

Shortest Paths PVD ORD SFO LGA HNL LAX DFW MIA Given a weighted graph and two vertices u and v, we want to find a path of minimum total weight between u and v. Length of a path is the sum of the weights of its edges. Example: Shortest path between Providence and Honolulu Applications Internet packet routing Flight reservations Driving directions 849 PVD 1843 ORD 142 SFO 802 LGA 1743 1205 337 1387 HNL 2555 1099 LAX 1233 DFW 1120 MIA Shortest Paths

Shortest Path Properties Property 1: The length (or weight) of a path is the sum of the weights of the edges composing that path Property 2: A subpath of a shortest path is itself a shortest path Property 3: There is a tree of shortest paths from a start vertex to all the other vertices Example: Tree of shortest paths from Providence 849 PVD 1843 ORD 142 SFO 802 LGA 1743 1205 337 1387 HNL 2555 1099 LAX 1233 DFW 1120 MIA Shortest Paths

Negative-Weight Cycles The distance from vertex v to vertex u in a graph G, denoted by d(v, u), is the length of the minimum length path (the shortest path) from v to u. d(v, u) = +∞ if there is no path between v and u. There are cases however where the distance between v and u may not be defined even if there is a path form v to u in G. This is the case if the graph has a negative-weight cycle (a cycle whose total weight is negative). A negative-weight edge is an edge whose weight is negative. For instance, assume that the edges in the airport graphs have weights that define the cost to travel between these cities. Further, assume that someone has offered to pay travelers an amount that is larger than the actual cost to travel between PVD and ORD. Shortest Paths

Negative-Weight Cycles Consequently, the cost from PVD to ORD (that edge e = (PVD, ORD)) would carry a negative value, which makes e = (PVD, ORD) a negative-weight edge. Further, assume that someone else has then offered travelers to pay an amount that is larger than the actual cost to travel between ORD and PVD. Consequently, there is now a negative-weight cycle (cycle with a total weight that is negative) between PVD and ORD. These cycles are very undesirable and must be avoided when edge weights are used to represent distances. Now, the distances of that graph can no longer be defined since someone can build a path from any city A to another B going through PVD and making as many cycles as desired between PVD and ORD before continuing to B. ORD PVD MIA DFW SFO LAX LGA HNL -200 590 818 690 410 788 820 767 400 458 345 560 -300 Shortest Paths

Dijkstra’s Algorithm Dijkstra’s algorithm computes the distances (the shortest paths) of all the vertices from a given start vertex s Assumptions: the graph is connected the edges are undirected the edge weights are nonnegative Shortest Paths

Dijkstra’s Algorithm Edge Relaxation: Let us define a label D[u] for each vertex u in the graph, where D[u] stores an approximation of the shortest path distance from a starting vertex v to u. In other words, D[u] will always store the length of the best path to u that could have been found so far. Initially, D[v] = 0, and D[u] = +∞ for each other vertex u in the graph. D[u] will afterwards be updated if a better path is found that results in a smaller shortest path value to u. The operation is known as relaxation; the idea is similar to stretching a spring more than needed then “relax” it back to its true resting distance. Shortest Paths

Dijkstra’s Algorithm The idea is to grow a “cloud” of vertices, beginning with the stating vertex s and eventually covering all the vertices In details, the algorithm starts by defining two sets; let us refer to them as the C (for Cloud) set and the S set. Initially, C is empty, while S has vertex s. Additionally, all the distance approximations, D[u], as initialized as ∞,with the exception of D[v], which is initialized as 0. Afterwards C will grow to include specific vertices one at a time (those are the ones with concrete shortest paths), while S will include all other vertices that are directly connected either to s or to other vertices already in C. Algorithm terminates once all vertices have been moved to C. Shortest Paths

Dijkstra’s Algorithm (continues…) In details, the algorithm performs the repeated moving operations from S to C as follows: Move one vertex from S to C. Let us refer to that vertex as X. This vertex, X, must satisfy the following: It is either directly connected to v or directly connected to another vertex that is already in C Has the smallest distance to v among all the potential vertices that are considered for movement to C Whenever the vertex X is moved to C, its distance is the smallest distance (shortest path) to v Additionally, once this vertex X is chosen, move it to C and recheck all distances of all vertices that have direct connection to X and are NOT yet in C. If a smaller value than what we already have is found, update this value In each step, we also update how to go to these vertices (that is, through which vertex); we refer to this vertex as the prior function The steps are repeated until all the graph vertices are moved to C. The final obtained distances represent the shortest paths, and the prior functions indicate the direction from v (through which vertex) to obtain these shortest paths to each of the other vertices in the graph Shortest Paths

Dijkstra’s Algorithm (continues…) v X u S C Distances yet to be determined Distances to potential vertices Potential vertex with shortest distance X Selected vertex to move to C Different routes to u. Is there a cheaper route through X to other vertices in S? If so, update distances to these vertices. Shortest Paths

Dijkstra’s Algorithm Example: What are the shortest paths from A to other vertices?* *Figure is © Understanding Data Communications and Networks by William A. Shay - PWS publishing company, Third Edition. ISBN-10: 0534383173, ISBN-13: 978-0534383176. Data Communications & Computer Networking, by: Aiman Hanna

Dijkstra’s Algorithm Cost function Prior function C Potential elements of S X A B D E F {} {A} ∞ - 1 {B,C} 2 {A, C} {B,D,E,F} 4 7 8 3 {A,B,C} {D,E,F} 6 {A,B,C,D} {E,F} 5 {A,B,C,D,E} {F} Notes: Here are the details of the algorithm’s progress: Start with the initial set of S including the potential nodes that can be moved (these are either vertices directly connected from A or directly connected through other vertices already in C). At the moment, only A connects to itself, so C:={A}. A is the smallest distance, so it moves to C. Information of A has already been determined (the highlighted parts in yellow will never change or be considered for distance comparison again) Continuing now, the set of S will include all nodes that are directly connected to A; so S:={B,C} Find the smallest cost from A to each one in S; that is B_2 & C_1 (This denotes VertexName_cost-to-it) Now update the cost function and the prior to function. The cost to B is 2, to C is 1 and to all the rest is ∞ since there is no possible connection to them from any node in C. The prior to (think of it as Through) will have A for B and A for C and nothing for the rest Find the node with the smallest cost; that will be X  step # 1 is now complete Move the node with the smallest cost to cloud set C. The node with smallest cost now is C, so C:={A,C} (Notice that the first C is the Cloud set while the later C is node C. The names could not have been any terrible!) Now reconstruct S with all nodes that either with a direct link to A or with a possible connection through one node in C, so S:={B, D, E, F} since B is connected to A and D, E and F are all connected to A through C Find every possible cost from A to the nodes in S; connected either directly from A or through any other node in C; in our case at this moment the only other node is C The costs are as follows: (VertexName_cost-through) B_2-a, D_4-c, E_7-c, F_8-c Choose the smallest distance and set its vertex as X; this is also what will move next to C. We must also update the prior function if we find a smallest cost for any of the vertices compared to the previous step. Now B has the smallest Cost and so it set as X. The smallest cost is now updated to be 2 for B, 1 for C, 4 for D, 7 for E and 8 for F. The prior function will also be reflected as a result, so the prior for B is A, for C is A, for D is C, for E is C and for F is C. Now move B to C and start Step 3 At that level, the costs are as follows: D_5-b, D_4-c, E_6-b, E_7-c, F_9-b, F_8-c. The smallest is D_4 through C. This concludes that D will move next to C. However if you look at E, now there is smaller cost through B (that is 6) compared to the last step, which was 7 through C. As a result, we must update the prior to E to be B instead of C. Now X is set to D and D will move to C and start step 4.  The rest is identical, and so it should be straight forward. In general, you will find in step 4 that the values are: E_6-b, E_7-c, E_12-cd (that is through D, which is through C as prior) is C), F_9-b, F_8-c, F_6-cd. Now we have a tie between E_6-b and F_6-cd; just pick up any of them as smallest. In our solutions we picked E_6-b as smallest and so we set it as X and moved it to C when step 5 starts. In the last step we have the following values: F_9-b, F_8-c, F_6-cd, F_12-be, and so the prior of F is set as D with a smallest cost of 6. Note: See comments attached to this slide for detailed information on how the operations progressed. Data Communications & Computer Networking, by: Aiman Hanna

Edge Relaxation Consider an edge e = (u,z) such that d(z) = 75 u is the vertex most recently added to the cloud z is not in the cloud The relaxation of edge e updates distance d(z) as follows: d(z)  min{d(z),d(u) + weight(e)} d(u) = 50 10 d(z) = 75 e u s z d(v) = 40 35 v d(u) = 50 10 d(z) = 60 u e s z d(v) = 40 35 v Shortest Paths

Example 2 – Visual Approach C B A E D F 3 2 8 5 4 7 1 9 A 4 8 2 8 2 4 7 1 B C D 3 9   2 5 E F A 4 A 4 8 8 2 2 8 2 3 7 2 3 7 1 7 1 B C D B C D 3 9 3 9 5 11 5 8 2 5 2 5 E F E F Shortest Paths

Example 2 (cont.) A 4 8 2 7 2 3 7 1 B C D 3 9 5 8 2 5 E F A 4 8 2 7 2 A 4 8 2 7 2 3 7 1 B C D 3 9 5 8 2 5 E F A 4 8 2 7 2 3 7 1 B C D 3 9 5 8 2 5 E F Shortest Paths

Dijkstra’s Algorithm Algorithm DijkstraDistances(G, s) Q  new heap-based priority queue for all v  G.vertices() if v = s setDistance(v, 0) else setDistance(v, ) l  Q.insert(getDistance(v), v) setEntry(v, l) while Q.isEmpty() l  Q.removeMin() u  l.getValue() for all e  G.incidentEdges(u) { relax e } z  G.opposite(u,e) r  getDistance(u) + weight(e) if r < getDistance(z) setDistance(z,r) Q.replaceKey(getEntry(z), r) A heap-based adaptable priority queue with location-aware entries stores the vertices outside the cloud Key: distance Value: vertex Recall that method replaceKey(l,k) changes the key of entry l We store two labels with each vertex: Distance Entry in priority queue Shortest Paths

Analysis of Dijkstra’s Algorithm Graph operations Method incidentEdges is called once for each vertex Label operations We set/get the distance and locator labels of vertex z O(deg(z)) times Setting/getting a label takes O(1) time Priority queue operations Each vertex is inserted once and removed once from the priority queue, where each insertion or removal takes O(log n) time The key of a vertex in the priority queue is modified at most deg(w) times, where each key change takes O(log n) time Recall that Sv deg(v) = 2m Dijkstra’s algorithm runs in O((n + m) log n) time provided the graph is represented by the adjacency list structure The running time can also be expressed as a function of n only as O(n2 log n) Recall that m <= n(n-1)/2 Shortest Paths

Shortest Paths Tree Using the template method pattern, we can extend Dijkstra’s algorithm to return a tree of shortest paths from the start vertex to all other vertices We store with each vertex a third label: parent edge in the shortest path tree In the edge relaxation step, we update the parent label Algorithm DijkstraShortestPathsTree(G, s) … for all v  G.vertices() setParent(v, ) for all e  G.incidentEdges(u) { relax edge e } z  G.opposite(u,e) r  getDistance(u) + weight(e) if r < getDistance(z) setDistance(z, r) setParent(z,e) Q.replaceKey(getEntry(z),r) Shortest Paths

Why Dijkstra’s Algorithm Works Dijkstra’s algorithm is based on a method called the greedy method. It adds vertices by increasing distance. The first wrong vertex with a path longer than the shortest path Assume that vertex u is the first vertex that was incorrectly picked up to enter the cloud By that time, both v (the starting vertex), and y (a correctly picked up vertex) are already in the cloud However, assume that the algorithm misbehaves such that it has incorrectly picked up u to enter the cloud without having a correct shortest path v P D[y] = d(v, y) C y u z D[z] = d(v, z) D[u] > d(v, u) Shortest Paths

Why Dijkstra’s Algorithm Works Since u was chosen by error, then D[u] > d(v, u) Let then back a bit on time to the moment when y was added to the cloud. At that time, we must have evaluated the relaxed distance to z, D[z], and possibly have updated it to reflect the current known shortest path to z. Consequently, at this point we had: D[z] <= D[y] + w((y, z))  D[z] <= d(v, y)+ w((y, z)) The first wrong vertex with a path longer than the shortest path v P D[y] = d(v, y) C y u z D[z] = d(v, z) D[u] > d(v, u) Shortest Paths

Why Dijkstra’s Algorithm Works But, since z is indeed having a correct shortest path (recall that u is he first incorrect vertex) then D[z] = d(v, z) But since now the algorithm is picking up u not z, it must have found that: D[u] <= D[z] Recall that a shortest path is composed of shortest sub-paths. Hence, since z is in the shortest path, P, from v to u then: d(v, z)+ d(z, u) = d(v, u) The first wrong vertex with a path longer than the shortest path v P D[y] = d(v, y) C y u z D[z] = d(v, z) D[u] > d(v, u) Shortest Paths

Why Dijkstra’s Algorithm Works Moreover, d(z, u) > 0 since there are no negative edges Therefore: D[u] <= D[z] = d(v, z)  d(v, z) <= d(v, z) + d(z, u)  d(v, z) <= d(v, u)  D[u] <= d(v, u) However this contradicts our original definition of u and assumption that D[u] > d(v, u)  Which means that there can never be such a vertex u. The first wrong vertex with a path longer than the shortest path v P D[y] = d(v, y) C y u z D[z] = d(v, z) D[u] > d(v, u) Shortest Paths

Why It Doesn’t Work for Negative-Weight Edges Dijkstra’s algorithm is based on the greedy method. It adds vertices by increasing distance. A 8 4 If a node with a negative incident edge were to be added late to the cloud, it could mess up distances for vertices already in the cloud. 6 7 5 4 7 1 B C D -8 5 9 2 5 E F C’s true distance is 1, but it is already in the cloud with d(C)=5 and possibly has already affected many other distances! Shortest Paths

Bellman-Ford Algorithm (not in book) Works even with negative-weight edges Must assume directed edges (for otherwise we would have negative-weight cycles) Iteration i finds all shortest paths that use i edges. Running time: O(nm). Can be extended to detect a negative-weight cycle if it exists How? Algorithm BellmanFord(G, s) for all v  G.vertices() if v = s setDistance(v, 0) else setDistance(v, ) for i  1 to n - 1 do for each e  G.edges() { relax edge e } u  G.origin(e) z  G.opposite(u,e) r  getDistance(u) + weight(e) if r < getDistance(z) setDistance(z,r) Shortest Paths

Bellman-Ford Example Nodes are labeled with their d(v) values 4 4 8 8 4 4 8 8 -2 -2 8 -2 4 7 1 7 1       3 9 3 9 -2 5 -2 5     4 4 8 8 -2 -2 5 7 1 -1 7 1 8 -2 4 5 -2 -1 1 3 9 3 9 6 9 4 -2 5 -2 5   1 9 Shortest Paths

DAG-based Algorithm (not in book) Algorithm DagDistances(G, s) for all v  G.vertices() if v = s setDistance(v, 0) else setDistance(v, ) { Perform a topological sort of the vertices } for u  1 to n do {in topological order} for each e  G.outEdges(u) { relax edge e } z  G.opposite(u,e) r  getDistance(u) + weight(e) if r < getDistance(z) setDistance(z,r) Works even with negative-weight edges Uses topological order Doesn’t use any fancy data structures Is much faster than Dijkstra’s algorithm Running time: O(n+m). Shortest Paths

DAG Example Nodes are labeled with their d(v) values 4 4 8 8 -2 -2 8 1 1 4 4 8 8 -2 -2 4 8 -2 4 3 7 2 1 3 7 2 1 4       3 9 3 9 -5 5 -5 5     6 5 6 5 1 1 4 4 8 8 -2 -2 5 3 7 2 1 4 -1 3 7 2 1 4 8 -2 4 5 -2 -1 9 3 3 9 1 7 4 -5 5 -5 5   1 7 6 5 6 5 Shortest Paths (two steps)