1. 1 Graphs: shortest paths 15-211 Fundamental Data Structures and Algorithms Ananda Guna April 3, 2003

2. 2 Announcements  Start working on Homework #5.  Quiz #3 will be made available Friday 4/4 for 24 hours.

3. Recap

4. 4 (Undirected) Graphs — an overview PIT BOS JFK DTW LAX SFO Vertices (aka nodes) Edges 1987 2273 344 2145 2462 618 211 318 190 Weights

5. 5 Definitions  Graph G = (V,E)  Set V of vertices (nodes)  Set E of edges  Elements of E are pair (v,w) where v,w  V.  An edge (v,v) is a self-loop. (Usually assume no self-loops.)  Weighted graph  Elements of E are ((v,w),x) where x is a weight.  Directed graph (digraph)  The edge pairs are ordered  Undirected graph  The edge pairs are unordered  E is a symmetric relation  (v,w)  E implies (w,v)  E  In an undirected graph (v,w) and (w,v) are the same edge

6. 6 Graph Questions  Is (x,y) an edge in the Graph?  Given x in V, does (x,x) in E?  Given x, y in V, what is the closest(cheapest) path from x to y (if any)?  What node v in V has the maximum(minimum) degree?  What is the largest connected sub-graph?  What is the complexity of algorithms for each of the above questions if  Graph is stored as an adjacency matrix?  Graph is stored as an adjacency list?

7. 7 Graph Traversals  One of the fundamental operations in graphs. Find things such as  Count the total edges  Output the content in each vertex  Identify connected components  Before/during the tour - mark vertices  Visited  Not-visited  explored

8. 8 Graph Traversals using..  Queue - Store the vertices in a first-in, first out (FIFO) queue as follows. Put the starting node in a queue first. Dequeue the first element and add all its neighbors to the queue. Continue to explore the oldest unexplored vertices first. Thus the explorations radiate out slowly from the starting vertex. This describes a traversal technique known as breadth-first search.  Stack - Store vertices in a last-in, first-out (LIFO) stack. Explore vertices by going along a path, always visiting a new neighbor if one is available, and backing up only if all neighbors are discovered vertices. Thus our explorations quickly move away from start. This is called depth-first search.

9. Shortest Paths

10. 10 Airline routes PVD BOS JFK ORD LAX SFO DFW BWI MIA 337 2704 1846 1464 1235 2342 802 867 849 740 187 144 1391 184 1121 946 1090 1258621

11. 11 Single-source shortest path  Suppose we live in Baltimore (BWI) and want the shortest path to San Francisco (SFO).  One way to solve this is to solve the single-source shortest path problem:  Find the shortest path from BWI to every city.

12. 12 Single-source shortest path  While we may be interested only in BWI-to-SFO, there are no known algorithms that are asymptotically faster than solving the single-source problem for BWI-to-every-city.

13. 13 Shortest paths  What do we mean by “shortest path”?  Minimize the number of layovers (i.e., fewest hops).  Unweighted shortest-path problem.  Minimize the total mileage (i.e., fewest frequent-flyer miles ;-).  Weighted shortest-path problem.

14. 14 Many applications  Shortest paths model many useful real-world problems.  Minimization of latency in the Internet.  Minimization of cost in power delivery.  Job and resource scheduling.  Route planning.

15. Unweighted Single-Source Shortest Path Algorithm

16. 16 Unweighted shortest path  In order to find the unweighted shortest path, we will augment vertices and edges so that:  vertices can be marked with an integer, giving the number of hops from the source node, and  edges can be marked as either explored or unexplored. Initially, all edges are unexplored.

17. 17 Unweighted shortest path  Algorithm:  Set i to 0 and mark source node v with 0.  Put source node v into a queue L 0.  While L i is not empty:  Create new empty queue L i+1  For each w in L i do: For each unexplored edge (w,x) do: –mark (w,x) as explored –if x not marked, mark with i and enqueue x into L i+1  Increment i.

18. 18 Breadth-first search  This algorithm is a form of breadth- first search.  Performance: O(|V|+|E|).  Q: Use this algorithm to find the shortest route (in terms of number of hops) from BWI to SFO.  Q: What kind of structure is formed by the edges marked as explored?

19. 19 Use of a queue  It is very common to use a queue to keep track of:  nodes to be visited next, or  nodes that we have already visited.  Typically, use of a queue leads to a breadth-first visit order.  Breadth-first visit order is “cautious” in the sense that it examines every path of length i before going on to paths of length i+1.

20. Weighted Single-Source Shortest Path Algorithm (Dijkstra’s Algorithm)

21. 21 Weighted shortest path  Now suppose we want to minimize the total mileage.  Breadth-first search does not work!  Minimum number of hops does not mean minimum distance.  Consider, for example, BWI-to-DFW:

22. 22 Three 2-hop routes to DFW PVD BOS JFK ORD LAX SFO DFW BWI MIA 337 2704 1846 1464 1235 2342 802 867 849 740 187 144 1391 184 1121 946 1090 1258621

23. 23 A greedy algorithm  Assume that every city is infinitely far away.  I.e., every city is  miles away from BWI (except BWI, which is 0 miles away).  Now perform something similar to breadth-first search, and optimistically guess that we have found the best path to each city as we encounter it.  If we later discover we are wrong and find a better path to a particular city, then update the distance to that city.

24. 24 Intuition behind Dijkstra’s alg.  For our airline-mileage problem, we can start by guessing that every city is  miles away.  Mark each city with this guess.  Find all cities one hop away from BWI, and check whether the mileage is less than what is currently marked for that city.  If so, then revise the guess.  Continue for 2 hops, 3 hops, etc.

25. 25 Dijkstra’s algorithm  Algorithm initialization:  Label each node with the distance , except start node, which is labeled with distance 0.  D[v] is the distance label for v.  Put all nodes into a priority queue Q, using the distances as labels.

26. 26 Dijkstra’s algorithm, cont’d  While Q is not empty do:  u = Q.removeMin  for each node z one hop away from u do:  if D[u] + miles(u,z) < D[z] then D[z] = D[u] + miles(u,z) change key of z in Q to D[z]  Note use of priority queue allows “finished” nodes to be found quickly (in O(log N) time).

27. 27 Shortest mileage from BWI PVD  BOS  JFK  ORD  LAX  SFO  DFW  BWI 0 MIA  337 2704 1846 1464 1235 2342 802 867 849 740 187 144 1391 184 1121 946 1090 1258621

28. 28 Shortest mileage from BWI PVD  BOS  JFK 184 ORD 621 LAX  SFO  DFW  BWI 0 MIA 946 337 2704 1846 1464 1235 2342 802 867 849 740 187 144 1391 184 1121 946 1090 1258621

29. 29 Shortest mileage from BWI PVD 328 BOS 371 JFK 184 ORD 621 LAX  SFO  DFW 1575 BWI 0 MIA 946 337 2704 1846 1464 1235 2342 802 867 849 740 187 144 1391 184 1121 946 1090 1258621

30. 30 Shortest mileage from BWI PVD 328 BOS 371 JFK 184 ORD 621 LAX  SFO  DFW 1575 BWI 0 MIA 946 337 2704 1846 1464 1235 2342 802 867 849 740 187 144 1391 184 1121 946 1090 1258621

31. 31 Shortest mileage from BWI PVD 328 BOS 371 JFK 184 ORD 621 LAX  SFO 3075 DFW 1575 BWI 0 MIA 946 337 2704 1846 1464 1235 2342 802 867 849 740 187 144 1391 184 1121 946 1090 1258621

32. 32 Shortest mileage from BWI PVD 328 BOS 371 JFK 184 ORD 621 LAX  SFO 2467 DFW 1423 BWI 0 MIA 946 337 2704 1846 1464 1235 2342 802 867 849 740 187 144 1391 184 1121 946 1090 1258621

33. 33 Shortest mileage from BWI PVD 328 BOS 371 JFK 184 ORD 621 LAX 3288 SFO 2467 DFW 1423 BWI 0 MIA 946 337 2704 1846 1464 1235 2342 802 867 849 740 187 144 1391 184 1121 946 1090 1258621

34. 34 Shortest mileage from BWI PVD 328 BOS 371 JFK 184 ORD 621 LAX 2658 SFO 2467 DFW 1423 BWI 0 MIA 946 337 2704 1846 1464 1235 2342 802 867 849 740 187 144 1391 184 1121 946 1090 1258621

35. 35 Shortest mileage from BWI PVD 328 BOS 371 JFK 184 ORD 621 LAX 2658 SFO 2467 DFW 1423 BWI 0 MIA 946 337 2704 1846 1464 1235 2342 802 867 849 740 187 144 1391 184 1121 946 1090 1258621

36. 36 Shortest mileage from BWI PVD 328 BOS 371 JFK 184 ORD 621 LAX 2658 SFO 2467 DFW 1423 BWI 0 MIA 946 337 2704 1846 1464 1235 2342 802 867 849 740 187 144 1391 184 1121 946 1090 1258621

37. 37 Shortest mileage from BWI PVD 328 BOS 371 JFK 184 ORD 621 LAX 2658 SFO 2467 DFW 1423 BWI 0 MIA 946 337 2704 1846 1464 1235 2342 802 867 849 740 187 144 1391 184 1121 946 1090 1258621

38. 38 Dijkstra’s algorithm, recap  While Q is not empty do:  u = Q.removeMin  for each node z one hop away from u do:  if D[u] + miles(u,z) < D[z] then D[z] = D[u] + miles(u,z) change key of z in Q to D[z]

39. 39 Quiz break  Would it be better to use an adjacency list or an adjacency matrix for Dijkstra’s algorithm?  What is the running time of Dijkstra’s algorithm, in terms of |V| and |E|?

40. 40 Complexity of Dijkstra  Adjacency matrix version Dijkstra finds shortest path from one vertex to all others in O(|V| 2 ) time  If |E| is small compared to |V| 2, use a priority queue to organize the vertices in V-S, where V is the set of all vertices and S is the set that has already been explored  So total of |E| updates each at a cost of O(log |V|)  So total time is O(|E| log|V|)

41. The All Pairs Shortest Path Algorithm (Floyd’s Algorithm)

42. 42 Finding all pairs shortest paths  Assume G=(V,E) is a graph such that c[v,w]  0, where C is the matrix of edge costs.  Find for each pair (v,w), the shortest path from v to w. That is, find the matrix of shortest paths  Certainly this is generalization of Dijkstra’s.  Note: For later discussions assume |V| = n and |E| = m

43. 43 Floyd’s Algorithm  A[i][j] = C(i,j) if there is an edge (i,j)  A[i][j] = infinity(inf) if there is no edge (i,j) Graph “adjacency” matrix A is the shortest path matrix that uses 1 or fewer edges

44. 44 Floyd ctd..  To find shortest paths that uses 2 or fewer edges find A 2, where multiplication defined as min of sums instead sum of products  That is (A 2 ) ij = min{ A ik + A kj | k =1..n}  This operation is O(n 3 )  Using A 2 you can find A 4 and then A 8 and so on  Therefore to find A n we need log n operations  Therefore this algorithm is O(log n* n 3 )  We will consider another algorithm next

45. 45 Floyd-Warshall Algorithm  This algorithm uses nxn matrix A to compute the lengths of the shortest paths using a dynamic programming technique.  Let A[i,j] = c[i,j] for all i,j & ij  If (i,j) is not an edge, set A[i,j]=infinity and A[i,i]=0  A k [i,j] = min (A k-1 [i,j], A k-1 [i,k]+ A k-1 [k,j])

46. 46 Example – Floyd’s Algorithm 1 23 8 2 3 5 Find the all pairs shortest path matrix  A k [i,j] = min (A k-1 [i,j], A k-1 [i,k]+ A k-1 [k,j])

47. 47 Floyd-Warshall Implementation  initialize A[i,j] = C[i,j]  initialize all A[i,i] = 0  for k from 1 to n for i from 1 to n for j from 1 to n if (A[i,j] > A[i,k]+A[k,j]) A[i,j] = A[i,k]+A[k,j];  The complexity of this algorithm is O(n 3 )

48. Negative Weighted Single-Source Shortest Path Algorithm (Bellman-Ford Algorithm)

49. 49 Bellman-Ford Algorithm Definition: An efficient algorithm to find the shortest paths from a single source vertex to all other vertices in a weighted, directed graph. Weights may be negative. The algorithm initializes the distance to the source vertex to 0 and all other vertices to. It then does V-1 passes (V is the number of vertices) over all edges relaxing, or updating, the distance to the destination of each edge. Finally it checks each edge again to detect negative weight cycles, in which case it returns false. The time complexity is O(VE), where E is the number of edges.shortest pathssource vertexweighted, directed graphedges cyclesO(VE) Source: NIST

50. 50 Bellman-Ford Shortest Paths  We want to compute the shortest path from the start to every node in the graph. As usual with DP, let's first just worry about finding the LENGTH of the shortest path. We can later worry about actually outputting the path.  To apply the DP approach, we need to break the problem down into sub-problems. We will do that by imagining that we want the shortest path out of all those that use i or fewer edges. We'll start with i=0, then use this to compute for i=1, 2, 3,....

51. 51 Example  For each edge (u,v), let's denote its length by C(u,v))  Let d[i][v] = distance from start to v using the shortest path out of all those that use i or fewer edges, or infinity if you can't get there with <= i edges.

52. 52 Example ctd..  How can we fill out the rows? 012345 00  1050  15  2050801545  V i

53. 53 Example ctd..  Can we get i th row from i-1 th row?  for v != start, d[v][i] = MIN d[x][i-1] + len(x,v) x->v  We know minimum path to come to x using < i nodes.So for all x that can reach v, find the minimum such sum (in blue) among all x  Assume d[start][i] = 0 for all i

54. 54 Completing the table 012345 00  1050  15  2050801545  30 40 50 d[v][i] = MIN d[x][i-1] + len(x,v) x->v

55. 55 Questions  Question: How large does i ever have to be? (Remember, there are no negative-weight cycles)  Question: what is the total running time in terms of |V| and |E|?

56. 56 Next Week  Work on Homework #5  We will talk more about graph traversals and topological sorting  We will discuss Minimum Spanning Trees (Prim’s Algorithm)  Read Chapter 14