1. Contest Algorithms January 2016 Describe shortest path trees, SSSP, APSP, and three algorithms: Dijkstra, Bellman-Ford, Floyd-Warshall 9. Shortest Paths 1Contest Algorithms: 9. Shortest Paths

2.  The length of a path = the sum of the weights of the edges in the path.  The shortest path between two verticies = the path having the minimum length. 1. Shortest Paths 2 ORD PVD MIA DFW SFO LAX LGA HNL 849 802 1387 1743 1843 1099 1120 1233 337 2555 142 1205

3.  A shortest path tree spans all the nodes in the graph, using edges that give the shortest paths from the starting node to all the other vertices. Shortest Path Tree (SPT) SPT start node is 0

4.  SSSP: single source shortest path  find the shortest path from a given vertex to all the other nodes  for undirected, unweighted graphs use BFS  for directed, weighted graph use:  Dijkstra's algoritm, Bellman-Ford  APSP: all pairs shortest path  find the shortest path between all pairs of nodes in the graphs  popular contest algorithm  use Floyd-Warshall for small graphs Types of Shortest Path Problems Contest Algorithms: 8 MSTs4

5.  Build a shortest path tree (SPT) from a source node  No negative edge weights allowed.  Similar to Prim's algorithm (Greedy)  Grow a tree gradually, advancing from vertices taken from a priority queue of "distances"  Instead of a MST, we are building a SPT 2. Dijkstra’s Algorithm O(|E| * log |V|)

6.  Each vertex has a entry in a distTo[] array.  Initialize dist[start] to 0 and all other distTo[] entries to positive infinity (  )  During the execution, the distTo[] values will get smaller. This is called (edge) relaxing.  At each iteration, the SPT is grown by adding a vertex with the smallest distTo[] value. Algorithm in Words

7. What is Relaxing? a b c d distTo[a] = 3 distTo[b] = 13 distTo[c] = 13 distTo[d] = 10 5 4 8 relax 'edges' linked to a a is in the SPT a b c d distTo[a] = 3 distTo[b] = 8 distTo[c] = 7 distTo[d] = 10 5 4 8 c will be the next vertex to join SPT if (distTo[b] > distTo[a] + weight()) distTo[b] = distTo[a] + weight();

8. An Example (directed, weighted) 0 1 2 7 6 5 34 2 2 14 3 2 4 73 5 6 1 continued Start Vertex: 0

9. Initialisation 0 1 2 7 6 5 34 2 2 14 3 2 4 73 5 6 1 continued Inf = changing distTo 0

10. First Iteration 0 1 2 7 6 5 34 2 2 14 3 2 4 73 5 6 1 continued 2 Inf 1 0 = final distTo

11. Second Iteration 0 1 2 7 6 5 34 2 2 14 3 2 4 73 5 6 1 continued 2 Inf 4 6 0 1

12. Third Iteration 0 1 2 7 6 5 34 2 2 14 3 2 4 73 5 6 1 4 Inf 4 6 6 0 1 2 continued

13. Fourth Iteration 0 1 2 7 6 5 34 2 2 14 3 2 4 73 5 6 1 5 6 6 0 1 24 4 Could have chosen ‘d’ instead. continued

14. Fifth Iteration 0 1 2 7 6 5 34 2 2 14 3 2 4 73 5 6 1 5 6 6 0 1 24 4 Choosing ‘3’ is a waste of time. continued

15. Sixth Iteration 0 1 2 7 6 5 34 2 2 14 3 2 4 73 5 6 1 6 6 0 1 2 4 4 5 Shortest path from node ‘0’ to ‘node 7’ is length 5.

16. 0 1 2 7 6 5 34 2 2 14 3 2 4 73 5 6 1 0 1 2 4 4 5 Final Result 6 6

17.  no. of vertices, no. of edges, start vertex  many lines, each containing one triple (u v, weight)  e.g. 8 12 0 0 1 2 1 2 2 2 7 1 0 5 1 5 6 5 6 7 6 1 3 2 1 4 4 2 4 3 3 4 4 3 5 3 4 6 7 Data Input Format Contest Algorithms17

18.  Adjacency list for a directed, weighted graph.  A list of distances from the starting vertex to the other vertices; this data structure is computed by the algorithm.  A priority queue of (distances, vertex) pairs sorted based on increasing distances  i.e. smallest distance is first Dijkstra Data Structures Contest Algorithms: 8 MSTs18

19. public static void main(String[] args) throws Exception { if (args.length != 1) { System.out.println("Usage: java Dijkstra "); return; } Scanner sc = new Scanner(new File(args[0])); int numVs = sc.nextInt(); int numEs = sc.nextInt(); int vStart = sc.nextInt(); // adj list contains lists of (vertex, weight) pairs ArrayList > adjList = new ArrayList<>(); for (int i = 0; i < numVs; i++) { ArrayList neighbors = new ArrayList (); adjList.add(neighbors); // add neighbor list to Adjacency List } for (int i = 0; i < numEs; i++) { int u = sc.nextInt(); int v = sc.nextInt(); int weight = sc.nextInt(); adjList.get(u).add(new IPair(v, weight)); //add (v,weight) to list for u } : Code Contest Algorithms: 8 MSTs19 see Dijkstra.java

20. // list of distances from vStart to the other vertices ArrayList distTo = new ArrayList<>(); distTo.addAll(Collections.nCopies(numVs, INF)); distTo.set(vStart, 0); // pri-queue of (distance, vertex) pairs PriorityQueue pq = new PriorityQueue (numVs); // sort based on increasing distances pq.offer(new IPair(0, vStart)); while (!pq.isEmpty()) { IPair nearest = pq.poll(); // pick shortest unvisited vertex int d = nearest.getX(); int u = nearest.getY(); if (d <= distTo.get(u)) // process vertex u only once relax(u, adjList, distTo, pq); } for (int i = 0; i < numVs; i++) System.out.println(" " + vStart + " --> " + i + " = " + distTo.get(i)); } // end of main() Contest Algorithms: 8 MSTs20

21. private static void relax(int u, ArrayList > adjList, ArrayList distTo, PriorityQueue pq) // examine all neighbors of u, trying to relax their distTo values { for (IPair neighbor : adjList.get(u)) { int v = neighbor.getX(); // for u-v edge int vWeight = neighbor.getY(); int uvDist = distTo.get(u) + vWeight; if (distTo.get(v) > uvDist) { // possibly relax distTo.set(v, uvDist); pq.offer(new IPair(uvDist, v)); } } // end of relax() Contest Algorithms: 8 MSTs21

22. Example (again) Contest Algorithms: 8 MSTs22 8 12 0 0 1 2 1 2 2 2 7 1 0 5 1 5 6 5 6 7 6 1 3 2 1 4 4 2 4 3 3 4 4 3 5 3 4 6 7 spData3.txt

23. Execution Contest Algorithms: 8 MSTs23

24. Example 2 (fig 4.17, CP) Contest Algorithms: 8 MSTs24 2 0 4 3 1 3 7 5 6 2 6 1 Start Vertex: 2 5 7 2 2 1 2 2 3 7 2 0 6 1 3 3 1 4 6 3 4 5 0 4 1 spData1.txt

25. Execution Contest Algorithms: 8 MSTs25 6 6 7 7 5 5 2 2

26.  Bellman-Ford algorithm can be used on graphs with negative edge weights  as long as the graph contains no negative cycles reachable from the source  Slower than Dijkstra's algorithm for the same problem  O(|V| * |E|) 3. Bellman-Ford Algorithm

27. Example This example can only be processed by Bellman-Ford since there are negative weights. There are no negative cycles. 0 1 3 2 4 6 5 -2 -3 7 -4 8 9 2 7

28.  Bellman-Ford relaxes all the edges |V | − 1 times.  The biggest path in the SPT is at most |V|-1 edges long. So |V|-1 loops is certain to process it Algorithm in Words

29. Example Execution 0 1 3 2 4 6 5 -2 -3 7 -4 8 9 2 7 0123401234 distTo[] 00 edges ---------- 0 1 3 2 4 6 5 -2 -3 7 -4 8 9 2 7 0123401234 distTo[] 067067 edges - 0->1 - 0->3 - The source node is 0 after each relax for-loop iteration

30. 0 1 3 2 4 6 5 -2 -3 7 -4 8 9 2 7 0123401234 distTo[] 0647206472 edges - 0->1 3->2 0->3 1->4 0 1 3 2 4 6 5 -2 -3 7 -4 8 9 2 7 0123401234 0247202472 - 2->3 3->2 0->3 1->4 0 1 3 2 4 6 -2 -3 7 -4 8 9 2 7 0123401234 0 2 4 7 -2 - 2->3 3->2 0->3 1->4 Finished

31.  The same as for Dijkstra:  no. of vertices, no. of edges, start vertex  many lines, each containing one triple (u v, weight) Input Data Format Contest Algorithms: 8 MSTs31

32.  Adjacency list for a directed, weighted graph.  A list of distances from the starting vertex to the other vertices; this data structure is computed by the algorithm.  Unlike Dijkstra, there is no need for a priority queue of (distances, vertex) pairs  this makes the code a little simpler Bellman-Ford Data Structures Contest Algorithms: 8 MSTs32

33. public static void main(String[] args) throws Exception { if (args.length != 1) { System.out.println("Usage: java BellmanFord "); return; } Scanner sc = new Scanner(new File(args[0])); int numVs = sc.nextInt(); int numEs = sc.nextInt(); int vStart = sc.nextInt(); // adj list contains lists of (vertex, weight) pairs ArrayList > adjList = new ArrayList<>(); for (int i = 0; i < numVs; i++) { ArrayList neighbor = new ArrayList<>(); adjList.add(neighbor); // add neighbor list to Adjacency List } for (int i = 0; i < numEs; i++) { int u = sc.nextInt(); int v = sc.nextInt(); int weight = sc.nextInt(); adjList.get(u).add(new IPair(v, weight)); //add(v, weight) to list for u } : Code Contest Algorithms: 8 MSTs33 see BellmanFord.java

34. // list of distances to vertices from vStart ArrayList distTo = new ArrayList (); distTo.addAll(Collections.nCopies(numVs, INF)); distTo.set(vStart, 0); for (int i = 0; i < numVs-1; i++) // relax all edges numVs-1 times, O(numVs) for (int u = 0; u < numVs; u++) relax(u, adjList, distTo); // negative cycle detection- boolean hasNegCycle = false; for (int u = 0; u < numVs; u++) // one more pass to check for (IPair v : adjList.get(u)) { // relax these edges if (distTo.get(v.getX()) > (distTo.get(u)+v.getY())) // should be false hasNegCycle = true; } System.out.println("Negative Cycle Exist? " + hasNegCycle); if (!hasNegCycle) // print distances from start vertex for (int i = 0; i < numVs; i++) System.out.println(" " + vStart + " --> " + i + " = " + distTo.get(i)); } // end of main() Contest Algorithms: 8 MSTs34

35. private static void relax(int u, ArrayList > adjList, ArrayList distTo) // examine all neighbors of u, trying to relax their distTo values // same as Dijsktra's relax(), but without the pri-queue { for (IPair neighbor : adjList.get(u)) { // relax these edges int v = neighbor.getX(); // for u-v edge int vWeight = neighbor.getY(); int uvDist = distTo.get(u) + vWeight; if (distTo.get(v) > uvDist) // possibly relax distTo.set(v, uvDist); } } // end of relax() Contest Algorithms: 8 MSTs35

36. DirectedEdge[] edgeTo; double[] distTo; void bellmanFord(Digraph graph, int start) { for (int v : graph.allVerts()) distTo[v] = Double.POSITIVE_INFINITY; distTo[start] = 0; for (int i=1; i < graph.numV(); i++) relax(graph); for (DirectedEdge e : graph.allEdges()) { int u = e.from(); // edge e is u --> v int v = e.to(); if (distTo[v] > distTo[u] + e.weight()) println("There's no solution"); } Code Initialize distTo[], which will reduce to shortest-path value Relaxation: Make |V|-1 passes, relaxing every edge Test for solution When do we not get a solution?

37. void relax(Digraph graph) // a simpler version than Dijkstra { for (DirectedEdge e : graph.allEdges()) { int u = e.from(); // edge e is u --> v int v = e.to(); if (distTo[v] > distTo[u] + e.weight()) { distTo[v] = distTo[u] + e.weight(); edgeTo[v] = e; }

38. Example Again Contest Algorithms: 8 MSTs38 5 10 0 0 1 6 0 3 7 1 3 8 1 4 -4 1 2 5 2 1 -2 3 2 -3 3 4 9 4 0 2 4 2 7 spData4.txt Start Vertex: 0

39. Execution Contest Algorithms: 8 MSTs39 Start Vertex: 0 2 2 4 4 7 7 -2

40.  Contains a negative cycle. Example 2 (fig 4.19, cp) Contest Algorithms: 8 MSTs40 1 0 1000 Start Vertex: 0 3 3 0 0 1 1000 1 2 15 2 1 -42 spData5.txt 2 15 -42

41. 4. Floyd-Warshall Algorithm  Used for APSP problems.  If you can get from A to B at cost c 1, and you can get from B to C with cost c 2, then you can get from A to C with at most cost c 1 +c 2  As the algorithm proceeds, if it finds a lower cost for A to C then it uses that instead.  Running time: O(|V| 3 ) A B C c1c1 c2c2 at most c 1 + c 2

42. Example 0 1 4 3 2 8 13 1 6 12 9 7 0 11 0813-1 -0-612 -90-- 7-00- ---110 0 1 2 3 4 0 12 34 from to D 0 = (d ij ) 0 d ij = shortest distance from i to j through nodes {0, …, k} k d ij = shortest distance from i to j through no nodes 0

43. D 0 = (d ij ) 0 0813-1 -0-612 -90-- 7-00- ---110 d ij = shortest distance from i to j through {0, …, k} k Before 0 1 2 3 4 0 12 34 D 1 = (d ij ) 1 0813-1 -0-612 -90-- 7 15 00 8 ---110 After 0 1 2 3 4 0 12 34

44. D 0 = (d ij ) 0 0813-1 -0-612 -90-- 7-00- ---110 0 1 2 3 4 D 1 = (d ij ) 1 d ij = shortest distance from i to j through {0, …, k} k 0813-1 -0-612 -90-- 7 15 00 8 ---110 Before After 0 12 34 0 1 2 3 4 0 12 34 + ) d ij = k d ij k-1, d ik k-1 d kj k-1 min(

45. 0813 14 1 -0-612 -901521 715008 ---110 0 1 2 3 4 D 2 = (d ij ) 2 0813141 -0-612 -901521 7 9 008 ---110 D 3 = (d ij ) 3 0813141 130 6 612 22901521 79008 182011 0 D 4 = (d ij ) 4 D 5 = (d ij ) 5 to store the path, another matrix can track the last intermediate vertex 0812 1 1306612 22901521 79008 182011 0 0 1 2 3 4 0 1 2 3 4 0 1 2 3 4

46.  The input format is the same as for Dijkstra's and Bellman- Ford's algorithms:  no. of vertices, no. of edges, start vertex  each line is a triple (u v, weight)  Since Floyd-Warshall is for APSP, there's no need for a start vertex, but I've included it so that the same data files can be used by all three algorithms  the Floyd-Warshall code ignores the start vertex input Data Input Format Contest Algorithms46

47.  An adjacency matrix for a directed, weighted graph.  There's no need for a priority queue (as in Dijkstra) or a distances list (as in Dijkstra and Bellman-Ford).  A matrix is a bit easier to implement than an adj. list. Floyd-Warshall Data Structures Contest Algorithms: 8 MSTs47

48. public static void main(String[] args) throws Exception { if (args.length != 1) { System.out.println("Usage: java FloydWarshall "); return; } Scanner sc = new Scanner(new File(args[0])); int numVs = sc.nextInt(); int numEs = sc.nextInt(); sc.nextInt(); // to skip start vertex int[][] adjMat = new int[numVs][]; for (int i = 0; i < numVs; i++) { adjMat[i] = new int[numVs]; for (int j = 0; j < numVs; j++) adjMat[i][j] = INF; adjMat[i][i] = 0; } for (int i = 0; i < numEs; i++) { int u = sc.nextInt(); int v = sc.nextInt(); int weight = sc.nextInt(); adjMat[u][v] = weight; } : Code Contest Algorithms: 8 MSTs48 see FloydWarshall.java

49. for (int k = 0; k < numVs; k++) for (int i = 0; i < numVs; i++) for (int j = 0; j < numVs; j++) if (adjMat[i][j] > (adjMat[i][k] + adjMat[k][j])) adjMat[i][j] = adjMat[i][k] + adjMat[k][j]; System.out.println("All-points Shortest Paths"); for (int i = 0; i < numVs; i++) System.out.println(" " + i + " --> " + Arrays.toString(adjMat[i])); } // end of main() Contest Algorithms: 8 MSTs49 short and simple!

50. Example Again Contest Algorithms: 8 MSTs50 5 9 0 0 1 8 0 2 13 0 4 1 1 3 6 1 4 12 2 1 9 3 0 7 3 2 0 4 3 11 spData6.txt not used by the code

51. Execution Contest Algorithms: 8 MSTs51

52.  Floyd-Warshall running time is best, O(|V| 3 ):  Dijkstra running time: |V| * O(|E| log |V|) = O(|V| 3 log|V|)  Bellman-Ford running time: |V| * O(|V|*|E|) = O(|V| 4 )  Easy to implement, but memory will become a problem for large matricies (|V| > 400)  For larger problems, use Dijkstra or Bellman-Ford by calling the algorithm repeatedly with every vertex as the start node Comparison with Dijkstra & Bellman-Ford Contest Algorithms52