1. Nondecreasing Paths in Weighted Graphs Or: How to optimally read a train schedule Virginia Vassilevska

2. Traveling? Tomorrow after 8amAs early as possible!

3. Nondecreasing Paths in Weighted Graphs Or: How to optimally read a train schedule Virginia Vassilevska flight

4. Routes with Multiple Stops London Frankfurt 7pm – 1:20pm 5:30pm – 10:40am 7:45pm – 8:30pm 11:40am – 4:15pm New York Newark 11:35am – 1pm 5:30pm – 7pm 9:25pm – 9:05am 10:30am – 6pm

5. Scheduling You might need to make several connections. There are multiple possible stopover points, and multiple possible schedules. How do you choose which segments to combine?

6. Talk Overview Graph-theoretic abstraction History Two algorithms Improved algorithm, linear time

7. 4:15*pm 11:40*am 2:20*pm 10:05*am 8pm 2pm A vertex for each flight;A vertex for each city;(Origin, flight) edges;(flight, Destination) edges;departure time weight;arrival time weight; Graph-Theoretic Abstraction LondonFrankfurt 7pm 5:30pm 7:45*pm 11:40*am New YorkNewark 11:35am 5:30pm 9:25pm 10:30*am 1pm 7pm 1:20*pm 9:05*am 10:40*am 6*pm 8:30*pm … … … … Graph:Nondecreasing path with minimum last edge?

8. Versions of the problem We’ll focus on single source SSNP. Single source – single destination Single source (every destination) All pairs ST

9. History G. Minty 1958: graph abstraction and first algorithm for SSNP E. F. Moore 1959: a new algorithm for shortest paths, and SSNP – polytime – cubic time

10. History Dijkstra 1959 Fredman and Tarjan 1987 – Fibonacci Heaps implementation of Dijkstra’s; until now asymptotically fastest algorithm for SSNP. Nowadays – experimental research on improving Dijkstra’s algorithm implementation O(m+n log n) m – number of edges n – number of vertices

11. Fibonacci Heaps Fredman and Tarjan’s Fibonacci heaps: Maintain a set of elements with weights d[·] Insert element v with d[v] = ∞ in constant time Update the weight d[v] of an element v in constant time Return and remove the element v of minimum weight d[v] in O(log N) time where N is the number of elements

12. Dijkstra’s Algorithm for SSNP Maintain conservative “distance” estimates for all vertices; d[S] = - , d[v] =  for all other v U contains vertices with undetermined distances. Use Fibonacci Heaps! Vertices outside of U are completed.

13. Dijkstra’s algorithm Set U = V and T = { }. At each iteration, pick u from U minimizing d[u]. T = T U {u}, U = U \ {u}. For all edges (u, v),  If w(u,v) ≥ d[u], set d[v] = min (d[v], w(u,v)) u T U min d[u] d[u] v w(u,v) S Iterate:

14. Example S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 U - Fibonacci Heap: S: -infinity P: infinity Q: infinity a: infinity b: infinity c: infinity d: infinity Other Distances: 5

15. Example S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 U - Fibonacci Heap: P: infinity Q: infinity a: 5 b: 1 c: 3 d: infinity Other Distances: S: -infinity 5 S – extract min from U

16. Example S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 U - Fibonacci Heap: P: 2 Q: infinity a: 5 c: 3 d: infinity Other Distances: S: -infinity b: 1 5 b – extract min from U

17. Example S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 U - Fibonacci Heap: Q: 3 a: 3 c: 3 d: 2 Other Distances: S: -infinity b: 1 P: 2 5 P – extract min from U

18. Example S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 U - Fibonacci Heap: Q: 3 a: 2 c: 3 Other Distances: S: -infinity b: 1 P: 2 d: 2 5 d – extract min from U

19. Example S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 U - Fibonacci Heap: Q: 3 c: 2 Other Distances: S: -infinity b: 1 P: 2 d: 2 a: 2 5 a – extract min from U

20. Example S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 U - Fibonacci Heap: Q: 3 Other Distances: S: -infinity b: 1 P: 2 d: 2 a: 2 c: 2 5 c – extract min from U

21. Example S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 U - Fibonacci Heap: Other Distances: S: -infinity b: 1 P: 2 d: 2 a: 2 c: 2 Q: 3 5 Q – extract min from U

22. Running time of Dijkstra Inserting all n nodes in the Fibonacci heap takes O(n) time. Each d[v] update is due to some edge (u, v), and each edge is touched at most once. d[v] updates take O(m) time overall.

23. Running time of Dijkstra Every node removed from Fibonacci heap at most once – at most n extract- mins. This takes O(n log n) time overall. Final runtime: O(m+n log n). Optimal for Dijkstra’s algorithm – nodes visited in sorted order of their distance. bottleneck

24. More on Dijkstra’s Suppose we only maintain F vertices in the Fibonacci heaps. The rest we maintain in some other way. Then the runtime due to the Fibonacci heaps would be O(F log F + N(F)) where N(F) is the number of edges pointing to the F vertices. For F = m/log n, this is O(m)! F N(F)

25. ALG2: Depth First Search DFS(v, d[v]): For all (v, u) with w(v, u) ≥ d[v]: Remove (v, u) from graph. d[u] = min (d[u], w(v,u)) DFS(u, d[u]) d[S] = - ∞, start with DFS(S, d[S]). v d[v]=2 3 3 1 u d[u]=4d[u]=3

26. Example S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 Distances: S: -infinity P: infinity Q: infinity a: infinity b: infinity c: infinity d: infinity 5

27. Example S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 Distances: S: -infinity P: infinity Q: infinity a: infinity b: 1 c: infinity d: infinity 5 DFS(S, -infinity)

28. Example S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 Distances: S: -infinity P: 2 Q: infinity a: infinity b: 1 c: infinity d: infinity 5 DFS(b, 1)

29. Example S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 Distances: S: -infinity P: 2 Q: infinity a: infinity b: 1 c: infinity d: 2 5 DFS(P, 2)

30. Example S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 Distances: S: -infinity P: 2 Q: infinity a: 2 b: 1 c: infinity d: 2 5 DFS(d, 2)

31. Example S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 Distances: S: -infinity P: 2 Q: infinity a: 2 b: 1 c: 2 d: 2 5 DFS(a, 2)

32. Example S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 Distances: S: -infinity P: 2 Q: infinity a: 2 b: 1 c: 2 d: 2 5 DFS(c, 2)

33. Example S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 Distances: S: -infinity P: 2 Q: 3 a: 2 b: 1 c: 2 d: 2 5 DFS(P, 2)

34. Example S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 Distances: S: -infinity P: 2 Q: 3 a: 2 b: 1 c: 2 d: 2 5 DFS(Q, 3)

35. Example S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 Distances: S: -infinity P: 2 Q: 3 a: 2 b: 1 c: 2 d: 2 5 DFS(Q, 3)

36. Example S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 Distances: S: -infinity P: 2 Q: 3 a: 2 b: 1 c: 2 d: 2 5 DFS(Q, 3)

37. Example S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 Distances: S: -infinity P: 2 Q: 3 a: 2 b: 1 c: 2 d: 2 5 DFS(P, 2)

38. Example S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 Distances: S: -infinity P: 2 Q: 3 a: 2 b: 1 c: 2 d: 2 5 DFS(S, -infinity)

39. Example S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 Distances: S: -infinity P: 2 Q: 3 a: 2 b: 1 c: 2 d: 2 5 DFS(S, -infinity)

40. Example S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 Distances: S: -infinity P: 2 Q: 3 a: 2 b: 1 c: 2 d: 2 5 DFS(S, -infinity)

41. Runtime of DFS The number of times we call DFS(v, d[v]) for any particular v is at most indegree(v). Every such time we might have to check all outedges (w(v,u)≥ ? d[v]). Worst case running time: Σ v (indegree(v) × outdegree(v)) ≤ n Σ v indegree(v) = O(mn). Σ v indegree(v)=m

42. More on DFS The runtime can be improved! Suppose for a node v and weight d[v] we can access each edge (v, u) with w(v, u)≥ d[v] in O(t) time. As each edge is accessed at most once, the runtime is O(m t).

43. DFS with Binary Search Trees For each vertex v, insert outedges in binary search tree sorted by weight. Splay trees, treaps, AVL trees etc. support the following on totally ordered sets of size k in O(log k) time:  insert,  delete,  find, predecessor, successor 1 2 3 5 6

44. DFS with Binary Search Trees Given any weight d[v], one can find an edge (v, u) with w(v, u) ≥ d[v] in O(log [deg(v)]) time. All inserts in the beginning take O(Σ v deg(v) log [deg(v)]) = O(m log n) time, and DFS takes O(m log n) time. Σ v deg(v)=2m

45. Combining Dijkstra with DFS Recall:  If F nodes used in Fibonacci heaps, then the runtime due to the heaps is O(F log F + N(F))  If DFS with binary search trees is run on a set of nodes T, the runtime is O(Σ v  T { deg(v) log (deg(v)) }) O(m log log n) for T = {v | deg(v)<log n} ◄ O(m+n) for F = m/log n

46. Idea! Run DFS on vertices of low degree < log n: O(m log log n) time. Put the rest in Fibonacci heaps and run Dijkstra on them. There are at most O(m/log n) high degree nodes. Time due to Fibonacci heaps: O(m). We get O(m log log n). Better than O(m+n log n) for m = o(n log n/log log n).

47. But we wanted linear time… Fredman and Willard atomic heaps: After O(n) preprocessing, a collection of O(n) sets of O(log n) size can be maintained so that the following are in constant time:  Insert  Delete  Given w, return an element of weight ≥ w. outedges of low degree vertices

48. Linear runtime Replace binary trees by atomic heaps. Time due to Dijkstra with Fibonacci Heaps on O(m/log n) elements is still O(m). Time due to DFS with atomic heaps:  inserting outedges into atomic heaps takes constant time per edge;  given d[v], accessing an edge with w(v,u) ≥ d[v] takes constant time. O(m+n) time overall! But how do we combine Dijkstra and DFS?

49. New Algorithm Stage 1: Initialize  Find all vertices v of degree ≥ log n and insert into Fibonacci Heaps with d[v] = ∞;  For all vertices u of degree < log n, add outedges into atomic heap sorted by weights.  This stage takes O(m+n) time. Insert S with d[S] = - ∞

50. 22 New Algorithm Stage 2: Repeat: 1. Extract vertex v from Fibonacci heaps with minimum d[v] 2. For all neighbors u of v, if w(u,v) ≥ d[v]: 1. Update d[u] if w(v,u) < d[u] 2. Run DFS(u, d[u]) on the graph spanned by low degree vertices until no more can be reached 3. If Fib.heaps nonempty, go to 1. 3 Fibonacci Heaps 4 5 1 4 4 3

51. Example S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 U - Fibonacci Heap: S: -infinity P: infinity Q: infinity Other Distances: a: infinity b: infinity c: infinity d: infinity 5

52. S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 U - Fibonacci Heap: P: infinity Q: infinity Other Distances: a: 5 b: 1 c: 3 d: infinity S: -infinity S – extract min from U 5 Example

53. S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 U - Fibonacci Heap: P: 2 Q: infinity Other Distances: a: 5 b: 1 c: 3 d: infinity S: -infinity DFS(b, 1) 5 Example

54. DFS(a, 5) S P Q a b c d 1 2 3 4 5 2 2 2 3 3 3 3 U - Fibonacci Heap: P: 2 Q: infinity Other Distances: a: 5 b: 1 c: 3 d: infinity S: -infinity DFS(c, 3) 3 5 Example

55. 2 3 P – extract min from U S P Q a b c d 1 2 3 4 5 2 2 3 3 3 U - Fibonacci Heap: Q: 3 Other Distances: a: 3 b: 1 c: 3 d: 2 S: -infinity P: 2 5 Example

56. 2 3 S P Q a b c d 1 2 3 4 5 2 2 3 3 3 U - Fibonacci Heap: Q: 3 Other Distances: a: 2 b: 1 c: 3 d: 2 S: -infinity P: 2 DFS(d, 2) 5 Example

57. 2 3 S P Q a b c d 1 2 3 4 5 2 2 3 3 3 U - Fibonacci Heap: Q: 3 Other Distances: a: 2 b: 1 c: 2 d: 2 S: -infinity P: 2 DFS(a, 2) 5 Example

58. 2 3 S P Q a b c d 1 2 3 4 5 2 2 3 3 3 U - Fibonacci Heap: Q: 3 Other Distances: a: 2 b: 1 c: 2 d: 2 S: -infinity P: 2 DFS(c, 2) 5 DFS(a, 3) Example

59. 2 3 S P Q a b c d 1 2 3 4 5 2 2 3 3 3 U - Fibonacci Heap: Other Distances: a: 2 b: 1 c: 2 d: 2 S: -infinity P: 2 Q: 3 5 Q – extract min from U Example

60. Summary We gave the first linear time algorithm for the single source nondecreasing paths problem. We did this by combining two known algorithms, and by using clever data structures. Now you can read a train schedule optimally!

61. Directions for future work What about the all-pairs version of the problem? That is, if we want to schedule the best trips between any two cities? Naïve algorithm: apply linear time algorithm for all possible sources – O(mn) time, O(n 3 ) in the worst case. Can we do anything better? O(n 2.9 )?

62. Conclusion With the aid of the right data structures simple algorithms can be fast. THANK YOU!