1. Engineering shortest path algorithms (a round trip between theory and experiments) Camil Demetrescu University of Rome “La Sapienza” Giuseppe F. Italiano University of Rome “Tor Vergata”

2. Outline of the Talk Conclusions Back from Experiments to Theory The Algorithm Engineering Cycle From Theory to Experiments 1 2 3 4

3. Why Engineering Algorithms? Theory In theory, theory and practice are the same.

4. Why Engineering Algorithms? The real world out there... In practice, theory and practice may be quite different…

5. Programs are first class citizens as well int, float, double Number types: N, R Only asymptotics matter Seconds do matter Abstract algorithm description Non-trivial implementation decisions, error-prone Unbounded memory, unit access cost Memory hierarchy / bandwidth Elementary operations take constant time Instruction pipelining, … TheoryPractice

6. The Algorithm Engineering Cycle Algorithm implementation Experimental analysisTheoretical analysis Bottlenecks, Heuristics Deeper insights More realistic models Hints to refine analysis Algorithm design

7. Fully Dynamic all-pairs shortest paths Given a weighted directed graph G=(V,E,w), perform any intermixed sequence of the following operations: return distance from x to y (or shortest path from x to y) Query(x,y): update cost of edge (u,v) to w Update(u,v,w): update edges incident to v [w( )] Update(v,w):

8. State of the Art First fully dynamic algorithms date back to the 60’s P. Loubal, A network evaluation procedure, Highway Research Record 205, 96-109, 1967. J. Murchland, The effect of increasing or decreasing the length of a single arc on all shortest distances in a graph, TR LBS-TNT-26, Transport Network Theory Unit, London Business School, 1967. V. Rodionov, A dynamization of the all-pairs least cost problem, USSR Comput. Math. And Math. Phys. 8, 233-277, 1968. …

9. State of the Art 65-6970-7475-7980-8485-8990-9495-9900- # papers

10. State of the Art Until 1999, none of them was better in the worst case than recomputing APSP from scratch (~ cubic time!) generalreal? < o(n 3 )O(1) QueryUpdateGraphWeight Ramaling.&Reps 96generalreal O(n 3 )O(1) King 99general[0,C]O(n 2.5 (C log n) 0.5 )O(1)D. & Italiano 01generalS realO(n 2.5 (S log 3 n) 0.5 )O(1)

11. Fully Dynamic APSP Edge insertions (edge cost decreases) Quite easy: O(n 2 ) 10 For each pair x,y check whether d(x,i) + w(i,j) + d(j,y) < d(x,y) xy i j

12. Fully Dynamic APSP Edge deletions (edge cost increases) Seem the hard operations When edge (shortest path) deleted: need info about second shortest path? (3rd, 4th, …) Now we take a little break from this problem… Edge insertions (edge cost decreases) Quite easy: O(n 2 )

13. Engineering Dijkstra’s AP Shortest Path y’ The algorithm: Run Dijkstra from all vertices “in parallel” Possibly insert (x,y’) into heap or decrease priority3. 2. Scan only y’ for which (a,y’) shortest (suboptimality) Can we do better? Extract shortest pair (x,y) from heap: xy 1. a Possibly insert (x,y’) into heap or decrease its priority3. y’ 2. Scan all neighbors y’ of y Edge scanning bottleneck for dense graphs [Goldberg] Extract shortest pair (x,y) from heap: xy 1.

14. Definition: Locally Shortest Paths [DI’03] A path is locally shortest if all of its proper subpaths are shortest paths Shortest path xy π xy Locally shortest path

15. Locally shortest paths Locally Shortest Paths Shortest paths By optimal-substructure property of shortest paths:

16. How much do we gain? Running time on directed graphs with real edge weights O(n 2 ) spaceO( #LS-paths + n 2 log n) time Q.: How many locally shortest paths ? Q.: How much can we gain in practice? A.: #LS-paths ≤ mn. No gain in asymptopia…

17. How many LS paths in a graph?

18. What about real-world graphs?

19. US road networks

20. Can we exploit this in practice?

21. Theoretical analysis Bottlenecks, Heuristics What we have seen so far Algorithm implementation Deeper insights More realistic models Hints to refine analysis Algorithm design Experimental analysis

22. Bottlenecks, Heuristics Return trip to Theory Algorithm implementation Theoretical analysis Deeper insights More realistic models Hints to refine analysis Algorithm design Experimental analysis

23. Back to Fully Dynamic APSP Given a weighted directed graph G=(V,E,w), perform any intermixed sequence of the following operations: return distance from x to y (or shortest path from x to y) Query(x,y): update cost of edge (u,v) to w Update(u,v,w):

24. Recall Fully Dynamic APSP Hard operations seem edge deletions (edge cost increases) When edge (shortest path) deleted: need info about second shortest path? (3rd, 4th, …) Shortest path xy π xy Hey… what about locally shortest paths? Candidate for being shortest path? Locally shortest path

25. Locally shortest paths for dynamic APSP Idea: Maintain all the locally shortest paths of the graph How do locally shortest paths change in a dynamic graph?

26. Assumptions behind the analysis Locally shortest paths π xy are internally vertex-disjoint xy π1π1 π3π3 π2π2 No two paths share the same weight.

27. Tie breaking Shortest paths are unique Assumptions In theory, tie breaking is not a problem Practice In practice, tie breaking can be subtle

28. Appearing Locally Shortest Paths Fact 1 At most mn (n 3 ) paths can START being locally shortest after an increase xy 10 20 30 40 xy 100 10 20 30 40

29. Disappearing Locally Shortest Paths Fact 2 At most n 2 paths can STOP being locally shortest after an increase if π stops being locally shortest after increase of e subpath of π (was shortest path) must contain e shortest paths are unique: at most n 2 contain e

30. Maintaining Locally Shortest Paths # Locally shortest paths appearing after increase: < n 3 # Locally shortest paths disappearing after increase: < n 2 The amortized number of changes in the set of locally shortest paths at each update in an increase-only sequence is O(n 2 )

31. An Increase-only update Algorithm This gives (almost) immediately: O(n 2 log n) amortized time per increase O(mn) space

32. Maintaining Locally Shortest Paths xy 10 20 30 40 xy 100 10 20 30 40 What about fully dynamic sequences?

33. xy How to pay only once? xy xy This path remains the same while flipping between being LS and non-LS: Would like to have update algorithm that pays only once for it until it is further updated...

34. xy Looking at the substructure xy …but if we removed the same edge it would be a shortest path again! It is not dead! This path remains a shortest path after the insertion This path is no longer a shortest path after the insertion…

35. Historical Paths xy A path is historical if it was shortest at some time since it was last updated historical path

36. Locally Historical Paths Shortest path x y π xy Locally shortest path Historical path x y π xy Locally historical path

37. Key idea for Partially Dynamic LSP SP

38. Key idea for Fully Dynamic LHP HP

39. Key Idea for Fully Dynamic LHP SP HP

40. Putting things into perspective: LHP HP LSP SP

41. The fully dynamic update algorithm O(n 2 log 3 n) amortized time per update Fully dynamic update algorithm very similar to partially dynamic, but maintains locally historical paths instead of locally shortest paths (+ performs some other operations) O(mn log n) space Idea: Maintain all the locally historical paths of the graph

42. Experiments (update time)

43. What about real graphs? (update time)

44. Relative time performance D-RLL/D-LHP Number of edges (x 100) D-LHP slower than D-RRL 12345678910 D-LHP faster than D-RRL Experiments on US road networks IBM Power 4 Sun UltraSPARC IIi AMD Athlon (32MB L3 cache) (256KB L2 cache) (2MB L2 cache) OH NC CA MO MN AL IA MI KY KS AR IN MS NE MD CO LA NJ MACT NM ND MT ID AZ ME NH NV DE IBM Power 4 Sun UltraSPARC IIi AMD Athlon (on different platforms)

45. Cache effects Cache size 0.69 0.84 1.00 1.17 1.30 1.41 1.83 0.75 0.87 0.92 0.71 11.12 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 128KB256KB512KB1MB2MB4MB8MB16MB32MB Simulated cache miss ratio D-RRL/D-LHP Performance ratio D-RRL/D-LHP on real architectures UltraSPARC IIi Xeon Athlon 1.59 Power 4 Colorado road network

46. Conclusions and open problems… C Code publicly available at http://www.dis.uniroma1.it/ ~ demetres/experim/dsp O(n 2 ) space too much for applications Trading-off space and (query) time? Fully dynamic single source shortest path problem?

47. The world is not perfect…