1. 1 Travelling Salesman Problem Algorithms and Networks 2015/2016 Hans L. Bodlaender Johan M. M. van Rooij

2. 2 Contents  TSP and its applications  Heuristics and approximation algorithms  Construction heuristics, a.o.: Christofides, insertion heuristics  Improvement heuristics, a.o.: 2-opt, 3-opt, Lin-Kernighan  Exact algorithms

3. 3 PROBLEM DEFINITION APPLICATIONS Travelling Salesman Problem – Algorithms and Networks

4. 4 Problem  Instance: n vertices (cities), distance between every pair of vertices.  Question: Find shortest (simple) cycle that visits every city? 1 1 2 2 3 3 4 4 4 5 2 2 3 2 1 1 2 2 3 3 4 4 4 5 2 2 3 2 1 1 2 2 3 3 4 4 4 5 2 2 3 2 13 11

5. 5 Applications  Pickup and delivery problems  Robotics  Board drilling / chip manufacturing  Lift scheduling

6. 6 Assumptions / Problem Variants  Complete graph vs underlying graph structure.  Complete graph: a given distance between all pairs of cities.  Directed edges vs undirected arcs.  Undirected: symmetric: w(u,v) = w(v,u).  Directed: not symmetric.  Asymmetric examples: painting/chip machine application.  Lengths:  Lengths are non-negative (or positive).  Triangle inequality: for all x, y, z: w(x,y) + w(y,z)  w(x,z)  Different assumptions lead to different problems.

7. 7 NP-complete  Instance: cities, distances, k.  Question: is there a TSP-tour of length at most k?  Is an NP-complete problem.  Relation with Hamiltonian Circuit problem.

8. 8 If triangle inequality does not hold  Theorem: If PNP, then there is no polynomial time algorithm for TSP without triangle inequality that approximates within a ratio c, for any constant c.  Proof: Suppose there is such an algorithm A. We build a polynomial time algorithm for Hamiltonian Circuit (giving a contradiction):  Take instance G=(V,E) of Hamiltonian Circuit.  Build instance of TSP: A city for each v  V. If (v,w)  E, then d(v,w) = 1, otherwise d(v,w) = nc+1.  A finds a tour with distance at most nc, if and only if, G has a Hamiltonian circuit.

9. 9 CONSTRUCTION HEURISTICS Travelling Salesman Problem – Algorithms and Networks

10. 10 Heuristics and approximations  Construction heuristics:  A tour is built from nothing.  Improvement heuristics:  Start with `some’ tour, and continue to change it into a better one as long as possible

11. 11 1 st Construction Heuristic: Nearest neighbor  Start at some vertex s; v=s;  While not all vertices visited  Select closest unvisited neighbor w of v  Go from v to w; v=w  Go from v to s.  Without triangle inequality:  Approximation ratio arbitrarily bad.  With triangle inequality:  Approximation ratio O(log n).

12. 12 2 nd Construction Heuristic: Heuristic with ratio 2  Find a minimum spanning tree  Report vertices of tree in preorder  Preorder: visit a node before its children  Approximation ratio 2 (with triangle inequality):  OPT ≥ MST  2 MST ≥ Result  Result / OPT ≤ 2MST / MST = 2

13. 13 3 rd Construction Heuristic: Christofides 1. Make a Minimum Spanning Tree T. 2. Set W = {v | v has odd degree in tree T}. 3. Compute a minimum weight matching M in the graph G[W]. 4. Look at the graph T+M.  Note that T+M is Eulerian! 5. Compute an Euler tour C’ in T+M. 6. Add shortcuts to C’ to get a TSP-tour.

14. 14 Ratio 1.5  Total length edges in T: at most OPT  Total length edges in matching M: at most OPT/2.  T+M has length at most 3/2 OPT.  Use -inequality.

15. 15 4 th Construction Heuristic: Closest insertion heuristic  Build tour by starting with one vertex, and inserting vertices one by one.  Always insert vertex that is closest to a vertex already in tour.

16. 16 Closest insertion heuristic has performance ratio 2  Build tree T: if v is added to tour, add to T edge from v to closest vertex on tour.  T is a Minimum Spanning Tree (Prim’s algorithm).  Total length of T  OPT.  Length of tour  2* length of T.

17. 17 Many variants  Closest insertion: insert vertex closest to vertex in the tour.  Farthest insertion: insert vertex whose minimum distance to a node on the cycle is maximum.  Cheapest insertion: insert the node that can be inserted with minimum increase in cost.  Gives also ratio 2.  Computationally expensive.  Random insertion: randomly select a vertex.  Each time: insert vertex at position that gives minimum increase of tour length.

18. 18 5 th Construction Heuristic: Cycle merging heuristic  Start with n cycles of length 1.  Repeat:  Find two cycles with minimum distance.  Merge them into one cycle.  Until 1 cycle with n vertices.  This has ratio 2: compare with algorithm of Kruskal for MST.

19. 19 6 th Construction Heuristic: Savings heuristic  Cycle merging heuristic where we merge tours that provide the largest “savings”:  Saving for a merge: merge with the smallest additional cost / largest savings.

20. 20 Some test results  In an overview paper, Junger et al. report on tests on set of instances (105 – 2392 vertices; city-generated TSP benchmarks)  Nearest neighbor: 24% away from optimal in average  Closest insertion: 20%;  Farthest insertion: 10%;  Cheapest insertion: 17%;  Random Insertion: 11%;  Preorder of min spanning trees: 38%;  Christofides: 19% with improvement 11% / 10%;  Savings method: 10% (and fast).

21. 21 IMPROVEMENT HEURISTICS Travelling Salesman Problem – Algorithms and Networks

22. 22 Improvement heuristics  Start with a tour (e.g., from heuristic) and improve it stepwise  2-Opt  3-Opt  K-Opt  Lin-Kernighan  Iterated LK  Simulated annealing, … Iterative improvement Local search

23. 23 Scheme  Rule that modifies solution to different solution  While there is a Rule(sol, sol’) with sol’ a better solution than sol  Take sol’ instead of sol  Cost decrease  Stuck in `local minimum’  Can use exponential time in theory…

24. 24 Very simple  Node insertion:  Take a vertex v and put it in a different spot in the tour.  Edge insertion:  Take two successive vertices v, w and put these as edge somewhere else in the tour.

25. 25 2-opt  Take two edges (v,w) and (x,y) and replace them by (v,x) and (w,y) if this improves the tour.  Costly: part of tour should be turned around.  On R 2 : get rid of crossings of tour.

26. 26 2-Opt improvements  Reversing shorter part of the tour.  We need a direction on the tour to decide what reconnect option leads to subtours.  Postpone correcting tour.  Clever search to improving moves.  Restrict distance function evaluations.  Prove that you do not need to consider some potential 2-opt moves, because these were not improving in earlier steps.  Look only to subset of candidate improvements.  Combine with node insertion.

27. 27 3-opt  Choose three edges from tour  Remove them, and combine the three parts to a tour in the cheapest way to link them

28. 28 3-opt  Costly to find 3-opt improvements: O(n 3 ) candidates  k-opt: generalizes 3-opt

29. 29 Lin-Kernighan  Idea: modifications that are bad can lead to enable something good  Tour modification:  Collection of simple changes  Some increase length  Total set of changes decreases length

30. 30 Lin-Kernighan  One LK step:  Make sets of edges X = {x 1, …, x r }, Y = {y 1,…,y r } If we replace X by Y in tour then we have another tour  Sets are built stepwise  Repeated until …  Variants on scheme possible

31. 31 One Lin-Kernighan step  Choose vertex t 1, and edge x 1 = (t 1,t 2 ) from tour.  i=1.  Choose edge y 1 =(t 2, t 3 ) not in tour with g 1 = w(x 1 )–w(y 1 ) > 0 (or, as large as possible).  Repeat a number of times, or until …  i++;  Choose edge x i = (t 2i-1,t 2i ) from tour, such that: x i not one of the edges y j. oldtour – X + (t 2i,t 1 ) +Y is also a tour.  if oldtour – X + (t 2i,t 1 ) +Y has shorter length than oldtour, then take this tour: done.  Choose edge y i = (t 2i, t 2i+1 ) such that: g i = w(x i ) – w(y i ) > 0. y i is not one of the edges x j. y i not in the tour.

32. 32 Iterated Lin-Kernighan  Construct a start tour.  Repeat the following r times:  Improve the tour with Lin-Kernighan until not possible.  Do a random 4-opt move that does not increase the length with more than 10 percent.  Report the best tour seen. Cost much time. Gives excellent results! Cost much time. Gives excellent results!

33. 33 Other methods  Simulated annealing and similar methods.  Problem specific approaches, special cases.  Iterated LK combined with treewidth/branchwidth approach:  Run ILK a few times (e.g., 5).  Take graph formed by union of the 5 tours.  Find minimum length Hamiltonian circuit in graph with clever dynamic programming algorithm.

34. 34 DYNAMIC PROGRAMMING Travelling Salesman Problem – Algorithms and Networks

35. 35 Held-Karp algorithm for TSP  O(n 2 2 n ) algorithm for TSP.  Uses dynamic programming.  Take some starting vertex s.  For set of vertices R (s  R), vertex w  R, let B(R,w) = minimum length of a path, that  Starts in s.  Visits all vertices in R (and no other vertices).  Ends in w.

36. 36 TSP: Recursive formulation  B({s},s) = 0  B({s},v) = ∞ if v ≠ s  If |R| > 1, then  B(R,x) = min vR\{x} B( R\{x}, v ) + w(v,x)  If we have all B(V,v) then we can solve TSP.  Gives requested algorithm using DP-techniques.

37. 37 Space Improvement for Hamiltonian Cycle  Dynamic programming algorithm uses:  O(n 2 2 n ) time.  O(n2 n ) space.  In practice space becomes a problem before time does.  Next, we give an algorithm for Hamiltonian Cycle that uses:  O(n 3 2 n ) time.  O(n) space.  This algorithm counts solutions using ‘Inclusion/Exclusion’.

38. 38 Counting (Non-)Hamiltonian Cycles  Computing/counting tours is much easier if we do not care about which cities are visited.  This uses exponential space in the DP algorithm.  Define: Walks[ v i, k ] = the number of ways to travel from v 1 to v i traversing k times an edge.  We do not care whether nodes/edges are visited (or twice).  Using Dynamic Programming:  Walks[ v i, 0 ]= 1if i = 1  Walks[ v i, 0 ]= 0otherwise  Walks[ v i, k ]= ∑ vj ∈ N(vi) Walks[ v j, k – 1 ]  Walks[ v 1, n ] counts all length n walks that return in v 1.  This requires O(n 3 ) time and O(n) space.

39. 39 How Does Counting ‘Arbitrary’ Cycles Help? A useful formula:  Number of cycles through v j = total number of cycles (some go through v j some don’t) -total number of cycles that do not go through v j.  Hamiltonian Cycle: cycle of length n that visits every city.  Keeping track of whether every vertex is visited is hard.  Counting cycles that do not go through some vertex is easy.  Just leave out the vertex in the graph.  Counting cycles that may go through some vertex is easy.  We do not care whether it is visited or not (or twice) anymore.

40. 40 Using the formula on every city A useful formula:  Number of cycles through v j = total number of cycles (some go through v j some don’t) -total number of cycles that do not go through v j.  What if we apply the above formula to every city (except v 1 )?  We get 2 n-1 subproblems, were we count walks from v 1 to v 1 where either the city is not allowed to be visited, or may be visited (not important whether it is visited, or twice, or not).  After combining the computed numbers from all 2 n-1 subproblems using the above formula, we get:  The number of cycles of length n visiting every vertex.  I.e, the number of Hamiltonian cycles.

41. 41 Apply the formula to every city: the Inclusion/Exclusion formula A useful formula:  Number of cycles through v j = total number of cycles (some go through v j some don’t) -total number of cycles that do not go through v j. Let CycleWalks( V’ ) be the number of walks from v 1 to v 1 using only vertices in V’, then:

42. 42 CONCLUSIONS Travelling Salesman Problem – Algorithms and Networks

43. 43 Conclusions  TSP has many applications.  Also many applications for variants of TSP.  Heuristics.  Construction heuristics  Improvement heuristics  Dynamic programming in O(n 2 2 n ) time and O(n2 n ) space.  Inclusion/Exclusion for Hamiltonian Cycle in O(n 3 2 n ).  Further reading:  M. Jünger, G. Reinelt, G. Rinaldi, The Traveling Salesman Problem, in: Handbooks in Operations Research and Management Science, volume 7: Network Models, North- Holland Elsevier, 1995.