1. Vittorio Maniezzo University of Bologna
TSP: an introduction
Vittorio Maniezzo
University of Bologna
Vittorio Maniezzo - University of Bologna - Transportation Logistics

2. Travelling Salesman Problem (TSP)
n locations given (home of salesperson and n-1 customers).
[dij] distance matrix given, contains distances (Km, time, … ) between each pair of locations.
The TSP asks for the shortest tour starting home, visiting each customer once and returning home
Symmetric / asymmetric versions.
Vittorio Maniezzo - University of Bologna - Transportation Logistics

3. Introduction Which is the shortest tour? Difficult to say …
Vittorio Maniezzo - University of Bologna - Transportation Logistics

4. TSP history
The general form of the TSP appears to have been first studied by mathematicians starting in the 1930s by Karl Menger in Vienna.
Mathematical problems related to the traveling salesman problem were treated in the 1800s by the Irish mathematician Sir William Rowan Hamilton.
The picture shows Hamilton's Icosian Game that requires players to complete tours through the 20 points using only the specified connections. (
Vittorio Maniezzo - University of Bologna - Transportation Logistics

5. Past practice …
Vittorio Maniezzo - University of Bologna - Transportation Logistics

6. Current practice
Actual tours of persons or vehicles (deliveries to customers)
Given: Graph of streets, where some nodes represent customers.
Find:
shortest tour among customers.
Precondition:
Solve shortest path problems  Distance matrix
Vittorio Maniezzo - University of Bologna - Transportation Logistics

7. Application: PCB
Minimization of time for drilling holes in PCBs (printed circuit boards)
Vittorio Maniezzo - University of Bologna - Transportation Logistics

8. Application: setup costs
Minimization of total setup costs (or setup times) when setup costs are sequence dependent
All product types must be produced and the setup must return to initial configuration
i \ j 1 2 3 4 5 18 17 20 21 16 98 19
Vittorio Maniezzo - University of Bologna - Transportation Logistics

9. Application: paint shop
Minimization of total setup costs (or setup times) when setup costs are sequence dependent
All colors must be processed
i \ j
White
Pink
Red
Blue 5 7 10 2 15
Vittorio Maniezzo - University of Bologna - Transportation Logistics

10. Solution Methods n nodes  (n-1)! possible tours
Some possible solution method:
Complete enumeration
IP: Branch and Bound (and Cut, and Price, … )
Dynamic programming
Approximation algorithms
Heuristics
Metaheuristics n
n! 120 3, ,31*10^12 2,43*10^18
Vittorio Maniezzo - University of Bologna - Transportation Logistics

11. Standard Example Example with 5 customers (1 home, 4 customers)
Zimmermann, W., O.R., Oldenbourg, 1990:
e.g. cost of : = 93
First: simplify by reducing costs  16 19 18 5 17 20 4 98 3 21 2 1
i \ j
Vittorio Maniezzo - University of Bologna - Transportation Logistics

12. Lower bound Subtract minimum from each row & column
Reduction constant = = 82
all tours have costs which are 82 greater,
e.g. cost of : = 11 [= ]
Optimal tour does not change (see [Toth, Vigo, 02]).  3 2 5 4 1 82
i \ j
Vittorio Maniezzo - University of Bologna - Transportation Logistics

13. Complete Enumeration 5 customers  4! = 234 = 24 tours
Vittorio Maniezzo - University of Bologna - Transportation Logistics

14. A wider scope
A combinatorial optimization problem is defined over a set C = {c1 , … , cn} of basic components.
A solution of the problem is a subset S  C;
F  2C is the subset of feasible solutions, (a solution S is feasible iff SF).
z: 2C   is the cost function,
the objective is to find a minimum cost feasible solution S°, i.e., to find S° F such that z(S°)  z(S), S  F.
Failing this, the algorithm anyway returns the best feasible solution found, S*  F.
Vittorio Maniezzo - University of Bologna - Transportation Logistics

15. TSP as a CO example
TSP is defined over a weighted undirected complete graph G=(V,E,D), where V is the set of vertices, E is the set of edges and D is the set of edge weights.
The component set corresponds to E (C=E),
F corresponds to the set of Hamiltonian cycles in G
z(S) is the sum of the weights associated to the edges belonging to the solution S.
Vittorio Maniezzo - University of Bologna - Transportation Logistics

16. LP-Formulation
LP formulation is a binary LP (IP) which is almost identical to the assignment problem:
plus conditions to avoid short cycles
Vittorio Maniezzo - University of Bologna - Transportation Logistics

17. Example for short cycles
Optimal solution of LP without conditions to avoid short cycles = solution of assignment problem
short cycles and
 conditions to avoid short cycles are needed 1 2 5 4 3
Vittorio Maniezzo - University of Bologna - Transportation Logistics

18. Conditions to avoid short cycles
Several formulations in the literature
e.g. Dantzig-Fulkerson-Johnson
exponentially many!
Vittorio Maniezzo - University of Bologna - Transportation Logistics

19. Example: avoid short cycles
Above example with n = 5 cities:
must consider subsets Q:
{1, 2}; {1, 3}; {1, 4}; {1, 5}; {2, 3}; {2, 4}; {2, 4}; {3, 4}; {3, 5}; {4, 5}
constraint for subset Q = {1, 2} and V-Q = {3, 4, 5} x13 + x14 + x15 + x23 + x24 + x25  1
n = 6  subsets (constraints)
exponentially many!
Vittorio Maniezzo - University of Bologna - Transportation Logistics

20. Vittorio Maniezzo University of Bologna
Heuristics for the TSP
Vittorio Maniezzo
University of Bologna
Vittorio Maniezzo - University of Bologna - Transportation Logistics

21. Computational issues
The size of the instances met in real world applications rules out the possibility of solving them to optimality in an acceptable time.
Nevertheless, these instances must be solved.
Thus the need to look for suboptimal solutions, provided that they are of “acceptable quality” and that they can be found in “acceptable time”.
Vittorio Maniezzo - University of Bologna - Transportation Logistics

22. Heuristic algorithms How to cope with NP-completeness:
small instances;
polynomial special cases;
approximation algorithms guarantee to find a solution within a known gap from optimality;
probabilistic algorithms guarantee that for instances big enough, the probability of getting a bad solution is very small;
heuristic algorithms: no guarantee, but historically, on the average, these algorithms have the best quality/time trade off for the problem of interest.
Vittorio Maniezzo - University of Bologna - Transportation Logistics

23. heuristics: focus on solution structure
Simplest heuristics exploit structural properties of feasible solutions in order to quickly come to a good one.
They belong to one of two main classes: constructive heuristics or local search heuristics.
Vittorio Maniezzo - University of Bologna - Transportation Logistics

24. Constructive heuristics
1. Sort the components in C by increasing costs.
2. Set S*= and i=1.
3. Repeat
If (S*  ci is a partial feasible solution) then S* = S*ci.
i=i+1.
Until S* F.
Vittorio Maniezzo - University of Bologna - Transportation Logistics

25. Constructive heuristics
A constructive approach can yield optimal solutions for certain problems, eg. the MST.
In other cases it could be unable to construct a feasible solution.
TSP: order all edges by increasing edge cost, take the least cost one and add to it increasing cost edges, provided they do not close subtours, until a Hamiltonian circuit is completed.
More involved constructive strategies give rise to well- known TSP heuristics, like the Farthest Insertion, the Nearest Neighbor, the Best Insertion or the Sweep algorithms.
Vittorio Maniezzo - University of Bologna - Transportation Logistics

26. Nearest Neighbor Heuristic
0. Start from any city; e.g. the first one
1. Find nearest neighbor (to the last city) not already visited
2. repeat 1 until all cities are visited.
Then connect last city with starting point
Ties can be broken arbitrarily.
Starting from different starting points gives different solutions! Some will be bad some will be good.
GREEDY  the last connections tend to be long and improvement heuristics should be used afterwards
Vittorio Maniezzo - University of Bologna - Transportation Logistics

27. Best Insertion Heuristic
0. Select 2 cities A, B and start with short cycle A-B-A
1. Insert next city (not yet inserted) in the best position in the short cycle
2. Repeat 1. until all cities are visited
Using different starting cycles and different choices of the next city different solutions are obtained
Often when symmetric: starting cycle = 2 cities with maximum distance next city = maximizes min distance to inserted cities
Vittorio Maniezzo - University of Bologna - Transportation Logistics

28. An approximation algorithm
In some cases, it is possible to construct solutions which are guaranteed to be “not too bad”.
Approximation algorithms provide a guarantee on worst possible value for the ratio (zh – z*)/z*.
TSP hypothesis: it is always cheaper to go straight from a node u to another node v instead of traversing an intermediate node w (triangular inequality).
Vittorio Maniezzo - University of Bologna - Transportation Logistics

29. Approx-TSP
An approx. Algorithm for the TSP with triangular inequality is:
Approx-TSP-Tour(G,w)
1 select a root vertex rV
2 construct a MST T for G rooted in r
3 let L be the list of vertices visited in a preorder tree walk of T
4 return the hamiltonian cycle H which visits all vertices in the order L
Vittorio Maniezzo - University of Bologna - Transportation Logistics

30. Approx-TSP: example a d e b f g c h
Vittorio Maniezzo - University of Bologna - Transportation Logistics

31. Approx-TSP: example a d e b f g c h
Vittorio Maniezzo - University of Bologna - Transportation Logistics

32. Approx-TSP: example a d e b f g c h
Vittorio Maniezzo - University of Bologna - Transportation Logistics

33. Approx-TSP: example a d e b f g c h
Vittorio Maniezzo - University of Bologna - Transportation Logistics

34. Approx-TSP: performance
Theorem
Approx-TSP-Tour is an approximation algorithm with a ratio bound of 2 for the traveling salesman problem with triangle inequality.
Proof
Because the optimal tour solution must have one more edge than the MST of T, so c(MST)  c(T*).
c(Tour) = 2c(MST)
Because of the assumption of triangle inequality, c(H)  c(Tour).
Therefore, c(H)  2c(T*).
Since the input is a complete graph, so the implementation of Prim’s algorithm runs in O(V2). Therefore the total time complexity of
Approx-TSP-Tour also is O(V2).
Vittorio Maniezzo - University of Bologna - Transportation Logistics

35. Local Search: neighborhoods
The neighborhood of a solution S, N(S), is a subset of 2C defined by a neighborhood function N: 2C  22c.
Often only feasible solutions considered, thus neighborhood functions N: F  2F.
The specific function used has a deep impact on the algorithm performance and its choice is left to the algorithm designer.
Vittorio Maniezzo - University of Bologna - Transportation Logistics

36. Local search 1. Generate an initial feasible solution S.
2. Find S'N(S), such that z(S')=min z(S^), S^ N(S).
3. If z(S') < z(S) then S=S'
go to step 2.
4. S* = S.
The update of a solution in step 3 is called a move from S to S'.
It could be made to the first improving solution found.
Vittorio Maniezzo - University of Bologna - Transportation Logistics

37. Local search
There are problems for which a local search approach guarantees to find an optimal solution (ex. the simplex algorithm).
For TSP, two LS are 2-opt and 3-opt, which take a solution (a list of n vertices) and exhaustively swap the elements of all pairs or triplets of vertices in it.
More sophisticated neighborhoods give rise to more effective heuristics, among which Lin and Kernighan [LK73].
Vittorio Maniezzo - University of Bologna - Transportation Logistics

38. Local search heuristics
2 – opt (arcs): try all inversions of some subsequence
if improvement  start again from beginning
3 – opt (arcs): try all shifts of some subsequence to different positions
if improvement 
start again from
beginning
3-opt (nodes): swap every triplet of nodes in the permutation representing the solution.
if improvement  start again from beginning
2-opt (nodes): swap every pair of nodes in the permutation representing the solution.
if improvement  start again from beginning
Vittorio Maniezzo - University of Bologna - Transportation Logistics

39. Metaheuristcs for the TSP (some of them)
Vittorio Maniezzo
University of Bologna, Italy
Vittorio Maniezzo - University of Bologna - Transportation Logistics

40. Metaheuristics: focus on heuristic guidance
Simple heuristics can perform very well, but can get trapped in poor quality solutions (local optima).
New approaches have been presented starting from the mid '70ies.
They are algorithms which manage the trade-off between search diversification, when search is going on in bad regions of the search space, and intensification, aimed at finding the best solutions within the region being analyzed.
These algorithms have been named metaheuristics.
Vittorio Maniezzo - University of Bologna - Transportation Logistics

41. Metaheuristics Metaheuristics include: Simulated Annealing Tabu Search
GRASP
Genetic Algorithms
Variable Neighborhood Search
ACO
Particle Swarm Optimization (PSO) …
Vittorio Maniezzo - University of Bologna - Transportation Logistics

42. Simulated Annealing
Simulated Annealing (SA) [AK89] modifies local search in order to accept, in probability, worsening moves.
1. Generate an initial feasible solution S,
initialize S* = S and temperature parameter T.
2. Generate S’N(S).
3. If z(S') < z(S) then S=S', if (z(S*) > z(S)) S* = S
else accept to set S=S' with probability p = e-(z(S')-z(S))/kT.
4. If (annealing condition) decrease T.
5. If not(end condition) go to step 2.
Vittorio Maniezzo - University of Bologna - Transportation Logistics

43. SA example: ulysses 16 Simulated annealing trace
Vittorio Maniezzo - University of Bologna - Transportation Logistics

44. Tabu search
Tabu Search (TS) [GL97] escapes from local minima by moving onto the best solution of the neighborhood at each iteration, even though it is worse than the current one.
A memory structure called tabu list, TL, forbids to return to already explored solutions.
1. Generate an initial feasible solution S,
set S* = S and initialize TL=.
2. Find S' N(S), such that
z(S')=min {z(S^), S^ N(S), S^ TL}.
3. S=S', TL=TL  {S}, if (z(S*) > z(S)) set S* = S.
4. If not(end condition) go to step 2.
Vittorio Maniezzo - University of Bologna - Transportation Logistics

45. TS example: ulysses 16 Tabu Search trace
Vittorio Maniezzo - University of Bologna - Transportation Logistics

46. GRASP
GRASP (Greedy Randomized Adaptive Search Procedure) [FR95] restarts search from another promising region of the search space as soon as a local optimum is reached.
GRASP consists in a multistart approach with a suggestion on how to construct the initial solutions.
1. Build a solution S (=S* in the first iteration) by a constructive greedy randomized procedure based on candidate lists.
2. Apply a local search procedure starting from S and producing S'.
3. If z(S')<z(S*) then S*=S'.
4. If not(end condition) go to step 2.
Vittorio Maniezzo - University of Bologna - Transportation Logistics

47. Genetic algorithms
Genetic Algorithms (GA) [G89] do not rely on construction or local search but on the parallel update of a set R of solutions.
This is achieved by recombining subsets of 2 elements in R, the parent sets, to obtain new solutions.
To be effective on CO problems they are often hybridized with SA, TS or local search.
Vittorio Maniezzo - University of Bologna - Transportation Logistics

48. Genetic algorithms 1. Initialize a set R of solutions.
2. Construct a set Rc of solutions by recombining pairs of randomly chosen elements in R.
3. Construct a set Rm of solutions by randomly modifying elements in Rc.
4. Construct the new set R by extracting elements from Rm, following a montecarlo strategy with repetitions.
5. If not(end condition) go to step 2.
Vittorio Maniezzo - University of Bologna - Transportation Logistics

49. Components of a GA A problem to solve, and ...
Encoding technique (gene, chromosome)
Initialization procedure (creation)
Evaluation function (fitness)
Selection of parents (reproduction)
Genetic operators (mutation, recombination)
Parameter settings (practice and art)
Vittorio Maniezzo - University of Bologna - Transportation Logistics

50. GA jargon
Procedure GA { initialize population; evaluate population; while not(End_condition) { select parents for reproduction; perform recombination and mutation; }
Vittorio Maniezzo - University of Bologna - Transportation Logistics

51. Populations Solutions are called “chromosomes”. Chromosomes could be:
Bit strings ( )
Real numbers ( )
Permutations of element (E11 E3 E7 ... E1 E15)
Lists of rules (R1 R2 R3 ... R22 R23)
Program elements (genetic programming)
... any data structure ...
Vittorio Maniezzo - University of Bologna - Transportation Logistics

52. Reproduction
Parents are selected at random with selection chances biased in relation to chromosome evaluations.
Only parents will reproduce.
Selection is with repetition.
Every pair of parents will generate two children.
Vittorio Maniezzo - University of Bologna - Transportation Logistics

53. Chromosome Modification
Modifications are stochastically triggered
Operator types are:
Mutation
Crossover (recombination)
Vittorio Maniezzo - University of Bologna - Transportation Logistics

54. Crossover: Recombination
P1 ( ) ( ) C1
P2 ( ) ( ) C2
One (or two) cutting points are randomly chosen.
The code between the points is swapped in the two parents.
Vittorio Maniezzo - University of Bologna - Transportation Logistics

55. Mutation: Local Modification
Before: ( )
After: ( )
Before: ( )
After: ( )
Causes movement in the search space.
Vittorio Maniezzo - University of Bologna - Transportation Logistics

56. Evaluation
The evaluator decodes a chromosome and assigns it a fitness measure.
The fitness measures “how good” a solution is: it is directly related to the objective function in maximization problems and inversely in minimization problems.
The evaluator is the only link between a GA and the problem it is solving.
Vittorio Maniezzo - University of Bologna - Transportation Logistics

57. An Abstract Example Fitness
Distribution of individuals in generation 0
Fitness
Distribution of individuals in generation n
Vittorio Maniezzo - University of Bologna - Transportation Logistics

58. TSP: Representation Representation is an ordered list of city
numbers known as an order-based GA.
1) Amsterdam 2) Berlin 3) Bologna 4) London
5) Madrid ) Paris ) Prague ) Vienna
CityList1 ( )
CityList2 ( )
Vittorio Maniezzo - University of Bologna - Transportation Logistics

59. Crossover Crossover can be defined in different ways. One example:
* *
Parent1 ( )
Parent2 ( )
Child ( )
This operator is called the Order1 crossover.
Vittorio Maniezzo - University of Bologna - Transportation Logistics

60. Mutation Mutation can be implemented as a 2-swap:
* *
Before: ( )
After: ( )
Vittorio Maniezzo - University of Bologna - Transportation Logistics

61. TSP Example: 30 Cities
Vittorio Maniezzo - University of Bologna - Transportation Logistics

62. Solutioni (Distance = 941)
Vittorio Maniezzo - University of Bologna - Transportation Logistics

63. Solutionj (Distance = 800)
Vittorio Maniezzo - University of Bologna - Transportation Logistics

64. Solutionn (Distance = 652)
Vittorio Maniezzo - University of Bologna - Transportation Logistics

65. Best Solution (Distance = 420)
Vittorio Maniezzo - University of Bologna - Transportation Logistics