1. Ant colonies for the traveling salesman problem Eliran Natan Seminar in Bioinformatics (236818) – Spring 2013 Computer Science Department Technion - Israel Institute of Technology

2. Content  Pheromone trails  Ant Colony Optimization algorithms  Travelling Salesman Problem And its ACS solution

3. Content  Pheromone trails  Ant Colony Optimization algorithms  Travelling Salesman Problem And its ACS solution

4. Content  Pheromone trails  Ant Colony Optimization algorithms  Travelling Salesman Problem And its ACS solution

5. Content  Pheromone trails  Ant Colony Optimization algorithms  Travelling Salesman Problem And its ACS solution

6. Pheromone trails It is well known that the primary means for ants to form and maintain the line is a pheromone trail. Ants deposit a certain amount of pheromone while walking, and each ant probabilistically prefers to follow a direction rich in pheromone.

7. Pheromone trails This elementary behavior of real ants can be used to explain how they can find the shortest path that reconnects a broken line after the sudden appearance of an unexpected obstacle has interrupted the initial path. Shortest path around an obstacle Let us consider the following scenario: Ants are moving on a straight line that connects a food source to their nest.

8. Pheromone trails Shortest path around an obstacle An obstacle appears on the path. This elementary behavior of real ants can be used to explain how they can find the shortest path that reconnects a broken line after the sudden appearance of an unexpected obstacle has interrupted the initial path.

9. Pheromone trails Shortest path around an obstacle Those ants which are just in front of the obstacle cannot continue to follow the pheromone trail and therefore they have to choose between turning right or left. In this situation we can expect half the ants to choose to turn right and the other half to turn left. This elementary behavior of real ants can be used to explain how they can find the shortest path that reconnects a broken line after the sudden appearance of an unexpected obstacle has interrupted the initial path.

10. Pheromone trails Shortest path around an obstacle Those ants which choose, by chance, the shorter path around the obstacle will more rapidly reconstitute the interrupted pheromone trail compared to those which choose the longer path. Thus, the shorter path will receive a greater amount of pheromone per time unit and in turn a larger number of ants will choose the shorter path. This elementary behavior of real ants can be used to explain how they can find the shortest path that reconnects a broken line after the sudden appearance of an unexpected obstacle has interrupted the initial path.

11. Pheromone trails Shortest path around an obstacle Shortest path is being obtained. This elementary behavior of real ants can be used to explain how they can find the shortest path that reconnects a broken line after the sudden appearance of an unexpected obstacle has interrupted the initial path.

12. Pheromone trails Ants are able, without using any spatial Information, to identify a sudden appearance of a food source around their nest, and to find the shortest available path to it. Shortest path from the nest to the food source Let us describe the algorithm: A small amount of ants travel randomly around the nest. N

13. Pheromone trails Shortest path from the nest to the food source One of the ants find food source. S N Ants are able, without using any spatial Information, to identify a sudden appearance of a food source around their nest, and to find the shortest available path to it.

14. Pheromone trails Shortest path from the nest to the food source When ant finds food, it returns to the nest while laying down pheromones trail. S N Ants are able, without using any spatial Information, to identify a sudden appearance of a food source around their nest, and to find the shortest available path to it.

15. Pheromone trails Shortest path from the nest to the food source When other ants find a pheromone trail, they are likely not to keep travelling at random, but to instead follow the trail. S N Ants are able, without using any spatial Information, to identify a sudden appearance of a food source around their nest, and to find the shortest available path to it.

16. Pheromone trails Shortest path from the nest to the food source If an ant eventually find food by following a pheromone trail, it returning to the nest while reinforcing the trail with more pheromones. S N Ants are able, without using any spatial Information, to identify a sudden appearance of a food source around their nest, and to find the shortest available path to it.

17. Pheromone trails Shortest path from the nest to the food source Due to their stochastic behavior, some ants are not following the pheromone trails, and thus uncover more possible paths. S N Ants are able, without using any spatial Information, to identify a sudden appearance of a food source around their nest, and to find the shortest available path to it.

18. Pheromone trails Shortest path from the nest to the food source Over time, however, the pheromones trails starts to evaporate, thus reducing its attractive strength. S N Ants are able, without using any spatial Information, to identify a sudden appearance of a food source around their nest, and to find the shortest available path to it.

19. Pheromone trails Shortest path from the nest to the food source Shortest path is being obtained. S N Ants are able, without using any spatial Information, to identify a sudden appearance of a food source around their nest, and to find the shortest available path to it.

20. Pheromone trails Shortest path from the nest to the food source An Intuitive proof of correctness Pheromone evaporation also has the advantage of avoiding the convergence to a locally optimal solution. If there were no evaporation at all, all paths chosen by the first ants would tend to be excessively attractive to the following ones. In that case, the exploration of the solution space would be constrained.  The more time it takes for an ant to travel down the path and back again, the more time the pheromones have to evaporate. Thus,  A short path, by comparison, get marched over more frequently, and thus its pheromone density becomes higher on shorter paths than longer ones. Thus, when one ant finds a short path from the colony to a food source, other ants are more likely to follow that path, and positive feedback eventually leads all the ants following a single path.

21. Ant Colony Optimization (ACO) studies artificial systems that take inspiration from the behavior of real ant colonies and which are used to solve discrete optimization problems. Ant Colony Optimization algorithms Some applications: Set partition problem Deciding whether a given multiset of positive integers can be partitioned into two subsets A and B such that the sum of the numbers in A equals the sum of the numbers in B.

22. Ant Colony Optimization algorithms Ant Colony Optimization (ACO) studies artificial systems that take inspiration from the behavior of real ant colonies and which are used to solve discrete optimization problems. Some applications: Job – Shop problems Given a number of jobs have to be done and every job consists of using a number of machines for a certain amount of time. Find the best planning to do all the jobs on all the different machines in the shortest period of time.

23. Ant Colony Optimization algorithms Ant Colony Optimization (ACO) studies artificial systems that take inspiration from the behavior of real ant colonies and which are used to solve discrete optimization problems. Some applications: Multiple knapsack problem Given a set of items, each with a weight and a value, determine the number of each item to include in a collection so that the total weight is less than or equal to a given limit and the total value is as large as possible.

24. Ant Colony Optimization algorithms Ant Colony Optimization (ACO) studies artificial systems that take inspiration from the behavior of real ant colonies and which are used to solve discrete optimization problems. Some applications: Protein folding The process by which a protein structure assumes its functional shape or conformation.

25. Ant Colony Optimization algorithms Ant Colony Optimization (ACO) studies artificial systems that take inspiration from the behavior of real ant colonies and which are used to solve discrete optimization problems. Some applications: Data mining the computational process of discovering patterns in large data sets.

26. Ant Colony Optimization algorithms Ant Colony Optimization (ACO) studies artificial systems that take inspiration from the behavior of real ant colonies and which are used to solve discrete optimization problems. Some applications: Travelling salesman problem Given a list of cities and the distances between each pair of cities, determine the shortest possible route that visits each city exactly once and returns to the origin city.

27. Travelling Salesman Problem Definitions For a complete undirected weighted graphs Problem Complexity class NP–Hard

28. Travelling Salesman Problem Algorithms for deterministic solution For an undirected complete weighted graphs  Brute force search  Dynamic programming  Greedy algorithm

29. Travelling Salesman Problem ACS solution

30. Travelling Salesman Problem ACS Algorithm description ACS solution

31. Travelling Salesman Problem ACS Algorithm description ACS solution

32. Travelling Salesman Problem ACS Algorithm description ACS solution

33. Travelling Salesman Problem ACS Algorithm description ACS solution

34. Travelling Salesman Problem ACS Algorithm description ACS solution

35. Travelling Salesman Problem ACS Algorithm description ACS solution Every time an edge is chosen by an ant, its amount of pheromone is changed according to the following formula:  Local updating is intended to avoid a very strong edge being chosen by all the ants.  Local trail updating is also motivated by trail evaporation in real ants. While traveling, each ant applies the local update rule for each visited edge.

36. Travelling Salesman Problem ACS Algorithm description ACS solution Update the weight of each edge that belongs to the shortest tour, according to the following formula:  Global updating is intended to reward edges belonging to shorter tours.  The amount of weight deposited on each edge is inversely proportional to the length of the tour: The shorter the tour the greater the amount of weight deposited on edges. While traveling, each ant applies the local update rule for each visited edge. Once ants have completed their tours, apply the global update rule.

37. Travelling Salesman Problem ACS Algorithm description ACS solution  Global updating is intended to emulate the property of differential pheromone trail accumulation, which in the case of real ants was due to the interplay between length of the path and continuity of time. Update the weight of each edge that belongs to the shortest tour, according to the following formula: While traveling, each ant applies the local update rule for each visited edge. Once ants have completed their tours, apply the global update rule.

38. Travelling Salesman Problem ACS Algorithm description ACS solution While traveling, each ant applies the local update rule for each visited edge. Once ants have completed their tours, apply the global update rule.

39. Travelling Salesman Problem ACS Algorithm description ACS solution While traveling, each ant applies the local update rule for each visited edge. Once ants have completed their tours, apply the global update rule. Return shortest tour.

40. Travelling Salesman Problem For an undirected complete weighted graphs Algorithm – Initialization A B C D E

41. Travelling Salesman Problem For an undirected complete weighted graphs Algorithm – Initialization Place each ant in a randomly chosen city. A B C D E

42. A B C D E Travelling Salesman Problem For an undirected complete weighted graphs Algorithm – Step 1

43. A B C D E Travelling Salesman Problem For an undirected complete weighted graphs Algorithm – Step 2 Apply the Global Update Rule.

44. Travelling Salesman Problem For an undirected complete weighted graphs Comparison with other algorithms Comparison of ACS tour length on a TSP problem with 50 vertexes with the following nature-inspired algorithms:  SA – Simulated Annealing  EN – Elastic Net  SOM – Self Organizing Map  FI – Farthest Insertion Heuristic

45. 5.86 5.88 5.98 6.06 6.03 ACS SA EN SOM FI Best Travelling Salesman Problem For an undirected complete weighted graphs Comparison with other algorithms Test A

46. Travelling Salesman Problem For an undirected complete weighted graphs Comparison with other algorithms 6.05 6.01 6.03 6.25 6.28 ACS SA EN SOM FI Test B Best

47. Travelling Salesman Problem For an undirected complete weighted graphs Comparison with other algorithms 5.7 5.81 5.86 5.87 5.96 ACS SA EN SOM FI Test C Best

48. Travelling Salesman Problem For an undirected complete weighted graphs Comparison with other algorithms 6.17 6.33 6.49 6.7 6.71 ACS SA EN SOM FI Test D Best

49. Travelling Salesman Problem For an undirected complete weighted graphs Comparison with other algorithms 6.05 6.01 6.03 6.25 6.28 ACS SA EN SOM FI Test E Best

50. Travelling Salesman Problem For an undirected complete weighted graphs Number of ants 0.05 0.1 0.13 0.15 0.16 0.18 0.2 0.22 0.23 1 23 4 56 7 89

51. Travelling Salesman Problem For an undirected complete weighted graphs An Intuitive proof of correctness  We can interpret the ant colony as a reinforcement learning system, in which reinforcements modify the strength (Pheromone trail) of connections between vertexes.  Each ant has two options: Exploit the experience accumulated by the ant colony in the form of pheromone trail (pheromone trail will tend to grow on those edges which belong to short tours, making them more desirable).

52. Reference Marco Dorigo IRIDIA, Université Libre de Bruxelles, Avenue Franklin Roosevelt 50, CP 194/6, 1050 Bruxelles, Belgium Luca Maria Gambardella IDSIA, Corso Elvezia 36, 6900 Lugano, Switzerland Marco Dorigo is a Research Associate with the FNRS. Luca Gambardella is Research Director at IDSIA. This research has been funded by the Swiss National Science Fund, contract 21–45653.95 titled “Cooperation and learning for combinatorial optimization”. “Ant colonies for the traveling salesman problem” Université Libre de Bruxelles, Belgium.