1. traveling salesman problem
Ant colonies for the
traveling salesman problem

2. Content Pheromone trails Ant Colony Optimization algorithms
Travelling Salesman Problem
And its ACS solution

3. Content Pheromone trails Ant Colony Optimization algorithms
Travelling Salesman Problem
And its ACS solution

4. Content Pheromone trails Ant Colony Optimization algorithms
Travelling Salesman Problem
And its ACS solution

5. Content Pheromone trails Ant Colony Optimization algorithms
Travelling Salesman Problem
And its ACS solution

6. Pheromone trails
It is well known that the primary means for ants to form and maintain the line is a pheromone trail. Ants deposit a certain amount of pheromone while walking, and each ant probabilistically prefers to follow a direction rich in pheromone.
ℙ 𝐶 <ℙ 𝐵 <ℙ 𝐴
ℙ(𝐴)
ℙ(𝐵)
ℙ(𝐶)

7. Pheromone trails Shortest path around an obstacle
Shortest path around an obstacle
This elementary behavior of real ants can be used to explain how they can find the shortest path that reconnects a broken line after the sudden appearance of an unexpected obstacle has interrupted the initial path.
Let us consider the following scenario:
Ants are moving on a straight line that connects a food source to their nest.

8. Pheromone trails Shortest path around an obstacle
Shortest path around an obstacle
This elementary behavior of real ants can be used to explain how they can find the shortest path that reconnects a broken line after the sudden appearance of an unexpected obstacle has interrupted the initial path.
An obstacle appears on the path.

9. Pheromone trails Shortest path around an obstacle
Shortest path around an obstacle
This elementary behavior of real ants can be used to explain how they can find the shortest path that reconnects a broken line after the sudden appearance of an unexpected obstacle has interrupted the initial path.
Those ants which are just in front of the obstacle cannot continue to follow the pheromone trail and therefore they have to choose between turning right or left. In this situation we can expect half the ants to choose to turn right and the other half to turn left.

10. Pheromone trails Shortest path around an obstacle
Shortest path around an obstacle
This elementary behavior of real ants can be used to explain how they can find the shortest path that reconnects a broken line after the sudden appearance of an unexpected obstacle has interrupted the initial path.
Those ants which choose, by chance, the shorter
path around the obstacle will more rapidly reconstitute the interrupted pheromone trail
compared to those which choose the longer path. Thus, the shorter path will receive a greater amount of pheromone per time unit and in turn a larger number of ants will choose the shorter path.

11. Pheromone trails Shortest path around an obstacle
Shortest path around an obstacle
This elementary behavior of real ants can be used to explain how they can find the shortest path that reconnects a broken line after the sudden appearance of an unexpected obstacle has interrupted the initial path.
Shortest path is being obtained.

12. Pheromone trails Shortest path from the nest to the food source
Shortest path from the nest to the food source
Ants are able, without using any spatial Information, to identify a sudden appearance of a food source around their nest, and to find the shortest available path to it.
Let us describe the algorithm:
A small amount of ants travel randomly around the nest. N

13. Pheromone trails Shortest path from the nest to the food source
Shortest path from the nest to the food source
Ants are able, without using any spatial Information, to identify a sudden appearance of a food source around their nest, and to find the shortest available path to it.
One of the ants find food source. S N

14. Pheromone trails Shortest path from the nest to the food source
Shortest path from the nest to the food source
Ants are able, without using any spatial Information, to identify a sudden appearance of a food source around their nest, and to find the shortest available path to it.
When ant finds food, it returns to the nest while laying down pheromones trail. S N

15. Pheromone trails Shortest path from the nest to the food source
Shortest path from the nest to the food source
Ants are able, without using any spatial Information, to identify a sudden appearance of a food source around their nest, and to find the shortest available path to it.
When other ants find a pheromone trail, they are likely not to keep travelling at random, but to instead follow the trail. S N

16. Pheromone trails Shortest path from the nest to the food source
Shortest path from the nest to the food source
Ants are able, without using any spatial Information, to identify a sudden appearance of a food source around their nest, and to find the shortest available path to it.
If an ant eventually find food by following a pheromone trail, it returning to the nest while reinforcing the trail with more pheromones. S N

17. Pheromone trails Shortest path from the nest to the food source
Shortest path from the nest to the food source
Ants are able, without using any spatial Information, to identify a sudden appearance of a food source around their nest, and to find the shortest available path to it.
Due to their stochastic behavior, some ants are not following the pheromone trails, and thus uncover more possible paths. S N

18. Pheromone trails Shortest path from the nest to the food source
Shortest path from the nest to the food source
Ants are able, without using any spatial Information, to identify a sudden appearance of a food source around their nest, and to find the shortest available path to it.
Over time, however, the pheromones trails starts to evaporate, thus reducing its attractive strength. S N

19. Pheromone trails Shortest path from the nest to the food source
Shortest path from the nest to the food source
Ants are able, without using any spatial Information, to identify a sudden appearance of a food source around their nest, and to find the shortest available path to it.
Shortest path is being obtained. S N

20. Pheromone trails Shortest path from the nest to the food source
Shortest path from the nest to the food source  
An Intuitive proof of correctness
The more time it takes for an ant to travel down the path and back again, the more time the pheromones have to evaporate.
Thus,
A short path, by comparison, get marched over more frequently, and thus its pheromone density becomes higher on shorter paths than longer ones.
Pheromone evaporation also has the advantage of avoiding the convergence to a locally optimal solution. If there were no evaporation at all, all paths chosen by the first ants would tend to be excessively attractive to the following ones. In that case, the exploration of the solution space would be constrained.
Thus, when one ant finds a short path from the colony to a food source, other ants are more likely to follow that path, and positive feedback eventually leads all the ants following a single path.

21. Ant Colony Optimization algorithms
Ant Colony Optimization (ACO) studies artificial systems that take inspiration from the behavior of real ant colonies and which are used to solve discrete optimization problems.
Some applications:
Set partition problem
Deciding whether a given multiset of positive integers can be partitioned into two subsets A and B such that the sum of the numbers in A equals the sum of the numbers in B.

22. Ant Colony Optimization algorithms
Ant Colony Optimization (ACO) studies artificial systems that take inspiration from the behavior of real ant colonies and which are used to solve discrete optimization problems.
Some applications:
Job – Shop problems
Given a number of jobs have to be done and every job consists of using a number of machines for a certain amount of time. Find the best planning to do all the jobs on all the different machines in the shortest period of time.

23. Ant Colony Optimization algorithms
Ant Colony Optimization (ACO) studies artificial systems that take inspiration from the behavior of real ant colonies and which are used to solve discrete optimization problems.
Some applications:
Multiple knapsack problem
Given a set of items, each with a weight and a value, determine the number of each item to include in a collection so that the total weight is less than or equal to a given limit and the total value is as large as possible.

24. Ant Colony Optimization algorithms
Ant Colony Optimization (ACO) studies artificial systems that take inspiration from the behavior of real ant colonies and which are used to solve discrete optimization problems.
Some applications:
Protein folding
The process by which a protein structure
assumes its functional shape or conformation.

25. Ant Colony Optimization algorithms
Ant Colony Optimization (ACO) studies artificial systems that take inspiration from the behavior of real ant colonies and which are used to solve discrete optimization problems.
Some applications:
Data mining
the computational process of discovering patterns in large data sets.

26. Ant Colony Optimization algorithms
Ant Colony Optimization (ACO) studies artificial systems that take inspiration from the behavior of real ant colonies and which are used to solve discrete optimization problems.
Some applications:
Travelling salesman problem
Given a list of cities and the distances between each pair of cities, determine the shortest possible route that visits each city exactly once and returns to the origin city.

27. Travelling Salesman Problem
For a complete undirected weighted graphs
Definitions
Undirected graph 𝐺 𝑉,𝐸 is complete if for every pear of vertexes 𝑣 1 , 𝑣 2 ∈𝑉 there exist an edge (𝑣 1 , 𝑣 2 )∈𝐸.
Path 𝑃 in 𝐺 is a Hamiltonian path if 𝑃 visits each vertex exactly once.
Problem
Given a complete undirected weighted graph G V,E , find the minimal Hamiltonian cycle in G.
Complexity class
NP–Hard

28. Travelling Salesman Problem
For an undirected complete weighted graphs
Algorithms for deterministic solution
Brute force search
Compute all possible cycles and get the minimal path.
Time complexity: 𝑂 𝑉 !
Exact solution
Dynamic programming
Recursive algorithms such as the Held–Karp algorithm.
Time complexity: 𝑂 𝑉 𝑉
Exact solution
Greedy algorithm
Such as the nearest neighbor (NN) algorithm.
Time complexity: 𝑂 𝑙𝑜𝑔 𝑉
Approximate solution

29. Travelling Salesman Problem
ACS solution
The Ant Colony System (ACS) is one of the ACO algorithms, which gives a non – deterministic approximate solution to the TSP problem with a time complexity of 𝑂 𝑉 3 ∙𝑐 , where 𝑐 is a constant.

30. Travelling Salesman Problem
ACS solution
ACS Algorithm description
Given a complete undirected weighted graph G V,E :
Place 𝑘≤𝑛 ants in a randomly chosen vertexes.

31. Travelling Salesman Problem
ACS solution
ACS Algorithm description
Given a complete undirected weighted graph G V,E :
Place 𝑘≤𝑛 ants in a randomly chosen vertexes.
Each ant preforms a Hamiltonian cycle according to the navigation function 𝐹 𝑠 .
Ant 𝑖 endowed with a working memory 𝑀 𝑘 𝑖 , which is used to memorize vertexes already visited. At initialization, 𝑀 𝑘 𝑖 =∅ for every 1≤𝑖≤𝑘, and when ant 𝑖 arrives to vertex 𝑣, vertex 𝑣 is being added to 𝑀 𝑘 𝑖 .

32. Travelling Salesman Problem
ACS solution
ACS Algorithm description
Given a complete undirected weighted graph G V,E :
Place 𝑘≤𝑛 ants in a randomly chosen vertexes.
Each ant preforms a Hamiltonian cycle according to the navigation function 𝐹 𝑠 .
An ant 𝑖 in city 𝑟 chooses the city 𝑠 to move to among those which do not belong to its working memory 𝑀 𝑘 𝑖 by applying the following probabilistic formula:
𝜏 𝑟,𝑢 - Amount of pheromone trail on edge 𝑟,𝑢
𝜂 𝑟,𝑢 - A heuristic function which was chosen to be the inverse of the distance between iiiiiiiiiiiiiiivertexes 𝑟 and 𝑢.
𝛽 - A parameter which weighs the relative importance of pheromone trail and of iiiiiiiiiiiiiiicloseness.
𝑓 𝑥 = arg max 𝑢∉ 𝑀 𝑘 𝜏 𝑟,𝑢 ∙ 𝜂 𝛽 𝑟,𝑢 𝑞< 𝑞 &𝑆 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

33. Travelling Salesman Problem
ACS solution
ACS Algorithm description
Given a complete undirected weighted graph G V,E :
Place 𝑘≤𝑛 ants in a randomly chosen vertexes.
Each ant preforms a Hamiltonian cycle according to the navigation function 𝐹 𝑠 .
An ant 𝑖 in city 𝑟 chooses the city 𝑠 to move to among those which do not belong to its working memory 𝑀 𝑘 𝑖 by applying the following probabilistic formula:
𝑞 - A value chosen randomly with uniform probability in 0,1 .
𝑞 0 - A parameter in 0,1 .
𝑓 𝑥 = arg max 𝑢∉ 𝑀 𝑘 𝜏 𝑟,𝑢 ∙ 𝜂 𝛽 𝑟,𝑢 𝑞< 𝑞 &𝑆 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

34. Travelling Salesman Problem
ACS solution
ACS Algorithm description
Given a complete undirected weighted graph G V,E :
Place 𝑘≤𝑛 ants in a randomly chosen vertexes.
Each ant preforms a Hamiltonian cycle according to the navigation function 𝐹 𝑠 .
An ant 𝑖 in city 𝑟 chooses the city 𝑠 to move to among those which do not belong to its working memory 𝑀 𝑘 𝑖 by applying the following probabilistic formula:
𝑓 𝑥 = arg max 𝑢∉ 𝑀 𝑘 𝜏 𝑟,𝑢 ∙ 𝜂 𝛽 𝑟,𝑢 𝑞< 𝑞 &𝑆 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
𝑆 - A random variable selected according to the following probability distribution, which IIIIIfavors edges which are shorter and have a higher level of pheromone trail:
ℙ 𝑘 𝑟,𝑠 = 𝜏 𝑟,𝑠 ∙ 𝜂 𝛽 𝑟,𝑠 𝑢∉ 𝑀 𝑘 𝜏 𝑟,𝑢 ∙ 𝜂 𝛽 𝑟,𝑢 𝑠∉ 𝑀 𝑘 & 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

35. Travelling Salesman Problem
ACS solution
ACS Algorithm description
Given a complete undirected weighted graph G V,E :
Place 𝑘≤𝑛 ants in a randomly chosen vertexes.
Each ant preforms a Hamiltonian cycle according to the navigation function 𝐹 𝑠 .
While traveling, each ant applies the local update rule for each visited edge.
Every time an edge is chosen by an ant, its amount of pheromone is changed according to the following formula:
𝜏 𝑟,𝑠 ⟵ 1−𝛼 ∙𝜏 𝑟,𝑠 +𝛼∙ 𝜏 0
𝜏 0 ,𝛼 - Parameters
Local updating is intended to avoid a very strong edge being chosen by all the ants.
Local trail updating is also motivated by trail evaporation in real ants.

36. Travelling Salesman Problem
ACS solution
ACS Algorithm description
Given a complete undirected weighted graph G V,E :
Place 𝑘≤𝑛 ants in a randomly chosen vertexes.
Each ant preforms a Hamiltonian cycle according to the navigation function 𝐹 𝑠 .
While traveling, each ant applies the local update rule for each visited edge.
Once ants have completed their tours, apply the global update rule.
Update the weight of each edge that belongs to the shortest tour, according to the following formula:
𝜏 𝑟,𝑠 ⟵ 1−𝛼 ∙𝜏 𝑟,𝑠 +𝛼∙ 𝐿 𝑚𝑖𝑛 −1
𝐿 𝑚𝑖𝑛 - The length of the shortest tour.
Global updating is intended to reward edges belonging to shorter tours.
The amount of weight deposited on each edge is inversely proportional to the length of the tour: The shorter the tour the greater the amount of weight deposited on edges.

37. Travelling Salesman Problem
ACS solution
ACS Algorithm description
Given a complete undirected weighted graph G V,E :
Place 𝑘≤𝑛 ants in a randomly chosen vertexes.
Each ant preforms a Hamiltonian cycle according to the navigation function 𝐹 𝑠 .
While traveling, each ant applies the local update rule for each visited edge.
Once ants have completed their tours, apply the global update rule.
Update the weight of each edge that belongs to the shortest tour, according to the following formula:
𝜏 𝑟,𝑠 ⟵ 1−𝛼 ∙𝜏 𝑟,𝑠 +𝛼∙ 𝐿 𝑚𝑖𝑛 −1
𝐿 𝑚𝑖𝑛 - The length of the shortest tour.
Global updating is intended to emulate the property of differential pheromone trail accumulation, which in the case of real ants was due to the interplay between length of the path and continuity of time.

38. Travelling Salesman Problem
ACS solution
ACS Algorithm description
Given a complete undirected weighted graph G V,E :
Place 𝑘≤𝑛 ants in a randomly chosen vertexes.
Each ant preforms a Hamiltonian cycle according to the navigation function 𝐹 𝑠 .
While traveling, each ant applies the local update rule for each visited edge.
Once ants have completed their tours, apply the global update rule.
𝑐 times

39. Travelling Salesman Problem
ACS solution
ACS Algorithm description
Given a complete undirected weighted graph G V,E :
Place 𝑘≤𝑛 ants in a randomly chosen vertexes.
Each ant preforms a Hamiltonian cycle according to the navigation function 𝐹 𝑠 .
While traveling, each ant applies the local update rule for each visited edge.
Once ants have completed their tours, apply the global update rule.
𝑐 times
Return shortest tour.

40. Travelling Salesman Problem
For an undirected complete weighted graphs
Algorithm – Initialization
Choose constant 𝑘 between 1 and 𝑉 to be the number of ants. A B C D E
𝑘=3

41. Travelling Salesman Problem
For an undirected complete weighted graphs
Algorithm – Initialization
Place each ant in a randomly chosen city. A B C D E
𝑘=3

42. Travelling Salesman Problem
For an undirected complete weighted graphs
Algorithm – Step 1
Each ant preforms a Hamiltonian cycle according to the navigation function 𝐹 𝑠 , while applying the local update rule for each visited edge. A B C D E

43. Travelling Salesman Problem
For an undirected complete weighted graphs
Algorithm – Step 2
Apply the Global Update Rule. A B C D E

44. Travelling Salesman Problem
For an undirected complete weighted graphs
Comparison with other algorithms
Comparison of ACS tour length on a TSP problem with 50 vertexes with the following nature-inspired algorithms:
SA – Simulated Annealing
EN – Elastic Net
SOM – Self Organizing Map
FI – Farthest Insertion Heuristic

45. Travelling Salesman Problem
For an undirected complete weighted graphs
Comparison with other algorithms
Test A
5.86
5.88
5.98
6.06
6.03
ACS SA EN
SOM FI
Best

46. Travelling Salesman Problem
For an undirected complete weighted graphs
Comparison with other algorithms
Test B
6.05
6.01
6.03
6.25
6.28
Best
ACS SA EN
SOM FI

47. Travelling Salesman Problem
For an undirected complete weighted graphs
Comparison with other algorithms
Test C
Best
5.7
5.81
5.86
5.87
5.96
ACS SA EN
SOM FI

48. Travelling Salesman Problem
For an undirected complete weighted graphs
Comparison with other algorithms
Test D
Best
6.17
6.33
6.49
6.7
6.71
ACS SA EN
SOM FI

49. Travelling Salesman Problem
For an undirected complete weighted graphs
Comparison with other algorithms
Test E
Best
6.05
6.01
6.03
6.25
6.28
ACS SA EN
SOM FI

50. Travelling Salesman Problem
For an undirected complete weighted graphs
Number of ants
ACS runtime on a TSP problem with 10 vertexes for 1≤𝑘<10, where 𝑘 is the number of ants.
0.05
0.1
0.13
0.15
0.16
0.18
0.2
0.22
0.23 1 2 3 4 5 6 7 8 9 𝑘
𝑡𝑖𝑚𝑒 [𝑚𝑠𝑒𝑐]

51. Travelling Salesman Problem
For an undirected complete weighted graphs
An Intuitive proof of correctness
We can interpret the ant colony as a reinforcement learning system, in which reinforcements modify the strength (Pheromone trail) of connections between vertexes.
Each ant has two options:
Apply a biased exploration of new paths (exploration is biased towards short and high trail edges) by choosing the vertex to move to randomly, with a probability distribution that is a function of both the accumulated pheromone trail, the heuristic function, and the working memory 𝑀 𝑘 .
Exploit the experience accumulated by the ant colony in the form of pheromone trail (pheromone trail will tend to grow on those edges which belong to short tours, making them more desirable).