1. Selected Topics in CI I Ant Colony Optimization Dr. Widodo Budiharto 2015

2. Agenda Introduction to Ant Colony Optimization Ant Algorithm Demo Exercise Bina Nusantara University 2

3. Implementation of ACO Shortest route Bina Nusantara University 3

4. An Ant Colony Optimization(ACO) algorithm is an artificial intelligence technique based on the pheromone- laying behavior of ants It can be used to find solutions to exceedingly complex problems that seek the optimal path through a graph. Bina Nusantara University 4 Introduction

5. Ants are social insects, living in colonies of 30 to millions of individuals. Ants are not the only social insects living in colonies. The complex behaviors that emerge from colonies of ants have intrigued humans, and there have been many studies of ant colonies aimed at a better understanding of these collective behaviors. One of the first behaviors studied by entomologists was the ability of ants to find the shortest path between their nest and a food source. Bina Nusantara University 5 Introduction

6. Pheromones Pheromones are chemicals ants place on their trails; they attract other ants. More ants will travel on a shorter trail to a food source— and deposit more pheromones—than on longer trails. The pheromones slowly evaporate over time. Bina Nusantara University 6

7. Foraging Behavior of Ants How do ants find the shortest path between their nest and food source, without any visible, central, active coordination mechanisms? Studies of the foraging behavior of several species of real ants revealed an initial random or chaotic activity pattern in the search for food. As soon as a food source is located, activity patterns become more organized with more and more ants following the same path to the food source. “Auto- magically”, soon all ants follow the same, shortest path. Bina Nusantara University 7

8. The recruitment mechanism differs for different species, and can either be in the form of direct contact, or indirect “communication.” Most ant species use the latter form of recruitment, where communication is via pheromone trails. When an ant locates a food source, it carries a food item to the nest and lays pheromone along the trail. Bina Nusantara University 8 Foraging Behavior of Ants

9. ACO is a meta-heuristic, meaning that it’s a general framework that can be used to create a specific algorithm to solve a specific graph path problem. Although ACO was proposed in a 1991 doctoral thesis by M. Dorigo, the first detailed description of the algorithm is generally attributed to a 1996 follow-up paper by M. Dorigo, V. Maniezzo and A. Colorni. Since then, ACO has been widely studied and modified, but, somewhat curiously, very few complete and correct implementations are available online. Bina Nusantara University 9 Introduction

10. Binary Bridge Experiment Forager ants decide which path to follow based on the pheromone concentrations on the different paths. Paths with a larger pheromone concentration have a higher probability of being selected. Bina Nusantara University 10

11. Initially, both branches were free of any pheromones. After a finite time period, one of the branches was selected, with most of the ants following the path, even with both branches being of the same length. The selection of one of the branches is due to random fluctuations in path selection, causing higher concentrations on the one path. From this experiment, referred to as the binary bridge experiment, a simple formal model was developed to characterize the path selection process. Bina Nusantara University 11 Binary Bridge Experiment

12. Let n A (t) and n B (t) denote the number of ants on paths A and B respectively at time step t. Pasteels et al. found empirically that the probability of the next ant to choose path A at time step t + 1 is given as: where c quantifies the degree of attraction of an unexplored branch, and α is the bias to using pheromone deposits in the decision process. Bina Nusantara University 12

13. Introduction Bina Nusantara University 13

14. The larger the value of c, the more pheromone deposits are required to make the choice of path non-random. It was found empirically that α ≈ 2 and c ≈ 20 provide a best fit to the experimentally observed behavior. Using the probability defined in equation (17.1), the decision rule of an ant that arrives at the binary bridge is expressed as follows: if U(0, 1) ≤ P A (t+1) then follow path A otherwise follow path B. Bina Nusantara University 14

15. Goss et al. found that the probability of selecting the shorter path increases with the length ratio between the two paths. This has been referred to as the differential path length effect by Dorigo et al. Although an ant colony exhibits complex adaptive behavior, a single ant follows very simple behaviors. An ant can be seen as a stimulus–response agent [629]: the ant observes pheromone concentrations and produces an action based on the pheromonestimulus. An ant can therefore abstractly be considered as a simple computational agent. Bina Nuantara University 15

16. Artificial Ant Decision Process The algorithm is executed at each point where the ant needs to make a decision. Bina Nusantara University 16

17. Stigmergy and Artificial Pheromone Stigmergy is a class of mechanisms that mediate animal- to-animal interactions. The term stigmergy was formally defined by Grass´e as a form of indirect communication mediated by modifications of the environment. This definition originated from observations of the nest- building behavior of the termite species Bellicositermes natalensis and Cubitermes. Grass´e observed that coordination and regulation of nest-building activities are not on an individual level, but achieved by the current nest structure. Bina Nusantara University 17

18. The word stigmergy is aptly constructed from the two Greek words stigma, which means sign, and ergon, which means work. Ant algorithms are population-based systems inspired by observations of real ant colonies. Cooperation among individuals in an ant algorithm is achieved by exploiting the stigmergic communication mechanisms observed in real ant colonies. Bina Nusantara University 18

19. Individuals observe signals, which trigger a specific response or action. The action may reinforce or modify signals to influence the actions of other individuals. Two forms of stigmergy have been defined: sematectonic and signbased. Sematectonic stigmergy refers to communication via changes in the physical characteristics of the environment. Example activities that are accomplished through sematectonic stigmergy include nest building, nest cleaning, and brood sorting. Bina Nusantara University 19

20. Simple Ant Colony Optimization The first ant algorithm developed was the ant system, and since then several improvements of the ant system have been devised.These algorithms have somewhat more complex decision processes than that illustrated in Algorithm 17.1. To provide a gentle, didactic introduction to ant algorithms, this section breaks the chronological order in which ant algorithms have been developed to first present the simple ACO (SACO). Bina Nusantara University 20

21. The general problem of finding the shortest path between two nodes on a graph, G = (V,E), where V is the set of vertices (nodes) and E is a matrix representing the connections between nodes. The graph has n G = |V | nodes. The length, L k, of the path constructed by ant k is calculated as the number of hops in the path from the origin to the destination node. Bina Nusantara University 21 Simple Ant Colony Optimization

22. Any of a number of termination criteria can be used in Algorithm 17.2 (and for the rest of the ant algorithms discussed later), for example, terminate when a maximum number of iterations, nt, has been exceeded; terminate when an acceptable solution has been found, with terminate when all ants (or most of the ants) follow the same path. Bina Nusantara University 22

23. If ant k is currently located at node i, it selects the next node based on the transition probability Where is the set of feasible nodes connected to node i, with respect to ant k. If, for any node i and ant k, = ∅, then the predecessor to node i is included in Note that this may cause loops to occur within constructed paths. These loops are removed once the destination node has been reached. Bina Nusantara University 23

24. ACO Algorithm Bina Nusantara University 24

25. In this case, the demo is solving an instance of the TSP with the goal of finding the shortest path that visits each of 60 cities exactly once. The demo program uses four ants; each ant represents a potential solution. ACO requires the specification of several parameters such as the pheromone influence factor (alpha) and the pheromone evaporation coefficient (rho). The four ants are initialized to random trails through the 60 cities; after initialization, the best ant has a shortest trail length of 245.0 units. Bina Nusantara University 25

26. The main processing loop alternates between updating the ant trails based on the current pheromone values and updating the pheromones based on the new ant trails. After the maximum number of times (1,000) through the main processing loop, the program displays the best trail found and its corresponding length (61.0 units). Bina Nusantara University 26

27. The 60-city graph is artificially constructed. There’s no easy way to solve the TSP. With 60 cities, assuming you can start at any city and go either forward or backward, and that all cities are connected, there are a total of (60 - 1)! / 2 = 69,341,559,272,844,917,868,969,509,860,194,703,172, 951,438,386,343,716,270,410,647,470,080,000,000,000,000 possible solutions. 2.2 * 1063 years !!! Bina Nusantara University 27

28. Make a graph distance static int[][] MakeGraphDistances(int numCities) { int[][] dists = new int[numCities][]; for (int i = 0; i < dists.Length; ++i) dists[i] = new int[numCities]; for (int i = 0; i < numCities; ++i) for (int j = i + 1; j < numCities; ++j) { int d = random.Next(1, 9); // [1,8] dists[i][j] = d; dists[j][i] = d; } return dists; } Bina Nusantara University 28

29. Pheromones are chemicals ants place on their trails; they attract other ants. More ants will travel on a shorter trail to a food source— and deposit more pheromones—than on longer trails. The pheromones slowly evaporate over time. Here’s method Bina Nusantara University 29

30. example The trail starts at city 1, then goes to city 3, and the update algorithm is determining the next city. Now suppose the pheromone and distance information is as shown in the image. The first step in determining the next city is constructing an array I’ve called “taueta” (because the original research paper used the Greek letters tau and eta). The taueta value is the value of the pheromone on the edge raised to the alpha power, times one over the distance value raised to the beta power. Bina Nusantara University 30

31. Recall that alpha and beta are global constants that must be specified. Here I’ll assume that alpha is 3 and beta is 2. The taueta values for city 1 and city 3 aren’t computed because they’re already in the current trail. Notice that larger values of the pheromone increase taueta, but larger distances decrease taueta. After all the taueta values have been computed, the next step is to convert those values to probabilities and place them in an array I’ve labeled probs. The algorithm sums the taueta values, getting 82.26 in this example, and then divides each taueta value by the sum. Bina Nusantara University 31

32. City 0 has a probability of 0.09 of being selected and so on. Next, the algorithm needs to select the next city based on the computed probabilities. The ACO algorithm I’m presenting in this article uses a neat technique called roulette wheel selection. The size of the augmented array is one greater than the probs array, and cell [0] is seeded with 0.0. Each cell in cumul is the cumulative sum of the probabilities. After the cumul array has been constructed, a random number p between 0.0 and 1.0 is generated. Suppose p = 0.538 as shown. That p value falls between the values at [2] and [3] in the cumul array, which means that [2], or city 2, is selected as the next city. Bina Nusantara University 32

33. Bina Nusantara University 33

34. Updating pheromone information is much easier than updating the ant trail information. The key lines of code in method UpdatePhermones are: double length = Length(ants[k], dists); double decrease = (1.0 - rho) * pheromones[i][j]; double increase = 0.0; if (EdgeInTrail(i, j, ants[k]) == true) increase = (Q / length); pheromones[i][j] = decrease + increase; Bina Nusantara University 34

35. Bina Nusantara University 35

36. Presentation (each 15 minutes) Ant colonies for the traveling salesman problem Bina Nusantara University 36 WAWAN YOGI

37. Adrian Ayu Bina Nusantara University 37 Ant colony Optimization: A Solution of Load balancing in Cloud