Artificial Intelligence Search Methodologies Dr Rong Qu School of Computer Science University of Nottingham Nottingham, NG8 1BB, UK Population Based Algorithms

Konstanz, May 2014 AI Search Algorithms – Population Based2 Optimisation Problems : Methods Meta-heuristics Guide an underlying heuristic/search to escape from being trapped in a local optima and to explore better areas of the solution space Single solution approaches Simulated Annealing, Tabu Search, Variable Neighbourhood Search, etc.; Population based approaches Genetic algorithm, Memetic algorithm, Ant Algorithms, Particle Swarm Intelligence, etc.;

AI Search Algorithms – Population Based3 Population Based Algorithms Local search Concerning only one solution at a particular time during the search Search is very much restricted to local regions, so called local Population based algorithms concern a population of solutions at a time Konstanz, May 2014

GENETIC ALGORITHMS Konstanz, May 2014 AI Search Algorithms – Population Based4 Charles Darwin

Konstanz, May 2014 AI Search Algorithms – Population Based5 GA Algorithm – basic idea Based on survival of the fittest Algorithm uses terms from genetics: population, chromosome and gene Developed extensively by John Holland in mid 70’s Three modules the evaluation module, the population module and the reproduction module Solutions (individuals) often coded as bit strings

Konstanz, May 2014 AI Search Algorithms – Population Based6 GA Algorithm – basic idea 1859 Origin of the Species Survival of the Fittest

Konstanz, May 2014 AI Search Algorithms – Population Based7 GA Algorithm – basic idea 1975 Genetic Algorithms Artificial Survival of the Fittest

Konstanz, May 2014 AI Search Algorithms – Population Based8 GA Algorithm – basic idea 1989 Genetic Algorithms Foundations and Applications

Konstanz, May 2014 AI Search Algorithms – Population Based9 GA Algorithm – basic steps Initial population Evaluations on individuals Breeding Choose suitable parents (proportion to evaluation rating) Produce two offspring (Probability of breeding) Mutation Domain knowledge – evaluation function

Konstanz, May 2014 AI Search Algorithms – Population Based10 GA Algorithm – basic steps 1. Initialise a population of chromosomes Ci 2. Evaluate each Ci (individual) in the population Create new C by using Ci in the current population (using crossover and mutation) Delete members of the existing population to make way for the new members Evaluate the new members and insert them into the population Repeat (evolve) until some termination condition is reached (normally based on time or number of populations produced) 3. Return the best Ci as the solution

Konstanz, May 2014 AI Search Algorithms – Population Based11 GA Algorithm – basic steps Generate Initial Population Generate Initial Population Generation 'n' Population Generation 'n' Crossover Population Crossover Population Mutate Population Mutate Population n = n + 1 n = 1 Final Population Final Population Selection n<20?

Konstanz, May 2014 AI Search Algorithms – Population Based12 GA Algorithm – encoding The decision variables of a problem are normally encoded into a finite length string This could be a binary string or a list of integers For example : or We could also represent numbers as coloured boxes

Konstanz, May 2014 AI Search Algorithms – Population Based13 Evaluation Module Responsible for evaluating a chromosome Only part of the GA that has domain knowledge. The rest of the GA modules are simply operating on (typically) bit strings with no information about the problem A different evaluation module is needed for each problem

Konstanz, May 2014 AI Search Algorithms – Population Based14 Population Module Responsible for maintaining the population Initialization Random Known Solutions Heuristics Population Size Elitism

Konstanz, May 2014 AI Search Algorithms – Population Based15 Population Module Deletion Delete-All : replaces all the members of the current population by the same number of chromosomes created Steady-State : replaces n old members by n new members. But: replace the worst, random or the parents individuals? Steady-State-No-Duplicates : Same as steady- state but also checks that no duplicate chromosomes are added to the population.

Konstanz, May 2014 AI Search Algorithms – Population Based16 Reproduction Module Parent selection Fitness techniques Crossover & mutation

Konstanz, May 2014 AI Search Algorithms – Population Based17 Parent Selection Roulette Wheel Selection Select the parents with a probability in proportion to their fitness Tournament Select two individuals at random. The individual with the highest evaluation becomes the parent. Repeat to find a second parent

Konstanz, May 2014 AI Search Algorithms – Population Based18 Fitness Techniques Fitness-Is-Evaluation : Simply use the fitness of the chromosome equal to its evaluation Linear Normalization : The chromosomes are sorted by decreasing the evaluation value. Then the chromosomes are assigned a fitness value that starts with a constant value and decreases linearly. The initial value and the decrement are parameters to the techniques

Konstanz, May 2014 AI Search Algorithms – Population Based19 Crossover Operators Order based crossover Cycle crossover Partially matched crossover

Konstanz, May 2014 AI Search Algorithms – Population Based20 Mutation A method of ensuring premature convergence does not occur Usually set to a small value Dynamic mutation and crossover rates

Konstanz, May 2014 AI Search Algorithms – Population Based21 Example I Crossover probability, P C = 1.0 Mutation probability, P M = 0.0 Maximise f(x) = x * x * x = 31 x can be represented using five binary digits

Konstanz, May 2014 AI Search Algorithms – Population Based22 Example I Generate random initial individuals Maximise f(x) = x^ * x^ * x +100 (0 <= x <= 31) chromosomebinary stringxf(x) P P P P Total7718 Average

Konstanz, May 2014 AI Search Algorithms – Population Based23 Example I Choose Parents, using roulette wheel selection Crossover point, 1, is chosen randomly Roulette wheelParents 4116P3 1915P P3P3 P2P C1C1 C2C P4P4 P2P C3C3 C4C4

Konstanz, May 2014 AI Search Algorithms – Population Based24 Example I New generation chromosomebinary stringxf(x) P P P P Total11735 Average Maximise f(x) = x^ * x^ * x +100 (0 <= x <= 31)

Konstanz, May 2014 AI Search Algorithms – Population Based25 Example I chromosomebinary string xf(x) P P P P Total7718 Average chromosomebinary string xf(x) P P P P Total11735 Average Two generations Mutation Maximise f(x) = x^ * x^ * x +100 (0 <= x <= 31) What problem do you see with the populations? what chance is there of finding the global optimum?

Konstanz, May 2014 AI Search Algorithms – Population Based26 GA - performance There are a number of factors which affect the performance of a genetic algorithm The size of the population The initial population Selection pressure (elitism, tournament) The cross-over probability The mutation probability Defining convergence Local optimisation

Konstanz, May 2012 AI Search Algorithms – Population Based27 GA - applications Combinatorial optimisation problems Cutting and packing problems vehicle routing problems job shop scheduling

Konstanz, May 2012 AI Search Algorithms – Population Based28 GA - applications Combinatorial optimisation problems portfolio optimization multimedia multicast routing knapsack problem

ANT ALGORITHMS Konstanz, May 2014 AI Search Algorithms – Population Based29 Ants are practically blind but they still manage to find their way to and from food. How do they do it?

Konstanz, May 2014 AI Search Algorithms – Population Based30 Ant Algorithms Ant systems are a population based approach. In this respect it is similar to genetic algorithms There is a population of ants, with each ant finding a solution and then communicating with the other ants

Konstanz, May 2014 AI Search Algorithms – Population Based31 Ant Algorithms A B C H D F E G

Konstanz, May 2014 AI Search Algorithms – Population Based32 Ant Algorithms A B C D F E d=0.5 d=1

Konstanz, May 2014 AI Search Algorithms – Population Based33 Ant Algorithms Time, t, is discrete At each time unit an ant moves a distance, d of 1 Once an ant moved, it lays down 1 unit of pheromone At t = 0, there is no pheromone on any edge

Konstanz, May 2014 AI Search Algorithms – Population Based34 Ant Algorithms At t=1 there will be 16 ants at B and 16 ants at D. At t=2 there will be 8 ants at D and 8 ants at B. There will be 16 ants at E The intensities on the edges will be as follows FD = 16, AB = 16, BE = 8, ED = 8, BC = 16 and CD = 16 A B C D F E ants are moving from A - F and another 16 are moving from F - A

Konstanz, May 2014 AI Search Algorithms – Population Based35 Ant Algorithms We need to allow the ants to explore paths and follow the best paths with some probability in proportion to the intensity of the pheromone trail We do not want them simply to follow the route with the highest amount of pheromone on it, else our search will quickly settle on a sub-optimal (and probably very sub-optimal) solution

Konstanz, May 2014 AI Search Algorithms – Population Based36 Ant Algorithms The probability of an ant following a certain route is a function, not only of the pheromone intensity but also a function of what the ant can see (visibility) The pheromone trail must not build unbounded. Therefore, we need “evaporation”

Konstanz, May 2014 AI Search Algorithms – Population Based37 Ant Algorithms – initial ideas Dorigo (1996) Based on real world phenomena Ants, despite almost blind, are able to find their way to the food source using the shortest route If an obstacle is placed, ants have to decide which way to take around the obstacle.

Konstanz, May 2014 AI Search Algorithms – Population Based38 Ant Algorithms – initial ideas Dorigo (1996) Initially there is a probability as to which way they will turn Assume one route is shorter than the other Ants taking the shorter route will arrive at a point on the other side of the obstacle before the ants which take the longer route.

Konstanz, May 2014 AI Search Algorithms – Population Based39 Ant Algorithms – initial ideas Dorigo (1996) As ants walk they deposit pheromone trail. Ants have taken shorter route will have already laid trail So ants from the other direction are more likely to follow that route with deposit of pheromone.

Konstanz, May 2014 AI Search Algorithms – Population Based40 Ant Algorithms – initial ideas Dorigo (1996) Over a period of time, the shortest route will have high levels of pheromone. The quantity of pheromones accumulates faster on the shorter path than on the longer one There is positive feedback which reinforces that behaviors so that the more ants follow a particular route, the more desirable it becomes.

Konstanz, May 2014 AI Search Algorithms – Population Based41 Ant Algorithms

Konstanz, May 2014 AI Search Algorithms – Population Based42 Ant Algorithms - TSP At the start of the algorithm, one ant is placed in each city Time, t, is discrete. t(0) marks the start of the algorithm. At t+1 every ant will have moved to a new city Assuming that the TSP is represented as a fully connected graph, each edge has an intensity of trail on it. This represents the pheromone trail laid by the ants Let T i,j (t) represent the intensity of trail edge (i,j) at time t Variations have been tested by Dorigo

Konstanz, May 2014 AI Search Algorithms – Population Based43 Ant Algorithms - TSP An ant decides which town to move to next, with a probability that is based on the distance to that city AND the amount of trail intensity on the connecting edge The distance to the next town, is known as the visibility, n ij, defined as 1/d ij, d ij is the distance between cities i and j. At each time unit evaporation takes place, at a rate p, a value between 0 and 1 Variations have been tested by Dorigo

Konstanz, May 2014 AI Search Algorithms – Population Based44 Ant Algorithms - TSP In order to stop ants visiting the same city in the same tour a data structure, Tabu, is maintained Tabu k is defined as the list for the k th ant and it holds the cities that have already been visited Variations have been tested by Dorigo

Konstanz, May 2014 AI Search Algorithms – Population Based45 Ant Algorithms - TSP After each ant tour the trail intensity on edge (i,j) is updated using the following formula T ij (t + n) = p. T ij (t) + ΔT ij Q is a constant L k is the tour length of the k th ant p is the evaporation coefficient

Konstanz, May 2014 AI Search Algorithms – Population Based46 Ant Algorithms - TSP Transition Probability where  and  are control parameters that control the relative importance of trail versus visibility

Konstanz, May 2014 AI Search Algorithms – Population Based47 Ant Algorithms - TSP Left: Trail distribution at the beginning; Right: Trail distribution after 100 cycles. (Dorigo et al., 1996)

Konstanz, May 2014 AI Search Algorithms – Population Based48 Ant Algorithms - Applications Travelling Salesman Problem (TSP) Facility Layout Problem Vehicle Routing Stock Cutting … Marco Dorigo maintains a page devoted to the subject at contains information about ant algorithms as well as links to the main papers published on the subject

APPENDIX Konstanz, May 2014 AI Search Algorithms – Population Based49 Examples of Genetic Algorithms

Genetic Algorithm Example II Traveling Salesman Problem a number of cities costs of traveling between cities a traveling sales man needs to visit all these cities exactly once and return to the starting city What’s the cheapest route? Konstanz, May AI Search Algorithms – Population Based

Traveling Salesman Problem Konstanz, May AI Search Algorithms – Population Based Genetic Algorithm Example II

Initial generation 581…… P1P …… P2P2 817…… P 30 … Any idea of other ways to generate the initial population? Konstanz, May AI Search Algorithms – Population Based Genetic Algorithm Example II

Choose pairs of parents …… P2P2 817…… P Crossover …… C2C2 817…… C1C Konstanz, May AI Search Algorithms – Population Based Genetic Algorithm Example II

Next generation …… P2P2 817…… P1P … Konstanz, May AI Search Algorithms – Population Based 782…… P2P2 Genetic Algorithm Example II

Traveling Salesman Problem No. of cities: 100 Population size: 30 Cost: 6.37 Generation: 88 Cost: 6.34 Generation: 1100 Konstanz, May AI Search Algorithms – Population Based Genetic Algorithm Example II

Many applications of genetic algorithms. Best suited to problems where the efficient solutions are not already known. Strength of GA's: ability to heuristically search for solutions when all else fails. If you can represent the solutions to the problem in a suitable format, such as a series of 1's and 0's, then the GA will do the rest. Konstanz, May AI Search Algorithms – Population Based Genetic Algorithm Example III

Applying Genetic Algorithms to Personnel Scheduling Applying Genetic Algorithms to Personnel Scheduling Personnel scheduling in healthcare is usually a very complex operation which has a profound effect upon the efficient usage of expensive resources. Konstanz, May AI Search Algorithms – Population Based Genetic Algorithm Example III

A number of nurses A number of shifts each day A set of constraints shift coverage one shift per day resting time workload per month consecutive shifts working weekends … Konstanz, May AI Search Algorithms – Population Based Genetic Algorithm Example III

Konstanz, May AI Search Algorithms – Population Based Genetic Algorithm Example III

Genetic Algorithm - Initial population - construct rosters - repair infeasible ones Konstanz, May AI Search Algorithms – Population Based Genetic Algorithm Example III

Genetic Algorithm - Select parents - Recombine rows in the two rosters - repair infeasible ones + Konstanz, May AI Search Algorithms – Population Based Genetic Algorithm Example III

Genetic Algorithm - Mutation - Local optimiser Genetic Algorithm Example III

Population Size Crossover Probability Mutation Probability Local Optimiser ON Konstanz, May AI Search Algorithms – Population Based Genetic Algorithm Example III