2. Artificial Intelligence Search Methodologies Dr Rong Qu School of Computer Science University of Nottingham Nottingham, NG8 1BB, UK rxq@cs.nott.ac.uk Population Based Algorithms

3. Konstanz, May 2014 AI Search Algorithms – Population Based2 Optimisation Problems : Methods Meta-heuristics Guide an underlying heuristic/search to escape from being trapped in a local optima and to explore better areas of the solution space Single solution approaches Simulated Annealing, Tabu Search, Variable Neighbourhood Search, etc.; Population based approaches Genetic algorithm, Memetic algorithm, Ant Algorithms, Particle Swarm Intelligence, etc.;

4. AI Search Algorithms – Population Based3 Population Based Algorithms Local search Concerning only one solution at a particular time during the search Search is very much restricted to local regions, so called local Population based algorithms concern a population of solutions at a time Konstanz, May 2014

5. GENETIC ALGORITHMS Konstanz, May 2014 AI Search Algorithms – Population Based4 Charles Darwin 1809 - 1882

6. Konstanz, May 2014 AI Search Algorithms – Population Based5 GA Algorithm – basic idea Based on survival of the fittest Algorithm uses terms from genetics: population, chromosome and gene Developed extensively by John Holland in mid 70’s Three modules the evaluation module, the population module and the reproduction module Solutions (individuals) often coded as bit strings

7. Konstanz, May 2014 AI Search Algorithms – Population Based6 GA Algorithm – basic idea 1859 Origin of the Species Survival of the Fittest

8. Konstanz, May 2014 AI Search Algorithms – Population Based7 GA Algorithm – basic idea 1975 Genetic Algorithms Artificial Survival of the Fittest

9. Konstanz, May 2014 AI Search Algorithms – Population Based8 GA Algorithm – basic idea 1989 Genetic Algorithms Foundations and Applications

10. Konstanz, May 2014 AI Search Algorithms – Population Based9 GA Algorithm – basic steps Initial population Evaluations on individuals Breeding Choose suitable parents (proportion to evaluation rating) Produce two offspring (Probability of breeding) Mutation Domain knowledge – evaluation function

11. Konstanz, May 2014 AI Search Algorithms – Population Based10 GA Algorithm – basic steps 1. Initialise a population of chromosomes Ci 2. Evaluate each Ci (individual) in the population Create new C by using Ci in the current population (using crossover and mutation) Delete members of the existing population to make way for the new members Evaluate the new members and insert them into the population Repeat (evolve) until some termination condition is reached (normally based on time or number of populations produced) 3. Return the best Ci as the solution

12. Konstanz, May 2014 AI Search Algorithms – Population Based11 GA Algorithm – basic steps Generate Initial Population Generate Initial Population Generation 'n' Population Generation 'n' Crossover Population Crossover Population Mutate Population Mutate Population n = n + 1 n = 1 Final Population Final Population Selection n<20?

13. Konstanz, May 2014 AI Search Algorithms – Population Based12 GA Algorithm – encoding The decision variables of a problem are normally encoded into a finite length string This could be a binary string or a list of integers For example : or 0 1 1 0 1 1 0 1 0 2341145 We could also represent numbers as coloured boxes

14. Konstanz, May 2014 AI Search Algorithms – Population Based13 Evaluation Module Responsible for evaluating a chromosome Only part of the GA that has domain knowledge. The rest of the GA modules are simply operating on (typically) bit strings with no information about the problem A different evaluation module is needed for each problem

15. Konstanz, May 2014 AI Search Algorithms – Population Based14 Population Module Responsible for maintaining the population Initialization Random Known Solutions Heuristics Population Size Elitism

16. Konstanz, May 2014 AI Search Algorithms – Population Based15 Population Module Deletion Delete-All : replaces all the members of the current population by the same number of chromosomes created Steady-State : replaces n old members by n new members. But: replace the worst, random or the parents individuals? Steady-State-No-Duplicates : Same as steady- state but also checks that no duplicate chromosomes are added to the population.

17. Konstanz, May 2014 AI Search Algorithms – Population Based16 Reproduction Module Parent selection Fitness techniques Crossover & mutation

18. Konstanz, May 2014 AI Search Algorithms – Population Based17 Parent Selection Roulette Wheel Selection Select the parents with a probability in proportion to their fitness Tournament Select two individuals at random. The individual with the highest evaluation becomes the parent. Repeat to find a second parent

19. Konstanz, May 2014 AI Search Algorithms – Population Based18 Fitness Techniques Fitness-Is-Evaluation : Simply use the fitness of the chromosome equal to its evaluation Linear Normalization : The chromosomes are sorted by decreasing the evaluation value. Then the chromosomes are assigned a fitness value that starts with a constant value and decreases linearly. The initial value and the decrement are parameters to the techniques

20. Konstanz, May 2014 AI Search Algorithms – Population Based19 Crossover Operators Order based crossover Cycle crossover Partially matched crossover

21. Konstanz, May 2014 AI Search Algorithms – Population Based20 Mutation A method of ensuring premature convergence does not occur Usually set to a small value Dynamic mutation and crossover rates

22. Konstanz, May 2014 AI Search Algorithms – Population Based21 Example I Crossover probability, P C = 1.0 Mutation probability, P M = 0.0 Maximise f(x) = x 3 - 60 * x 2 + 900 * x +100 0 = 31 x can be represented using five binary digits

23. Konstanz, May 2014 AI Search Algorithms – Population Based22 Example I Generate random initial individuals Maximise f(x) = x^3 - 60 * x^2 + 900 * x +100 (0 <= x <= 31) chromosomebinary stringxf(x) P11110028212 P201111153475 P310111231227 P40010042804 Total7718 Average1929.50

24. Konstanz, May 2014 AI Search Algorithms – Population Based23 Example I Choose Parents, using roulette wheel selection Crossover point, 1, is chosen randomly Roulette wheelParents 4116P3 1915P2 10111 01111 P3P3 P2P2 11111 00111 C1C1 C2C2 00100 01111 P4P4 P2P2 00111 01100 C3C3 C4C4

25. Konstanz, May 2014 AI Search Algorithms – Population Based24 Example I New generation chromosomebinary stringxf(x) P11111131131 P20011173803 P30011173803 P401100123889 Total11735 Average2933.75 Maximise f(x) = x^3 - 60 * x^2 + 900 * x +100 (0 <= x <= 31)

26. Konstanz, May 2014 AI Search Algorithms – Population Based25 Example I chromosomebinary string xf(x) P11110028212 P201111153475 P310111231227 P40010042804 Total7718 Average1929.50 chromosomebinary string xf(x) P11111131131 P20011173803 P30011173803 P401100123889 Total11735 Average2933.75 Two generations Mutation Maximise f(x) = x^3 - 60 * x^2 + 900 * x +100 (0 <= x <= 31) What problem do you see with the populations? what chance is there of finding the global optimum?

27. Konstanz, May 2014 AI Search Algorithms – Population Based26 GA - performance There are a number of factors which affect the performance of a genetic algorithm The size of the population The initial population Selection pressure (elitism, tournament) The cross-over probability The mutation probability Defining convergence Local optimisation

28. Konstanz, May 2012 AI Search Algorithms – Population Based27 GA - applications Combinatorial optimisation problems Cutting and packing problems vehicle routing problems job shop scheduling

29. Konstanz, May 2012 AI Search Algorithms – Population Based28 GA - applications Combinatorial optimisation problems portfolio optimization multimedia multicast routing knapsack problem

30. ANT ALGORITHMS Konstanz, May 2014 AI Search Algorithms – Population Based29 Ants are practically blind but they still manage to find their way to and from food. How do they do it?

31. Konstanz, May 2014 AI Search Algorithms – Population Based30 Ant Algorithms Ant systems are a population based approach. In this respect it is similar to genetic algorithms There is a population of ants, with each ant finding a solution and then communicating with the other ants

32. Konstanz, May 2014 AI Search Algorithms – Population Based31 Ant Algorithms A B C H D F E G

33. Konstanz, May 2014 AI Search Algorithms – Population Based32 Ant Algorithms A B C D F E d=0.5 d=1

34. Konstanz, May 2014 AI Search Algorithms – Population Based33 Ant Algorithms Time, t, is discrete At each time unit an ant moves a distance, d of 1 Once an ant moved, it lays down 1 unit of pheromone At t = 0, there is no pheromone on any edge

35. Konstanz, May 2014 AI Search Algorithms – Population Based34 Ant Algorithms At t=1 there will be 16 ants at B and 16 ants at D. At t=2 there will be 8 ants at D and 8 ants at B. There will be 16 ants at E The intensities on the edges will be as follows FD = 16, AB = 16, BE = 8, ED = 8, BC = 16 and CD = 16 A B C D F E 0.5 1 1 1 1 16 ants are moving from A - F and another 16 are moving from F - A

36. Konstanz, May 2014 AI Search Algorithms – Population Based35 Ant Algorithms We need to allow the ants to explore paths and follow the best paths with some probability in proportion to the intensity of the pheromone trail We do not want them simply to follow the route with the highest amount of pheromone on it, else our search will quickly settle on a sub-optimal (and probably very sub-optimal) solution

37. Konstanz, May 2014 AI Search Algorithms – Population Based36 Ant Algorithms The probability of an ant following a certain route is a function, not only of the pheromone intensity but also a function of what the ant can see (visibility) The pheromone trail must not build unbounded. Therefore, we need “evaporation”

38. Konstanz, May 2014 AI Search Algorithms – Population Based37 Ant Algorithms – initial ideas Dorigo (1996) Based on real world phenomena Ants, despite almost blind, are able to find their way to the food source using the shortest route If an obstacle is placed, ants have to decide which way to take around the obstacle.

39. Konstanz, May 2014 AI Search Algorithms – Population Based38 Ant Algorithms – initial ideas Dorigo (1996) Initially there is a 50-50 probability as to which way they will turn Assume one route is shorter than the other Ants taking the shorter route will arrive at a point on the other side of the obstacle before the ants which take the longer route.

40. Konstanz, May 2014 AI Search Algorithms – Population Based39 Ant Algorithms – initial ideas Dorigo (1996) As ants walk they deposit pheromone trail. Ants have taken shorter route will have already laid trail So ants from the other direction are more likely to follow that route with deposit of pheromone.

41. Konstanz, May 2014 AI Search Algorithms – Population Based40 Ant Algorithms – initial ideas Dorigo (1996) Over a period of time, the shortest route will have high levels of pheromone. The quantity of pheromones accumulates faster on the shorter path than on the longer one There is positive feedback which reinforces that behaviors so that the more ants follow a particular route, the more desirable it becomes.

42. Konstanz, May 2014 AI Search Algorithms – Population Based41 Ant Algorithms

43. Konstanz, May 2014 AI Search Algorithms – Population Based42 Ant Algorithms - TSP At the start of the algorithm, one ant is placed in each city Time, t, is discrete. t(0) marks the start of the algorithm. At t+1 every ant will have moved to a new city Assuming that the TSP is represented as a fully connected graph, each edge has an intensity of trail on it. This represents the pheromone trail laid by the ants Let T i,j (t) represent the intensity of trail edge (i,j) at time t Variations have been tested by Dorigo

44. Konstanz, May 2014 AI Search Algorithms – Population Based43 Ant Algorithms - TSP An ant decides which town to move to next, with a probability that is based on the distance to that city AND the amount of trail intensity on the connecting edge The distance to the next town, is known as the visibility, n ij, defined as 1/d ij, d ij is the distance between cities i and j. At each time unit evaporation takes place, at a rate p, a value between 0 and 1 Variations have been tested by Dorigo

45. Konstanz, May 2014 AI Search Algorithms – Population Based44 Ant Algorithms - TSP In order to stop ants visiting the same city in the same tour a data structure, Tabu, is maintained Tabu k is defined as the list for the k th ant and it holds the cities that have already been visited Variations have been tested by Dorigo

46. Konstanz, May 2014 AI Search Algorithms – Population Based45 Ant Algorithms - TSP After each ant tour the trail intensity on edge (i,j) is updated using the following formula T ij (t + n) = p. T ij (t) + ΔT ij Q is a constant L k is the tour length of the k th ant p is the evaporation coefficient

47. Konstanz, May 2014 AI Search Algorithms – Population Based46 Ant Algorithms - TSP Transition Probability where  and  are control parameters that control the relative importance of trail versus visibility

48. Konstanz, May 2014 AI Search Algorithms – Population Based47 Ant Algorithms - TSP Left: Trail distribution at the beginning; Right: Trail distribution after 100 cycles. (Dorigo et al., 1996)

49. Konstanz, May 2014 AI Search Algorithms – Population Based48 Ant Algorithms - Applications Travelling Salesman Problem (TSP) Facility Layout Problem Vehicle Routing Stock Cutting … Marco Dorigo maintains a page devoted to the subject at http://iridia.ulb.ac.be/~mdorigo/ACO/ACO.html contains information about ant algorithms as well as links to the main papers published on the subject

50. APPENDIX Konstanz, May 2014 AI Search Algorithms – Population Based49 Examples of Genetic Algorithms

51. Genetic Algorithm Example II Traveling Salesman Problem a number of cities costs of traveling between cities a traveling sales man needs to visit all these cities exactly once and return to the starting city What’s the cheapest route? Konstanz, May 2014 50AI Search Algorithms – Population Based

52. Traveling Salesman Problem Konstanz, May 2014 51AI Search Algorithms – Population Based Genetic Algorithm Example II

53. Initial generation 581……8432275467 P1P1 788127……91174424 P2P2 817……916362419 P 30 … 6.5 7.8 6.0 Any idea of other ways to generate the initial population? Konstanz, May 2014 52AI Search Algorithms – Population Based Genetic Algorithm Example II

54. Choose pairs of parents 788127……91174424 P2P2 817……916362419 P 30 6.0 7.8 Crossover 788127……916362419 C2C2 817……91174424 C1C1 5.9 6.2 13 Konstanz, May 2014 53AI Search Algorithms – Population Based Genetic Algorithm Example II

55. Next generation 788127……916362419 P2P2 817……91174424 P1P1 5.9 6.2 6.0 … Konstanz, May 2014 54AI Search Algorithms – Population Based 782……51076479 P2P2 Genetic Algorithm Example II

56. Traveling Salesman Problem No. of cities: 100 Population size: 30 Cost: 6.37 Generation: 88 Cost: 6.34 Generation: 1100 Konstanz, May 2014 55AI Search Algorithms – Population Based Genetic Algorithm Example II

57. Many applications of genetic algorithms. Best suited to problems where the efficient solutions are not already known. Strength of GA's: ability to heuristically search for solutions when all else fails. If you can represent the solutions to the problem in a suitable format, such as a series of 1's and 0's, then the GA will do the rest. Konstanz, May 2014 56AI Search Algorithms – Population Based Genetic Algorithm Example III

58. Applying Genetic Algorithms to Personnel Scheduling Applying Genetic Algorithms to Personnel Scheduling Personnel scheduling in healthcare is usually a very complex operation which has a profound effect upon the efficient usage of expensive resources. Konstanz, May 2014 57AI Search Algorithms – Population Based Genetic Algorithm Example III

59. A number of nurses A number of shifts each day A set of constraints shift coverage one shift per day resting time workload per month consecutive shifts working weekends … Konstanz, May 2012 58AI Search Algorithms – Population Based Genetic Algorithm Example III

60. Konstanz, May 2012 59AI Search Algorithms – Population Based Genetic Algorithm Example III

61. Genetic Algorithm - Initial population - construct rosters - repair infeasible ones Konstanz, May 2012 60AI Search Algorithms – Population Based Genetic Algorithm Example III

62. Genetic Algorithm - Select parents - Recombine rows in the two rosters - repair infeasible ones + Konstanz, May 2012 61AI Search Algorithms – Population Based Genetic Algorithm Example III

63. Genetic Algorithm - Mutation - Local optimiser Genetic Algorithm Example III

64. Population Size Crossover Probability Mutation Probability Local Optimiser 50 0.7 0.001 ON Konstanz, May 2014 63AI Search Algorithms – Population Based Genetic Algorithm Example III