1. When A* doesn’t work CIS 391 – Intro to Artificial Intelligence A few slides adapted from CS 471, Fall 2004, UBMC (which were adapted from notes by Charles R. Dyer, University of Wisconsin-Madison)

2. CIS 391 - Intro to AI 2 Outline  Local Search: Hill Climbing  Escaping Local Maxima: Simulated Annealing  Genetic Algorithms (if time allows)

3. CIS 391 - Intro to AI 3 Local search and optimization  Local search: Use single current state and move to neighboring states.  Idea: start with an initial guess at a solution and incrementally improve it until it is one  Advantages: Use very little memory Find often reasonable solutions in large or infinite state spaces.  Also useful for pure optimization problems. Find best state according to some objective function. e.g. survival of the fittest as a metaphor for optimization.

4. Hill Climbing

5. CIS 391 - Intro to AI 5 Hill climbing on a surface of states Height Defined by Evaluation Function

6. CIS 391 - Intro to AI 6 Hill-climbing search: Take I & II I.While (  uphill points): Move in the direction of increasing value, lessening distance to goal II.If (  a successor s i for the current state n such that —h(s i ) < h(n) —h(s i )  h(s j ) for all successors s j of n, j  i,): then move from n to s i. Otherwise, halt at n.  Properties: Terminates when a peak is reached. Does not look ahead of the immediate neighbors of the current state. Chooses randomly among the set of best successors, if there is more than one. Doesn’t backtrack, since it doesn’t remember where it’s been  a.k.a. greedy local search "Like climbing Everest in thick fog with amnesia"

7. CIS 391 - Intro to AI 7 Hill-climbing search: Take III function HILL-CLIMBING( problem) return a state that is a local maximum input: problem, a problem local variables: current, a node. neighbor, a node. current  MAKE-NODE(INITIAL-STATE[problem]) loop do neighbor  a highest valued successor of current if VALUE [neighbor] ≤ VALUE[current] then return STATE[current] current  neighbor

8. CIS 391 - Intro to AI 8 Hill climbing Example I 2 345 678 1 start goal 5 h = 4 h = 3 h = 2 h = 1 h = 0 h = 5 5 4 45 2 h(n) = (number of tiles out of place) 2 45 678 1 3 2 4 5 678 1 3 3 2 45 678 1 3 2 458 67 1 3 2 458 6 7 1

9. CIS 391 - Intro to AI 9 Hill-climbing Example: n-queens  Put n queens on an n × n board with no two queens on the same row, column, or diagonal

10. CIS 391 - Intro to AI 10 Hill-climbing example: 8-queens h = number of pairs of queens that are attacking each other a) A state with h=17 and the h-value for each possible successor. b) A local minimum of h in the 8-queens state space (h=1). a) b)

11. CIS 391 - Intro to AI 11 Search Space features

12. CIS 391 - Intro to AI 12 Drawbacks of hill climbing  Problems: Local Maxima (foothills): peaks that aren’t the highest point in the space Plateaus: the space has a broad flat region that gives the search algorithm no direction (random walk) Ridges: flat like a plateau, but with dropoffs to the sides; steps to the North, East, South and West may go down, but a step to the NW may go up.

13. CIS 391 - Intro to AI 13 Example of a local maximum 12 345 678 1 2 2 2 0 startgoal 412 3 5 678 412 35 678 412 375 6 8 4 2 315 678

14. CIS 391 - Intro to AI 14 The Shape of an Easy Problem

15. CIS 391 - Intro to AI 15 The Shape of a Harder Problem

16. CIS 391 - Intro to AI 16 The Shape of a Yet Harder Problem

17. CIS 391 - Intro to AI 17 Remedies to drawbacks of hill climbing  Random restart  Problem reformulation  In the end: Some problem spaces are great for hill climbing and others are terrible.

18. Simulated Annealing

19. CIS 391 - Intro to AI 19 Simulated annealing (SA)  Annealing: the process by which a metal cools and freezes into a minimum-energy crystalline structure (the annealing process)  SA exploits an analogy between annealing and the search for a minimum [or maximum] in a more general system.

20. CIS 391 - Intro to AI 20 Simulated annealing  Idea: Escape local maxima by allowing “bad” moves. But gradually decrease their size and frequency.  Bouncing ball analogy: Shaking hard (= high temperature). Shaking less (= lower the temperature).  Control parameter T By analogy with the original application is known as the system “temperature.” T starts out high and gradually decreases toward 0. If T decreases slowly enough, then finds a global optimum with probability approaching 1.  Applied for VLSI layout, airline scheduling, etc.

21. CIS 391 - Intro to AI 21 The Simulated Annealing Algorithm function SIMULATED-ANNEALING( problem, schedule) return a solution state input: problem, a problem schedule, a mapping from time to temperature local variables: current, a node. next, a node. T, a “temperature” controlling the probability of downward steps current  MAKE-NODE(INITIAL-STATE[problem]) for t  1 to ∞ do T  schedule[t] if T = 0 then return current next  a randomly selected successor of current ∆E  VALUE[next] - VALUE[current] if ∆E > 0 then current  next else current  next only with probability e ∆E /T

22. CIS 391 - Intro to AI 22 Local beam search  Keep track of k states instead of one Initially: k random states Next: determine all successors of k states If any of successors is goal  finished Else select k best from successors and repeat.  Major difference with random-restart search Information is shared among k search threads.  Can suffer from lack of diversity. Stochastic variant: choose k successors at proportionally to state success.

23. Genetic Algorithms (only if time allows)

24. CIS 391 - Intro to AI 24 Genetic algorithms  Start with k random states (the initial population)  New states are generated by either 1.“Mutation” of a single state or 2.“Sexual Reproduction” (combining) of two parent states (selected according to their fitness)  Encoding used for the “genome” of an individual strongly affects the behavior of the search  Similar (in some ways) to stochastic beam search

25. CIS 391 - Intro to AI 25 Representation: Strings of genes  Each chromosome represents a possible solution made up of a string of genes  Each gene encodes some property of the solution  There is a fitness metric on phenotypes of chromosomes Evaluation of how well a solution with that set of properties solves the problem.  New generations are formed by Crossover: sexual reproduction Mutation: asexual reproduction

26. CIS 391 - Intro to AI 26 Encoding of a Chromosome  The chromosome encodes characteristics of the solution which it represents, often as a string of binary digits. Chromosome 11101100100110110 Chromosome 21101111000011110  Each bit or set of bits in this string represents some aspect of the solution.

27. CIS 391 - Intro to AI 27 Example: Genetic Algorithm for Drive Train Genes for:  Number of Cylinders  RPM: 1 st -> 2 nd  RPM 2 nd -> 3 rd  RPM 3 rd -> Drive  Rear end gear ratio  Size of wheels A Chromosome specifies a full drive train design

28. CIS 391 - Intro to AI 28 Reproduction  Reproduction by crossover selects genes from two parent chromosomes and creates two new offspring.  To do this, randomly choose some crossover point (perhaps none).  For the first child, everything before this point comes from the first parent and everything after a from the second parent.  Crossover can then look like this ( | is the crossover point): Chromosome 111001 | 00100110110 Chromosome 210011 | 11000011110 Offspring 111001 | 11000011110 Offspring 210011 | 00100110110

29. CIS 391 - Intro to AI 29 Mutation  Mutation randomly changes genes in the new offspring.  For binary encoding we can switch a few randomly chosen bits from 1 to 0 or from 0 to 1. Original offspring110 1 1110000 1 1110 Mutated offspring110 0 1110000 0 1110

30. CIS 391 - Intro to AI 30 The Basic Genetic Algorithm 1.Generate random population of chromosomes 2.Until the end condition is met, create a new population by repeating following steps 1.Evaluate the fitness of each chromosome 2.Select two parent chromosomes from a population, weighed by their fitness 3.With probability p c cross over the parents to form a new offspring. 4.With probability p m mutate new offspring at each position on the chromosome. 5.Place new offspring in the new population 3.Return the best solution in current population

31. CIS 391 - Intro to AI 31 Genetic algorithm function GENETIC_ALGORITHM( population, FITNESS-FN) return an individual input: population, a set of individuals FITNESS-FN, a function which determines the quality of the individual repeat new_population  empty set loop for i from 1 to SIZE(population) do x  RANDOM_SELECTION(population, FITNESS_FN) y  RANDOM_SELECTION(population, FITNESS_FN) child  REPRODUCE(x,y) if (small random probability) then child  MUTATE(child ) add child to new_population population  new_population until some individual is fit enough or enough time has elapsed return the best individual

32. CIS 391 - Intro to AI 32 Genetic algorithms:8-queens