1. An Introduction to Artificial Life Lecture 4b: Informed Search and Exploration Ramin Halavati (halavati@ce.sharif.edu) In which we see how information about the state space can prevent algorithms from blundering about the dark.

2. Local search algorithms Some times the path to goal matters… –Shortest route to a city. –The solution to 8-Puzzle. –Robot’s route in a building. –A Check-Mate And some times, not –8 Queens. –Job-Shop Scheduling –Automatic program generation –Check-Mate in Barareh!

3. Local search algorithms When path doesn’t matter… State space = set of "complete" configurations Find configuration satisfying constraints Keep a single "current" state or a fixed number of independent current states, try to improve it or them.

4. Objective Landscape

5. Hill-Climbing Search "Like climbing Everest in thick fog with amnesia" Greedy Local Search

6. Hill-Climbing Search: 8-Queens h = number of pairs of queens that are attacking each other, either directly or indirectly h = 17 for the above state

7. Hill Climbing Problems Local Maxima/Minima

8. Hill Climbing Problems A local minimum with h = 1

9. Hill Climbing Problems Ridges

10. Hill Climbing Problems Plateaux –A state whose all neighbors have similar fitness.

11. Hill Climbing, Now What? 8-Queens, random examples: –86% Failure, 14% Success. –In average, 4 moves. –State Space: 8 8 > 17,000,000 Solution 1: Random Restart: –Restart from a random point if failed. Almost 7 tries for 8-queens. 3,000,000 queens in less than 1 minute.

12. Hill Climbing, Now What? Solution 2: –Sideway moves for plateaux. / Limited. Solution 3: –Stochastic Hill Climbing, random selection among up hills. Slower convergence, sometimes better solutions. Solution 4: –First Choice Hill Climbing.

13. Practical State Spaces

14. Simulated Annealing Search Hill Climbing: –Just move to a better state. –Efficient, but can stuck in local maxima Random Walk: –Move to a random neighbor. –Complete, but extremely inefficient. Idea: Escape local maxima by allowing some "bad" moves but gradually decrease their frequency.

15. Simulated Annealing Search

16. One can prove: If T decreases slowly enough, then simulated annealing search will find a global optimum with probability approaching 1 Widely used in VLSI layout, airline scheduling, etc.

17. Local Beam Search Keep track of k states rather than just one Start with k randomly generated states At each iteration, all the successors of all k states are generated If any one is a goal state, stop; else select the k best successors from the complete list and repeat.

18. Local Beam Search Looks similar to Parallel Random Start Hill Climbers, but it’s not. Stochastic Beam Search

19. Genetic Algorithms Beagle Voyage Animals adopt to environmental.

20. Genetic Algorithms Natural Evolution: –Given these requirements: 1.A creature whose features affects his reproduction rate. 2.Offspring features are very much, but not exactly similar to the parent(s). 3.There is a competition on resources. –We will have Gradual Progress –The Blind Watchmaker - Dawkins

21. Genetic algorithms Start with k randomly generated states (population) A successor state is generated by combining two parent states Produce the next generation of states by selection, crossover, and mutation. Evaluation function (fitness function). Higher values for better states. –Selection of Higher Fitnesses

22. Genetic algorithms Cross Over: –To select some part of the solution (state) from one person and the rest from another. Mutation: –To change a small part of one solution with a small probability.

23. Genetic algorithms

24. L.S. in Continues Spaces Infinite number of successor states. –To select three best locations for airports. (x 1,y 1 ), (x 2,y 2 ), (x 3,y 3 ) Approach 1: –To discretize Just change them by ±  Approach 2: –To compute gradient! J

25. Online Search / Unknown Env. Offline Search –Generate as many nodes as you wish, in any order. Online Search –Interleave computation and action Dynamic Domains Stochastic Domains. Unknown Domains. –Exploration

26. Online Search Problems You can see the successors just by applying an action. You may have different results from repeating a similar action at a certain state. You just see one step ahead. G S

27. Algorithms for Online Search Nothing General Enough Limitation to Local Search Competitive Ratio Safely Explorable

28. Hill Climbing, once more … Hill Climbing stores just one state, so it is an online searcher But can not Randomly Restart. –Random Walk –Adding Memory: Learning Real Time A*

29. To remember and update the costs of all visited nodes.

30. Learning Real Time A*

31. Essay Proposals What is MA*? Compare it with SMA* and A*. What is Tabu Search? What is Viterbi Search? What is Tree-Trellis? What are variants of Genetic Algorithms? What is Immunity System Search? What is Simulated DNA Computing Search? Compare Natural Evolution and Genetic Algorithms.

32. Essay Proposals What is Game of Life? What other search method exist that are inspired from nature?

33. Essay Proposals Bring a survey of any of the proposed algorithms in real applications. Bring a detailed usage of one of the algorithms in a real application. Choose 5 exercises and do them in details, from any chapter, but tell me first.

34. Exercises Choose one of 4.15 – 4.18 Due: Esfand 30 th Email To: ghanbari@ce.sharif.edu Subject: AIEX-415.. AIEX-418

35. Project Proposals Make an improvement in one of the algorithms, compare your results with original one on different domains. …

36. Project Proposals Choose a real problem. Design and implement a suitable algorithm for it. Compare your results with required results or other solutions.

37. Project Proposals Choose a one player game, find some other people who agree on yours. Write a common engine and perform a competition, with or without time limit. –Mahjongg, Solitaire, Open Tetris, …

38. That ’ s all.