1. Oregon State University – CS430 Intro to AI (c) 2003 Thomas G. Dietterich and Devika Subramanian1 Search-Based Agents  Appropriate in Static Environments where a model of the agent is known and the environment allows prediction of the effects of actions evaluation of goals or utilities of predicted states  Environment can be partially-observable, stochastic, sequential, continuous, and even multi-agent, but it must be static!  We will first study the deterministic, discrete, single-agent case.

2. Oregon State University – CS430 Intro to AI (c) 2003 Thomas G. Dietterich and Devika Subramanian2 Computing Driving Directions Arad Oradea Zerind Timisoara Lugoj Dobreta Sibiu Rimnicu Vilcea Fagaras Pitesti Mehadia Craiova Bucharest Giurgia Urzicenl Neamt Iasi Vasini Hirsova Eforie 71 75 118 140 151 99 80 111 70 75 120 146 97 138 101 211 85 90 142 92 87 98 86 You are here You want to be here

3. Oregon State University – CS430 Intro to AI (c) 2003 Thomas G. Dietterich and Devika Subramanian3 Search Algorithms  Breadth-First  Depth-First  Uniform Cost  A*  Dijkstra’s Algorithm

4. Oregon State University – CS430 Intro to AI (c) 2003 Thomas G. Dietterich and Devika Subramanian4 Oradea (146) 71 Breadth-First Arad (0) 75 Zerind (75) 118 Timisoara (118) Lugoj (229) 111 Mehadia (299) 70 75 Dobreta (486) 120 140 Sibiu (140) 151 Rimnicu Vilcea (220) 80 Fagaras (239) 99 Craiova (366) 146 Pitesti (317) 97 138 Bucharest (450) 211 101 Detect duplicate path (291) Detect new shorter path (418) Detect duplicate path (455) Detect duplicate path (504) Detect new shorter path (374) Detect duplicate path (494) Detect duplicate path (197)

5. Breadth-First Oradea (146) 71 Arad (0) 75 Zerind (75) 118 Timisoara (118) Lugoj (229) 111 Mehadia (299) 70 75 Dobreta (486) 120 140 Sibiu (140) 151 Rimnicu Vilcea (220) 80 Fagaras (239) 99 Craiova (366) 146 Pitesti (317) 97 138 Bucharest (450) 211 101 Arad (0) Sibiu (140)Zerind (75) Timisoara (118) Arad (150) Oradea (146) Arad (280) Fagaras (239) Oradea (291) Arad (236) Lugoj (229) Sibiu (197)Sibiu (338)Bucharest (450) Rimnicu Vilcea (220) Timisoara (340) Mehadia (299) Sibiu (300) Pitesti (317)Craiova (366) Lugoj (369) Dobreta (374) Craiova (455)Rimnicu Vilcea (317)Bucharest (418) Rimnicu Vilcea (482) Dobreta (486)Pitesti (504)Medhadia (449)Craiova (494) Zerind (217)

6. Oregon State University – CS430 Intro to AI (c) 2003 Thomas G. Dietterich and Devika Subramanian6 Formal Statement of Search Problems  State Space: set of possible “mental” states cities in Romania  Initial State: state from which search begins Arad  Operators: simulated actions that take the agent from one mental state to another traverse highway between two cities  Goal Test: Is current state Bucharest?

7. Oregon State University – CS430 Intro to AI (c) 2003 Thomas G. Dietterich and Devika Subramanian7 General Search Algorithm  Strategy: first-in first-out queue (expand oldest leaf first)

8. Oregon State University – CS430 Intro to AI (c) 2003 Thomas G. Dietterich and Devika Subramanian8 Leaf Selection Strategies  Breadth-First Search: oldest leaf (FIFO)  Depth-First Search: youngest leaf (LIFO)  Uniform Cost Search: cheapest leaf (Priority Queue)  A* search: leaf with estimated shortest total path length g(x) + h(x) = f(x) where g(x) is length so far and h(x) is estimate of remaining length (Priority Queue)

9. Oregon State University – CS430 Intro to AI (c) 2003 Thomas G. Dietterich and Devika Subramanian9 A* Search  Let h(x) be a “heuristic function” that gives an underestimate of the true distance between x and the goal state Example: Euclidean distance  Let g(x) be the distance from the start to x, then g(x) + h(x) is an lower bound on the length of the optimal path

10. Oregon State University – CS430 Intro to AI (c) 2003 Thomas G. Dietterich and Devika Subramanian10 Euclidean Distance Table Arad 366 Mehadia 241 Bucharest 0 Neamt 234 Craiova 160 Oradea 380 Dobreta 242 Pitesti 100 Eforie 161 Rimnicu Vilcea 193 Fagaras 176 Sibiu 253 Giurgiu 77 Timisoara 329 Hirsova 151 Urziceni 80 Iasi 226 Vaslui 199 Lugoj 244 Zerind 374

11. Oregon State University – CS430 Intro to AI (c) 2003 Thomas G. Dietterich and Devika Subramanian11 A* Search Arad (0+366=366) Sibiu (140+253=393)Zerind (75+374=449) Timisoara (118+329=447) Fagaras (239+176=415)Oradea (291+380=671) Bucharest (450+0=450) Rimnicu Vilcea (220+193=413) Pitesti (317+100=417) Craiova (366+160=526) Craiova (455+160=615) Bucharest (418+0=418) All remaining leaves have f(x) ¸ 418, so we know they cannot have shorter paths to Bucharest

12. Oregon State University – CS430 Intro to AI (c) 2003 Thomas G. Dietterich and Devika Subramanian12 Dijkstra’s Algorithm  Works backwards from the goal  Each node keeps track of the shortest known path (and its length) to the goal  Equivalent to uniform cost search starting at the goal  No early stopping: finds shortest path from all nodes to the goal

13. Oregon State University – CS430 Intro to AI (c) 2003 Thomas G. Dietterich and Devika Subramanian13 Local Search Algorithms  Keep a single current state x  Repeat Apply one or more operators to x Evaluate the resulting states according to an Objective Function J(x) Choose one of them to replace x (or decide not to replace x at all)  Until time limit or stopping criterion

14. Oregon State University – CS430 Intro to AI (c) 2003 Thomas G. Dietterich and Devika Subramanian14 Hill Climbing  Simple hill climbing: apply a randomly-chosen operator to the current state  If resulting state is better, replace current state  Steepest-Ascent Hill Climbing:  Apply all operators to current state, keep state with the best value  Stop when no successors state is better than current state

15. Oregon State University – CS430 Intro to AI (c) 2003 Thomas G. Dietterich and Devika Subramanian15 Gradient Ascent  In continuous state spaces, x = (x 1, x 2, …, x n ) is a vector of real values  Continuous operator: x := x+  x for any arbitrary vector  x (infinitely many operators!)  Suppose J(x) is differentiable. Then we can compute the direction of steepest increase of J by the first derivative with respect to x, the gradient:

16. Oregon State University – CS430 Intro to AI (c) 2003 Thomas G. Dietterich and Devika Subramanian16 Gradient Descent Search  Repeat Compute Gradient rJ Update x := x +  rJ  Until rJ ¼ 0   is the “step size”, and it must be chosen carefully  Methods such as conjugate gradient and Newton’s method choose  automatically

17. Oregon State University – CS430 Intro to AI (c) 2003 Thomas G. Dietterich and Devika Subramanian17 Visualizing Gradient Ascent If  is too large, search may overshoot and miss the maximum or oscillate forever

18. Oregon State University – CS430 Intro to AI (c) 2003 Thomas G. Dietterich and Devika Subramanian18 Problems with Hill Climbing  Local optima  Flat regions  Random restarts can give good results

19. Oregon State University – CS430 Intro to AI (c) 2003 Thomas G. Dietterich and Devika Subramanian19 Simulated Annealing  T = 100 (or some large value)  Repeat Apply randomly-chosen operator to x to obtain x′. Let  E = J(x′) – J(x) If  E > 0, /* J(x′) is better */ switch to x′ Else /* J(x′) is worse */ switch to x′ with probability exp [  E/T] /* large negative steps are less likely */ T := 0.99 * T */ “cool” T */  Slowly decrease T (“anneal”) to zero  Stop when no changes have been accepted for many moves  Idea: Accept “down hill” steps with some probability to help escape from local minima. As T ! 0 this probability goes to zero.