Informed Search Methods Chapter 4 Fall 2009 Copyright, 1996 © Dale Carnegie & Associates, Inc.

What we’ll learn Informed search algorithms are more efficient in most cases What are informed search methods How to use problem-specific knowledge How to optimize a solution CSE 471/598 by H. Liu

Best-First Search Evaluation functions It gives a measure about which node to expand Minimizing the path cost g(n) – a true cost Expands the node based on the past Minimizing estimated cost to reach a goal Greedy search at node n heuristic function h(n) an example is straight-line distance between cities (Fig 4.1) The simple Romania map Finding the route using greedy search – example (Fig 4.2) CSE 471/598 by H. Liu

Best-first search (2) h(n) is independent of the path cost g(n) Minimizing the total path cost f(n) = g(n) + h(n) estimated cost of the cheapest solution via n Admissible heuristic function h never overestimates the cost What is the most useless h? optimistic One never overestimates CSE 471/598 by H. Liu

A* search How it works (Fig 4.3) Characteristics of A* Monotonicity (Consistency) - h is nondescreasing How to check – using triangle inequality Tree-search to ensure monotonicity Contours (Fig 4.4) - from circle to oval (ellipse) Proof of the optimality of A* The completeness of A* (Fig 4.4 Contours) Complexity of A* (time and space) For most problems, the number of nodes within the goal contour search space is still exponential in the length of the solution The proof sketch is: assuming it reaches a suboptimal goal (its cost is bigger than the optimal cost), then assuming a node along the optimal path, since this node is not chosen over the suboptimal goal, we can arrive at the contradiction. CSE 471/598 by H. Liu

Improving A* - memory-bounded heuristic search Iterative-deepening A* (IDA*) Using f-cost(g+h) rather than the depth Cutoff value is the smallest f-cost of any node that exceeded the cutoff on the previous iteration; keep these nodes only Space complexity O(bd) Recursive best-first search (RBFS) Best-first search using only linear space complexity (Fig 4.5) It replaces the f-value of each node along the path with the best f-value of its children (Fig 4.6) Space complexity O(bd) with excessive node regeneration Simplified memory bounded A* (SMA*) IDA* and RBFS use too little memory – excessive node regeneration Expanding the best leaf until memory is full Dropping the worst leaf node (highest f-value) by backing up to its parent CSE 471/598 by H. Liu

Different Search Strategies Uniform-cost search minimize the path cost so far Greedy search minimize the estimated path cost A* minimize the total path cost Time and space issues of A* Designing good heuristic functions A* usually runs out of space long before it runs out of time CSE 471/598 by H. Liu

Heuristic Functions An example (the 8-puzzle, Fig 4.7) How simple can a heuristic be? The distance to its correct position Using Manhattan distance What is a good heuristic? Effective branching factor - close to 1 (Why?) Value of h not too large - must be admissible (Why?) not too small - ineffective (oval to circle) (expanding all nodes with f (n) < f*) Goodness measures - no. of nodes expanded and branching factor (Fig 4.8) CSE 471/598 by H. Liu

Domination translates directly into efficiency Larger h means smaller branching factor If h2 >= h1, is h2 always better than h1? Proof? (h1 <= h2 <= C* - g) Inventing heuristic functions – An important component of A* How to invent One way is to work on relaxed problems Simplify the problem Remove some constraints C* is the optimal cost, being better means expanding fewer number of nodes How to work collaboratively on a project, yet individual effort is still recognized? CSE 471/598 by H. Liu

8-puzzle revisited Definition: A tile can move from A to B if A is horizontally or vertically adjacent to B and B is blank Relaxation by removing one or both the conditions A tile can move from A to B if A ~ B A tile can move from A to B if B is blank A tile can move from A to B Deriving a heuristic from the solution cost of a sub-problem Fig 4.9 CSE 471/598 by H. Liu

Feature selection and combination If we have admissible h1 … hm and none dominates, we can have for node n h = max(h1, …, hm) Feature selection and combination use only relevant features “number of misplaced tiles” as a feature The cost of heuristic function calculation <= the cost of expanding a node otherwise, we need to rethink. Learning heuristics from experience Each optimal solution to 8-puzzle provides a learning example CSE 471/598 by H. Liu

Local Search Algorithms and Optimization Problems Sometimes the path to the goal constitutes a solution; sometimes, the path to the goal is irrelevant (e.g., 8-queen) Local search algorithms operate using a single current state and generally move only to neighbors of that state. The paths followed by the search are not retained Key advantages: little memory usage; can find reasonable solutions in large or infinite state space where systematic search is not suitable Global and local optima Fig 4.10, from current state to global maximum CSE 471/598 by H. Liu

Some local-search algorithms Hill-climbing (maximization) Well know drawbacks (Fig 4.13) Local maxima, Plateaus, Ridges Random-restart Simulated annealing Gradient descent (minimization) Escaping the local minima by controlled bouncing Local beam search Keeping track of k states instead of just one Is it similar to have k random-start of Hill-climbing? Genetic algorithms Selection, cross-over, and mutation Local bean search is different from k random-start of Hill-climbing. Basically, it keeps the best k states at any time. CSE 471/598 by H. Liu

Online Search Offline search – computing a complete solution before acting Online search – interleaving computation and action Solving an exploration problem where the states and actions are unknown to the agent Good for domains where there is a penalty for computing too long, or for stochastic domains An example – A robot is placed in a new building: explore it to build a map that it can use for getting A to B Any additional examples? Please send it to me if you find one. CSE 471/598 by H. Liu

Online search problems An agent knows: e.g., Fig 4.18 Actions(s) in state s Step-cost function c(s,a,s’) c() cannot be used until the agent knows s’ is the outcome In order to know c(), a must be actually tried Goal-Test(s) Others: with memory of states visited, and admissible heuristic from current state to the goal state Objective: Reaching a goal state while minimizing cost CSE 471/598 by H. Liu

Measuring its performance Competitive ratio: the true path cost over the path cost if it knew the search space in advance The best achievable competitive ratio can be 1 If some actions are irreversible, it may reach a dead-end (Fig 4.19 (a)) An adversary argument – Fig 4.19 (b) No bounded competitive ratio can be guaranteed if there are paths of unbounded cost CSE 471/598 by H. Liu

Online search agents It can expand only a node that it physically occupies, so it should expand nodes in a local order Online Depth-First Search (Fig 4.20) Backtracking requires that actions are reversible Hill-climbing search keeps one current state in memory It can get stuck in a local minimum Random restart does not work here (Why?) Random walk selects at random one of the available actions from the current state It can be very slow, Fig 4.21 Augmenting hill climbing with memory rather than randomness is more effective Learning real-time agent, Fig 4.22 H(s) is updated as the agent gains experience Encourages to explore new paths An agent has to be transferred to a randomly started position. Random restart would work only if the agent could teleport itself. CSE 471/598 by H. Liu

Summary Heuristics are the key to reducing research costs f(n) = g(n)+h(n) Understand their variants A* is complete, optimal, and optimally efficient among all optimal search algorithms, but ... Iterative improvement algorithms are memory efficient, but ... Local search There is a cost associated with it Online search is different from offline search Mainly for exploration problems CSE 471/598 by H. Liu