B EYOND C LASSICAL S EARCH Instructor: Kris Hauser 1

A GENDA Local search, optimization Branch and bound search Online search

L OCAL S EARCH Light-memory search methods No search tree; only the current state is represented! Applicable to problems where the path is irrelevant (e.g., 8-queen) For other problems, must encode entire paths in the state Many similarities with optimization techniques 3

I DEA : M INIMIZE H (N) …Because h(G)=0 for any goal G An optimization problem! 4

S TEEPEST D ESCENT 1. S  initial state 2. Repeat: 3. S’  arg min S’  SUCCESSORS(S) {h(S’)} 4. if GOAL?(S’) return S’ 5. if h(S’)  h(S) then S  S’ else return failure Similar to: hill climbing with –h gradient descent over continuous space 5

A PPLICATION : 8-Q UEEN Pick an initial state S at random with one queen in each column Repeat k times: If GOAL?(S) then return S Pick an attacked queen Q at random Move Q in its column to minimize the number of attacking queens  new S [min-conflicts heuristic] Return failure

A PPLICATION : 8-Q UEEN Pick an initial state S at random with one queen in each column Repeat k times: If GOAL?(S) then return S Pick an attacked queen Q at random Move Q in its column to minimize the number of attacking queens  new S [min-conflicts heuristic] Return failure Repeat n times:

A PPLICATION : 8-Q UEEN Pick an initial state S at random with one queen in each column Repeat k times: If GOAL?(S) then return S Pick an attacked queen Q at random Move Q in its column to minimize the number of attacking queens  new S [min-conflicts heuristic] Return failure Repeat n times: Why does it work ??? 1)There are many goal states that are well-distributed over the state space 2)If no solution has been found after a few steps, it’s better to start it all over again. Building a search tree would be much less efficient because of the high branching factor 3)Running time almost independent of the number of queens Why does it work ??? 1)There are many goal states that are well-distributed over the state space 2)If no solution has been found after a few steps, it’s better to start it all over again. Building a search tree would be much less efficient because of the high branching factor 3)Running time almost independent of the number of queens

S TEEPEST D ESCENT S  initial state Repeat: S’  arg min S’  SUCCESSORS(S) {h(S’)} if GOAL?(S’) return S’ if h(S’)  h(S) then S  S’ else return failure may easily get stuck in local minima Random restart (as in n-queen example) Monte Carlo descent 9

G RADIENT D ESCENT IN C ONTINUOUS S PACE Minimize y=f(x) Move in opposite direction of derivative  df/dx(x) y x x1x1 df/dx(x 1 )

G RADIENT D ESCENT IN C ONTINUOUS S PACE Minimize y=f(x) Move in opposite direction of derivative  df/dx(x) y x x1x1 df/dx(x 1 ) x2x2

G RADIENT D ESCENT IN C ONTINUOUS S PACE Minimize y=f(x) Move in opposite direction of derivative  df/dx(x) y x x1x1 df/dx(x 2 ) x2x2

G RADIENT D ESCENT IN C ONTINUOUS S PACE Minimize y=f(x) Move in opposite direction of derivative  df/dx(x) y x x1x1 df/dx(x 2 ) x2x2 x3x3

G RADIENT D ESCENT IN C ONTINUOUS S PACE Minimize y=f(x) Move in opposite direction of derivative  df/dx(x) y x x1x1 df/dx(x 3 ) x2x2 x3x3

G RADIENT D ESCENT IN C ONTINUOUS S PACE Minimize y=f(x) Move in opposite direction of derivative  df/dx(x) y x x1x1 x2x2 x3x3

f Gradient : analogue of derivative in multivariate functions f(x 1,…,x n ) Direction that you would move x 1,…,x n to make the steepest increase in f x1x1 x2x2

17 f f GD works well GD works poorly

A LGORITHM FOR G RADIENT D ESCENT Input: continuous objective function f, initial point x 0 =(x 1 0,…,x n 0 ) For t=0,…,N-1: Compute gradient vector g t =(  f/  x 1 ( x t ),…,  f/  x n ( x t )) If the length of g t is small enough [convergence] Return x t Pick a step size  t Let x t+1 = x t -  t g t Return failure [convergence not reached] “Industrial strength” optimization software uses more sophisticated techniques to use higher derivatives, handle constraints, deal with particular function classes, etc.

P ROBLEMS FOR D ISCRETE O PTIMIZATION … 19 Plateau Ridges NP-hard problems typically have an exponential number of local minima

M ONTE C ARLO D ESCENT S  initial state Repeat k times: If GOAL?(S) then return S S’  successor of S picked at random if h(S’)  h(S) then S  S’ else ∆h = h(S’)-h(S) with probability ~ exp(  ∆h/T), where T is called the “temperature”, do: S  S’ [Metropolis criterion] Return failure Simulated annealing lowers T over the k iterations. It starts with a large T and slowly decreases T 20

“P ARALLEL ” L OCAL S EARCH T ECHNIQUES They perform several local searches concurrently, but not independently: Beam search Genetic algorithms Tabu search Ant colony/particle swarm optimization 21

E MPIRICAL S UCCESSES OF L OCAL S EARCH Satisfiability (SAT) Vertex Cover Traveling salesman problem Planning & scheduling Many others… 22

R ELATION TO N UMERICAL O PTIMIZATION Optimization techniques usually operate on a continuous state space Example: stitch point clouds together into a global model Same major issues, e.g., local minima, apply

D EALING WITH I MPERFECT K NOWLEDGE

Classical search assumes that: World states are perfectly observable,  the current state is exactly known Action representations are perfect,  states are exactly predicted How an agent can cope with adversaries, uncertainty, and imperfect information? 25

26 Distance, speed, acceleration? Intent? Personality?

O N -L INE S EARCH Sometimes uncertainty is so large that actions need to be executed for the agent to know their effects On-line search: repeatedly observe effects, and replan A proactive approach for planning A reactive approach to uncertainty Example: A robot must reach a goal position. It has no prior map of the obstacles, but its vision system can detect all the obstacles visible from a the robot’s current position 27

28 Assuming no obstacles in the unknown region and taking the shortest path to the goal is similar to searching with an admissible (optimistic) heuristic

29 Assuming no obstacles in the unknown region and taking the shortest path to the goal is similar to searching with an admissible (optimistic) heuristic

30 Assuming no obstacles in the unknown region and taking the shortest path to the goal is similar to searching with an admissible (optimistic) heuristic Just as with classical search, on-line search may detect dead-ends and move to a more promising position (~ node of search tree)

31 D* ALGORITHM FOR M OBILE R OBOTS Tony Stentz

R EAL - TIME REPLANNING AMONG UNPREDICTABLY MOVING OBSTACLES

N EXT CLASS Uncertain and partially observable environments Game playing Read R&N HW1 due at end of next class