Search CSE When you can’t use A* Hill-climbing Simulated Annealing Other strategies 2 person- games

Search CSE Hill-climbing Just keep current state Generate successors If best successor is better than current state, move to it. Otherwise, you’re stuck – local maxima.

Search CSE N-queens NxN board N queens Place the queens on the board so that all queens are safe. No queen is on the same row, column or diagonal as another queen.

Search CSE queens Initial state: no queens Successor function: add a queen to any empty square Goal: are all queens safe? 64 x 63 x 62 ….= 3 x possible sequences

Search CSE queens – almost a solution, 4 steps

Search CSE queens Initial state: no queens Successor function: add a queen to a safe empty square Goal: are all queens safe? 2057 possible sequences

Search CSE Applying hill climbing to 8 queens Initial state: 8 queens placed randomly Successor function: move queen in its column (8 x 7 = 56 successors) 8 8 Goal state: all queens are safe Works 14% of the time, gets stuck 86%

Search CSE queens – a random start state

Search CSE queens – h = # of pairs of queens attacking each other

Search CSE queens – almost a solution, 4 steps Almost a solution - 5 moves

Search CSE Applying hill climbing to 8 queens Initial state: 8 queens placed randomly Successor function: move queen in its column (8 x 7 = 56 successors) 8 8 or allow up to 100 sideways moves… Goal state: all queens are safe Works 94% of the time, gets stuck 6%

Search CSE Hill-climbing Local maxima: a peak which is higher than each of its neighbors but lower than the global maximum. Ridge: a sequence of local maxima Plateau: function value is flat –Flat local maxima, no uphill exit –Shoulder, can find uphill path

Search CSE Hill-climbing Just keep current state Generate successors If best successor is better than current state, move to it. Otherwise, you’re stuck – local maxima. Random restart.

Search CSE Simulated annealing Modeled after “annealing” in metallurgy –Heated metal cools slowly so that crystals can form Closer to a purely random walk Starts by picking random moves (high temp) As the “temperature” cools, shifts to preferring “best” moves. Applications: –VLSI layouts, factory scheduling, airline scheduling

Search CSE Other strategies Local beam search –Same as hill-climbing, but keeps k nodes in memory instead of just 1. Genetic algorithms –Generate a successor by combining two parent states, instead of just modifying one. –Apply random mutations –Evaluate with “fitness function”

Search CSE person games – more complex Opponent introduces non-determinism –Minimax, alpha-beta Space and time limitations can introduce inaccessibility

Search CSE Problem Formulation Initial state Operators Terminal test (goal state) Utility function (payoff) –Minimax: back up from terminal state, high values for max, low values for min

Search CSE Tic-Tac-Toe Initial state –Representation? matrix Successor functions –Placing x’s and o’s Goal states –Explicit Utility function - Minimax x o x o x x o x x x x o

Search CSE Straightforward Mimimax Utility function (payoff) –Minimax: back up from terminal state, high values for max, low values for min

Search CSE x x x o x x o x x o x x o x o x x o x o x x o x x o x o x o x o x x o x o x x o x o x o x x o x x x o x x o o x x o x o x x o x o Tic-Tac-Toe x x o x o x o x x x o o x

Search CSE A More “Intelligent” Minimax Backing up wins and losses –Requires entire search tree – Often infeasible Can use heuristic estimates, e(p), instead –pick “best next move” based on limited search –After opponent’s move, extend search further and estimate again

Search CSE Tic-Tac-Toe Evaluation Function If p is a win for MAX then e(p) = If p is a win for MIN then e(p) = - Else h(n) = ( X’s # of complete open rows,columns, diagonals - O’s # of complete open rows, columns,diagonals) e(p) = = 2 o x

Search CSE Heuristic Minimax for Tic-Tac-Toe x o x o x x o ox ox x o x x o xoxo x o xo xo x o x o x o x 6-5=1 5-5=0 4-5=-1 5-6=-1 5-5=0 5-6=-1 6-6=0 4-6=-2 5-4=1 6-4=

Search CSE o x o x o x o x x o x x o x o x o x o x o o x x o x o o x x o x o x x o o x o x o o x x o x o x o o x o x o x o x o o o x o x o x x o o x x o o o x x o x o x 4-2=2 3-2=1 5-2=3 3-2=1 4-2=2 3-2=1 4-3=1 3-3=0 5-3=2 3-3= 0 4-3=1 4-3=1 4-2=2 4-2=2 5-2=3 3-2=1 4-2=2 4-2=2 4-3=1 4-3=1 3-3= Minimax - 2

Search CSE Minimax-3 o o x 2-1=1 3-1=2 - = - 2-2=0 2-2=0 3-2=1 -=- 2-1=1 2-1=1 2-1= x o o x o o x x o o o x x x o o x X o x o o x x o x o o x o x o o x o x o o x x x o o o o x x o o o x o o o x x x o o x x x o o o x x x o o o x x x o o o x x x - -

Search CSE Alpha/Beta The value of a MAX node = current largest final backed-up value of its successors The value of a MIN node = current smallest final backed-up value of its successors

Search CSE Alpha/Beta Pruning Stop search below any MIN node where Stop search below any MAX node where

Search CSE Alpha/Beta for Tic-Tac-Toe x o x o x x o ox ox x o x x o xoxo x o xo xo x o x o x o x 6-5=1 5-5=0 4-5=-1 5-6=-1 5-5=0 5-6=-1 6-6=0 4-6=-2 5-4=1 6-4=2 MIN=-1, BETA=-1 MIN=-2, BETA=-1 MIN=1, BETA=1 MAX=1, ALPHA= -1 X X X X ALPHA= 1

Search CSE Alpha/Beta Pruning o o x 2-1=1 3-1=2 - = - 2-2=0 2-2=0 3-2=1 x o o x o o x x o o o x x x o o x X o x o o x x o x o o x o x o o x o x o o x x x o o o o x x o o o x XX X o o x x x o o o x x x X X X - = - MIN= -, BETA= - MIN= - BETA= - MIN=1, BETA=1 MAX=1 ALPHA=1