Games: Expectimax Andrea Danyluk September 25, 2013

Announcements Programming Assignment 2: Games – Anyone still looking for a partner? – Partner can’t be the same as for Assignment 1

Today’s Lecture Just a bit more on α-β pruning Games with more than 2 players Expectimax Pacman

α = 8 Β = +inf α = 8 Β = +inf 8 Is there a problem here???

function αβSEARCH(state) returns an action a v = MAX-VALUE(state, -infinity, +infinity) return action a in ACTIONS(state) with value v function MAX-VALUE(state, α, β) returns a utility value v if TERMINAL-TEST(state) then return UTILITY(state) v = -infinity for each a in ACTIONS(state) do v = MAX(v, MIN-VALUE(RESULT(state, a), α, β)) if v ≥ β then return v α = MAX(α, v) return v function MIN-VALUE(state, α, β) returns a utility value v if TERMINAL-TEST(state) then return UTILITY(state) v = infinity for each a in ACTIONS(state) do v = MIN(v, MAX-VALUE(RESULT(state, a), α, β)) if v ≤ α then return v β = MIN(β, v) return v

Node Order Matters B B D D C C A A MAX min Best ordering for maximum pruning?

Move Ordering Can we order moves in such a way that α-β will prune more rather than less? Chess? Connect 4? Don’t worry about this for Pacman assignment

Properties of α-β Pruning does not affect the final result (but be careful) Good move ordering improves effectiveness of pruning If search depth is d, what is the time complexity of minimax? – O(b d ) With perfect pruning, get down to O(b d/2 ) – Doubles solvable depth [Adapted from Russell]

Games with > 2 players Up to 4 players Player tries to place all 21 of his/her pieces Hopes to block opponents from placing their pieces

A Four-Player Game Blue to move Yellow to move Red Green … … … ………… (5,7,2,9)(4,3,1,1)(2,6,10,2)(6,4,19,8) (5,7,2,9)(3,5,1,8)(8,7,6,5)(9,7,1,3)(4,3,2,1)(4,4,4,4)(6,4,19,8)(3,5,7,9) (5,7,2,9)(8,7,6,5)(4,4,4,4)(6,4,19,8) (5,7,2,9)(4,4,4,4) (5,7,2,9)

Multi-Player Games Evaluation function returns a vector of utilities Each player chooses the move that maximizes its utility. Is Pacman with 2 ghosts a multi-player game?

Stochastic Games

Expectimax Introduce chance nodes into a minimax tree

Expectimax Introduce chance nodes into a minimax tree

Expectimax Introduce chance nodes into a minimax tree 0.5

Expectimax Introduce chance nodes into a minimax tree

Expectimax Introduce chance nodes into a minimax tree

Expectimax Introduce chance nodes into a minimax tree

Expectimax Introduce chance nodes into a minimax tree

Expectimax Introduce chance nodes into a minimax tree

Expectimax Introduce chance nodes into a minimax tree

Expectimax Introduce chance nodes into a minimax tree

Expectimax Introduce chance nodes into a minimax tree

Expectimax Introduce chance nodes into a minimax tree

Expectimax Introduce chance nodes into a minimax tree [From Russell]

Expectimax …

Evaluation Functions Revisited Behavior is preserved under any monotonic transformation of EVAL [From Russell]

Evaluation Functions Revisited Behavior is preserved only under positive linear transformation of EVAL [From Russell]

Expectimax Pacman def value(s) if s is a max node return maxValue(s) if s is an exp node return expValue(s) if s is a terminal node return eval(s) def maxValue(s) values = [value(s 1 ) for s 1 in succ(s)] return max(values) def expValue(s) values = [value(s 1 ) for s 1 in succ(s)] weights = [prob(s,s 1 ) for s 1 in succ(s)] return expectation(values, weights) [Verbatim from CS 188]

Depth-limited Expectimax ply Don’t forget: Magnitudes of the utilities / heuristic evaluations need to be meaningful.

Why Pacman Can Starve He knows his score can go up by eating now. He knows his score can go up by eating later. Within this search window there are no other eating opportunities.