1. Games: Expectimax Andrea Danyluk September 25, 2013

2. Announcements Programming Assignment 2: Games – Anyone still looking for a partner? – Partner can’t be the same as for Assignment 1

3. Today’s Lecture Just a bit more on α-β pruning Games with more than 2 players Expectimax Pacman

4. 8 8 4 4 7 7 3 3 8 8 19 11 2 2 6 6 1 1 9 9 5 5 α = 8 Β = +inf 8 8 9 2 α = 8 Β = +inf 8 Is there a problem here???

5. function αβSEARCH(state) returns an action a v = MAX-VALUE(state, -infinity, +infinity) return action a in ACTIONS(state) with value v function MAX-VALUE(state, α, β) returns a utility value v if TERMINAL-TEST(state) then return UTILITY(state) v = -infinity for each a in ACTIONS(state) do v = MAX(v, MIN-VALUE(RESULT(state, a), α, β)) if v ≥ β then return v α = MAX(α, v) return v function MIN-VALUE(state, α, β) returns a utility value v if TERMINAL-TEST(state) then return UTILITY(state) v = infinity for each a in ACTIONS(state) do v = MIN(v, MAX-VALUE(RESULT(state, a), α, β)) if v ≤ α then return v β = MIN(β, v) return v

6. Node Order Matters 5 5 7 7 2 2 4 4 8 8 10 8 8 7 7 3 3 B B D D C C A A MAX min Best ordering for maximum pruning?

7. Move Ordering Can we order moves in such a way that α-β will prune more rather than less? Chess? Connect 4? Don’t worry about this for Pacman assignment

8. Properties of α-β Pruning does not affect the final result (but be careful) Good move ordering improves effectiveness of pruning If search depth is d, what is the time complexity of minimax? – O(b d ) With perfect pruning, get down to O(b d/2 ) – Doubles solvable depth [Adapted from Russell]

9. Games with > 2 players Up to 4 players Player tries to place all 21 of his/her pieces Hopes to block opponents from placing their pieces

10. A Four-Player Game Blue to move Yellow to move Red Green … … … ………… (5,7,2,9)(4,3,1,1)(2,6,10,2)(6,4,19,8) (5,7,2,9)(3,5,1,8)(8,7,6,5)(9,7,1,3)(4,3,2,1)(4,4,4,4)(6,4,19,8)(3,5,7,9) (5,7,2,9)(8,7,6,5)(4,4,4,4)(6,4,19,8) (5,7,2,9)(4,4,4,4) (5,7,2,9)

11. Multi-Player Games Evaluation function returns a vector of utilities Each player chooses the move that maximizes its utility. Is Pacman with 2 ghosts a multi-player game?

12. Stochastic Games

13. Expectimax Introduce chance nodes into a minimax tree 2 2 4 4 7 7 4 4 6 6 0 0 5 5 -2 240 0.5 3

14. Expectimax Introduce chance nodes into a minimax tree

15. Expectimax Introduce chance nodes into a minimax tree 0.5

16. Expectimax Introduce chance nodes into a minimax tree 2 2 4 4 2 0.5

17. Expectimax Introduce chance nodes into a minimax tree 2 2 4 4 2 0.5 1+

18. Expectimax Introduce chance nodes into a minimax tree 2 2 4 4 7 7 4 4 24 0.5 1+

19. Expectimax Introduce chance nodes into a minimax tree 2 2 4 4 7 7 4 4 24 0.5 3

20. Expectimax Introduce chance nodes into a minimax tree 2 2 4 4 7 7 4 4 24 0.5 3

21. Expectimax Introduce chance nodes into a minimax tree 2 2 4 4 7 7 4 4 6 6 0 0 240 0.5 3

22. Expectimax Introduce chance nodes into a minimax tree 2 2 4 4 7 7 4 4 6 6 0 0 240 0.5 30+

23. Expectimax Introduce chance nodes into a minimax tree 2 2 4 4 7 7 4 4 6 6 0 0 5 5 -2 240 0.5 3 0+

24. Expectimax Introduce chance nodes into a minimax tree 2 2 4 4 7 7 4 4 6 6 0 0 5 5 -2 240 0.5 3

25. Expectimax Introduce chance nodes into a minimax tree 2 2 4 4 7 7 4 4 6 6 0 0 5 5 -2 240 0.5 3 [From Russell]

26. Expectimax …

27. Evaluation Functions Revisited 1 1 20 400 1 1 2 2 2 2 4 4 Behavior is preserved under any monotonic transformation of EVAL [From Russell]

28. Evaluation Functions Revisited Behavior is preserved only under positive linear transformation of EVAL [From Russell] 20 30 11400 22331144 231420301400.9.1.9.1.9.1.9.1 2.1 1.321 40.9

29. Expectimax Pacman def value(s) if s is a max node return maxValue(s) if s is an exp node return expValue(s) if s is a terminal node return eval(s) def maxValue(s) values = [value(s 1 ) for s 1 in succ(s)] return max(values) def expValue(s) values = [value(s 1 ) for s 1 in succ(s)] weights = [prob(s,s 1 ) for s 1 in succ(s)] return expectation(values, weights) [Verbatim from CS 188]

30. Depth-limited Expectimax 1 1 7 7 3 3 4 4 6 6 2 2 5 5 1 1 1 ply Don’t forget: Magnitudes of the utilities / heuristic evaluations need to be meaningful.

31. Why Pacman Can Starve He knows his score can go up by eating now. He knows his score can go up by eating later. Within this search window there are no other eating opportunities.