1. Games: Efficiency Andrea Danyluk September 23, 2013

2. Announcements Programming Assignment 1: Search – Autograding done – Please sign up for a code review Programming Assignment 2: Games – Remove your print statements before turning it in – Need help with buggy code? If I’m not in the office, email me your code and a description of the problem.

3. Today’s Lecture Heuristic evaluation functions for minimax Making the search more efficient: – α-β pruning

4. Minimax Reality Can rarely explore entire search space to terminal nodes. Choose a depth cutoff – i.e., a maximum ply Need an evaluation function – Returns an estimate of the expected utility of the game from a given position – Must order the terminal states in the same way as the true utility function – Must be efficient to compute Trading off plies for heuristic computation More plies makes a difference Consider iterative deepening

5. Evaluation Functions Ideal: returns the utility of the position In practice: typically weighted linear sum of features: Eval(s) = w 1 f 1 (s) + w 2 f 2 (s) + … + w n f n (s)

6. Exercise Evaluation function for Connect Four?

7. An earlier example revisited 7 7 2 2 6 6 3 3 0 0 6 6 -2 5 5 2 2 9 9 6 6 2 2

8. An earlier example revisited 7 7 2 2 6 6 3 3 0 0 6 6 -2 5 5 2 2 9 9 6 6 2 2 7

9. An earlier example revisited 7 7 2 2 6 6 3 3 0 0 6 6 -2 5 5 2 2 9 9 6 6 2 2 73

10. An earlier example revisited 7 7 2 2 6 6 3 3 0 0 6 6 -2 5 5 2 2 9 9 6 6 2 2 73 3

11. An earlier example revisited 7 7 2 2 6 6 3 3 0 0 6 6 -2 5 5 2 2 9 9 6 6 2 2 73 3 0 Wow! 0 is a great score!

12. An earlier example revisited 7 7 2 2 6 6 3 3 0 0 6 6 -2 5 5 2 2 9 9 6 6 2 2 73 3 0 I wonder if I can do even better!

13. An earlier example revisited 7 7 2 2 6 6 3 3 0 0 6 6 -2 5 5 2 2 9 9 6 6 2 2 73 3 0 Too bad the maximizer won’t ever let me get here!

14. An earlier example revisited 7 7 2 2 6 6 3 3 0 0 6 6 -2 5 5 2 2 9 9 6 6 2 2 73 3 0 Better not to waste my time!

15. An earlier example revisited 7 7 2 2 6 6 3 3 0 0 6 6 -2 5 5 2 2 9 9 6 6 2 2 73 3 0

16. An earlier example revisited 7 7 2 2 6 6 3 3 0 0 6 6 -2 5 5 2 2 9 9 6 6 2 2 73 3 0 0

17. An earlier example revisited 7 7 2 2 6 6 3 3 0 0 6 6 -2 5 5 2 2 9 9 6 6 2 2 73 3 0 0 6

18. An earlier example revisited 7 7 2 2 6 6 3 3 0 0 6 6 -2 5 5 2 2 9 9 6 6 2 2 73 3 0 0 6 9 Wow! 9!

19. An earlier example revisited 7 7 2 2 6 6 3 3 0 0 6 6 -2 5 5 2 2 9 9 6 6 2 2 73 3 0 0 6 9 Oh, wait!

20. An earlier example revisited 7 7 2 2 6 6 3 3 0 0 6 6 -2 5 5 2 2 9 9 6 6 2 2 73 3 0 0 6 9 Min won’t let me go here!

21. An earlier example revisited 7 7 2 2 6 6 3 3 0 0 6 6 -2 5 5 2 2 9 9 6 6 2 2 73 3 0 0 6 9 Min won’t let me go here! 6

22. An earlier example revisited 7 7 2 2 6 6 3 3 0 0 6 6 -2 5 5 2 2 9 9 6 6 2 2 73 3 0 0 6 9 Min won’t let me go here! 6

23. α-β Pruning If something looks too good to be true, it probably is. One example of the class of branch and bound algorithms with two bounds – α : the value of the best choice for Max – β : the value of the best choice for min

24. α-β Pruning Given these two bounds – α : the value of the best choice for Max – β : the value of the best choice for min Basic idea of the algorithm – On a minimizing level, if you find a value < α, cut the search – On a maximizing level, if you find a value > β, cut the search

25. function αβSEARCH(state) returns an action a v = MAX-VALUE(state, -infinity, +infinity) return action a in ACTIONS(state) with value v function MAX-VALUE(state, α, β) returns a utility value v if TERMINAL-TEST(state) then return UTILITY(state) v = -infinity for each a in ACTIONS(state) do v = MAX(v, MIN-VALUE(RESULT(state, a), α, β)) if v ≥ β then return v α = MAX(α, v) return v function MIN-VALUE(state, α, β) returns a utility value v if TERMINAL-TEST(state) then return UTILITY(state) v = infinity for each a in ACTIONS(state) do v = MIN(v, MAX-VALUE(RESULT(state, a), α, β)) if v ≤ α then return v β = MIN(β, v) return v

26. 8 8 4 4 7 7 3 3 8 8 19 11 2 2 6 6 1 1 9 9 5 5 α = -inf Β = +inf

27. 8 8 4 4 7 7 3 3 8 8 19 11 2 2 6 6 1 1 9 9 5 5 α = -inf Β = +inf α = -inf Β = +inf

28. 8 8 4 4 7 7 3 3 8 8 19 11 2 2 6 6 1 1 9 9 5 5 α = -inf Β = +inf α = -inf Β = +inf α = -inf Β = +inf

29. 8 8 4 4 7 7 3 3 8 8 19 11 2 2 6 6 1 1 9 9 5 5 α = -inf Β = +inf α = -inf Β = +inf α = 8 Β = +inf

30. 8 8 4 4 7 7 3 3 8 8 19 11 2 2 6 6 1 1 9 9 5 5 α = -inf Β = +inf α = -inf Β = +inf 8

31. 8 8 4 4 7 7 3 3 8 8 19 11 2 2 6 6 1 1 9 9 5 5 α = -inf Β = +inf α = -inf Β = 8 8

32. 8 8 4 4 7 7 3 3 8 8 19 11 2 2 6 6 1 1 9 9 5 5 α = -inf Β = +inf α = -inf Β = 8 8 α = -inf Β = 8

33. 8 8 4 4 7 7 3 3 8 8 19 11 2 2 6 6 1 1 9 9 5 5 α = -inf Β = +inf α = -inf Β = 8 8 9

34. 8 8 4 4 7 7 3 3 8 8 19 11 2 2 6 6 1 1 9 9 5 5 α = -inf Β = +inf 8 8 9

35. 8 8 4 4 7 7 3 3 8 8 19 11 2 2 6 6 1 1 9 9 5 5 α = 8 Β = +inf 8 8 9

36. 8 8 4 4 7 7 3 3 8 8 19 11 2 2 6 6 1 1 9 9 5 5 α = 8 Β = +inf 8 8 9 α = 8 Β = +inf

37. 8 8 4 4 7 7 3 3 8 8 19 11 2 2 6 6 1 1 9 9 5 5 α = 8 Β = +inf 8 8 9 2

38. 8 8 4 4 7 7 3 3 8 8 19 11 2 2 6 6 1 1 9 9 5 5 α = 8 Β = +inf 8 8 9 2 α = 8 Β = +inf

39. 8 8 4 4 7 7 3 3 8 8 19 11 2 2 6 6 1 1 9 9 5 5 α = 8 Β = +inf 8 8 9 2 α = 8 Β = +inf α = 8 Β = +inf

40. 8 8 4 4 7 7 3 3 8 8 19 11 2 2 6 6 1 1 9 9 5 5 α = 8 Β = +inf 8 8 9 2 α = 8 Β = +inf α = 8 Β = +inf

41. 8 8 4 4 7 7 3 3 8 8 19 11 2 2 6 6 1 1 9 9 5 5 α = 8 Β = +inf 8 8 9 2 α = 8 Β = +inf 8

42. 8 8 4 4 7 7 3 3 8 8 19 11 2 2 6 6 1 1 9 9 5 5 α = 8 Β = +inf 8 8 9 2 α = 8 Β = +inf 8 Is there a problem here???

43. function αβSEARCH(state) returns an action a v = MAX-VALUE(state, -infinity, +infinity) return action a in ACTIONS(state) with value v function MAX-VALUE(state, α, β) returns a utility value v if TERMINAL-TEST(state) then return UTILITY(state) v = -infinity for each a in ACTIONS(state) do v = MAX(v, MIN-VALUE(RESULT(state, a), α, β)) if v ≥ β then return v α = MAX(α, v) return v function MIN-VALUE(state, α, β) returns a utility value v if TERMINAL-TEST(state) then return UTILITY(state) v = infinity for each a in ACTIONS(state) do v = MIN(v, MAX-VALUE(RESULT(state, a), α, β)) if v ≤ α then return v β = MIN(β, v) return v

44. Properties of α-β Pruning does not affect the final result (but be careful) Good move ordering improves effectiveness of pruning If search depth is d, what is the time complexity of minimax? – O(b d ) With perfect pruning, get down to O(b d/2 ) – Doubles solvable depth [Adapted from Russell]