1. AD FOR GAMES Lecture 4

2. M INIMAX AND A LPHA -B ETA R EDUCTION Borrows from Spring 2006 CS 440 Lecture Slides

3. M OTIVATION Want to create programs to play games Want to play optimally Want to be able to do this in a reasonable amount of time

4. T YPES OF G AMES Backgammon Monopoly Chess Checkers Go Othello Card GamesBattleship Fully Observable Partially Observable Deterministic Nondeterministic (Chance) Minimax is for deterministic, fully observable games

5. H OW TO PLAY A GAME BY SEARCHING General Scheme Consider all legal moves, each of which will lead to some new state of the environment (‘board position’) Evaluate each possible resulting board position Pick the move which leads to the best board position. Wait for your opponent’s move, then repeat. Key problems Representing the ‘board’ Representing legal next boards Evaluating positions Looking ahead

6. G AME T REES Represent the problem space for a game by a tree Nodes represent ‘board positions’; edges represent legal moves. Root node is the position in which a decision must be made. Evaluation function f assigns real-number scores to `board positions.’ Terminal nodes represent ways the game could end, labeled with the desirability of that ending (e.g. win/lose/draw or a numerical score)

7. B ASIC I DEA ( SEARCHING ) Search problem Searching a tree of the possible moves in order to find the move that produces the best result Depth First Search algorithm Assume the opponent is also playing optimally Try to guarantee a win anyway!

8. R EQUIRED P IECES FOR M INIMAX An initial state The positions of all the pieces Whose turn it is Operators Legal moves the player can make Terminal Test Determines if a state is a final state Utility Function Other name: static evaluation function

9. U TILITY F UNCTION Gives the utility of a game state utility(State) Examples -1, 0, and +1, for Player 1 loses, draw, Player 1 wins, respectively Difference between the point totals for the two players Weighted sum of factors (e.g. Chess) utility(S) = w 1 f 1 (S) + w 2 f 2 (S) +... + w n f n (S) f 1 (S) = (Number of white queens) – (Number of black queens), w 1 = 9 f 2 (S) = (Number of white rooks) – (Number of black rooks), w 2 = 5...

10. T WO A GENTS MAX Wants to maximize the result of the utility function Winning strategy if, on MIN's turn, a win is obtainable for MAX for all moves that MIN can make MIN Wants to minimize the result of the evaluation function Winning strategy if, on MAX's turn, a win is obtainable for MIN for all moves that MAX can make

11. E XAMPLE Coins game There is a stack of N coins In turn, players take 1, 2, or 3 coins from the stack The player who takes the last coin loses

12. C OINS G AME : F ORMAL D EFINITION Initial State: The number of coins in the stack Operators: 1. Remove one coin 2. Remove two coins 3. Remove three coins Terminal Test: There are no coins left on the stack Utility Function: F(S) F(S) = 1 if MAX wins, 0 if MIN wins

13. N = 4 K = N = 3 K = N = 2 K = N = 1 K = N = 0 K = N = 1 K = N = 2 K = N = 0 K = N = 0 K = N = 1 K = N = 0 K = N = 1 K = N = 0 K = N = 0 K = 1 2 3 32 1 211 1211 N = 0 K = 1F(S)=0 F(S)=1 MAX MIN

14. N = 4 K = 1 N = 3 K = 0 N = 2 K = 0 N = 1 K = 1 N = 0 K = 1 N = 1 K = 0 N = 2 K = 1 N = 0 K = 1 N = 0 K = 1 N = 1 K = 0 N = 0 K = 0 N = 1 K = 1 N = 0 K = 0 N = 0 K = 0 1 2 3 32 1 211 1211 N = 0 K = 1 1F(S)=0 F(S)=1 MAX MIN Solution

15. Basic Algorithm

16. D EMO T IME See: WinMinimaxCS project demo

17. E XERCISE T IME See board ….

18. A NALYSIS Max Depth: 5 Branch factor: 3 Number of nodes: 15 Even with this trivial example, you can see that these trees can get very big Generally, there are O(b d ) nodes to search for Branch factor b: maximum number of moves from each node Depth d: maximum depth of the tree Exponential time to run the algorithm! How can we make it faster??????

19. A LPHA -B ETA P RUNING Main idea: Avoid processing subtrees that have no effect on the result Two new parameters α : The best value for MAX seen so far β : The best value for MIN seen so far α is used in MIN nodes, and is assigned in MAX nodes β is used in MAX nodes, and is assigned in MIN nodes

20. A LPHA -B ETA P RUNING MAX (Not at level 0) If a subtree is found with a value k greater than the value of β, then we do not need to continue searching subtrees MAX can do at least as good as k in this node, so MIN would never choose to go here! MIN If a subtree is found with a value k less than the value of α, then we do not need to continue searching subtrees MIN can do at least as good as k in this node, so MAX would never choose to go here!

21. Algorithm

22. CSE 391 - Intro to AI 22 A LPHA -B ETA P RUNING A way to improve the performance of the Minimax Procedure Basic idea: “If you have an idea which is surely bad, don’t take the time to see how truly awful it is” ~ Pat Winston >=2

23. CSE 391 - Intro to AI 23 A LPHA -B ETA P RUNING Traverse the search tree in depth-first order For each MAX node n, α(n)=maximum child value found so far Starts with –  Increases if a child returns a value greater than the current α(n) Lower-bound on the final value For each MIN node n, β(n)=minimum child value found so far Starts with +  Decreases if a child returns a value less than the current β(n) Upper-bound on the final value MAX cutoff rule: At a MAX node n, cut off search if α(n)>=β(n) MIN cutoff rule: At a MIN node n, cut off search if β(n)<=α(n) Carry α and β values down in search

24. function Max-Value (s,α,β) inputs: s: current state in game, A about to play α: best score (highest) for A along path to s β: best score (lowest) for B along path to s output: min(β, best-score (for A) available from s) if ( s is a terminal state ) then return ( terminal value of s ) else for each s’ in Succ(s) α:= max( α, Min-value(s’,α,β)) if ( α≥β) then return β return α function Min-Value (s’,α,β) output: max(α, best-score (for B) available from s ) if ( s is a terminal state ) then return ( terminal value of s) else for each s’ in Succs(s) β:= min( β, Max-value(s’,α,β)) if (β≤α) then return α return β

25. CSE 391 - Intro to AI 25

26. CSE 391 - Intro to AI 26

27. CSE 391 - Intro to AI 27

28. CSE 391 - Intro to AI 28

29. CSE 391 - Intro to AI 29

30. CSE 391 - Intro to AI 30

31. CSE 391 - Intro to AI 31

32. CSE 391 - Intro to AI 32

33. CSE 391 - Intro to AI 33

34. CSE 391 - Intro to AI 34

35. CSE 391 - Intro to AI 35

36. CSE 391 - Intro to AI 36

37. CSE 391 - Intro to AI 37

38. CSE 391 - Intro to AI 38

39. CSE 391 - Intro to AI 39

40. CSE 391 - Intro to AI 40

41. CSE 391 - Intro to AI 41

42. CSE 391 - Intro to AI 42

43. E FFECTIVENESS OF A LPHA -B ETA P RUNING Guaranteed to compute same root value as Minimax Worst case: no pruning, same as Minimax (O(bd)) Best case: when each player’s best move is the first option examined, you examine only O(bd/2) nodes, allowing you to search twice as deep!

44. C ONCLUSION Minimax finds optimal play for deterministic, fully observable, two-player games Alpha-Beta reduction makes it faster Interesting Link(s): http://www.ocf.berkeley.edu/~yosenl/extras/alphabeta/ alphabeta.html http://www.ocf.berkeley.edu/~yosenl/extras/alphabeta/ alphabeta.html

45. Y OUR KBS 2 P ROJECT We will focus on deterministic, two-player, fully observable games You will use minimax, alpha beta pruning and transposition tables ( next college )

46. H OMEWORK Study chapter 10.5.2 Games form your study book Study this presentation

47. H OMEWORK AND P RACTICE Implement “ Tic Tac Toe with a Twist ” with two human players. See DEMO now During practice you are asked to implement an AI player using minimax for this game.