1. How computers play games with you CS161, Spring ‘03 Nathan Sturtevant

2. Outline Historic Examples Classes of Games Algorithms  Minimax   -  pruning Other techniques Multi-Player Games

3. Successful Game Programs Checkers  Chinook 1992 Tinsley won 40-game match, 4-2-34 1994 Tinsley withdrew due to health reasons 444 billion move end-game database Chess  Kasparov is currently the best human  1997 Deep Blue won exhibition match 2-1-3  2003 Deep Junior played to a draw

4. Game Programs (continued) Othello (Reversi)  1997, Logistello beat Murakami 6-0 (264/120) Scrabble  Maven 1998 played Adam Logan, won 9-5 Came back from down 98 to win with MOUTHPART Awari (Mancala)  Solved in 2002 - draw  http://awari.cs.vu.nl/

5. Overview - Types of Games Single-Agent Search  1 player v. a difficult problem  Defined by: Start state Successor function Heuristic function Goal test

6. Overview - Types of Games Game Search (Adversary Search)  Defined by: Initial State Successor function Terminal Test Utility / payoff function  Similar to heuristics in single agent problems

7. Chinese Checkers Based on European game Halma Americans called it Chinese Checkers  1 player game?  2 player game?  Multi-player game?

8. Classes of Games Deterministic v. Non-deterministic  Chess v. Backgammon Perfect Information v. Imperfect information  Checkers v. Bridge Zero-sum (strictly competitive)  Prisoners dilemna Non-zero sum

9. Classes of Games DeterministicChance Perfect information Imperfect information

10. Classes of Games DeterministicChance Perfect information Chess, checkers, go, othello, chinese checkers Imperfect information

11. Classes of Games DeterministicChance Perfect information Chess, checkers, go, othello, chinese checkers Backgammon, monopoly, risk Imperfect information

12. Classes of Games DeterministicChance Perfect information Chess, checkers, go, othello, chinese checkers Backgammon, monopoly, risk Imperfect information Stratego

13. Classes of Games DeterministicChance Perfect information Chess, checkers, go, othello, chinese checkers Backgammon, monopoly, risk Imperfect information StrategoBridge, poker, scrabble, (real life)

14. How do we simulate games? Build a game tree  Start state at the root  All possible moves as children

15. Tic-Tac-Toe Me Opponent

16. How do we choose our move? Apply utility function at the leaves of the tree  In tic-tac-toe, count how many rows and columns are occupied by each player and subtract

17. Tic-Tac-Toe Me Opponent x: 2r 2c 2d o: 2r 1c 1d Utility = 2Utility = 3Utility = ∞Utility = 2 x: 2r 3c 2d o: 2r 1c 1d x: 3r 3c 2d o: 2r 2c 0d x: 2r 2c 2d o: 2r 2c 0d Utility = 3Utility = ∞ Utility = 3

18. What is our algorithm? Apply utility function at the leaves of the tree  In tic-tac-toe, count how many rows and columns are occupied by each player and subtract  Back-up values in the tree This calculates the “minimax” value of a tree

19. Minimax Maximizer Minimizer 23∞2 3 3 ∞ 1 - ply Minimizers strategy

20. Minimax - Properties Complete?  Yes - if tree is finite Optimal?  Yes - against an optimal opponent Time Complexity?  O(b d ) Space Complexity?  O(bd)

21. Minimax Assume our computer can expand 10 5 nodes/sec Assume we have 100 seconds to move 10 7 nodes/move Tic-tac-toe  9! = 362880 (naïve) ways to play a game (b=4)  3 9 = 19683 possible states (upper bound) on a board Chess  b = 35, d = 100, must search 2 154 nodes

22. Minimax - issues Evaluation function  Where does it come from? Expert knowledge  Chess: material value  Othello (reversi): positional strength Learned information Pre-computed tables Quiescence

23. quiescence

24. Minimax - issues Quiescence  We don’t see the consequences of our bad choices  quiescence search Horizon problem  We avoid dealing with a bad situation

25. Minimax In Chess  b = 35  10 7 nodes/move  Can search 4-ply into tree (human novice)  Good humans can search 8-ply  Kasparov searches about 12-ply What to do?   -  pruning

26. Minimax Maximizer Minimizer 23∞2 3 3 ∞ 1 - ply Minimizers strategy

27.  -  pruning  = lower bound on Maximizer’s score  Start at -∞  = upper bound on Minimizer’s score  Start at ∞

28. Maximizer Minimizer 1 -∞ ∞  = -∞  = ∞  = -∞  = ∞  = -∞  = ∞ ≥1 

29. Maximizer Minimizer 12 2 -∞ ∞  = -∞  = ∞  = -∞  = ∞  = 1  = ∞ ≥1

30. Maximizer Minimizer 12 2 -∞ ∞  = -∞  = ∞  = -∞  = ∞  = 2  = ∞ 

31. Maximizer Minimizer 1 2 3 2 -∞ ∞  = -∞  = ∞ ≥ 3  = -∞  = 2  = -∞  = 2 ≤ 2 

32. Maximizer Minimizer 1 2 3 2 2 -∞ ∞  = -∞  = ∞ ≥ 3 ≥ 2  = -∞  = 2  = 3  = 2 

33. Maximizer Minimizer 123 2 2 5 ≥ 5 -∞ ∞  = 2  = ∞ ≥ 3 ≥ 2  = 2  = ∞  = 2  = ∞ 

34. Maximizer Minimizer 123 2 2 56 6 -∞ ∞  = 2  = ∞ ≥ 3 ≥ 2  = 2  = ∞  = 5  = ∞  ≥ 5

35. Maximizer Minimizer 123 2 2 567 6 -∞ ∞  = 2  = ∞ ≥ 3 ≥ 2  = 2  = 6 ≤ 6 ≥7  = 2  = 6 

36. Maximizer Minimizer 123 2 2 567 6 -∞ ∞  = 2  = ∞ ≥ 3 ≥ 2  = 2  = 6 ≤ 6 ≥7  = 7  = 6  6 6

37.  -  pruning Complete?  Yes - if tree is finite Optimal?  Computes same value as minimax Time Complexity?  Best case O(b d/2 )  Average case O(b 3d/4 )

38.  -  pruning Effectiveness depends on order of moves in tree  In practice, we can usually get best-case performance Chess  Before we could search 4-ply into tree  Now we can search 8-ply into tree

39. Other Techniques Transposition tables Opening / Closing book

40. Transposition Tables Only using linear about of memory  Search only takes a few kb of memory Most games aren’t trees but graphs

41. Transposition Tables

42. A lot of duplicated effort Transposition tables hash game states into table  Store saved minimax value in table Pre-compute & store values  Opening book  Closing book

43. Multi-Player Games 2-Player game trees have a single minimax value Games with ≥ 2 players use a n-tuple of scores  ie (3, 2, 5) The sum of values in every tuple should be constant

44. Max n 1 3 (7, 3, 0) 3 (3, 2, 5) (7, 3, 0)(0, 10, 0)(1, 4, 5) (7, 3, 0) 3 (0, 10, 0) 3 (4, 2, 4) 222 3 (1, 4, 5) 3 (4, 3, 3) (3 Players)

45. Can we prune max n trees In minimax we bound the game tree value In max n we bound based on sum of values  All scores sum to 10  If Player 1 gets 7 points…  Player 2-3 will get ≤ 3 points

46. Shallow Max n Pruning 1 3 (7, 3, 0) 3 (3, 2, 5) (7, 3, 0)(0, 10, 0) (≤6, ≥4, ≤6) (7, 3, 0) 3 (0, 10, 0) 222 3 (1, 4, 5) (3 Players) (≥7, ≤3, ≤3)

47. Shallow Max n Pruning Complete?  Yes Optimal?  Yes* Time Complexity?  Best-case**: b d/2  Average-case: b d Space Complexity?  bd

48. Max n Pruning Why is max n weak in practice?  Only compares 2 scores out of n players  Relies on game evaluation properties, not ordering Last-Branch Pruning Speculative Pruning

49. Last-Branch/Speculative Pruning 1 (3, 3, 4) 2 (3 Players) 2 3 3 (1, 4, 5) 1 (2, 4, 4) 2

50. Last Branch/Spec. Pruning Best case: O(b d·(n-1)/n )  As b gets large  Dependent only on node ordering in tree  http://www.cs.ucla.edu/~nathanst/ for more info

51. Imperfect Information Most card games have imperfect information We can use monte-carlo simulation  Create many consistent samples of possible opponent hands  Solve using perfect-information methods  Combine results together to make next move