1. Adversarial Search and Game Playing
Russell and Norvig: Chapter 5
Russell and Norvig: Chapter 6
CS121 – Winter 2003
Adversarial Search and Game Playing

2. Adversarial Search and Game Playing
Game-Playing Agent
environment
agent ?
sensors
actuators
Environment
Adversarial Search and Game Playing

3. Perfect Two-Player Game
Two players MAX and MIN take turn (with MAX playing first)
State space
Initial state
Successor function
Terminal test
Score function, that tells whether a terminal state is a win (for MAX), a loss, or a draw
Perfect knowledge of states, no uncertainty in successor function
Adversarial Search and Game Playing

4. Example: Grundy’s Game
Initial state: a stack of 7 coins
State: a set of stacks
Successor function: Break one stack of coins into two unequal stacks
Terminal state: All stacks contain one or two coins
Score function: terminal state is a win for MAX if it was generated by MAX, and a loss otherwise
Adversarial Search and Game Playing

5. Adversarial Search and Game Playing
Game Graph/Tree +1 -1
Adversarial Search and Game Playing

6. Partial Tree for Tic-Tac-Toe
Adversarial Search and Game Playing

7. Uncertainty in Action Model
Adversarial Search and Game Playing

8. Uncertainty in Action Model?
Make the best decision assuming
the worst-case outcome of each action
Adversarial Search and Game Playing

9. Adversarial Search and Game Playing
AND/OR Tree
Adversarial Search and Game Playing

10. Labeling of AND/OR Tree
Adversarial Search and Game Playing

11. Example: Grundy’s Game-1 -1 -1 -1 +1 -1 -1 +1 +1 -1 +1 -1 +1
Adversarial Search and Game Playing

12. Adversarial Search and Game Playing
But in general the search tree is
too big to make it possible to reach
the terminal states!
Adversarial Search and Game Playing

13. Adversarial Search and Game Playing
But in general the search tree is
too big to make it possible to reach
the terminal states!
Examples:
Checkers: ~1040 nodes
Chess: ~10120 nodes
Adversarial Search and Game Playing

14. Evaluation Function of a State
e(s) = + if s is a win for MAX
e(s) = - if s is a win for MIN
e(s) = a measure of how “favorable” is s for MAX > 0 if s is considered favorable to MAX < 0 otherwise
Adversarial Search and Game Playing

15. Adversarial Search and Game Playing
Example: Tic-Tac-Toe
e(s) = number of rows, columns, and diagonals open for MAX
- number of rows, columns, and diagonals open for MIN
8-8 = 0
6-4 = 2
3-3 = 0
Adversarial Search and Game Playing

16. Adversarial Search and Game Playing
Example
Tic-Tac-Toe with
horizon = 2 1 -1 1 -2
6-5=1
5-6=-1
5-5=0
5-5=1
4-5=-1
6-4=2
5-4=1
6-6=0
4-6=-2
Adversarial Search and Game Playing

17. Adversarial Search and Game Playing
Example 1 1 1 3 2 1 2 1 2 3 1 1
Adversarial Search and Game Playing

18. Adversarial Search and Game Playing
Minimax procedure
Expand the game tree uniformly from the current state (where it is MAX’s turn to play) to depth h
Compute the evaluation function at every leaf of the tree
Back-up the values from the leaves to the root of the tree as follows:
A MAX node gets the maximum of the evaluation of its successors
A MIN node gets the minimum of the evaluation of its successors
Select the move toward the MIN node that has the maximal backed-up value
Horizon of the procedure
Needed to limit the size of
the tree or to return a
decision within allowed time
Adversarial Search and Game Playing

19. Adversarial Search and Game Playing
Game Playing (for MAX)
Repeat until win, lose, or draw
Select move using Minimax procedure
Execute move
Observe MIN’s move
Adversarial Search and Game Playing

20. Adversarial Search and Game Playing
Issues
Choice of the horizon
Size of memory needed
Number of nodes examined
Adversarial Search and Game Playing

21. Adversarial Search and Game Playing
Adaptive horizon
Wait for quiescence
Extend singular nodes /Secondary search
Note that the horizon may not then be the same on every path of the tree
Adversarial Search and Game Playing

22. Adversarial Search and Game Playing
Issues
Choice of the horizon
Size of memory needed
Number of nodes examined
Adversarial Search and Game Playing

23. Adversarial Search and Game Playing
Alpha-Beta Procedure
Generate the game tree to depth h in depth-first manner
Back-up estimates (alpha and beta values) of the evaluation functions whenever possible
Prune branches that cannot lead to changing the final decision
Adversarial Search and Game Playing

24. Adversarial Search and Game Playing
Example
Adversarial Search and Game Playing

25. Adversarial Search and Game Playing
Example
The beta value of a MIN
node is a higher bound on
the final backed-up value.
It can never increase
b = 1 1
Adversarial Search and Game Playing

26. Adversarial Search and Game Playing
Example
b = 0
The beta value of a MIN
node is a higher bound on
the final backed-up value.
It can never increase 1 1
Adversarial Search and Game Playing

27. Adversarial Search and Game Playing
Example
b = -1
The beta value of a MIN
node is a higher bound on
the final backed-up value.
It can never increase 1 -1 1
Adversarial Search and Game Playing

28. Adversarial Search and Game Playing
Example
a = -1
b = -1
The alpha value of a MAX
node is a lower bound on
the final backed-up value.
It can never decrease 1 -1 1
Adversarial Search and Game Playing

29. Adversarial Search and Game Playing
Example
a = -1
b = -1 1 -1 1
Adversarial Search and Game Playing

30. Adversarial Search and Game Playing
Example
a = -1
b = -1
b = -1 1 1 -1 -1
Search can be discontinued below
any MIN node whose beta value is
less than or equal to the alpha value
of one of its MAX ancestors
Adversarial Search and Game Playing

31. Adversarial Search and Game Playing
Alpha-Beta Example 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

32. Adversarial Search and Game Playing
Alpha-Beta Example 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

33. Adversarial Search and Game Playing
Alpha-Beta Example 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

34. Adversarial Search and Game Playing
Alpha-Beta Example -3 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

35. Adversarial Search and Game Playing
Alpha-Beta Example -3 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

36. Adversarial Search and Game Playing
Alpha-Beta Example -3 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

37. Adversarial Search and Game Playing
Alpha-Beta Example 3 -3 3 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

38. Adversarial Search and Game Playing
Alpha-Beta Example 3 -3 3 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

39. Adversarial Search and Game Playing
Alpha-Beta Example 3 -3 3 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

40. Adversarial Search and Game Playing
Alpha-Beta Example 3 -3 3 5 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

41. Adversarial Search and Game Playing
Alpha-Beta Example 3 2 -3 3 2 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

42. Adversarial Search and Game Playing
Alpha-Beta Example 3 2 -3 3 2 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

43. Adversarial Search and Game Playing
Alpha-Beta Example 2 2 3 2 -3 3 2 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

44. Adversarial Search and Game Playing
Alpha-Beta Example 2 2 3 2 -3 3 2 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

45. Adversarial Search and Game Playing
Alpha-Beta Example 2 2 3 2 -3 3 2 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

46. Adversarial Search and Game Playing
Alpha-Beta Example 2 2 3 2 -3 3 2 5 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

47. Adversarial Search and Game Playing
Alpha-Beta Example 2 2 3 2 1 -3 3 2 1 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

48. Adversarial Search and Game Playing
Alpha-Beta Example 2 2 3 2 1 -3 3 2 1 -3 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

49. Adversarial Search and Game Playing
Alpha-Beta Example 2 2 3 2 1 -3 3 2 1 -3 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

50. Adversarial Search and Game Playing
Alpha-Beta Example 2 1 2 1 3 2 1 -3 3 2 1 -3 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

51. Adversarial Search and Game Playing
Alpha-Beta Example 2 1 2 1 3 2 1 -3 3 2 1 -3 -5 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

52. Adversarial Search and Game Playing
Alpha-Beta Example 2 1 2 1 3 2 1 -3 3 2 1 -3 -5 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

53. Adversarial Search and Game Playing
Alpha-Beta Example 2 1 2 1 -5 3 2 1 -5 -3 3 2 1 -3 -5 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

54. Adversarial Search and Game Playing
Alpha-Beta Example 2 1 2 1 -5 3 2 1 -5 -3 3 2 1 -3 -5 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

55. Adversarial Search and Game Playing
Alpha-Beta Example 1 1 2 1 2 1 -5 3 2 1 -5 -3 3 2 1 -3 -5 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

56. Adversarial Search and Game Playing
Alpha-Beta Example 1 1 2 1 2 2 1 -5 2 3 2 1 -5 2 -3 3 2 1 -3 -5 2 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

57. Adversarial Search and Game Playing
Alpha-Beta Example 1 1 2 1 2 2 1 -5 2 3 2 1 -5 2 -3 3 2 1 -3 -5 2 5 -3 3 3 -3
Adversarial Search and Game Playing 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

58. Adversarial Search and Game Playing
How Much Do We Gain? 1
Size of tree = O(bh)
In the worst case all
nodes must be examined
In the best case, only O(bh/2) nodes need to be examined 1 2 1 2 2 1 -5 2 3 2 1 -5 2
Exercise: In which order should the
node be examined in order to achieve
the best gain? -3 3 2 1 -3 -5 2 5 -3 3 3 -3 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2
Adversarial Search and Game Playing

59. Adversarial Search and Game Playing
Alpha-Beta Procedure
The alpha of a MAX node is a lower bound on the backed-up value
The beta of a MIN node is a higher bound on the backed-up value
Update the alpha/beta of the parent of a node N when all search below N has been completed or discontinued
Adversarial Search and Game Playing

60. Adversarial Search and Game Playing
Alpha-Beta Procedure
The alpha of a MAX node is a lower bound on the backed-up value
The beta of a MIN node is a higher bound on the backed-up value
Update the alpha/beta of the parent of a node N when all search below N has been completed or discontinued
Discontinue the search below a MAX node N if its alpha is  beta of a MIN ancestor of N
Discontinue the search below a MIN node N if its beta is  alpha of a MAX ancestor of N
Adversarial Search and Game Playing

61. Adversarial Search and Game Playing
Alpha-Beta + …
Iterative deepening
Singular extensions
Adversarial Search and Game Playing

62. Adversarial Search and Game Playing
Checkers
Adversarial Search and Game Playing
© Jonathan Schaeffer

63. Adversarial Search and Game Playing
Chinook vs. Tinsley
Name: Marion Tinsley
Profession: Teach
mathematics
Hobby: Checkers
Record: Over 42 years
loses only 3 (!)
games of checkers
Adversarial Search and Game Playing
© Jonathan Schaeffer

64. Adversarial Search and Game Playing
Chinook
First computer to win human world championship!
Adversarial Search and Game Playing

65. Adversarial Search and Game Playing
Chess
Adversarial Search and Game Playing

66. Adversarial Search and Game Playing
Man vs. Machine
Kasparov
5’10”
176 lbs
34 years
50 billion neurons
2 pos/sec
Extensive
Electrical/chemical
Enormous
Deep Blue
6’ 5”
2,400 lbs
4 years
512 processors
200,000,000 pos/sec
Primitive
Electrical
None
Name
Height
Weight
Age
Computers
Speed
Knowledge
Power Source
Ego
Adversarial Search and Game Playing
© Jonathan Schaeffer

67. Adversarial Search and Game Playing
Reversi/Othello
Adversarial Search and Game Playing
© Jonathan Schaeffer

68. Adversarial Search and Game Playing
Othello
Name: Takeshi
Murakami
Title: World
Othello Champion
Crime: Man crushed
by machine
Adversarial Search and Game Playing
© Jonathan Schaeffer

69. Adversarial Search and Game Playing
Go: On the One Side
Name: Chen Zhixing
Author: Handtalk (Goemate)
Profession: Retired
Computer skills: self-
taught assembly
language programmer
Accomplishments:
dominated computer go
for 4 years.
Adversarial Search and Game Playing
© Jonathan Schaeffer

70. Adversarial Search and Game Playing
Go: And on the Other
Gave Handtalk a 9 stone
handicap and still easily
beat the program,
thereby winning $15,000
Adversarial Search and Game Playing
© Jonathan Schaeffer

71. Perspective on Games: Pro
“Saying Deep Blue doesn’t really think about chess is like saying an airplane doesn't really fly because it doesn't flap its wings”
Drew McDermott
Adversarial Search and Game Playing
© Jonathan Schaeffer

72. Perspective on Games: Con
“Chess is the Drosophila of artificial intelligence. However, computer chess has developed much as genetics might have if the geneticists had concentrated their efforts starting in 1910 on breeding racing Drosophila. We would have some science, but mainly we would have very fast fruit flies.”
John McCarthy
Adversarial Search and Game Playing
© Jonathan Schaeffer

73. Adversarial Search and Game Playing
Other Games
Multi-player games, with alliances or not
Games with randomness in successor function (e.g., rolling a dice)
Incompletely known states (e.g., card games)
Adversarial Search and Game Playing

74. Adversarial Search and Game Playing
Summary
Two-players game as a domain where action models are uncertain
Optimal decision in the worst case
Game tree
Evaluation function / backed-up value
Minimax procedure
Alpha-beta procedure
Adversarial Search and Game Playing