1. Adversarial Environments/ Game Playing
Why?
Fun 
Many real-world applications in game theory
Economics
Industry/Operations Research

2. Types of games we’ll consider
2 player
Zero-sum: one player’s loss is the other’s gain
Fully observable environment
Deterministic
Exs: chess, checkers, tic-tac-toe
Not: dice, black jack, poker

3. Strategic thinking ?= intelligence
Two-player games have been a focus of AI since its inception…
Clear success criteria
Begs the question: Is strategic thinking the same as intelligence?

4. How humans play games…
An experiment (by Adriann deGroot) was performed where valid chess boards were shown to novice and expert players…
- experts could reconstruct these perfectly
- novice players did far worse…

5. How humans play games…
An experiment (by Adriann deGroot) was performed where valid chess boards were shown to novice and expert players…
- experts could reconstruct these perfectly
- novice players did far worse…
Random chess positions (non-legal ones) were then shown to the two groups
- experts and novices did just as
badly at reconstructing them!

6. What makes a human play well?
Implications?
What makes a human play well?

7. Implications? What makes a human play well? Problem solving
Visual Memory

8. Strategic thinking ?= intelligence
Humans and computers have different relative strengths in these games:
humans
computers
good at evaluating the strength of a board
good at looking ahead in the game to find winning combinations of moves

9. Tic Tac Toe as search
How can we pose this as a search problem?

10. Baby Nim For winning strategy, should I take (1 or 2 matches)? MAX MIN
Action: Take 1 or 2 matches at each turn
Goal: Take the last match
MAX
MIN
For winning strategy,
should I take (1 or 2 matches)?
MAX

11. Baby Nim MAX MIN MAX MIN MAX 1 2 W L MAX (5) MIN (4) (3) 2 1 2 1
Take 1 or 2 at each turn
Goal: take the last match
(2)
(3)
(1)
MAX
MAX
(2)
MAX wins 1 2 1 2 1 2 1
MIN
(2)
(1)
MIN
(1)
(1) W
= 1.0 1 1 1 2 1
MAX
MAX
(1) L
= -1.0
MAX loses/
MIN wins
( ) = # of remaining matches

12. Baby Nim +1 +1 +1 -1 -1 -1 -1 +1 MAX MIN MAX MIN MAX 1 2 W MAX (5) MIN
(4)
(3)
MIN 2 1 2 1
Take 1 or 2 at each turn
Goal: take the last match
(2)
(1)
MAX
(3)
MAX
(2)
MAX wins 1 2 1 2 1 2 1
MIN
(2)
(1)
MIN
(1)
(1) +1 +1 +1 W
= 1.0 1 1 1 2 1
MAX
MAX
(1) L L L L L
= -1.0 -1 -1 -1 -1
MAX loses/
MIN wins +1
( ) = # of remaining matches

13. Baby Nim +1 +1 +1 +1 -1 -1 -1 -1 +1 MAX MIN MAX MIN MAX 1 2 W MAX (5)
(4)
(3)
MIN 2 1 2 1
Take 1 or 2 at each turn
Goal: take the last match
(2)
MAX
(3)
(1)
MAX
(2)
MAX wins 1 2 1 2 1 2 1
MIN
(2)
(1)
MIN
(1)
(1) +1 +1 +1 W
= 1.0 1 1 1 2 1
MAX
MAX
(1) L L L L L
= -1.0 +1 -1 -1 -1 -1
MAX loses/
MIN wins +1
( ) = # of remaining matches

14. Baby Nim -1 -1 -1 +1 -1 +1 +1 +1 -1 -1 -1 -1 +1 MAX MIN MAX MIN MAX 12 W
MAX
(5)
MAX
MIN
(4)
(3)
MIN 2 1 2 1
Take 1 or 2 at each turn
Goal: take the last match
(2)
(1)
MAX
(3)
MAX
(2)
MAX wins 1 2 1 2 1 2 1
MIN
(2)
(1)
MIN
(1)
(1) -1 -1 -1 +1 -1 +1 +1 W
= 1.0 1 1 1 2 1
MAX
MAX
(1) L
= -1.0 +1 -1 -1 -1 -1
MAX loses/
MIN wins +1
( ) = # of remaining matches

15. Baby Nim +1 +1 -1 +1 -1 -1 -1 +1 -1 +1 +1 +1 -1 -1 -1 -1 +1 MAX MIN2 W
MAX
(5)
MAX
MIN
(4)
(3)
MIN 2 1 2 1
Take 1 or 2 at each turn
Goal: take the last match
(2)
(1)
MAX
(3)
MAX
(2) +1 +1 -1 +1
MAX wins 1 2 1 2 1 2 1
MIN
(2)
(1)
MIN
(1)
(1) -1 -1 -1 +1 -1 +1 +1 W
= 1.0 1 1 1 2 1
MAX
MAX
(1) L
= -1.0 +1 -1 -1 -1 -1
MAX loses/
MIN wins +1
( ) = # of remaining matches

16. Baby Nim +1 -1 +1 +1 -1 +1 -1 -1 -1 +1 -1 +1 +1 +1 -1 -1 -1 -1 +1 MAX2 W
MAX
(5)
MAX
MIN
(4)
(3) +1
MIN -1 2 1 2 1
Take 1 or 2 at each turn
Goal: take the last match
(2)
(3)
(1)
MAX
MAX
(2) +1 +1 -1 +1
MAX wins 1 2 1 2 1 2 1
MIN
(2)
(1)
MIN
(1)
(1) -1 -1 -1 +1 -1 +1 +1 W
= 1.0 1 1 1 2 1
MAX
MAX
(1) W
= -1.0 +1 -1 -1 -1 -1
MIN wins/
MAX loses +1
( ) = # of remaining matches

17. Baby Nim +1 +1 -1 +1 +1 -1 +1 -1 -1 -1 +1 -1 +1 +1 +1 -1 -1 -1 -1 +12 W
MAX
(5)
MAX +1
MIN
(4)
(3) +1
MIN -1 2 1 2 1
Take 1 or 2 at each turn
Goal: take the last match
(2)
MAX
(3)
(1)
MAX
(2) +1 +1 -1 +1
MAX wins 1 2 1 2 1 2 1
MIN
(2)
(1)
MIN
(1)
(1) -1 -1 -1 +1 -1 +1 +1 W
= 1.0 1 1 1 2 1
MAX
MAX
(1) L
= -1.0 +1 -1 -1 -1 -1
MAX loses/
MIN wins +1
( ) = # of remaining matches

18. Baby Nim +1 +1 -1 +1 +1 -1 +1 -1 -1 -1 +1 -1 +1 +1 +1 -1 -1 -1 -1 +1
MAX can always win by taking 2 on first turn 1 2 W
MAX
(5)
MAX +1
MIN
(4)
(3) +1
MIN -1 2 1 2 1
Take 1 or 2 at each turn
Goal: take the last match
(2)
(3)
(1)
MAX
MAX
(2) +1 +1 -1 +1
MAX wins 1 2 1 2 1 2 1
MIN
(2)
(1)
MIN
(1)
(1) -1 -1 -1 +1 -1 +1 +1 W
= 1.0 1 1 1 2 1
MAX
MAX
(1) L
= -1.0 +1 -1 -1 -1 -1
MAX loses/
MIN wins +1
( ) = # of remaining matches

19. Tree Size Tree size gets very large quickly 0 Ply Branching factor = 7

20. Tree Size Tree gets very large quickly 0 Ply Branching factor = 7
Branching Factor Estimates
for different two-player games
2 Ply
Tic-tac-toe
Connect Four
Checkers
Othello
Chess Go

21. Search Times On average: Tic Tac Toe: b ≈ 4, d ≈9  solvable
Chess, b ≈ 35, d ≈100 for reasonable games  exact solution is infeasible!

22. Tree Size Tree gets very large quickly “solved” games
0 Ply
Branching factor = 7
1 Ply
Branching Factor Estimates
for different two-player games
2 Ply
Tic-tac-toe
Connect Four
Checkers
Othello
Chess Go
“solved” games
computer-dominated
Google’s AlphaGo (2015)

23. Tree Size Tree gets very large quickly How to reduce search time?
0 Ply
Branching factor = 7
1 Ply
Branching Factor Estimates
for different two-player games
2 Ply
Tic-tac-toe
Connect Four
Checkers
Othello
Chess Go
“solved” games
How to reduce search time?
computer-dominated
Google’s AlphaGo (2015)