1. Game tree search Thanks to Andrew Moore and Faheim Bacchus for slides!

2. Game tree search In this lecture we will cover some basics of two player zero sum discrete finite deterministic games of perfect information

5. More on the meaning of “Zero-Sum” We will focus on “Zero Sum” games. “Zero-sum” games are fully competitive if one player wins, the other player loses more specifically, the amount of money I win (lose) at poker is the amount of money you lose (win) More general games can be cooperative some outcomes are preferred by both of us, or at least our values aren’t diametrically opposed

6. Scissors cut paper, paper covers rock, rock smashes scissors Represented as a matrix: Player I chooses a row, Player II chooses a column Payoff to each player in each cell (P I / P II) 1: win, 0: tie, -1: loss is this game “zero-sum”? RPS 0/00/0 0/00/0 0/00/0 -1/1 1/-1 R P S Player II Player I Is Rock Paper Scissors “Zero-Sum”?

7. Dilemma: Two prisoners are in separate cells and there is not enough evidence to convict them If one confesses, while the other doesn’t: the confessor goes free the other sentenced to 4 years If both confess both are sentenced to 3 years If neither confess: both are sentenced to 1 year on minor charge Is this game “zero sum”? DoDon’t 3/33/3 1/11/1 0/40/4 4/04/0 Do Don’t Is The Prisoner’s Dilemma “Zero-Sum”? Prisoner II: confess? Prisoner I: confess?

8. Is The Coffee-Bot Dilemma “Zero-Sum”? Two robots: Green (Craig’s), Red (Fahiem’s) one cup of coffee and tea left both Craig and Faheim prefer coffee (value 10) but, tea is acceptable (value 8) Both robot’s go for coffee they collide and get no payoff Both go for tea: collide and get no payoff One goes for coffee, other for tea: coffee robot gets 10 tea robot gets 8 CoffeeTea 0/00/0 8/10 0/00/0 10/0 Coffee Tea

13. With a search space defined for II-Nim, we can define a Game Tree. A game tree looks like a search tree Layers reflect alternating moves between A and B Player A doesn’t decide where to go alone after Player A moves to a state, B decides which of the state’s children to move to. Thus, A must have a strategy: A must know what to do for each possible move of B. “What to do” will depend on how B plays.

23. Question: what happens if there are loops in the tree? How would looping influence your determination of the minimax value for a node?

24. Imagine you have a game with N states, that each state has b successors, and the length of the game is usually D moves. Minimax will expand O(b D ) states, which is both BEST and WORSE case scenario. This is different than regular DFS! But, what if N is less than b D ? In chess, for example, b D = 10 120, but N = 10 40... Managing games with fewer states than game tree nodes. Note: info on this slide WON’T be on exams!

25. Managing games with fewer states than game tree nodes. Make a huge array of size N. Give each element in the array one of the following values: ?: We don’t know who wins from this state W: We know white wins from this position B: We know black wins from this position. Mark all terminal states with their values (‘W’ or ‘B’). Suppose we have 4 pieces left at the end of a chess game. With enough computing power, we can compute, for all such game states, if the position is a win for White, Black or a Draw. Note: info on this slide WON’T be on exams!

26. Managing games with fewer states than game tree nodes. Look through all states that remain marked with a ‘?’. For states where W is about to move: If all successors are marked ‘B’, mark the current state ‘B’. If any successors is marked ‘W’, mark the current state ‘W’. Otherwise leave the state unchanged. For states where B is about to move: If all successors are marked ‘W’, mark the current state ‘W’. If any successors is marked ‘B’, mark the current state ‘B’. Otherwise leave the state unchanged. Repeat! Until there are none of the elements in the array change their value. Any state remaining at ‘?’ is a state from which no one can force a win. Note: to turn this algorithm into a strategy, you also need to record pointers from given states to their best successors. This is a DYNAMIC PROGRAMMING technique. Note: info on this slide WON’T be on exams!

33. Question: what happens if there are loops in the tree?

34. A s1s2 s3 14 128 β = 8 24 α = 2, then 4, then.... s4 s5 B 911 2 Example 1: We are currently expanding possible moves for player A, from left to right. Which of the node expansions above could we prune, and why?

35. A s1s2 s3 14 128 β = 8 24 α = 9 s4 s5 B 911 2 Once we discover a node with value ‘9’, there is no need to expand the nodes to the right!

36. B s1s2 s3 6 27 α = 7 93 β = 9, then 3, then.... s4 s5 A 42 8 Example 2: We are currently expanding possible moves for player B, from left to right. Which of the node expansions above could we prune, and why?

37. B s1s2 s3 6 27 α = 7 93 β = 3 s4 s5 A 42 8 Once we discover a node with value ‘3’, there is no need to expand the nodes to the right!

38. Rational Opponents This all assumes that your opponent is rational e.g., will choose moves that minimize your score Storing your strategy is a potential issue: you must store “decisions” for each node you can reach by playing optimally if your opponent has unique rational choices, this is a single branch through game tree if there are “ties”, opponent could choose any one of the “tied” moves: must store strategy for each subtree What if your opponent doesn’t play rationally? will it affect quality of outcome? will your stored strategies work?

40. Heuristic evaluation functions in games Some issues in heuristic search How far should we search in our game tree to determine the value of a node, if we only have a fixed amount of time? What if we stop our search at a level in the search tree where subsequent moves dramatically change our evaluation?

41. Heuristic evaluation functions in games Think of a few games and suggest some heuristics for estimating the “goodness” of a position chess? checkers? your favorite video game? “find the last parking spot”?

42. Question: is there an alpha beta version you can use to search this tree?