1. 1 Chapter 6 Game Playing

2. 2 Chapter 6 Contents l Game Trees l Assumptions l Static evaluation functions l Searching game trees l Minimax l Bounded lookahead l Alpha-beta pruning l Checkers

3. 3 Chapter 6 Contents (2) l Chess l Go l Games of chance

4. 4 Game Trees l Game trees are used to represent two- player games. l Alternate moves in the game are represented by alternate levels in the tree (plies). l Nodes in the tree represent positions. l Edges between nodes represent moves. l Leaf nodes represent won, lost or drawn positions. l For most games, the game tree can be enormous.

5. 5 Game Trees This is an example of a partial game tree for the game tic-tac-toe. Even for this simple game, the game tree is very large.

6. 6 Assumptions l In talking about game playing systems, we make a number of assumptions: nThe opponent is rational – will play to win. nThe game is zero-sum – if one player wins, the other loses. nUsually, the two players have complete knowledge of the game. For games such as poker, this is clearly not true.

7. 7 Static Evaluation Functions l Since game trees are too large to be fully searched, it is important to have a function to statically evaluate a given position in the game. l A static evaluator assigns a score to a position: nHigh positive = computer is winning nZero = even game nHigh negative = opponent is winning l It is most important that a static evaluator will give a better score to a better position – the actual values are not so important.

8. 8 Searching Game Trees l Exhaustively searching a game tree is not usually a good idea. l Even for a game as simple as tic-tac-toe there are over 350,000 nodes in the complete game tree. l An additional problem is that the computer only gets to choose every other path through the tree – the opponent chooses the others.

9. 9 Minimax l Minimax is a method used to evaluate game trees. l A static evaluator is applied to leaf nodes, and values are passed back up the tree to determine the best score the computer can obtain against a rational opponent.

10. 10 Minimax – Animated Example 5 1362270 Max Min Max 5 5 6 70 6 6 3 3 31 The computer can obtain 6 by choosing the right hand edge from the first node.

11. 11 Bounded Lookahead l For trees with high depth or very high branching factor, minimax cannot be applied to the entire tree. l In such cases, bounded lookahead is applied: nWhen search reaches a specified depth, the search is cut off, and the static evaluator applied. l Must be applied carefully: In some positions a static evaluator will not take into account significant changes that are about to happen.

12. 12 Alpha-beta Pruning l A method that can often cut off a large part of the game tree. l Based on the idea that if a move is clearly bad, there is no need to follow the consequences of it.

13. 13 Alpha-beta Pruning – Example. l In this tree, having examined the nodes with values 7 and 1 there is no need to examine the final node.

14. 14 Alpha-beta Pruning – Better Ex a b c de f g 123457102615 max min max

15. 15 Checkers l In 1959, Arthur Samuel published a paper in which he described a computer program that could play checkers to a high level using minimax and alpha-beta pruning. l Chinook, developed in Canada defeated the world champion: nUses alpha-beta pruning. nHas a database of millions of end games. nAlso has a database of openings. nUses heuristics and knowledge about the game.

16. 16 Chess l In 1997, Deep Blue defeated world champion, Garry Kasparov. l This has not yet been repeated. l Current systems use parallel search, alpha-beta pruning, databases of openings and heuristics. l The deeper in a search tree the computer can search, the better it plays.

17. 17 Go l Go is a complex game played on a 19x19 board. l Average branching factor in search tree around 360 (compared to 38 for chess). l The best computer programs cannot compete yet with the best human players. l Methods use pattern matching or selective search to explore the most appropriate parts of the search tree.

18. 18 Games of Chance l The methods described so far do not work well with games of chance such as poker or backgammon. l Expectiminimax is a variant of minimax designed to deal with chance. l Nodes have expected values based on probabilities.

19. 19 Games of Chance, Example MAX CHANCE MIN 43123523 2313.9.1.8.2 1.4 2.1