1. Game Playing Revision Mini-Max search Alpha-Beta pruning General concerns on games

2. 2 Why study board games ?  One of the oldest subfields of AI (Shannon and Turing, 1950)  Abstract and pure form of competition that seems to require intelligence  Easy to represent the states and actions  Very little world knowledge required !  Game playing is a special case of a search problem, with some new requirements.

3. 3 Types of games Bridge, poker, scrabble, nuclear war Backgammon, monopoly Chess, checkers, go, othello ChanceDeterministic Imperfect information Perfect information Sea battle

4. 4 Why new techniques for games?  “Contingency” problem:  We don’t know the opponents move !  The size of the search space:  Chess : ~15 moves possible per state, 80 ply  15 80 nodes in tree  Go : ~200 moves per state, 300 ply  200 300 nodes in tree  Game playing algorithms:  Search tree only up to some depth bound  Use an evaluation function at the depth bound  Propagate the evaluation upwards in the tree

5. 5 MINI MAX  Restrictions:  2 players: MAX (computer) and MIN (opponent)  deterministic, perfect information  Select a depth-bound (say: 2) and evaluation function MAX MIN MAX - Construct the tree up till the depth-bound the depth-bound - Compute the evaluation function for the leaves function for the leaves2531443 - Propagate the evaluation function upwards: function upwards: - taking minima in MIN - taking minima in MIN 213 - taking maxima in MAX 3 Select this move

6. 6 The MINI-MAX algorithm: Initialise depthbound; Minimax (board, depth) = IF depth = depthbound THEN return static_evaluation(board); THEN return static_evaluation(board); ELSE IF maximizing_level(depth) ELSE IF maximizing_level(depth) THEN FOR EACH child child of board THEN FOR EACH child child of board compute Minimax(child, depth+1); compute Minimax(child, depth+1); return maximum over all children; return maximum over all children; ELSE IF minimizing_level(depth) ELSE IF minimizing_level(depth) THEN FOR EACH child child of board THEN FOR EACH child child of board compute Minimax(child, depth+1); compute Minimax(child, depth+1); return minimum over all children; return minimum over all children; Call: Minimax(current_board, 0)

7. 7 Alpha-Beta Cut-off  Generally applied optimization on Mini-max.  Instead of:  first creating the entire tree (up to depth-level)  then doing all propagation  Interleave the generation of the tree and the propagation of values.  Point:  some of the obtained values in the tree will provide information that other (non-generated) parts are redundant and do not need to be generated.

8. 8 MIN MAXMAX2 Alpha-Beta idea:  Principles:  generate the tree depth-first, left-to-right  propagate final values of nodes as initial estimates for their parent node. 2222 5 =2 2222 1 1111 - The MIN-value (1) is already smaller than the MAX-value of the parent (2) - The MIN-value can only decrease further, decrease further, - The MAX-value is only allowed to increase, to increase, - No point in computing further below this node below this node

9. 9 Terminology: - The (temporary) values at MAX-nodes are ALPHA-values - The (temporary) values at MIN-nodes are BETA-values MIN MAX MAX 2 2222 5 =2 2222 1 1111 Alpha-value Beta-value

10. 10 The Alpha-Beta principles (1): - If an ALPHA-value is larger or equal than the Beta-value of a descendant node: stop generation of the children of the descendant MIN MAX MAX 2 2222 5 =2 2222 1 1111 Alpha-value Beta-value 

11. 11 The Alpha-Beta principles (2): - If an Beta-value is smaller or equal than the Alpha-value of a descendant node: stop generation of the children of the descendant MIN MAX MAX 2 2222 5 =2 2222 3 1111 Alpha-value Beta-value 

12. 12 873916241135392652123972864 Mini-Max with  at work: 1 2  8 8 8 83 5 = 8 4 6  8 8 8 87 8  9 9 9 991113171921242628323436 10  2 2 2 2 12  4 4 4 4 14 = 4 15  4 4 4 416 18  1 1 1 1 20  3 3 3 3 22 = 5 30  5 5 5 523  5 5 5 531 25  3 3 3 3 27  9 9 9 9 29  6 6 6 6 33  1 1 1 1 35  2 2 2 2 37 = 3  3 3 3 338 39 = 5 MAX MIN MAX 11 static evaluations saved !!

13. 13 “DEEP” cut-offs - For game trees with at least 4 Min/Max layers: the Alpha - Beta rules apply also to deeper levels. 4  4 4 4 4  4 4 4 4  4 4 4 4  4 4 4 4 2  2 2 2 2

14. 14 The Gain: Best case: MAXMIN MAX - If at every layer: the best node is the left-most one Only THICK is explored

15. 15 Example of a perfectly ordered tree MAXMIN MAX 21 20 19 24 23 22 27 26 25 12 11 10 15 14 13 18 17 16 3 2 1 6 5 4 9 8 7 21 24 27 12 15 18 3 6 9 21 12 3 21

16. 16 # (static evaluations saved) = How much gain ? - Alpha / Beta : best case : 2 b d/2 - 1 (if d is even) b (d+1)/2 + b (d-1)/2 - 1 (if d is odd) - The proof is by induction. - In the running example: d=3, b=3 : 11 !

17. 17 Best case gain pictured: 10 100 1000 10000 100000 1234567 # Static evaluations Depth No pruning b = 10 Alpha-Beta Best case - Note: algorithmic scale. - Conclusion: still exponential growth !! - Worst case?? For some trees alpha-beta does nothing, For some trees: impossible to reorder to avoid cut-offs

18. 18 The horinzon effect. Queen lost Pawn lost Queen lost horizon = depth bound of mini-max Because of the depth-bound we prefer to delay disasters, although we don’t prevent them !! we prefer to delay disasters, although we don’t prevent them !!  solution: heuristic continuations

19. 19 Heuristic Continuation In situations that are identifies as strategically crucial e.g: king in danger, imminent piece loss, pawn to become as queens,... extend the search beyond the depth-bound ! depth-bound

20. 20 How to organize the continuations? How to control (and stop) extending the tree? Tapering search (or: heuristic pruning) Order the moves in 1 layer by quality. b(child) = b(parent) - (rank child among brothers) b = 4 3210 21010...

21. 21 Time bounds: How to play within reasonable time bounds? Even with fixed depth-bound, times can vary strongly! Solution: Iterative Deepening !!! Remember: overhead of previous searches = 1/b Good investment to be sure to have a move ready.

22. 22 Games of chance Ex.: Backgammon: Form of the game tree:

23. 23 “Utility” propagation with chances: Utility function for a Maximizing node C : MAX s1s2s3s4 d1 d2d3d4d5 S(C,d3) C Min