PENGANTAR INTELIJENSIA BUATAN (64A614) Kuliah 5 : Game Playing Fakultas Teknik Jurusan Informatika Universitas Surabaya

Outline Game Playing Game Trees Minimax Alpha-Beta Pruning

Games Some games: Ball games Card games Board games Computer games ...

Human Game Playing

Garry Kasparov and Deep Blue, 1997 Computer Game Playing Garry Kasparov and Deep Blue, 1997

Game-Playing Agent sensors actuators agent environment ? Environment

Type of games

Typical Case 2-person game Players alternate moves Zero-sum: one player’s loss is the other’s gain Perfect information: both players have access to complete information about the state of the game. No information is hidden from either player.

Typical Case No chance (e.g., using dice) involved Examples: Tic-Tac-Toe, Checkers, Chess, Go, Nim, Othello Not : Bridge, Solitaire, Backgammon, ...

Two-player games A game formulated as a search problem: Initial state: board position and turn Successor function(Action) : definition of legal moves Terminal state: conditions for when game is over Utility function: a numeric value that describes the outcome of the game. E.g., -1, 0, 1 for loss, draw, win. (payoff function)

The Minimax Algorithm Basic idea : Choose move with highest minimax value = Best achievable payoff against best play. Algorithm : Generate game tree completely. Determine utility of each terminal state. Propagate the utility values upward in the tree by applying MIN and MAX operators on the nodes in the current level. At the root node use minimax decision to select the move with the max (of the min) utility value. Steps 2 and 3 in the algorithm assume that the opponent will play perfectly.

Tic-Tac-Toe

Generate Game Tree

Generate Game Tree x x x x

Generate Game Tree x x o x o x o

Generate Game Tree x 1 ply 1 move x o x o x o

A subtree win lose draw x o x o x o x o x x o x o x o x o x o x o o o

What is a good move? win lose draw x o x o x o x o x x o x o x o x o x

Minimax Minimize opponent’s chance Maximize your chance 3 8 12 4 6 14 5 Minimize opponent’s chance Maximize your chance

Minimax Minimize opponent’s chance Maximize your chance 3 2 8 12 4 6 14 5 MIN Minimize opponent’s chance Maximize your chance

Minimax Minimize opponent’s chance Maximize your chance 3 2 8 12 4 6 14 5 MAX MIN Minimize opponent’s chance Maximize your chance

Minimax Minimize opponent’s chance Maximize your chance 3 2 8 12 4 6 14 5 MAX MIN Minimize opponent’s chance Maximize your chance

Minimax = Maximum of the minimum 1st ply 2nd ply

MAX A B C MIN D E F G MAX 1 1 -3 4 1 2 -3 4 -5 1 -7 2 -3 -8 Select this move B C MIN 1 -3 D E F G MAX 4 1 2 -3 4 -5 1 -7 2 -3 -8 = terminal position = agent = opponent

MiniMax search on Tic-Tac-Toe Evaluation function Eval(n) for A infinity if n is a win state for A (Max) -- infinity if n is a win state for B (Min) (# of 3-moves for A) -- (# of 3-moves for B) a 3-move is an open row, column, diagonal

MiniMax search on Tic-Tac-Toe A is X Eval(s) = 6 - 4

Tic-Tac-Toe MiniMax search, d=2

Tic-Tac-Toe MiniMax search, d=4

Tic-Tac-Toe MiniMax search, d=6

Partial Game Tree for Tic-Tac-Toe f(n) = +1 if the position is a win for X. f(n) = -1 if the position is a win for O. f(n) = 0 if the position is a draw.

Properties of minimax Complete? Yes (if tree is finite) Optimal? Yes (against an optimal opponent) Time complexity? O(bm) Space complexity? O(bm) (depth-first exploration) For chess, b ≈ 35, m ≈100 for "reasonable" games  exact solution completely infeasible

Minimax Exercise 5 -3 3 3 -3 2 -2 3 5 2 5 -5 1 5 1 -3 -5 5 -3 3 2

Alpha-Beta Prunning A disadvantage of MinMax search is that it has to expand every node in the subtree to depth m. Can we make a correct MinMax decision without looking at every node? We want to prune the tree: stop exploring subtrees for which their value will not influence the final MinMax root decision Observation: all nodes whose values are greater (smaller) that the current minimum (maximum) need not be explored !

Do We Have To Do All That Work ? MAX MIN 3 12 8

Do We Have To Do All That Work ? 3 MAX 3 MIN 3 12 8

Do We Have To Do All That Work ? 3 MAX 3 2 MIN 3 12 8 2 Since 2 is smaller than 3, then there is no need for further search

Do We Have To Do All That Work ? 3 MAX 3 X 2 MIN 3 12 8 14 5 2

Alpha-Beta Pruning Generally applied optimization on Mini-max. Instead of: first creating the entire tree (up to depth-level) then doing all propagation Interleave the generation of the tree and the propagation of values. Point: some of the obtained values in the tree will provide information that other (non-generated) parts are redundant and do not need to be generated.

Alpha-Beta Pruning MIN MAX 2 2 1 2 =2 1 5

- The (temporary) values at MAX-nodes are ALPHA-values - The (temporary) values at MIN-nodes are BETA-values Alpha-value MIN MAX 2 2 5 =2 2 1 1 Beta-value

Alpha-Beta pruning Example

Alpha-Beta pruning Example

Alpha-Beta pruning Example

Alpha-Beta pruning Example

Alpha-Beta pruning Example

Properties of Alpha-Beta pruning Pruning does not affect final result. Good move ordering improves effectiveness of pruning. With "perfect ordering," time complexity = O(bm/2)  doubles depth of search

Alpha-Beta pruning Example  6 MAX MIN 6

Alpha-Beta pruning Example  6 MAX MIN 6  2 6 12 8 2

Alpha-Beta pruning Example  6 MAX MIN 6  2  5 6 12 8 2 5

Alpha-Beta pruning Example  6 MAX Selected move MIN 6  2  5 6 12 8 2 5

A B C MAX D E F G MIN H I J K L M MAX 6 6 2 beta cutoff 6 >=8 alpha cutoff 2 H I J K L M MAX 6 5 8 2 1 = agent = opponent

Alpha-Beta pruning Example 8 7 3 9 1 6 2 4 5  4 16  5 31 39 = 5 MAX 6  8  5 23 15 = 4 30 = 5  3 38 MIN 33  1 2  8 10  2 18  1 25  3 35  2 12  4 20  3 5 = 8 8  9 27  9 29  6 37 = 3 14 = 4 22 = 5 MAX 1 3 4 7 9 11 13 17 19 21 24 26 28 32 34 36 11 static evaluations saved !!

Example of a perfectly ordered tree MAX MIN 21 21 12 3 21 24 27 12 15 18 3 6 9 21 20 19 24 23 22 27 26 25 12 11 10 15 14 13 18 17 16 3 2 1 6 5 4 9 8 7

Exercise Minimax Alpha Beta Pruning A B C D E F G H I J K L M N O P Q V W Y X 2 3 8 5 7 6 1 4 10 Max Min