Artificial Intelligence in Game Design Heuristics and Other Ideas in Board Games

Good and Bad Heuristics Heuristic for evaluating board must be accurate –Directly related to “likelihood” of win –Inversely related to “distance” from win Example: TicTacToe heuristic H(board) = 2 × # of possible rows/columns/diagonals where X could win in one move × # of possible rows/columns/diagonals where X could win in two moves × # of possible rows/columns/diagonals where O could win in one move × # of possible rows/columns/diagonals where O could win in two moves Gives these two boards same measure -2 X OX OXO OX X OXO Guaranteed loss for AI

Good and Bad Heuristics Better heuristic must take this into account! H(board) = –if my move next and I have 2 in row  MAXINT –if opponent move next and they have 2 in row  -MAXINT –if my move and opponent has > 1 instance of 2 in row  -MAXINT –if opponent move and I have > 1 instance of 2 in row  MAXINT –else (above function) Will work better, but more complex to compute! Speed to compute heuristic value Accuracy of heuristic value tradeoff

Linear Heuristic Functions Heuristic is some function of individual pieces on board –Usually weighted in some way –Overall board value = Σ H(piece i ) i Very fast to compute Example: Chess –Pieces have “standard values” –Heuristic = sum of AI pieces – sum of player pieces 95331

Linear Heuristic Functions Often based on position of pieces on board Examples: –Games where purpose is to move all pieces to some goal Backgammon Sorry Can often send other player back to “start” –Heuristic value = total distance of AI pieces from goal – total distance of player pieces from goal

Linear Heuristic Functions Reversi (Othello) –Board value of based entirely on piece positions Corners very valuable (can’t be flipped) Sides somewhat valuable (very difficult to flip) Middle little value (will easily be flipped) –H(board) = C 1 * number of pieces in corner + C 2 * number of pieces on side + C 3 * number of pieces in middle + C 1 >> C 2 >> C 3 Reversi easiest type of game for AI –Low branching factor (5 to 15 possible moves) –Good heuristics –Single move can greatly change board Hard for human player to see Easy for MinMax lookahead to see

Nonlinear Heuristics Based on relationships between pieces on board Simple example: –Prefer chess pieces to protect one another –Piece value = piece value * 1.5 if protected by another –Piece value = piece value * 0.5 if not protected and threatened by opponent piece

Nonlinear Heuristics Drawback: Usually much more expensive to compute –n pieces  O (n 2 ) relationships between them May be able to explore more moves with simpler (linear) heuristic Example: –Simple linear heuristic takes k ms to evaluate per board –Complex nonlinear heuristic takes 400k ms to evaluate per board –Average of 20 possible next moves per board 400 possible next two moves –On average, could explore two additional moves if use linear heuristic

Linear vs. Nonlinear Heuristics Nonlinear heuristics often detect future changes in board –Pieces threatened by others might be captured –Pieces protected by others less likely to be captured Often see effect of nonlinear heuristic with additional levels of linear heuristic Example: “knight fork” in chess –Can look for this using nonlinear heuristic

Linear vs. Nonlinear Heuristics Searching 2 more moves in game tree will show result of knight fork –Shows state where rook now gone –Higher value for linear heuristic based on piece values –MinMax then gives “knight fork” state a high value Board with knight fork Board with knight threatening rook Black moves king Other moves give checkmate (value = MAXINT) Board with black rook gone Knight takes rook High value h Max level Min level

Horizon Effect Major weakness of purely linear heuristics based on piece values –“Throwing good money after bad” May need to recognize when move does not improve board position Chess example: Queen pinned by bishopsCould move rook in wayBut rook captured and will still lose queen

Horizon Effect Board with queen pinned Queen takes bishop Board with queen lost Bishop takes queen White down 6 points Move rook in front Board with rook lost Bishop takes rook White down 5 points Queen takes bishop Board with queen and rook lost Bishop takes queen White now down 11 points! This looks worse at cutoff level, but is actually best in long run!

Trappy Minmax Idea: Use minmax to set traps for player –Branch appears to have short-term benefit for player –In long run, branch has benefit for AI AI move player move Ok result for AIGreat result for AI This state looks good to player This state looks ok to player Good chance player might choose this branch So worth considering making this move

Trappy Minmax Queen can capture rook Board with rook ignored Some other move White unchanged Board with rook captured Queen takes rook White up 5 points Board with queen lost Knight captures queen White now down 4 points! Player did not lookahead far enough to see this Takes advantage of player horizon effect

Factors in Trappy Minmax Trappiness: Estimate of how likely player will choose branch corresponding to trap –Usually computed based on median or maximum of player heuristics down best branch Possible player move Path player thinks game will follow if they make this move 6 Median = 7 2

Factors in Trappy Minmax Profitability: Score if player follows “trap” branch – score if player follows their best branch Trap Quality: Trappiness * Profitability Use trap quality to adjust heuristic measure of a state: Trappy_heuristic(state) = normal_heuristic(state) + f(trap_quality(state))

Data-Driven Approaches Basing actions on known strategies rather than tree search Opening books of initial moves (Chess) –Often moves at start of Grandmaster match Each book consists of: –List of moves –Evaluation of final outcome Should we follow this strategy Allows faster processing –No need to search game tree until end of sequence –Can just use evaluation as heuristic Current board Boards in sequence End of sequence Now start branching

Opening Books Choose as own opening strategy Must have good final evaluation! –Make moves according to script –If opponent follows script, keep following –If opponent leaves script, start MinMax Will probably be in same way that benefits us Must also recognize when opponent uses an opening book –Keep database of moves in opening books –Match current board to those in database to find whether it is part of a sequence –If so, make decision about whether following script is good idea

Other Set Plays End Games –Many games have different strategies when few pieces left Forcing checkmate in chess Kings vs. kings in checkers Getting last pieces home in backgammon –Recognize based on pieces left –Follow set strategies Set Evaluation Values –No heuristic evaluation of board – instead, match board to database to get evaluation –Works best if can match subboards Example: Edge configurations in Othello This edge has a known value

Alternative Approaches Go –19  19 board  branching factor of 361 –Impossible for MinMax Only known approaches based on template matching –Look in local area for configurations that match known strategies –Still very open problem Best AI for Go only plays at amateur level