Adversarial Search CMPT 463

When: Tuesday, April 5 3:30PM Where: RLC 105 Team based: one, two or three people per team Languages: Python, C++ and Java IDEs: Python IDLE, Visual Studio, Eclipse, NetBeans Event Schedule 3:30 – 5:30 pm – Contest 5:30 pm – Award Ceremony 5:30 pm – Pizza Party Register your team online at or in RLC Contact Dr. Tina Tian for questions.

Outline Game playing Game trees o Minimax o Alpha-beta pruning

Games vs. search problems competitive environments: agents’ goals are in conflict adversarial search problems (games) Games are interesting b/c: Time limits o Search tree of chess has about nodes. o How to make the best possible use of time o Choose a good move when time is limited

Types of Games chess, checkers, go, othello backgammon, monopoly bridge, poker perfect information imperfect information deterministicchance

Games deterministic, fully-observable, turn-taking, two–player, zero-sum games o Utility values at the end are equal and opposite o Tic-tac-toe

Game Search Formulation Two players MAX and MIN take turns (with MAX playing first) S 0 : Player(s): Action(s): Result(s,a): Terminal-test(s): Utility(s,p):

Game Search Formulation S 0 : initial state Player(s): Action(s): Result(s,a): Terminal-test(s): Utility(s,p):

Game Search Formulation S 0 : initial state Player(s): which player has the move in a state Action(s): Result(s,a): Terminal-test(s): Utility(s,p):

Game Search Formulation S 0 : initial state Player(s): which player has the move in a state Action(s): set of legal moves in a state Result(s,a): Terminal-test(s): Utility(s,p):

Game Search Formulation S 0 : initial state Player(s): which player has the move in a state Action(s): set of legal moves in a state Result(s,a): transition model Terminal-test(s): Utility(s,p):

Game Search Formulation S 0 : initial state Player(s): which player has the move in a state Action(s): set of legal moves in a state Result(s,a): transition model Terminal-test(s): true/false (terminal states) Utility(s,p):

Game Search Formulation S 0 : initial state Player(s): which player has the move in a state Actions(s): set of legal moves in a state Result(s,a): transition model Terminal-test(s): true/false (terminal states) Utility(s,p): utility function defines the final value of a game that ends in terminal state s for a player p o zero-sum games: same total payoff

Game tree (1-player)

Partial Game Tree for Tic-Tac-Toe

Optimal strategies MAX uses search tree to determine next move. Assumption: Both players play optimally!! Given a game tree, the optimal strategy can be determined by using the minimax value of each node

Minimax The minimax value of a node is the utility (for Max) of being in the corresponding state, assuming that both players play optimally. Minimax(s) = o Utility (s)if Terminal-test(s) o max of Minimax(Result(s,a)) if Player(s) = Max o min of Minimax(Result(s,a))if Player(s) = Min

Optimal Play MAX MIN This is the optimal play

Two-Ply Game Tree

The minimax decision Minimax maximizes the worst-case outcome for max.

Minimax Tree MAX node MIN node f value value computed by minimax Minimax decision (backed up )

What if MIN does not play optimally? Definition of optimal play for MAX assumes MIN plays optimally: maximizes worst-case outcome for MAX. But if MIN does not play optimally, MAX can do even better.

Minimax Algorithm function MINIMAX-DECISION(state) returns an action inputs: state, current state in game v  MAX-VALUE(state) return the action in SUCCESSORS(state) with value v function MIN-VALUE(state) returns a utility value if TERMINAL-TEST(state) then return UTILITY(state) v  ∞ for a,s in SUCCESSORS(state) do v  MIN(v,MAX-VALUE(s)) return v function MAX-VALUE(state) returns a utility value if TERMINAL-TEST(state) then return UTILITY(state) v  -∞ for a,s in SUCCESSORS(state) do v  MAX(v,MIN-VALUE(s)) return v

Properties of minimax Complete? o Yes (if tree is finite) Optimal? o Yes (against an optimal opponent) Time complexity? o O(b m ) Space complexity? o O(bm) (depth-first exploration) o For chess, b ≈ 35, m ≈100 for "reasonable" games  exact solution is infeasible

Alpha-Beta Pruning Problem with minimax search: exponential in the depth of the tree Can we cut it in half? It is possible to compute the minimax decision without looking at every node. o pruning : eliminate some parts of the tree

Alpha-beta pruning We can improve on the performance of the minimax algorithm through alpha-beta pruning 271? MAX MIN

Alpha-beta pruning We can improve on the performance of the minimax algorithm through alpha-beta pruning 271? We don’t need to compute the value at this node. No matter what it is, it can’t affect the value of the root node. MAX MIN

Alpha-Beta Example [-∞, +∞] Range of possible values Do DFS until the first leaf

Alpha-Beta Example [-∞, +∞] Range of possible values Do DFS until first leaf

Alpha-Beta Example (continued) [-∞,3] [-∞,+∞]

Alpha-Beta Example (continued) [-∞,3] [-∞,+∞]

Alpha-Beta Example (continued) [3,3] [-∞,+∞]

Alpha-Beta Example (continued) [3,+∞] [3,3]

Alpha-Beta Example (continued) [-∞, ∞] [3,+∞] [3,3]

Alpha-Beta Example (continued) [-∞,2] [3,+∞] [3,3]

Alpha-Beta Example (continued) [-∞,2] [3,+∞] [3,3] This node is worse for MAX

Alpha-Beta Example (continued) [-∞,2] [3,14] [3,3][-∞, ∞],

Alpha-Beta Example (continued) [-∞,2] [3,14] [3,3][-∞,14],

Alpha-Beta Example (continued) [−∞,2] [3,5] [3,3][-∞,5],

Alpha-Beta Example (continued) [2,2] [−∞,2][3,3]

Alpha-Beta Example (continued) [2,2] [-∞,2] [3,3]

α-β pruning example Minimax(root) = max(min(3,12,8),min(2,x,y),min(14,5,2)) = max(3,min(2,x,y),2) = 3

α-β pruning We made the same minimax decision without ever evaluating two of the leaf nodes! o They are independent. It is possible to prune entire subtrees.

Why is it called α-β? α = value of the best choice found so far at any choice point along the path for max If v is worse than α, max will avoid it  prune that branch Define β similarly for min

Alpha-Beta Algorithm function ALPHA-BETA-SEARCH(state) returns an action inputs: state, current state in game v  MAX-VALUE(state, - ∞, + ∞ ) return the action in SUCCESSORS(state) with value v function MAX-VALUE(state, ,  ) returns a utility value if TERMINAL-TEST(state) then return UTILITY(state) v  - ∞ for a,s in SUCCESSORS(state) do v  MAX(v,MIN-VALUE(s, ,  )) if v ≥  then return v   MAX( ,v) return v

Alpha-Beta Algorithm function MIN-VALUE(state, ,  ) returns a utility value if TERMINAL-TEST(state) then return UTILITY(state) v  + ∞ for a,s in SUCCESSORS(state) do v  MIN(v,MAX-VALUE(s, ,  )) if v ≤  then return v   MIN( ,v) return v

Alpha-Beta Example

Alpha-Beta Example

Comments: Alpha-Beta Pruning Pruning does not affect the final results. Entire subtrees can be pruned. Good move ordering improves effectiveness of pruning. With “perfect ordering,” time complexity is O(b m/2 ) o Branching factor of sqrt(b) !! o Alpha-beta pruning can look twice as far as minimax in the same amount of time

Deterministic games in practice Checkers : Chinook ended 40-year-reign of human world champion Marion Tinsley in Used a precomputed endgame database defining perfect play for all positions involving 8 or fewer pieces on the board, a total of 444 billion positions. Chess : Deep Blue defeated human world champion Garry Kasparov in a six-game match in Deep Blue searches 200 million positions per second, uses very sophisticated evaluation, and undisclosed methods for extending some lines of search up to 40 ply. Othello : Logistello defeated the human world champion. It is generally acknowledged that human are no match for computers at Othello.