Artificial Intelligence for Games and Puzzles1 Games in the real world Many real-world situations and problems can be viewed as games of a sort: Players have a choice of actions, Players have conflicting goals, Players may move sequentially or simultaneously, Alliances may prosper, Treachery may occur, Understanding of the goals of others may be useful in predicting their actions and planning actions of one’s own. Parlour games offer environments in which various kinds of simplification can be made in order to focus attention on particular AI issues: perception, representation, reasoning, learning, opponent modelling, and risk assessment. Stock Market War Passing legislation Hustling (“Bangneki” in Go) Cartels Fight for market share Biological evolution Industrial relations Democratic elections Takeover negotiations

Artificial Intelligence for Games and Puzzles2 Some Game Theory Game theory is the formal study of conflict and cooperation, applicable when several “players” may act independently. Much of the terminology in use today was introduced in 1944 in the book Theory of Games and Economic Behaviour, by von Neumann & Morgenstern. It is an important analytic tool for economics, and is also widely applied in politics, war, sociology, psychology, evolutionary biology. It allows one to reason about the strategies that rational players may adopt, whether acting competitively, cooperatively, or in isolation of others, in deciding upon courses of action intended to yield some payoff: even when the payoff is merely the winning of a parlour game.

Artificial Intelligence for Games and Puzzles3 Adversarial Games Of Strategy Players (decision maker) Choices (feasible actions) Payoffs (benefits - or penalties -that might accrue) Combining whatever factors are needed to make players risk neutral Objectives (preferences among payoffs) When a player must make a choice this should be done Not by blind chance But by reasoning about which action is best Though curiously this may involve chance after all: see mixed strategies Using knowledge of what is good for the player Also using knowledge of what is good for other players, esp. adversaries

Artificial Intelligence for Games and Puzzles4 Static vs. Dynamic games In dynamic games the order of moves (performance of choices by players) is important:eg shopkeeper sets a price, customer then decides whether to buy. In static games, choices are unordered (perhaps simultaneous): like Roshambo, Diplomacy. Games may be drawn in extensive form: Consider (dynamic version) Roshambo: Player 1 Player 2 paper rock scissors prs prs prs (0, 0) (+1, -1) (-1, +1) (-1, +1) (0, 0) (+1, -1) (+1, -1) (-1, +1) (0, 0)

Artificial Intelligence for Games and Puzzles5 Games of perfect information, games of imperfect information If Player 2 knows Player 1’s move when making his own choice, the game is one of perfect information (Roshambo is then a PIG to play for player 1!). When the game is static, moves are simultaneous, player 2 has imperfect information. In extensive form, ovals denote information sets - nodes in the game tree which cannot be distinguished by the player making a choice. Player 1 Player 2 paper rock scissors prs prs prs (0, 0) (+1, -1) (-1, +1) (-1, +1) (0, 0) (+1, -1) (+1, -1) (-1, +1) (0, 0)

Artificial Intelligence for Games and Puzzles6 Strategies, and Strategic Form A strategy is a function which determines which choice a player makes at every choice point. (Even those which do not arise on a particular occasion.) Games may be described also in an alternative, so-called strategic form: Player2: Paper Player2: Rock Player2: Scissors Player1: Paper(0,0)(+1,-1)(-1,+1) Player1: Rock(-1,+1)(0,0)(+1,-1) Player1: Scissors(+1,-1)(-1,+1)(0,0) A game is zero-sum iff at each terminal of the extensive form, the sum of the players’ payoffs is zero.

Artificial Intelligence for Games and Puzzles7 Two-Person Zero Sum Games & Normal Form “Normal Form” is a compact notation for the payoffs of two players in a zero sum game: Roshambo = The cells of the matrix show the payoffs for player1; the payoff for player2 is trivially deduced. 0+1Paper 0+1Rock +10Scissors PaperRockScissors

Artificial Intelligence for Games and Puzzles8 Preferences over payoffs Suppose now one does not just win or lose, but instead hurts the loser using the winner’s chosen item (and gets enjoyment proportional to the hurt inflicted) Ow-ha-ha = How should the players behave? One thing player1 could do is ask for each move I could make, what is for me the worst thing player2 could do? 0+1-5Paper 0+4Rock +5-40Scissors PaperRockScissors

Artificial Intelligence for Games and Puzzles9 Max Min For each row i, player1 looks for the minimum payoff: min i G i,j He then looks for the move(s) i* which make this as good as possible: G i*,j = arg max i min j G i,j In this case, i* = {2}(rock) Similarly, player2 looks for the minimum payoff in each column j, and picks the j* that maximises the minimum payoff for player2: equivalently, minimises the maximum payoff for player1 (zero sum means p1’s gain = p2’s loss) G i,j* = arg min j j max i G i,j In this case, j* = {2}(rock again)

Artificial Intelligence for Games and Puzzles10 Stable and Unstable Solutions In this game, and in any zero-sum game, trying as hard as possible to be sure to hurt the other is equivalent to trying to profit as much as one can. Each player may rationally do this by selecting the move with the least potential for loss. But if one player knows that the other is doing this, they have a better choice: player 2 for example will do better with move 1 (paper). But if player 1 knows that, he will prefer move 3 (scissors). And so on … This is an unstable solution: a solution is stable if no player wants to move away unilaterally from the solution. At such a stable solution, a game is in equilibrium.

Artificial Intelligence for Games and Puzzles11 Gain Floor, Loss Ceiling, Saddle Point, Value of Game The Gain Floor of a two-player zero-sum game A, written  (A), is the worst outcome for player 1 if he picks his best strategy i*:  (A) =min j A i*,j The Loss Ceiling (likewise) is the best outcome for player 1 (hence worst for player 2) if player2 picks his best strategy j*:  (A) = max i A i,j* When Gain Floor equals Loss Ceiling, the game is said to have a saddle point. In this case, both players will do no better by unilaterally picking a different strategy than i*, j*. This pair of strategies is a stable solution to the game. A kind of cooperation emerges, even though the game is adversarial. The Value of game A is the payoff of its stable solution,  (A) =  (A) = Value(A)

Artificial Intelligence for Games and Puzzles12 Games without saddle points Zero-Sum games do not always have saddle points: eg Ow-ha-ha: Here i* = 2, so  (Ow-ha-ha) = -1, while j* = 2,  (Ow-ha-ha) = +1 The pair of strategies {i=2, j=2} is not stable. A player who can find out what the other will do can get a better result from a different strategy Paper 0+4Rock +5-40Scissors PaperRockScissors

Artificial Intelligence for Games and Puzzles13 Mixed strategies In a game with a saddle point, each player has a best strategy (or a set of them). The strategies remain best even if the other players knows what they are. In a game without a saddle point, and hence without a stable solution, each pure strategy one player might adopt, if known to an opponent, can be refuted A mixed strategy involves a random choice among pure strategies Each pure strategy is picked with its own particular probability Mixed strategies may be found which, even if known, cannot be refuted

Artificial Intelligence for Games and Puzzles14 Selecting the mixed strategy A player P will pick among the possible pure strategies S1…Sn using probabilities X1…Xn, such that  Xi = 1 P should pick these probabilities to give the highest expected payoff. For the very simple game G = For player 1: If p2 picks column 1 If p2 picks column 2 If X 1 = 0 & X 2 = 1 If X 1 =1 & X 2 =0 X 1 =1/7, X 2 =6/7

Artificial Intelligence for Games and Puzzles15 Selecting the mixed strategy - 2 With pure strategies mixed in the right proportion, player 2 gains no systematic advantage from choosing either pure strategy even when he knows the mixed strategy of player 1. Player 2 can pick a mixed strategy of probabilities Y1, Y2 according to the same principles: If p1 picks row 1 If p1 picks row 2 If Y 1 = 0 & Y 2 = 1 If Y 1 =1 & Y 2 =0 Y 1 =4/7, Y 2 =3/7

Artificial Intelligence for Games and Puzzles16 Stability with mixed strategies For each player, there is an optimal mixed strategy. One player’s choice of strategy does not depend either on knowing the other player’s strategy Keeping the strategy secret The expected payoff for one player does not depend on the other’s choice of strategy. The Value of the game is fixed. There is a kind of stability. These observations generalise to the case of multiple pure strategies per player. Von Neumann’s Minimax Theorem shows that a pair of mixed strategies can always be found for which the players have the same expected value for the game (which is not to say they can both expect to gain!)

Artificial Intelligence for Games and Puzzles17 Some game-theory-reading [3] [39]