Constraint Satisfaction Problems and Games S Kameshwaran Oct 22, 2002

Outline Introduction to CSP Introduction to N-Person Non-cooperative Games Nash Equilibrium revisited: Mixed Strategies Mapping CSP to a Game Tracing Procedure (Evolutionary Process) to find NE

CSP Given a set of variables, find possible values to the variables which simultaneously satisfy a set of constraints Applications: Scheduling Resource allocation Computational Molecular Biology Vision…

CSP Given: Find: X: Variables ( X1, X2, …, Xn) D: Domain Di for variable Xi R: Set of constraints r (can be logical) Find: Solution: Assignment of value to Xi from its domain Di such that all constraints are satisfied

CSP A constraint is called k-ary constraint if it connects k variables k(r): arity of constraint r (d1, d2, …, dn) r: The assignment di to Xi satisfies constraint r Characteristic Function: r (d1, d2, …, dn) 1 if (d1, d2, …, dn) r 0 otherwise

CSP Example: 8 Queens Problem Place the 8 queens on the chess board such that no queen attacks the others No two queens should be placed on the same row, or on the same column or on the same diagonal 8 variables Xi=j: Queen on ith row is placed on jth column Constraint r: No two queens should be placed in the same column Binary Constraint: k(r)=2 Xi is not equal to Xj

CSP Solution Approaches Search algorithms Graph based algorithms Backtracking Forward checking Graph based algorithms Neural Networks

N-Person Non-cooperative Games N players Non-cooperative vs. Cooperative: : Players cannot make binding commitments Players join and split the gains out of cooperation Solution concept: Nash Equilibrium

N-Person Non-cooperative Games Normal Form Games N players Si=Strategy set of player i (Pure Strategy) Single simultaneous move: each player i chooses a strategy siSi Nobody observes others’ move The strategy combination (s1, s2, …, sN) gives payoff (u1, u2, …, uN) to the N players All the above information is known to all the players and it is common knowledge

Nash Equilibrium Nash Equilibrium is a strategy combination s*= (s1*, s2*, …, sN*), such that si* is a best response to (s1*, …,si-1*,si+1*,…, sN*), for each i (s1*, s2*, s3*) is a Nash Equilibrium (3 player game) iff s1* is the best response of 1, if 2 chooses s2* and 3 chooses s3* s2* is the best response of 2, if 1 chooses s1* and 3 chooses s3* s3* is the best response of 3, if 1 chooses s1* and 2 chooses s2* Note: It is a simultaneous game and nobody knows what exactly the choice of other agents Nash Equilibrium assumes correct and consistent beliefs

Nash Equilibrium: Battle of the Sexes (Prize Fight, Prize Fight) is a NE: Best responses to each other (Ballet, Ballet) is a NE: Best responses to each other Man Woman Prize Fight Ballet 2, 1 0, 0 1, 2 Man Woman Prize Fight Ballet 2, 1 0, 0 1, 2

The Welfare Game Government wishes to aid a pauper if he searches for work but not otherwise Pauper searches for work only if he cannot depend on government aid Government Pauper Try to Work Be Idle Aid 3, 2 -1, 3 No Aid -1, 1 0, 0

The Welfare Game (Aid, Try to Work) is not NE: Pauper prefers Be Idle Government Pauper Try to Work Be Idle Aid 3, 2 -1, 3 No Aid -1, 1 0, 0

The Welfare Game (Aid, Try to Work) is not NE: Pauper prefers Be Idle (Aid, Be Idle) is not NE: Govt prefers No Aid Government Pauper Try to Work Be Idle Aid 3, 2 -1, 3 No Aid -1, 1 0, 0

The Welfare Game (Aid, Try to Work) is not NE: Pauper prefers Be Idle (Aid, Be Idle) is not NE: Govt prefers No Aid (No Aid, Be Idle) is not NE: Pauper prefers Try to Work Government Pauper Try to Work Be Idle Aid 3, 2 -1, 3 No Aid -1, 1 0, 0

The Welfare Game (Aid, Try to Work) is not NE: Pauper prefers Be Idle (Aid, Be Idle) is not NE: Govt prefers No Aid (No Aid, Be Idle) is not NE: Pauper prefers Try to Work (No Aid, Try to Work) is not NE: Govt prefers Aid No NE Government Pauper Try to Work Be Idle Aid 3, 2 -1, 3 No Aid -1, 1 0, 0

Mixed Strategies Pure Strategy: Player i chooses strategy sij from set Si Mixed Strategy: Player i chooses strategy sij with probability qij (qij>=0, j qij=1) Every pure strategy is also a mixed strategy Payoff in mixed strategies is the expected payoff

Mixed Strategies: Advantages Mathematical point of view: Convexifies the set: Convex sets are nice to play around as the terrain is well understood

Mixed Strategies: Advantages Mathematical point of view: Convexifies the set: Convex sets are nice to play around as the terrain is well understood Existence of Nash Equilibrium for finite games: Kakutani fixed point theorem

Mixed Strategies: Advantages Mathematical point of view: Convexifies the set: Convex sets are nice to play around as the terrain is well understood Existence of Nash Equilibrium for finite games: Kakutani fixed point theorem Practical point of view: Yes and No (depends on the situation)

Mixed Strategies: Interpretation Expected payoff: Let payoff with strategy si1 be 1 and si2 be 4 Mixed strategy (½, ½) gives the expected payoff ½+2=2.5

Mixed Strategies: Interpretation Expected payoff: Let payoff with strategy si1 be 1 and si2 be 4 Mixed strategy (½, ½) gives the expected payoff ½+2=2.5 It means a sure payoff of 2.5 is equivalent to a gamble where the payoffs are 1 and 4, each with probability ½

Mixed Strategies: Interpretation Expected payoff: Let payoff with strategy si1 be 1 and si2 be 4 Mixed strategy (½, ½) gives the expected payoff ½+2=2.5 It means a sure payoff of 2.5 is equivalent to a gamble where the payoffs are 1 and 4, each with probability ½ The above interpretation will not make sense if the payoff is money It is true only for utilities

Mixed Strategies: Interpretation Games where multiple strategies can be simultaneously employed Betting on more than one horse

Mixed Strategies: Interpretation Games where multiple strategies can be simultaneously employed Betting on more than one horse Multiple instances of the same game War Scenario: qij% of pilots use strategy sij

Mixed Strategies: Interpretation Games where multiple strategies can be simultaneously employed Betting on more than one horse Multiple instances of the same game War Scenario: qij% of pilots use strategy sij Same game repeated infinitely

Mixed Strategies: Interpretation Games where multiple strategies can be simultaneously employed Betting on more than one horse Multiple instances of the same game War Scenario: qij% of pilots use strategy sij Same game repeated infinitely For a single game: The probability distribution is the opponents’ estimation of player i’s decision

CSP as Games CSP C=< X, D, R > Game induced by C: GC=(S1, …, Sn; U1, …,Un) n = |X| Si = Di Ui(d1, …, dn) = rR[i] k(r) r (d1, d2, …, dn) R[i] = Constraint set that includes variable i The payoff function counts the number of satisfied constraints connecting that variable, taking every constraint along with its arity Instead of arity one can use different weights

Equilibria and Solutions Every solution of C is a Nash Equilibrium of GC For a solution, all constraints are satisfied, so no agent can improve its payoff by assuming a different value All Nash Equilibriums are not solutions C may not have a solution but still GC will have a NE

Equilibria and Solutions X

Equilibria and Solutions Not a Solution X

Equilibria and Solutions Not a Solution Nash Equilibrium: A better solution is not possible by moving a single queen in one move – Non-cooperative X

Trivia: Non-cooperative Games A solution better to some agents may be available, but cannot be reached by a decision of single agent alone Cooperation or non-cooperation depends on the game Non-cooperation need not be due to conflict in goal, but may be due to communication costs

Trivia: Non-cooperative Games Prisoner’s Dilemma in reality: Should IISc water its garden when there is drought in Mysore? Consider the following situation: Drought in Mysore but not in Bangalore Saved water from Bangalore can be transported to Mysore Decision making of an agent in Bangalore: If all saves water, his saving will not contribute much If nobody saves water, his saving will not contribute much So, better not to save

Equilibria and Solutions The complexity of the CSP depends on its structure Finding solution to C  Finding NE to GC Complexity of finding NE is not known It is unlikely to be in P It is also unlikely to be NP-hard as existence of solution is guaranteed

Equilibrium Selection Tracing Procedure (Evolutionary process) For agent i, there is a probability distribution (mixed strategy) pi, which the other agents expect that i will use p=(p1, …, pn) Assumption Limited computational power Agents are Bayesian decision makers Each agent estimates its best strategy depending on p Value of p is updated based on the previous outcome

Equilibrium Selection BR(p)=Best response strategy to the distribution p Synchronous Process p: Initial distribution p0 = p pk =  BR(pk-1)+(1- )pk-1 0<<=1 If pk converges then the limit point is NE

Equilibrium Selection Computation of BR(p) for an agent is computationally taxing if it is connected with large number of variables The process may converge to a NE that may not be a solution Can be considered as the best possible quasi solution No general proof that the process will always converge Susceptible to initial probability distributions

Trivia: Solution to 8 Queens Problem 10 distinct solutions http://www.math.utah.edu/~alfeld/queens/queens.html X

References Equilibrium Theory and Constraint Networks, Francesco Ricci, 1992 Games and Decisions, Luce and Raiffa, Dover Publications, 1957 Games and Information: An Introduction to Game Theory, Eric Rasmusen, Basil Blackwell Publishers, 1989

Nash Equilibrium: P or NP? Next.. 25/10/02 Nash Equilibrium: P or NP?