Modeling Reasoning in Strategic Situations Avi Pfeffer MURI Review Monday, December 17 th, 2007

Strategic Situations Scenarios involving multiple agents, all of which make decisions, and receive rewards based on their decisions May be competitive, cooperative, or anything in between May have interesting structure May be extended over time

Strategic Situations Abound Countering terrorist threats Robotic soccer Disaster response Diplomatic relations Auctions Trading

A Key Question Can we model how agents reason in strategic situations?

Why this Question is Important Modeling how agents reason will allow us to: predict their behavior develop counter-strategies develop computer systems that help people in their strategic decision making analyze situations to determine optimal strategies allow people to explain their reasoning

Possible Approaches Classical game theory Opponent modeling Behavioral economics Psychological theories

Our Approach Identify the basic reasoning patterns that can be used to justify decisions underlies sophisticated behavior such as sacrificing, retaliation and tempting Model and learn the factors underlying decision-making in particular games Use the models to develop strategies that work well

Characterizing Reasoning Patterns Informally, a reasoning pattern is a form of argument that leads to a decision We characterize reasoning patterns as structures in a graph describing a strategic situation the reasoning patterns capture the way information is used and manipulated

Reasoning Pattern #1: Direct Effect An agent takes a decision because of its direct effect on its utility without being mediated by other agents’ actions Drill Profit

Reasoning Pattern #2: Manipulation Child knows about parent’s action Parent does not care about reading, but wants child to brush teeth Child dislikes brushing teeth but likes being read to  Parent can manipulate child Offer to Read Parent Brush Teeth Child

Reasoning Pattern #3: Signaling A communicates something that she knows to B, thus influencing B’s behavior Recommendation Alice Choice Bob Better Restaurant

Reasoning Pattern #4: Revealing/Denying Driller cares about oil Tester receives fee if driller drills Tester causes driller to find out (or not) about information tester herself does not know Seismic Structure Oil Test Result Drill Test Tester’s ProfitDriller’s Profit

A Question For each reasoning pattern, we provide a graphical criterion to determine if the pattern holds Intuitively, a node is motivated if the agent owning the node cares about its decision If a node is motivated, does the graphical criterion characterizing one of the reasoning patterns necessarily hold?

Answering the question Answer: it depends what strategies we allow for other agents If we allow arbitrary strategies, any directed path from a decision node to a utility causes the node to be motivated But if we restrict attention to a “highly justifiable” class of strategies, we get a more interesting answer

Well-Distinguishing (WD) Strategies: Intuition A strategy is well-distinguishing if all distinctions that it makes really make a difference whenever the strategy distinguishes between two states of parents, the agent should receive different utility in the different states

Completeness Result Theorem: If other agents are playing WD strategies, then a node is motivated only if at least one of the reasoning patterns holds i.e., the four patterns of reasoning are sufficient to characterize all cases in which an agent cares about a decision

Relationship with Game Theory Theorem: The set of WD strategies always includes a Nash equilibrium We can view WD equilibrium as refinement of Nash Completeness theorem holds for all WD strategies, not just equilibria different assumption from rationality WDWD equilibriaNash

Modeling Key Situations An imam exhorts his students to commit terrorist acts. Can we model his strategic reasoning? manipulation? signalling? probably a combination of both can we capture how each argument contributes to the actions?

Learning how People Reason Reasoning patterns give us a theoretical basis for what arguments a person might make Can we learn what people actually do in particular games? Can we use what we learn to develop automatic strategies that perform well?

Learning How to Negotiate Can we learn how people trade off factors such as self-interest, altruism, etc. in negotiations? Yes we developed a computer agent using learned models it performed much better than game- theoretic agents, and also better than people

Learning Reciprocal Reasoning Do people use reciprocal reasoning in repeated interactions? retrospective reasoning prospective reasoning Yes models that factor in reciprocal reasoning perform better than those that don’t but prospective may not be as important as retrospective

Reasoning Under Uncertainty When people have uncertainty about other players, do they use models of the other players? Yes modeling people as reasoning about the potential actions of others leads to better performance but recursive modeling has diminishing returns

Distinguishing Beliefs from Preferences A person’s behavior may be influenced by both beliefs and preferences can we distinguish between them? Yes we have created models that are uniquely identifiable in this scenario, people have almost correct beliefs

Next Steps Algorithms and analysis tools for identifying relevant arguments in particular situations Analyze arguments for key behaviors recruitment to terrorism diplomatic relations, e.g. North Korea

Questions?