1. Modeling Agents’ Reasoning in Strategic Situations Avi Pfeffer Sevan Ficici Kobi Gal

2. The Big Question Why do agents (people or computers) do things in strategic situations? Possible directions –social norms –cognitive or computational limitations –beliefs –preferences –reasoning patterns

3. Outline Characterizing the reasoning patterns agents use (Avi) The Colored Trails framework (Kobi) Modeling people’s preferences and reasoning in dynamic situations (Kobi) Modeling how people reason about other people (Sevan) Modeling the subjective beliefs people have about other people (Sevan)

4. Why Understand Reasoning Patterns? Understand people’s strategies: people are likely to prefer some reasoning patterns to others Explanation: if we want to explain people’s strategies, or we want to explain computational agents’ strategies to people, it is useful to know what reasoning patterns are being employed Computational benefits Analogy to logic

5. Multi-Agent Influence Diagrams (MAIDs) Graphical representation of strategic situations, based on Bayesian networks Compact representation –exploits independences in decision making –naturally captures limited observations –capture structure in multi-attribute utility functions Tell the “story” of a game Ideal representation for thinking about reasoning patterns

6. Bayesian networks Represent probability distribution Nodes are random variables Edges represent direct influence Each node has conditional probability of the node given its parents Seismic Structure Oil Test Drill Profit Test Result P(Oil | Seismic Structure)

7. Influence Diagrams Represent single-agent decision problems –Chance nodes (ellipses) –Decision nodes (rectangles) –Utility nodes (diamonds) Parents of decision node represent information available to decision maker at time of making decision Seismic Structure Oil Test Result Drill Test Profit

8. Multi-Agent Influence Diagrams Represent multi-agent strategic situations Like influence diagrams, except that decision and utility nodes are associated with specific agents Seismic Structure Oil Test Result Drill Test Tester’s ProfitDriller’s Profit

9. Reasoning Pattern #1: Direct Effect An agent takes a decision because of its direct effect on its utility –without being mediated by other agents’ actions DADA UAUA

10. Influencing All other reasoning patterns fall under the category of influencing: trying to get another agent to do something that is beneficial to you The possible reasoning patterns depend on what strategies are considered for other agents Let’s restrict attention to those strategies where the other agent has “good reason” to play them

11. Reasoning Pattern #2: Manipulation B knows about A’s action A cares about B’s action A’s action influences B’s outcome, so B has to react to what A does A can manipulate B to respond to her in a favorable way DADA UAUA DBDB UBUB

12. Reasoning Pattern #3: Signaling A communicates something that she knows to B, thus changing B’s behavior Interesting point: A must care about the thing she is communicating, otherwise B won’t believe her DADA UAUA DBDB UBUB C

13. Reasoning Pattern #4: Revealing/Denying A causes B to find out about information A herself does not know Tiger example Also works the other way round DADA UAUA DBDB UBUB C E

14. Key Question Are these all the reasoning patterns? Answer: it depends what strategies you allow for other agents If you allow general strategies, any pattern in which there is a directed path from a decision node to a utility node is a pattern But if we restrict attention to a more restricted class of strategies, we get a more nuanced answer

15. Well-Distinguishing Strategies Intuition: if a strategy makes a distinction, the distinction should make a difference A well-distinguishing (WD) strategy is one in which –if the strategy distinguishes between two values of the parents of a node, the expected utility is different for the two values –if the strategy assigns different probability to two actions, the expected utility of the two actions is different

16. Reassuring Fact Theorem: The set of WD strategies always includes a Nash equilibrium But not all Nash equilibria are WD And not all WD strategies are Nash equilibria

17. Completeness Result Theorem: If other agents are playing WD strategies, then the four patterns of reasoning described earlier are the only patterns in which an agent cares about its decision

18. Converse When one of the reasoning patterns holds, does an agent necessarily care about its decision? Answer: not in general But we can prove a weak converse: Theorem: If one of the reasoning patterns holds in a MAID, there exist parameter values for the MAID such that the agent cares about its decision Similar situation to Bayesian networks

19. Opportunities for Synergistic Research What reasoning patterns do people use? Do people prefer some reasoning patterns to others? How can we study this in the lab or the field? What about non-WD strategies (which I think people use) – can we find a good solution concept that adequately capture’s people’s behavior?