1 Multiagent Teamwork: Analyzing the Optimality and Complexity of Key Theories and Models David V. Pynadath and Milind Tambe Information Sciences Institute and Department of Computer Science University of Southern California

2 Agent Teamwork Agents, robots, sensors, spacecraft, etc.  Performing a common task  Operating in an uncertain environment  Distinct, uncertain observations  Distinct actions with uncertain effects  Limited, costly communication Battlefield SimulationSatellite ClustersDisaster Rescue

3 Motivation Performance Complexity Optimal New algorithm Theoretical Approaches No communication Practical Systems ? ? ? ? Optimal Outline of Results 1)Unified teamwork framework 2)Complexity of optimal teamwork 3)New coordination algorithm 4)Optimality-Complexity evaluation of existing methods

4 Enemy Radar Example Domain: Helicopter Team Goal Did they see that? I destroyed the enemy radar.

5 Communicative Multiagent Team Decision Problem (COM-MTDP) S: states of the world  e.g., position of helicopters, position of the enemy A: domain-level actions  e.g., fly below radar, fly normal altitude P: transition probability function  e.g., world dynamics, effects of actions  : communication capabilities, possible “speech acts”  e.g., “I have destroyed enemy radar.”

6 COM-MTDPs (cont’d)  : observations  e.g., enemy radar, position of other helicopter O: probability (for each agent) of observation  Maps state and actions into distribution over observations (e.g., sensor noise model) R: reward (over states, actions, messages)  e.g., good if we reach destination, better if we reach it earlier  e.g., saying, “I have destroyed enemy,” has a cost Teamwork Definition:  All members share same preferences (i.e., R)

7 Problem Complexity COM-MTDPs Free communication Collectively Observable Individually Observable No communication

8 To Communicate or Not To Communicate Local decision of one agent at a single point in time:  “I have achieved a joint goal.”  “Should I tell my teammate?” Joint intentions theory: “I must attain mutual belief.”  Always communicate [Jennings] STEAM:  “I must communicate if the expected cost of miscoordination outweighs the cost of communication.” [Tambe]  Each cost is a fixed parameter specified by designer

9 Communicate if and only if:  E[R | communicate]  E[R | do not communicate] Locally Optimal Criterion for Communication Expectation over possible histories of states and beliefs up to current time Expected reward over future trajectories of states and beliefs WITH communication Expected reward over future trajectories of states and beliefs WITHOUT communication Expected cost of communicating

10 Empirical Results Communication Cost Observability V_opt-V

11 Empirical Results

12 Empirical Results

13 Silent Optimality vs. Complexity Optimality Complexity Globally Optimal Locally Optimal STEAM Jennings seconds (log) ,000 E[R] Observability = 0.2 Comm. Cost = 0.7

14 Jennings Optimality vs. Complexity Optimality Complexity Globally Optimal Locally Optimal STEAM Silent seconds (log) ,000 E[R] Observability = 0.2 Comm. Cost = 0.3

15 Summary COM-MTDPs provide a unified framework for agent teamwork  Representation subsumes many existing agent models  Policy space subsumes many existing prescriptive theories This framework supports deeper analyses of teamwork problems  Quantitative characterization of optimality-efficiency tradeoff, for different policies, in different domains  Derivation of novel coordination algorithms  Detailed proofs  Source code  JAIR article