1. Texas A&M University CAST: Collaborative Agents for Simulating Teamwork John Yen, Jianwen Yin, Thomas R. Ioerger, Michael Miller, Dianxiang Xu, Richard Volz Department of Computer Science Texas A&M University Acknowledgement: DoD/AFOSR MURI grant

2. Texas A&M University Our Goal: Building Intelligent Team-Training Systems Not just better individual performance, but better coordination, load-balancing, situation awareness Examples: fire fighters, air traffic controllers, TOCs, CICs, AWACS, shuttle mission control Scenario-based training via distributed simulation The need for agents - automated coaches and role players (virtual team members) Agents must understand the team structure and teamwork process

3. Texas A&M University Studies on Team Training from Psychology Literature teamwork vs. taskwork (Salas & Cannon-Bowers) process vs. outcome measures communications frequency, type (Serfaty) situation awareness (Endsley) adaptiveness of team to workload (also personality, leadership...) training protocols like cross-training to learn other roles (Salas & Cannon-Bowers)

4. Texas A&M University Roles for Agents in Team-Training Role players/Virtual Team members –reduce cost of training (e.g. need for human role players) –need to know when to act and when to defer –need to know whom to coordinate with –need to know whom to share info with –must describe team task (plan) and role assignments Automated Coaches –build student model (observe actions; infer their view) –compare to “expert” model (what should the trainee do ideally, given the team structure?) –give feedback, critique, instruction, more training...

5. Texas A&M University Key Concept: Shared Mental Model Various components –static: structure of the team, comm. policies... –goals and plans –dynamic: current situation, others’ workloads/status Needed for virtual team members –not just “domain” knowledge –also roles, responsibilities, capabilities, team plans –need to know who should act and when –need to know when to communicate for sync. & coordination, disambiguation, infomation sharing, etc.

6. Texas A&M University Shared Mental Models, continued Needed for user modeling –how to interpret incorrect actions? –alternative explanations of observed failures of action: thought it was another’s responsibility? waiting for synchronization or approval? –meaning of shared responsibilities & delegation for backup behavior (important for robustness)

7. Texas A&M University Related Work STEAM (Tambe) STEVE (Rickel & Johnson) REACT (Hill & Johnson) SWARMM/dMARS (Tidhar; Jennings) lots of others...

8. Texas A&M University The CAST Agent Architecture MALLET - team KR language –team structure –team process CAST kernel (interpreter) –convert to Petri nets (track progress, select actions) –use back-chaining theorem-prover for inference –dynamic role selection - make choices in context DIARG - information exchange algorithm –proactive: offer new info to those who need it

9. Texas A&M University MALLET Multi-Agent Logical Language for Encoding Teamwork syntax based on S-expressions (symbolic, nested lists) basic definitions –(team ( *)) –(role ) –(plays-role ) –(capable ) conditions: ( *) with variables prefixed by ‘?’ –e.g. ((forward-scout ?unit) (location ?unit ?x ?y)) operators: –(indiv-oper ( *) (pre-cond ) (effects )) –(team-oper ( *) (share-type [AND|OR|XOR]) (pre-cond ) (effects ))

10. Texas A&M University team plans: –(team-plan ( *) (pre-cond ) (effects ) (term-cond [SUCCESS|FAILURE]) [(role-select ( *) (constraint ))]* (process )) process expressions: – ::= (seq [ ]*) | (par [ ]+) | (while do ) | (if then else ) | (do | ) where ::= ( | *) semantics of responsibilities - similar to joint intentions (mutual belief) but asymmetric (Ioerger and Johnson, IC- AI 2001)

11. Texas A&M University (team-plan T1 () (process (par (kill-wumpuses) (collect-gold)))) (team-plan kill-wumpuses () (role ?s (scout)) (role ?f (fighter)) (process (while ((wumpus ?x) (not (dead ?x)))) (seq (do ?s (find-wumpus ?x)) (do ?f (move-to-wumpus ?x)) (do ?f (shoot-wumpus ?x))))) (team-plan find-gold () (role ?c (carrier)) (process (while (true) (if (see ?any-agent glitter) (do ?c (carrier-pickup gold)))) start findshootmove wumpus exists glitter done no wumpuses left pickup

12. Texas A&M University CAST Kernel compile team plans into Petri nets (incl. expanding sub-tasks) cycle: sense/decide/act loop 1. update beliefs about environment in self’s KB 2. check for any incoming messages from other agents 3. find active steps in plan (transitions with tokens in all input places) 4. if self is uniquely resp., consider executing oper. 5. if oper is XOR and resp. is ambiguous, offer 6. if oper is AND, broadcast READY and wait for others 7. randomly choose among remaining actions and execute 8. inform others of completed steps Dynamic Role Selection (DRS) –check role definitions, must satisfy any constraints, capable? –communicate when ambiguity exists –sync. for AND operators; select for XOR operators –could also allow individuals to vote/negotiate

13. Texas A&M University DIARG Dynamic Inter-Agent Rule Generator Info. sharing is a key to flexible teamwork more generally: distributed SA training target: learning what is relevant to whom? Want to capture information flow in team, including proactive distribution of information Want to restrict to only the most relevant cases (Tambe) Ideal criteria: (Bel A I) ^ (Bel A  (Bel B I)) ^ (Bel A (Goal B G) ^ [  (Bel B I)   (Done B G)] ^ [(Bel B I)    (Done B G)]  (Goal A (Inform B I)) where is the temporal operator for ‘always’

14. Texas A&M University DIARG, continued Explanation - A should send message I to B iff: –A believes I is true –A believes B does not believe I (or believes it is false) –I is relevant to one of B’s goals i.e. pre-cond of current action that B is resp. for in team plan, and that action would not succeed without knowing the info. Algorithm: 1. check for transitions which other agents are responsible for that can fire (pre-conds satisfied) 2. infer whether other agent might not believe pre-conds are true (currently, beliefs based on post-conditions of executed steps, i.e. tokens in output places) 3. send proactive message with information

15. Texas A&M University Experimental Testbed Wumpus World (Russell and Norvig) extended to teamwork environment –several agents exploring a 10x10 cave –goals: collect gold, kill wumpuses assume variable # wumpuses, but stationary –Roles: scouts/climbers - look for (smell for) wumpuses, can climb over pits shooters/fighters - have bow to shoot wumpus (from adjacent room), must collect arrows carriers - strong for carrying gold

16. Texas A&M University Experiment 1 Evaluate effect of teamwork and information exchange on team performance Team A: CAST, using DRS and DIARG Team B: CAST - no DIARG, just broadcast all new info. Team C: no teamwork - agents wander randomly 138

17. Texas A&M University Experiment 2 Effect of Dynamic Role Selection –constraint: choose closest fighter to wumpus –test scalability via: increasing # of pits (in 10x10 cave) makes it harder for agents to navigate

18. Texas A&M University Future Work Add delegation (role/resp change over time) –contract about communication between A and B (maintain mutual belief about goal status) Info. exchange (DIARG) depends on frequency of change and observability More complex model of agent capabilities –degrees of success (performance scores) –depends on workload, individual’s skill, deadlines, dual-tasks –affects team decision-making

19. Texas A&M University Future Work, continued Dynamic planning –generate plan if pre-conditions of current action are unsatisfied (instead of just waiting) –how will this impact team modeling? Interaction with humans –must infer their view of the team’s progress –deception, untruthfulness –failures due to forgetting/overload –conflicts with private goals?

20. Texas A&M University Conclusion Introduced CAST agent architecture MALLET team-representation language Major algorithms: –dynamic role selection (in kernel) –proactive information exchange (in DIARG) Simulation of a shared mental model Working toward support of intelligent team- training systems (user-modeling coaches, and virtual team members)