Modeling Teamwork and Command and Control in Multi-Agent Systems Thomas R. Ioerger Department of Computer Science Texas A&M University

TaskableAgents Architecture similar to process networks, or HTN’s has a customized knowledge representation language (TRL) for encoding knowledge about tasks and methods (doctrine, mission) agents run as independent processes each may have multiple parallel activities agents represent staff positions (S2, S3...) communicate with each other for teamwork interact with humans (via forms: info/cmds) interact with OTB for scenario simulation

Simulation DIS Agents OneSAF Testbed KB PDUs Translated To Facts (speed, location, unit type, etc.) Cache Periodic Updates From Simulation

High-Level Architecture of DBST text, forms, map actions inform, request, direct, approve, respond RFS, CFF mouse PDUs OTBAgents Puckster Interface BDE Interface PDUs

TaskableAgents Architecture Written in Java TRL Knowledge Representation Language - For Capturing Procedural Knowledge (Tasks & Methods) APTE Method Selection-Algorithm - responsible for building, maintaining, and repairing task-decomposition trees Inference Engine JARE - Java Automated Reasoning Engine - Knowledge Base with Facts and Horn Clauses - back-chaining (like Prolog) - Updating World With Facts

TaskableAgents sensing messages JARE KB: facts & Horn-clauses OTB (simulation) operators results assert, query, retract messages APTE Algorithm TRL Task Decomposition Hierarchy Process Nets Other Agents TRL KB: tasks & methods

Task Representation Language (TRL) Provides descriptors for: goals, tasks, methods, and operators Tasks: “what to do” –Can associate alternative methods, with priorities or preference conditions –Can have termination conditions Methods: “how to do it” –Can define preference conditions for alternatives –Process Net - Procedural language for specifying how to do things - While loops, if conditionals, sequential, parallel constructs - Can invoke sub-tasks or operators - Semantics based on Dynamic Logic Operators: lowest-level actions that can be directly executed in the simulation environment, e.g. move unit, send message, fire on enemy –Each descriptor is a schema with arguments and variables –Conditions are evaluated as queries to JARE

Example TRL Knowledge (:Task Monitor (?unit) (:Term-cond (destroyed ?unit)) (:Method (Track-with-UAV ?unit) (:Pref-cond (not (weather cloudy)))) (:Method (Follow-with-scouts ?unit) (:Pref-cond (ground-cover dense)))) (:Method Track-with-UAV (?unit) (:Pre-cond (have-assets UAV)) (:Process (:seq (:if(:cond(not(launched UAV)))(launch UAV)) (:let((x y)(loc ?unit ?x ?y))(fly UAV ?x ?y)) (circle UAV ?x ?y))))

useful for describing multiple ways of accomplishing tasks may encode preference conditions APTE algorithm will automatically try another if one method fails examples: –use of UAV vs. ATK helicopters vs. scouts for recon –suppression of direct fire with Arty/CAS –use of FASCAM to slow or re-direct advancing enemy –maintaining security: flank guard, patrols neighboring units use of terrain features electronic surveillance Alternative Methods

Task-Decomposition Hierarchy T1 M1 T2 T5 T3 T4 M7 M12 M92 M60 T15 T18 T40 T45 T40 C T45 T2 level 1 level 2 level 3 level 4 level 5 Tx =Task Mx = Method C = Condition

TOC Staff - Agent Decomposition CDR FSO S3 S2 Companies Scouts Control indirect fire, Artillery, Close Air, ATK Helicopter Maintain enemy situation, Detect/evaluate threats, Evaluate PIRs Maintain friendly situation, Maneuver sub-units Maneuver, React to enemy/orders, Move along assigned route Move to OP, Track enemy Move/hold, Make commands/decisions, RFI to Brigade

S2 Agent and Interactions DP approval Move to OP Threat level, PIRs RFI/RFS SALT/ INTSUM intel Enemy info S2 BDE/DIV Sensors/ Recon BDE S2 Scou t CD R CCIR S3 spot reports

Vignette 2 – Decision Point 1 [Shift Main Effort] Variation A: Enemy major threat is on main route (AA) as in route (planned AA). Company Size forces 3-66=1-22 4ID X X 1CD1CD 4ID X X 1CD1CD CoC TmB CoC TmA TmB TmA AA3 AA4 AA5a PL IB PL AA3 AA4 AA5a AA5 AA6c AA5 AA5c Company Size forces Main Effort(ME) Main Effort(ME) Shift ME from Tm B to Co C? DP1 2 Companies of heading along AA lead Bn of 234 Regt. Situation unclear on AA5 INTEL N Y Variation B: Enemy major threat changes to secondary approach 2 Companies of 234 heading along AA3 & AA lead Bn of 234 Regt intent is unclear. Lead Bn (1-235) of 235 Regt on AA5a INTEL Shift ME from Tm B to Co C? DP1 N Y Company Size forces ME Switch

Vignette 3 – Decision Point 2 [Commit TF Reserve] Variation A: Heavy enemy threat across across entire sector. Company Size forces 3-66=1-22 4ID X X 1CD1CD 4ID X X 1CD1CD CoC TmB CoC TmB TmA AA3 AA4 AA5a PL IB PL Y AA3 AA4 AA5a AA5 AA6c AA5 AA5c Company Size forces Main Effort(ME) Main Effort(ME) Commit the TF Reserve platoon? DP2 Company units of 3 different Bns on all 3 AAs Estimate enemy will reach PL Y at same time 238 Regt lead units not committed INTEL N Y Variation B: Major enemy movement on one avenue (AA). 2 Companies of 234 heading down AA3 Uniform pressure on AA’s 4 & 5ar. Calculations indicate Tm A unit can move to PL Y prior to lead of enemy unit. INTEL DP2 Y Res … Go to DP 3 Res … Commit the TF Reserve platoon? TmA(-) Blocking Positions TAI Company Size forces (+) N

Modeling Teamwork TOC is more than just collection of staff members helping/backup behavior, information fusion, resource sharing, and joint decision-making how to model collaborative behavior? CAST (extension to TaskableAgents) adds features to TRL language for encoding team structure and process (MALLET) adds algorithms for coordination and communication within teams semantics based on joint intention theory and mutual awareness (beliefs)

Key Concept: Shared Mental Models Various components –static: structure of the team, communication policies... –goals and plans –dynamic: current situation, others’ workloads/status Team knowledge needed by agent team members: –roles, responsibilities, capabilities, team plans –need to know who should act and when –need to reason about each other –need to know when to communicate for synchronization, coordination, disambiguation, infomation sharing, etc.

The CAST Agent Architecture MALLET - team knowledge repres. language –team structure (roles, capabilities, responsibilities) –team process (plans, policies) CAST kernel (interpreter) –convert to Petri nets (track progress, select actions) –use back-chaining theorem-prover for inference –dynamic role selection - make choices in context DIARG - information exchange algorithm –proactive: offer new info to those who need it primary references: (Yen et al., IJCAI, 2001), (Yin et al., Autonomous Agents Conf., 2000)

MALLET Multi-Agent Logical Language for Encoding Teamwork extension of TRL basic definitions –(team search-and-rescue (bill ted)) –(role pilot) (role spotter) –(plays-role bill pilot) –(capable spotter use-IR-binoculars) conditions: ( *) with variables prefixed by ‘?’ –e.g. ((forward-scout ?unit) (location ?unit ?x ?y)) team operators: (team-oper lift-heavy-object (?obj) (pre-cond (at ?obj) (num-agents >= 2)) (share-type AND))) share types: AND=together, OR=any, XOR=only 1 (excl.)

team plans: can select certain agents or roles to do steps (like SharedPlans of Kraus and Grosz) (team-plan indirect-fire (?target) (select-role (scout ?s) (in-visibility-range ?s ?target)) (process (do S3 (verify-no-friendly-units-in-area ?target)) (while (not (destroyed ?target)) (do FSO (enter-CFF ?target)) (do ?s (perform-BDA ?target)) (if (not (hit ?target)) (do ?s (report-accuracy-of-aim FSO)) (do FSO (adjust-coordinates ?target)))))) “compile” these into TRL using methods of Biggers and Ioerger (2001) other scouts can take over as backup in case of failure of ?s responsibilities (such as monitoring, reporting); semantics similar to joint intentions (Johnson and Ioerger, 2001)

CAST Kernel compile team plans into Petri nets (expand sub-tasks) cycle: sense/decide/act loop 1. update beliefs about environment in self’s KB 2. check for any incoming messages from other agents 3. find active steps in plan (transitions with tokens in all input places) 4. if self is uniquely resp., consider executing oper. 5. if oper is XOR and resp. is ambiguous, offer 6. if oper is AND, broadcast READY and wait for others 7. randomly choose among remaining actions and execute 8. inform others of completed steps Dynamic Role Selection (DRS) –check role definitions, must satisfy any constraints, capable? –communicate when ambiguity exists –sync. for AND operators; select for XOR operators –could also allow individuals to vote/negotiate

DIARG Dynamic Inter-Agent Rule Generator Info. sharing is a key to flexible teamwork Want to capture information flow in team, including proactive distribution of information Want to restrict to only the most relevant cases Ideal criterion: (Bel A I) ^ (Bel A  (Bel B I)) ^ (Bel A (Goal B G) ^ [  (Bel B I)   (Done B G)] ^ [(Bel B I)    (Done B G)]  (Goal A (Inform B I)) where is the temporal operator for ‘always’

DIARG, continued Explanation - A should send message I to B iff: –A believes I is true –A believes B does not believe I (or believes it is false) –I is relevant to one of B’s goals i.e. pre-cond of current action that B is resp. for in team plan, and that action would not succeed without knowing the info. Reasoning about observability –agents can sometimes infer that other team members already believe certain information –e.g. based on common observability in environment –use this to filter out superfluous messages –recent work: (Rozich and Ioerger, submitted)

Command and Control Need for tactical decision-making –more flexibility in unplanned situation –commander agent How to represent of “tactics”? –battlefield geometry, relative force strength, combined arms theory –terrain, effects on mobility –discovery of enemy intent

Naturalistic Decision-Making NDM (Klein) - a cognitive model of human decision-making in complex environments based heavily on situation assessment (SA) –3 stages (Endsley): acquisition of factual information comprehension (abstraction, relevance, goal impact) projection (prediction of consequences) NDM is “satisficing” –take first adequate match; don’t extensively evaluate and compare alternatives; respond

Evidence for NDM in TOCs verbal protocol analyses –characterize types of utterances and interactions representative studies –Serfaty, Entin, et al. (NDM, 1997) expertise as independent variable –Pascual and Henderson (NDM, 1997) reliance on recall from experience –Schmitt and Klein (CCRTS, 1999) recognitional processes in MDMP/COA –Endsley (ARL report) information flow in infantry platoon/urban combat lots of other similar C2 environments... –CIC/AAW, AWACS, fire fighting, ATC

Recognition-Primed Decision Making (RPD) a model of NDM characterized by “feature-matching” look for enough cues to trigger recognition features could be weighted for each situation, there is a typical response (doctrinal, or learned from experience) role of “mental simulation”?

RPD Flow Chart Len Adelman and Denny Leedom –integrate RPD within battalion TOC staff situation clear? status quo acceptable? generate new options reduce uncertainty current mission plan monitor progress modify No Yes generate response, mental sim, compat. test, modify plan...

Methods to Deal With Uncertainty often features are unknown, can’t evaluate options include: 1. suppress uncertainty 2. make default assumptions 3. confirmation bias (expectations) 4. take “probing” actions 5. forestalling until situation is more clear

Implementation of RPD in TaskableAgents task DetectSituation runs in parallel with other routines... loop until match enough features for one situation while some features are unknown, try various methods to findout (e.g. UAV, scouts, radar, JSTARS, Bde Int, feint...) drive information collection to discriminate situations trigger plans for response maneuvers once situation is determined (takes priority) go back to original mission once threat is handled all of this can be implemented as tasks and methods in TRL

Characterization of Situations Situations must have lists of typical features Characterize by: –location of nearby enemy, combat strength –in regions relative to local axis (frame of ref.) –distance, speed, direction –intent? (e.g. attacking, bypassing, objective) –cover, uncertainty –effect of terrain on mobility/reachability roads, mountains, streams, bridges, marshes, forests also minefields, targeted areas of interest...

Types of Situations Defensive –getting: flanked, ambushed, enveloped, etc. response: shift, request for support, withdraw... –over-whelming enemy maneuver/main effort response: impede, divert, CAS, inform Bde... –forked maneuvers (intent?); bypass attempt Offensive (opportunities worth recognizing) –exploit gaps, isolate enemy units, envelopment, use fixing force + flank attack, canalize enemy, bypass

Practical Issues Managing priorities –when to abandon mission plan & react tactically? –return to mission plan once done? Dependence of response on goals –ROE, aggression/initiative vs. defense/security –should SA involve impact on goals? (= threat?) Need to have “critics” to revise responses –avoid enemy TAI’s, minefields, contact... –stay near adjacent friendly units, air defense...

Conclusion TaskableAgents architecture CAST extensions for teamwork TOC staff agent model modeling command and control (on-going) HLA interoperability (on-going)