1. 9/10/98USC-ISI / ASTT IPR1 Flexible Group Behavior Randall Hill, USC-ISI Jonathan Gratch, USC-ISI ASTT Interim Progress Review September 10, 1998

2. 9/10/98USC-ISI / ASTT IPR2 Hypotheses The key to flexible behavior is handling situation interrupts –Understand the nature of the situation and adjust behavior appropriately –Achieve goals in spite of unexpected obstacles Flexible group behavior requires the ability to: –Understand behavior of groups of other agents Maintain situation awareness of friendly and adversarial groups Recognize when a situation does not match expectations –Plan a mission for groups against groups Collaborate with peers and superiors Perform adversarial reasoning –Execute mission plan in a coordinated manner React to situation interrupts, as a team Repair plans, when necessary, and continue executing mission

3. 9/10/98USC-ISI / ASTT IPR3 Hypotheses (2) Flexible group behavior interleaves the processes of situation awareness, planning, and execution –Requires concurrent, continuous integration of each process –Spans individuals, groups, and echelons –Affects how plans are generated and repaired –Need methods for integrating the component processes Group behavior requires a theory of multi-agent interaction –Provides a framework for understanding others’ behavior –Addresses issues of authority & autonomy –Enables collaborative & adversarial reasoning –Provides protocol for coordination and communication

4. 9/10/98USC-ISI / ASTT IPR4 Towards Flexible Group Behavior UnderstandExecutePlanExecuteUnderstand PlanExecute Battalion Company Learn Entity

5. 9/10/98USC-ISI / ASTT IPR5 State of the Art Current research on flexible group behavior is fragmented Group Understanding & Execution –Teamwork model Developed for RWA-Soar and participated in STOW-97 ACTD –Attack Helicopter Company + Command entity –Marine transport / Marine escort teams Executes plans with reactive behavior Understands own team’s roles, activities, and goals Provides coordination and communication protocol during execution –Limitations of teamwork model Does not understand other groups’ behavior Does not perform deliberative planning Does not reason about task dependencies when resolving conflicts

6. 9/10/98USC-ISI / ASTT IPR6 State of the Art AI planning systems –Hierarchical task network / partial order planners (IPEM,X11,DPLAN) Generate, execute, repair plans Example: Soar-CFOR RWA company command entity Not designed for multi-agent, collaborative planning Multi-agent planning –Joint Intentions / Shared Plans (Cohen & Levesque; Grosz & Kraus) Focuses on collaboration and reasoning about intentions Lacks situation awareness, execution, repair, authority, adversarial planning. –Generalized Partial Global Planner (Lesser & Decker) Focuses on coordination strategies and collaborative scheduling Lacks notion of authority, adversarial planning

7. 9/10/98USC-ISI / ASTT IPR7 Approach Build on Soar-CFOR model of planning, execution & repair –Need more flexibility in responding to situation interrupts –Need to extend model to higher echelons Develop more situation awareness –Detailed awareness of commander’s intent –Detailed awareness of other friendly activities –Detailed awareness of enemy situation –Constant tracking and updates of friendly and adversarial forces

8. 9/10/98USC-ISI / ASTT IPR8 Approach (2) Develop a more flexible approach to planning –Take advantage of enhanced situation awareness –Enable multi-agent reasoning Collaboration Authority Adversarial reasoning –Implement as a meta-reasoning capability on top of standard planner Don’t re-invent technology –Uses a standard planning paradigm –Builds on existing systems (RWA-Soar and Soar-CFOR) –The meta-reasoning layer integrates fragments of related research

9. 9/10/98USC-ISI / ASTT IPR9 Technology and R&D Architecture Situation Awareness Planning Situation Interrupts and Planning Multi-agent Planning

10. 9/10/98USC-ISI / ASTT IPR10 System Architecture Battalion Commander Company A Commander Company X Commander Company A Pilot Helicopter Pilot Helicopter Pilot Helicopter ModSAF Company X Pilot Helicopter Pilot Helicopter Pilot Helicopter …. Operations Order (plan) Operations Order (plan) Operations Order (plan) Situation Report (understanding) Situation Report (understanding) Situation Report (understanding) Percepts Actions Percepts Actions

11. 9/10/98USC-ISI / ASTT IPR11 Command Agent Architecture Understanding –Perceive and report groups of enemy entities –Understand plans from superiors/subordinates –Monitor execution of plans by subordinates Planning –Generate plans (collaboratively) at battalion and company levels –Battalion commander resolves conflicts in company plans Execution –Battalion executes plans –Repair plans when situation interrupt occurs

12. 9/10/98USC-ISI / ASTT IPR12 Plans Command Agent Architecture Environment Current Situation Domain Theory Planner Situation Awareness Radio Vision Radio Platform Commands Expectations

13. 9/10/98USC-ISI / ASTT IPR13 Situation Awareness Hide information gathering details from Planner Derives consolidated picture of current situation from: –Radio reports (via 16 CCSIL message types) OpOrders, SitReps, Status Reps, Replacement Reqs, Flight Advisory, BDA, Request Passage Coordination, etc... –Vehicle Sensors (via MITRE CFOR platform services) –Expectations expected enemy contact (derived from OpOrder) frequency of subordinate Status Reps Rule-based reasoning Can perform limited sensing actions –e.g.. Request situation reports

14. 9/10/98USC-ISI / ASTT IPR14 Situation Awareness Output List of facts currently true in the world –16B11 at holding_area ha11 –16B14 presumed dead –Enemy ADA platoon threatening battle_position bp141 –Target in EA nelson has been attritted –I’ve communicated order76 to 16C11 –I’ve received new orders from my commander Facts are echelon and unit type specific –Battalion tracks different information than company –CSS unit tracks different information than RWA unit –Determined by domain theory

15. 9/10/98USC-ISI / ASTT IPR15 Planner Implements basic command and control functions –Generates plans –Controls execution and coordination of subordinates –Recognizes Situation Interrupts and makes repairs GIVEN: –Domain theory (tasks, plan fragments, assets) –Mission objectives, friendly/enemy plans (from OPORDER) –Existing plans –Current situation (from Situation Awareness)

16. 9/10/98USC-ISI / ASTT IPR16 What are Plans? Plans are partially ordered sequences of tasks Plans capture assumptions –Column movement assumes enemy contact unlikely Plans capture dependencies between task –Move_to_Holding_Area results in unit being at the HA, which is a precondition to moving to the Battle_Position –Enemy and Company must be at the Engagement_area at the same time Different plans associated with different groups –Battalion plan, company plan, enemy plans

17. 9/10/98USC-ISI / ASTT IPR17 Planning Basics Plan generation –Sketch basic structure via decomposition –Fill in details with causal-link planning Plan execution –Explicitly initiate and terminate tasks –Initiate tasks whose preconditions unify with the current world –Terminate tasks whose effects unify with the current world

18. 9/10/98USC-ISI / ASTT IPR18 Example of Plan Generation Destroyed(Enemy) Attack(A, Enemy) Move(A,BP) Engage(A,Enemy) at(A,BP)at(A,FARP) at(Enemy,EA) at(A,BP)Destroyed(Enemy) at(A,FARP) at(Enemy,EA) Current World... init

19. 9/10/98USC-ISI / ASTT IPR19 Situation Interrupts Happen! What if the world changes unexpectedly? –Agents exist for extended time periods –Plans and goals may change over time –Actions have duration and may fail –Environment may change unexpectedly –Other agent may take unanticipated action To the planner this means –Effect of an action in the plan is deleted even though no task deletes it –Effect of an action is added even though no task adds it Need method for handling situation interrupts

20. 9/10/98USC-ISI / ASTT IPR20 Example destroyed(Enemy) Attack(A, Enemy) Move(A,BP) Engage(A,Enemy) at(A,BP)at(A,FARP)at(A,BP)destroyed(Enemy) at(A,FARP) at(Enemy,EA) Current World active(A) …. init Unexpected Event -active(A) active(A)

21. 9/10/98USC-ISI / ASTT IPR21 Reacting to Situation Interrupt Situation interrupt may force retraction of plan decisions –Example: Bn commander generates plan sending Company A to engage enemy tanks Company A is destroyed on way to the battle position Movement tasks of Company A are now invalid and must be retracted. Plan dependencies capture the ramifications of interrupts Repair plans in response to ramifications –Retract steps –Add new steps –Add additional constraints

22. 9/10/98USC-ISI / ASTT IPR22 Multi-Agent Planning Want to plan in context of other agents Want to explicitly reason about their plans Want to plan in a “socially appropriate” manner –Treat my friends with respect (collaborate or avoid negative interactions) –Treat my enemies with contempt –Understand what I am required to do –Understand where I have discretion to make decision –Understand who I have authority to command

23. 9/10/98USC-ISI / ASTT IPR23 Multi-Agent Planning Must represent goals and activities of other agents Must reason about interactions / conflicts across agents Cannot treat all plans equally –Must understand your relationship to other agents during planning –Modulate behavior of planner based on these relationships

24. 9/10/98USC-ISI / ASTT IPR24 Interaction Example Move(A,BP) Engage(A,Y) Dead(Y) Move(CSS,HQ) at(CSS,HQ) at(CSS,FAA) at(gas,FAA) at(gas,HQ) at(A,BP) at(A,FAA) at(A,BP) at(gas,FAA) Initial State Combat Service Support Plan Attack Helicopter Company Plan resupplied(HQ)

25. 9/10/98USC-ISI / ASTT IPR25 Planning Stances Authoritative: modify/generate other agents’ plans –Tell CSS to abandon re-supply operation Subordinate: use other’s plan to defer to their actions –Find a way to work around re-supply activity do I have time to do the engagement first? Collaborative: use other’s plan to reach mutual solution –Consult to see if some other resource could achieve the resupply Adversarial: use other’s plan against them –Try to introduce threats into other agent’s plan

26. 9/10/98USC-ISI / ASTT IPR26 Coordination and Control Joint commitments –Commander develops and communicates a plan –Commander must be committed to maintaining the plan Don’t make capricious changes –If plan must be changed, need to inform affected units Framing effects –Commander has one plan approved by his commander –Commander given subsequent mission to plan –Use the first plan to “frame” the second Consider the first plan fixed Develop second plan in the context of the first

27. 9/10/98USC-ISI / ASTT IPR27 Approach Add domain-independent theory of multi-agent interaction –Represent stances, authority relationships, joint commitments Add multi-agent controller that modulates general planner general purpose Reasoner (Planner) MultiAgent Controller Meta Plans Base Plans Meta Domain Theory domain independent Base Domain Theory

28. 9/10/98USC-ISI / ASTT IPR28 Plan Properties Associate properties with plans Multi-agent controller uses the properties as input Example: Modifiability –If a plan is unmodifiable, the planner cannot add or retract constraints to the plan in response to flaws –Unmodifiability is a component of modeling authority my bosses plans should be unmodifiable by me

29. 9/10/98USC-ISI / ASTT IPR29 Interaction Example Move(A,BP) Move(CSS,HQ) at(CSS,HQ) at(CSS,FAA) at(gas,FAA) at(gas,HQ) at(A,BP) at(A,FAA) at(gas,FAA) Initial State Planner Retract unmodifiable

30. 9/10/98USC-ISI / ASTT IPR30 Plan Properties Executability –Can I initiate tasks in this plan? Threat resolution –If a decision in one plan introduces a threat in another, is this ok? Role assignments –Tasks have an agent variable that planner can assign If I’m helpful I can add myself to tasks in your plan If I’m authoritative I can add you to tasks in my plan Commitment –Plans can be either conjectured or committed –Two conjectured plans can’t create flaws in each other Allows us to consider alternative courses of action Allows us to represent proposed changes to other plans

31. 9/10/98USC-ISI / ASTT IPR31 Meta Domain Theory Theory of authority –Battalion commander has authority over Company commander –Therefore Co cannot modify Bn’s plans; Bn can change Co’s Co’s plans must defer to Bn’s plans Bn can assign Co to tasks; Co can’t assign Bn to tasks

32. 9/10/98USC-ISI / ASTT IPR32 Meta Domain Theory (2) Military Decision Making Process (MDMP) modeled as meta domain theory COA Development COA Execution Create a plan free of flaws Don’t execute it until approved Run the plan till completion assuming there are no unpredicted flaws COA Analysis Take adversarial stance toward own plans (war gaming)

33. 9/10/98USC-ISI / ASTT IPR33 Meta tasks change the properties of other plans Executing meta-tasks changes state of multi-agent controller Meta-tasks COA Development -Flawed(p) Goal(g) -Modifiable(p) -Executable(p) Plan-for(g, p) Situation Interrupt Flawed(p) -CommonlyKnown(p) -Flawed(p) Modifiable(p)

34. 9/10/98USC-ISI / ASTT IPR34 Summary Reason about activities of other agents –Represent plans as subsets of planning decisions Tracks flaws across individuals’ activities –Uses standard threat detection Varies the way flaws are treated –Via plan properties Reasons about common knowledge – Also via plan properties Interleaves planning, execution, repair

35. 9/10/98USC-ISI / ASTT IPR35 Measures of Success Collective Measure: –Ability of a group of entities (RWA Battalion) to achieve mission objectives in scenarios containing a wide range of situation interrupts Individual Measures –Scalability: size of groups that can act autonomously –Flexibility: classes of situation interrupts handled by group behavior effectiveness of situation awareness, planning, execution, plan repair –Types of multi-agent reasoning integrated into framework i.e., collaborative, adversarial, temporal,... –Breadth and depth of domain knowledge e.g., # of tasks, echelon levels, functional categories (battlefield operating sys) –...

36. 9/10/98USC-ISI / ASTT IPR36 Evaluation Issues Need a systematic evaluation framework –Automatic scenario generator –Library of situation interrupts –How do we evaluate what happened? –Can situations be partitioned into classes?

37. 9/10/98USC-ISI / ASTT IPR37 Significant Results to Date Implemented RWA battalion commander Added a combat service support company to battalion Improved situation awareness –Enhanced pilot’s perceptual capabilities to handle larger groups –Implemented awareness of enemy situation at company/battalion –Improved tracking of subordinates Reorganized company/battalion cdr’s domain theory –Task hierarchy now matches with Logicon documentation Made C2 reasoning more accessible to user –Visualization tools –Built-in links from domain theory & plans to documentation

38. Scope of Task Coverage ATKHB Attack Mission Achieve Tactical Disposition Reduce Enemy Posture Achieve Culminating Task Consolidate 1-4-1101: Personnel (S1) planning (C 2 ) 1-4-1201: Intelligence (S2) planning (C 2 ) 1-4-1301: Operations (S3) planning (C 2 ) 1-4-1401: Logistics (S4) planning (C 2 ) 1-4-1302: Establish and maintain tactical operations center (C 2 ) 1-4-1305: Coordinate maneuver with CSS and rear ops (C 2 ) --------------------------------------------------- 1-2-0320: Provide supply support (CSS) 1-2-7723: Perform maintenance (CSS) 1-2-7728: Process ammo and fuel (CSS) 1-4-1103: Replacement operations (CSS) 1-4-1402: Coordinate supply/equip. (CSS) 1-4-1405: Plan and coordinate transport assets (CSS) Achieve Readiness 1-3:0001: Plan and organize move (Mnv) 1-2-0101: Move to and occupy assembly area (Mnv) 1-4-1306: Establish and maintain tactical command post (C 2 ) 1-2-7726: Conduct FARP operations (CSS) Achieve Physical Posture 1-4-1305 (Section 6.1.2): Integrate fire support Attack (METL task)1-4-1206: 1-2-xxxx: Establish satellite comm. (C 2 ) 1-2-xxx0: Establish ground comm (C 2 ) 1-2-7509: Establish voice comm (C 2 ) 11-5-0104: Establish FM radio (C 2 ) 1-4-1001: Perform C 2 operations (C 2 ) 1-4-1303: Control tactical operations (C 2 ) ------------------------------------------------------------ 1-4-1202: Implement security measures (Int) 1-4-1203: Process intelligence information (Int) 1-4-1311: Liaison operations (Int) ------------------------------------------------------------ 1-4-1105: Provide admin services (CSS) 1-2-7708: Provide food support (CSS) 1-2-7710: Operate field mess (CSS) 1-2-7720: Establish med support (CSS) 1-2-7721: Conduct med activities (CSS) 1-4-1102: Perform strength management (CSS) 1-4-1104: Conduct casualty reporting (CSS) 1-4-1308: Direct army airspace C 2 (CSS) 1-4-1310: Civil-military operations (CSS) 1-4-1403: Monitor equipment readiness (CSS) 1-4-1406: Provide logistic services (CSS) Continuous Tasks Legend Implemented Partially implemented Desire to implement Less relevant

39. 9/10/98USC-ISI / ASTT IPR39 Significant Results (2) Progressed toward theory for flexible group behavior Papers –“Metaplanning for Multiple Agents” in Proceedings of AIPS Plan Execution Workshop, Pittsburgh, PA –“Soar-RWA: Planning, Teamwork and Intelligent Behavior for Synthetic Rotary Wing Aircraft” in Proceedings of Seventh Conference on Computer Generated Forces and Behavioral Representation –“Reasoning about Multiple Plans in Dynamic Multi-agent Domains,” AAAI Fall 1998 Symposium on Continuous Distributed Planning

40. 9/10/98USC-ISI / ASTT IPR40 Expected Results Domain Independent Technology –Theory of flexible group behavior covering: Situation awareness Planning Execution –Integrated architecture for flexible group behavior Implementation of Technology –RWA Soar Attack Helicopter Battalion Evaluation of effectiveness of technology

41. 9/10/98USC-ISI / ASTT IPR41 Problem Areas Limiting factors –Change in personnel: Milind Tambe and Paul Rosenbloom –Impact: Re-focused approach to group understanding Transition to new PI (Randall Hill) Should not impact end result of project

42. 9/10/98USC-ISI / ASTT IPR42 Schedule Milestone 3: 9/98 –Technology POP Demonstration 2 RWA Attack Battalion Demonstrate Advanced Group Understanding –Scale up to larger groups of entities Demonstrate Advanced Group Planning –Group understanding (monitoring only) –Collaborative planning (added) Demonstrate advanced group execution –Deliver software and domain independent descriptions of new capabilities

43. 9/10/98USC-ISI / ASTT IPR43 Schedule (2) Milestone 4: 12/98 –Design Review 2 Approach to learning improved group models Approach to temporal planning

44. 9/10/98USC-ISI / ASTT IPR44 Schedule (3) Milestone 5: 9/99 –Technology POP Demonstration 3 RWA Attack Battalion Demonstrate advanced group understanding Demonstrate more advanced group planning –Temporal planning –Group understanding: plan recognition Demonstrate advanced group execution –Commander utilizes teamwork model (scaled down) Demonstrate group learning –Improve group models through experience –Deliver software and domain independent descriptions of new capabilities