1. Effective Coordination of Multiple Intelligent Agents for Command and Control
The Robotics Institute
Carnegie Mellon University
PI: Katia Sycara
Notes from Katia:
- some bullets have very long sentences. Feel free to shorten them as long as the message contained in the sentence is delivered
CURRENT SITUATION
Vast amounts of data from distributed and heterogeneous sources
Uncertain and evolving tactical situation
Shrinking decision cycles
Decision makers distributed in space and time
Overall Goal:
To develop effective agent-based system technology to support command and control decision making in time stressed and uncertain situations
Key Personnel: Onn Shehory
Terry Payne

2. Current Situation
Vast amounts of data from distributed and heterogeneous sources
Uncertain and evolving tactical situation
Shrinking decision cycles
Decision makers distributed in space and time

3. Overall Goal
To develop effective agent-based system technology to support command and control decision making in time stressed and uncertain situations

4. Research Objectives
Develop an adaptive, self-organizing collection of intelligent agents that interact with the humans and each other to
integrate information management and decision support
anticipate and satisfy human information processing and problem solving needs
perform real-time synchronization of domain activities
notify users and other each other about significant changes in the environment
adapt to user, task and situation
INNOVATIVE CLAIMS:
Scalable, robust and adaptive coordination and control multi-agent strategies
Sophisticated individual agent control
Reusable and customizable agent components
Multi-agent infrastructure coordination tools and environment
POTENTIAL IMPACT:
Reduce time for commanders to arrive at a decision
Allow commanders to consider a broader range of alternatives
Enable commanders to flexibly manage contingencies (replan, repair)
Improve battle field awareness
Enable in-context information filtering

5. Technical Challenges
What coordination mechanisms are effective for large numbers of sophisticated agents?
What are the scaling up properties of these coordination mechanisms?
How do they perform with respect to dimensions, such as task complexity, interdependence, agent heterogeneity, solution quality?
What guarantees do these mechanisms provide regarding predictability of overall system behavior?
Do they mitigate against harmful system behaviors?
How to achieve effective human-agent coordination?

6. Potential Impacts Reduce time for commanders to arrive at a decision
Allow commanders to consider a broader range of alternatives
Enable commanders to flexibly manage contingencies (replan, repair)
Improve battle field awareness
Enable in-context information filtering

7. Innovative Claims
Scalable, robust and adaptive coordination and control multi-agent strategies
Sophisticated individual agent control
Reusable and customizable agent components
Multi-agent infrastructure coordination tools and environment

8. Major Project Deliverables
Prototype multiagent system that aids human military planners to perform effective “in context” information gathering, execution monitoring, and problem solving
reusable “agent shell” that includes domain independent components for representing and controlling agent functionality, so that agents can be easily produced for different types of tasks
effective multiagent coordination protocols, that are scalable, efficient and adaptive to user task and planning context
multi agent coordination infrastructure consisting of a suite of tools for reliable and low cost building and experimenting with flexible multiagent systems

9. The RETSINA Multi-Agent Architecture
User 1
User 2
User u
Info
Source 1
Interface Agent 1
Interface Agent 2
Interface Agent i
Task Agent 1
Task Agent 2
Task Agent t
MiddleAgent 2
Info Agent n
Source 2
Source m
Goal and Task
Specifications
Results
Solutions
Tasks
Info & Service
Requests
Information Integration
Conflict Resolution
Replies
Advertisements
Info Agent 1
Queries
Answers
distributed adaptive collections of information agents that coordinate to retrieve, filter and fuse
information relevant to the user, task and situation, as well as anticipate user's information needs.

10. RETSINA Individual Agent Architecture

11. Capability-Based Coordination
Open, uncertain environment:
Agents leave and join unpredictably
Agents have heterogeneous capabilities
Replication increases robustness
Agent location via Middle agents:
Matchmakers match advertised capabilities
Blackboard agents collect requests
Broker agents process both

12. Capability-Based Coordination (cont)
Advertisement:
Includes agent capability, cost, etc.
Supports interoperability
Agent interface to the agent society independent of agent internal structure
We will test scale-up properties of capability-based coordination

13. Cooperation Problems with current methods:
Mechanisms not tested in real-world MAS
Simulations?size small (~20 agents)
Complex mechanism do not scale up
We will provide algorithms for efficient group formation

14. Cooperation - Solutions (continued)
Approach:
Very large systems (millions of agents):
Constant complexity cooperation method
Based on models of multi-particle interaction
Structural organization:
Relation of organization structure and autonomy
Effect on system flexibility, robustness

15. Cooperation - Solutions (continued)
Communication planning:
Change communication patterns to reduce eavesdropping risk
Bundle small message together
Use networks when less congested
Katia: if there is no enough time, you can omit this slide.

16. Competition and Markets
Limited resources result in competition
Market-based approaches:
Assume that agents can find one another
Assume centralized auctioneer
Otherwise, convergence results do not hold
Approach:
Utilize financial option pricing:
Prioritize tasks by dynamic valuation
Allows flexible contingent contracting
Analysis of large MAS via economics methods
Katia: again, if time is a problem this slide and the following slides can be combined into one.

17. Competition and Markets (contd)
Combine our capability-based coordination with market mechanisms
Mechanism design:
Design enforceable mechanisms for self-interested agents
Resolve Tragedy of Commons by pricing schemes.
Devise mechanisms to motivate truthful behavior

18. RETSINA: Testbed for Agent-Based Systems
Continuing development of general purpose multi-agent infrastructure
Agents built from domain-independent, reusable components
Agent behaviors specified in declarative manner
New agent configurations easily built and empirically tested.

19. Coordinating Agents With Human Users
Problem: Commanders’ already overloaded
For task delegation to be effective, communication with agents should be
natural
flexible: providing planning information when appropriate
concise: providing as little detail as possible
interactive:
before and during task execution, agents:
provide explanations of plans
assist users in revising plans
during task execution, agents:
report plan’s progress

20. Agent Task Delegation Languages for task description and delegation
Reconciling human and agent representation of tasks
Structured Natural Language/Graphical task description
Interactive Planning and Execution
user input as constraints on plan formation
execution monitor brings user into loop
Extending RETSINA’s
graphical task editor
planner and execution monitor

21. In-Context Information Management for C2
Agent-Based Information Management
Dependent on
user preferences
decision-making tasks
evolving situation
Agents’ responsibilities
Represent users’ task environment
Monitor significant changes
Provide appropriate notification to user or responsible agent
Learn to track and anticipate user’s information needs
Learn appropriate times and methods for presenting information

22. Agent Coordination in RETSINA
Build information management agents for C2 based on RETSINA mechanisms for agent coordination
Goal and task structures provide user and agent context
Information agents form and execute plans that
involve queries for future information monitoring
take situational constraints into account
work around notification deadlines
Build upon existing base of information management agents

23. Research Plan Agent Control Agent Coordination
mapping of task model and requirements to the appropriate coordination strategy
mapping of constraints of the environment, other agents and available resources to appropriate coordination strategy
experimental evaluation, analysis and refinement
Agent Coordination
design/refine coordination algorithm
implement appropriate experimental infrastructure
implement the coordination strategy and evaluate along different dimensions
analyze the results and refine algorithm design and experimental process

24. Research Plan (contd.) User-Agent Coordination
enhance the functionality of the current agent command language
develop and implement techniques for acquisition and maintenance of user tasks preferences and intentions
develop and implement protocols to enable an agent to accept task-related queries before, during or after task execution and generate natural descriptions of the unfolding execution of its plans
evaluate and refine
Information Management and Decision Support
develop mechanisms for information management (e.g., filtering, integration) in the context of the current problem solving task
develop mechanisms for in-context information monitoring and notification

25. Major Project Deliverables
Prototype multiagent system that aids human military planners to perform effective “in context” information gathering, execution monitoring, and problem solving
Reusable “agent shell” that includes domain independent components for representing and controlling agent functionality, so that agents can be easily produced for different types of tasks
Effective multiagent coordination protocols, that are scalable, efficient and adaptive to user task and planning context
Multi agent coordination infrastructure consisting of a suite of tools for reliable and low cost building and experimenting with flexible multiagent systems

26. Dimensions of Evaluation
Individual Agent
eg:
reasoning sophistication
control sophistication
learning capability
degree of self-interestedness
knowledge
data available to the agent
Task
task complexity
task interdependence
task temporal
resource constraints
frequency of task arrival

27. Dimensions of Evaluation (cont)
Environment
eg:
number of agents
system load
degree of uncertainty
resource availability
Coordination Mechanism
degree of agent coupling
richness of communication
task delegation mechanism
degree of agent cooperation/competition
Organizational Structure
hierarchy
heterarchy
federation

28. Evaluation Metrics Individual Agent Performance
eg:
accuracy of information returned by an agent
agent service responsiveness
resource consumption
MAS Aggregate Performance
System efficiency
Solution quality
System robustness
System stability
Predictability
Scalability

29. Examples of Experimental Hypothesis

30. Process for Experimentation
1. Formulation of the distributed coordination algorithm
2. Development of experimental infrastructure (eg: simulation tools, making appropriate modifications ro RETSINA components)
3. Running the experiment and collecting statistics
4. Analysis of the results
5. Inter-mechanism evaluation; the results of the simulations of the various mechanisms will be compared to determine performance landscapes of the different coordination mechanisms

31. Inter-Agent World Communications
1. The OAA Facilitator is started, followed by OAA Startit and OAA Monitor.
2. Start the InterOperator.
a. We verify its registration as a Retsina agent with the Retsina ANS entry, "OAA_InterOperator".
b. We verify its registration as an OAA agent via its registry and advertisement with the OAA Facilitator and by its name, "Retsina_InterOperator", and icon showing in the OAA Monitor.
3. Start the Retsina agent, "KQMLMessageSenderGUI" and register it with the Retsina ANS under the name, "Retsina_Matchmaker".
4. Using OAA Startit, start the other OAA agents. As those agents come online they will register and advertise with the OAA Facilitator. Each registry and advertisement will generate an event which is captured by the InterOperator and forwarded to the "Retsina_Matchmaker". In the future, the real Retsina Matchmaker will be the actual recipient of those messages.
5. Via the "Retsina_Matchmaker", it is possible to send messages to the OAA Facilitator.
6. OAA agents may be disconnected from the OAA Facilitator, or shutdown, and their status change will also be transmitted to the "Retsina_Matchmaker".

32. Inter-Agent World Communications
1. The "OAA Facilitator" is started, followed by "OAA Startit" (cf. <LV>/OAA_Start-It.gif) and "OAA Monitor" (cf. <LV>/OAA_Monitor.gif).
2. Start the InterOperator.
a. We verify its registration as a Retsina agent with the Retsina ANS entry, "OAA_InterOperator" (cf. <LV>/TestANS_lookup.gif).
b. We verify its registration as an OAA agent via its registry and advertisement with the OAA Facilitator, Ex. OAA Facilitator> Knowledge source connected: OAA Facilitator> 6 (Retsina_InterOperator) can solve: OAA Facilitator> [update_data(_6771,_6788)]
and by its name, "Retsina_InterOperator", and icon showing in the "OAA Monitor" (cf. <LV>/OAA_Monitor_InterOp.gif).

33. Inter-Agent World Communications (cont)
3. Start the Retsina agent, "KQMLMessageSenderGUI" and register it with the "Retsina ANS" under the name, "Retsina_Matchmaker" (cf. <LV>/Test_Retsina_Matchmaker.gif).
4. Using OAA Startit, start the other OAA agents (cf. <LV>/OAA_Start-It_AllAgentsUp.gif, <LV>/OAA_Monitor_AllAgentsUp.gif). As those agents come on-line they will register and advertise with the OAA Facilitator. Each registry and advertisement will generate an event which is captured by the InterOperator (cf. <LV>/OAA_Monitor_StartOaaWebL.gif) and forwarded to the "Retsina_Matchmaker" (cf. <LV>/Test_Retsina_Matchmaker_Updates.gif). In the future, the real Retsina Matchmaker will be the actual recipient of those messages.
5. Via the "Retsina_Matchmaker", it is possible to send messages to the OAA Facilitator (cf. <LV>/Hypothetical_MsgSend.gif).
6. OAA agents may be disconnected from the OAA Facilitator, or shutdown, and their status change will also be transmitted to the "Retsina_Matchmaker" (cf. <LV>/Agent_Shutdown.gif).