0. Software Agent - MAS: multi-agent systems-

1. Outline Definition Issues and elements of MAS Applications
MAS architectures
Coordination
Collaboration
Several issues in designing competitive MAS
Applications
MAS research direction
Summary

2. Multi-agent Systems A multi-agent system contains a number of agents…
…which interact through communication…
…are able to act in an environment…
…have different “spheres of influence” (which may coincide)…
…will be linked by other (organizational) relationships
MAS as seen from distributed AI
A loosely coupled network of entities that work together to find answers to problems that are beyond the individual capabilities or knowledge of each entity
A more general meaning
systems composed of autonomous components that exhibit the following characteristics:
each agent has incomplete capabilities to solve a problem
there is no global system control
data is decentralized
computation is asynchronous

3. Overview of MAS Aspects of multi-agent systems Types of MAS
Cooperative vs. competitive
Homogeneous vs. heterogeneous
Macro vs. micro
Interaction protocols and languages
Organizational structure
Mechanism design / market economics
Learning
Types of MAS
Cooperative MAS
Distributed problem solving: Less autonomy
Distributed planning: Models for cooperation and teamwork
Typical (cooperative) MAS domains
Distributed sensor network establishment
Distributed vehicle monitoring
Distributed delivery
Competitive or self-interested MAS
Distributed rationality: Voting, auctions
Negotiation: Contract nets

4. Comparison with Traditional Approaches
Client-server
Low-level messages
Synchronous
Can not do the job!
Agent breakthroughs
Peer-to-peer topology
Blackboard coordination model
Encapsulated messaging
High-level message protocols
Traditional Software
Client
Function(Parameters)
Server
Return(Parameters)
Agents
Intelligent
Agents
Intelligent
Agents
Intelligent
Agents
Intelligent
Agents
Blackboard
Intelligent
Agents
Intelligent
Agents
Message
Reply
Intelligent
Agents
Intelligent
Agents
Intelligent
Agents
A14<MAS>-4

5. Main Points in MAS
MAS researchers develop communications languages, interaction protocols, and agent architectures that facilitate the development of multi-agent systems
MAS researcher can tell you how to program each ant in a colony in order to get them all to bring food to the nest in the most efficient manner, or how to set up rules so that a group of selfish agents will work together to accomplish a given task
MAS researchers draw on ideas from many disciplines outside of AI, including biology, sociology, economics, organization and management science, complex systems, and philosophy

6. Key Elements of MAS
A coordination mechanism supported by a common agent communication language and protocol
A collaboration mechanism supported by agent community architecture (including agent and interaction architecture) to support the organization goal
A shared ontology
Popular MAS architectures
Object Manager Group (OMG)
Foundation for Intelligent Physical Agents (FIPA)
Knowledgeable Agent-oriented System (KAoS)
Open Agent Architecture (OAA)
General Magic group

7. MAS Architectures (1) OMG’s Model FIPA’s Model
Composed of agents and agencies that collaborate using general patterns and policies
Agents are characterized by: capabilities, type of interaction and mobility
Agencies support:
concurrent execution of agents
security
agent mobility
FIPA’s Model
Agents
Agent Platform (AP)
Directory Facilitator (DF)
Agent Management System (AMS)
Agent Communication Channel (ACC)
Agent Communication Language (ACL)

8. MAS Architectures (2) KAoS’s Model OAA Model
An Open Distributed Architecture for Software agents
Defines various agent implementations
Uses conversation policies to elaborate on agent-to-agent communication
OAA Model

9. MAS Architectures (3) General Magic’s Model
A commercial agent technology for electronic commerce
Views MAS as an electronic marketplace
The marketplace is modeled as a network of computers supporting a collection of places that offer services to mobile agents
The mobile agents:
can travel, meet other agents, create connections to other places
they have authority
Zeus: a MAS development toolkit

10. MAS Architectures (4) Geo-Agents (GIS agents) Architecture
Other Agent Systems
User
Query agent
Task(GeoScript)
Reply
Geo-Agents
Administrator
UI Agent
Query agent
Exchange registry
Facilitator
Query agent
Pass task
Reply
Query agent
Coordinate
Coordinate
Domain (Service) Agent
Task Agent
Control/Reply
Domain (Service) Agent
Task Agent
Collaborate
Retrieve
Collaborate
Data sources

11. Coordination
Coordination: a process to manage dependencies among activities
Three aspects of coordination
Activity aspect
What activity to execute?
When an activity should be executed?
Model to coordinate distributed tasks: Statecharts, Flowcharts, Process algebra, Lotos, SDL, Estelle …
Conversation (state) aspect
What is the structure of the conversation among the coordinating entities?
FSM, Petri-Nets, State Transition Diagrams
Implementation aspect
How to implement distributed software systems where software components coordinate their actions

12. Coordination
KQML
Knowledge Query and Manipulation Language (KQML) is both a message format and a message-handling protocol to support run-time knowledge sharing among agents
KQML comprise a substrate on which to develop higher-level models of inter-agent interaction such as contract nets
KQML is a coordination mechanism from the conversation aspect
KQML contains an extensible set of performatives, which defines the permissible speech acts agents may use
Example performative:
(ask-all /* message layer */
:content "price(IBM, [?price, ?time])“
/* content layer */
:receiver stock-server
/* communication layer */
:language standard_prolog
:ontology NYSE-TICKS
:sender me)

13. KQML: Types of Performatives
Coordination
KQML: Types of Performatives
Basic informative performatives: tell, deny, …
Database performatives: insert, delete, …
Basic responses: error, sorry, …
Basic query performatives: ask-one, ask-all, evaluate,…
Multi-response query performatives: stream-all, …
Basic effector performatives: achieve, …
Generator performatives: standby, ready, next, …
Capability-definition performatives: advertise
Notification performatives: subscribe
Networking performatives: register, forward, pipe, broadcast, …
Facilitation performatives: broker-one (all), recommend-one (all), recruit-one (all)

14. Collaboration
Collaboration refers to cooperative effort among agents to reach a single goal by exchanging knowledge built upon the underlying coordination mechanism
Example mechanism: Contract Net Protocol (CNP)
Negotiation as a collaboration mechanism
Negotiation on how tasks should be shared
A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)
An agent may subcontract another agent to perform a (sub)task.
Contract
Bid
agent

15. Collaboration
Phase 1: Task Announcement
- The contractor agent publicly announces a task.
- Potential candidates evaluate the task according to their wo n skills and availability.
Phase 2: Submission of Bids / Proposals
- Agents that satisfy the requiremenst, i.e., are able to perform the task, send their bid / proposal to the contractor.

16. Collaboration
Phase 3: Selection
- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates.
Phase 4: Contract awarding
A contract is established betwe en the contractor and the selecte d candidate.
- A privileged bilateral communic ation channel is established betw een the two agents.

17. Several Issues in Designing Competitive MAS
Distributed rationality
Pareto optimality
Stability

18. Distributed Rationality
Competitive MAS
Distributed Rationality
Techniques to encourage/coax/force self-interested agents to play fairly in the sandbox
Voting: Everybody’s opinion counts (but how much?)
Auctions: Everybody gets a chance to earn value (but how to do it fairly?)
Contract nets: Work goes to the highest bidder
Issues:
Global utility
Fairness
Stability
Cheating and lying

19. Pareto Optimality S is a Pareto-optimal solution iff
Competitive MAS
Pareto Optimality
S is a Pareto-optimal solution iff
 S’ ( x Ux(S’) > Ux(S) →  y Uy(S’) < Uy(S))
i.e., if X is better off in S’, then some Y must be worse off
Social welfare, or global utility, is the sum of all agents’ utility
If S maximizes social welfare, it is also Pareto-optimal (but not vice versa)
Which solutions
are Pareto-optimal?
Y’s utility
Which solutions
maximize global utility
(social welfare)?
X’s utility

20. Competitive MAS
Stability
If an agent can always maximize its utility with a particular strategy (regardless of other agents’ behavior) then that strategy is dominant
A set of agent strategies is in Nash equilibrium if each agent’s strategy Si is locally optimal, given the other agents’ strategies
No agent has an incentive to change strategies
Hence this set of strategies is locally stable
Prisoner’s dilemma
Pareto-optimal and social welfare maximizing solution: Both agents cooperate
Dominant strategy and Nash equilibrium: Both agents defect
Cooperate
Defect
3, 3
0, 5
5, 0
1, 1

21. Development of MAS
Define the organization of the MAS according to the problem specification (or solution structure)
Decide the coordination mechanism
Select a MAS implementation framework, e.g., Zeus, that supports the coordination mechanism
Implement the collaborative mechanism which support the MAS organization
Implement shared ontology
Implement each task agent (including customizing associated communication module)
Customize middle agents
Facilitators
Mediators
Brokers
Matchmakers and yellow pages
Blackboards

22. Applications of MAS Advanced Manufacturing Management Systems
Agents as representatives of machines, users, business processes, etc.
Intelligent Information Search on Internet
Some agents may show learning capabilities (learn the preferences of their users, ..)
Intelligent security enforcement on Internet
Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerce
With search, price comparison, and bargaining capabilities
Multi-agent auction in E-commerce
Distributed Surveillance
For information search or to look for special events informing their users of relevant news
Distributed Signal Processing
For problem diagnosis, situation assessment, etc. in the network
Distributed Problem Solving
Collaborative design, scheduling, and planning

23. Agent Organizations Multiple (human and/or artificial) agents
MAS Research Directions
Agent Organizations
Multiple (human and/or artificial) agents
Goal-directed (goals may be dynamic and/or conflicting)
Affects and is affected by the environment
Has knowledge, culture, memories, history, and capabilities (distinct from individual agents)
Legal standing is distinct from single agent
Q: How are MAS organizations different from human organizations?

24. Organizational Structures
MAS Research Directions
Organizational Structures
Exploit structure of task decomposition
Establish “channels of communication” among agents working on related subtasks
Organizational structure:
Defines (or describes) roles, responsibilities, and preferences
Use to identify control and communication patterns:
Who does what for whom: Where to send which task announcements/allocations
Who needs to know what: Where to send which partial or complete results

25. Communication Communication models Communication strategies
MAS Research Directions
Communication
Communication models
Theoretical models: Speech act theory
Practical models:
Shared languages like KIF, KQML, DAML
Service models like DAML-S
Social convention protocols
Communication strategies
Connectivity (network topology) strongly influences the effectiveness of an organization
Changes in connectivity over time can impact team performance:
Move out of communication range  coordination failures
Changes in network structure  reduced (or increased) bandwidth, increased (or reduced) latency

26. MAS Research Directions
Learning in MAS
Emerging field to investigate how teams of agents can learn individually and as groups
Distributed reinforcement learning
Behave as an individual, receive team feedback, and learn to individually contribute to team performance
Iteratively allocate “credit” for group performance to individual decisions
Genetic algorithms: Evolve a society of agents (survival of the fittest)
Strategy learning: In market environments, learn other agents’ strategies

27. Adaptive Organizational Dynamics
MAS Research Directions
Adaptive Organizational Dynamics
Potential for change:
Change parameters of organization over time
That is, change the structures, add/delete/move agents, …
Adaptation techniques:
Genetic algorithms
Neural networks
Heuristic search / simulated annealing
Design of new processes and procedures
Adaptation of individual agents

28. Summary “Agent” means many different things
Different types of “multi-agent systems”:
Cooperative vs. competitive
Heterogeneous vs. homogeneous
Micro vs. macro
Lots of interesting/open research directions:
Effective cooperation strategies
“Fair” coordination strategies and protocols
Learning in MAS
Resource-limited MAS (communication, …)
Next lecture
Communication & Platform