The Robotics Institute Intelligent Agents Katia Sycara katia@cs.cmu.edu The Robotics Institute Joseph Giampapa garof@cs.cmu.edu www.cs.cmu.edu/~softagents

What is an agent? An agent is an autonomous computational entity, which: is reactive and proactive is goal driven is intelligent: able to reason, plan and sometimes learn has domain specific intelligence interacts with humans, other agents, and the environment via sensors and effectors in a high level language/protocol anticipates user needs and reacts based on them wish list: friendly, understands natural lang.,etc

Multi-Agent Systems (MAS) An agent is more useful in the context of others: can concentrate on tasks of its expertise can delegate other tasks to other experts can take advantage of its ability to intelligently communicate, coordinate, negotiate But, a MAS is not just a collection of agents it needs meaningful ways for agents to interact it needs some system design and performance evaluation

MAS - Two Approaches 1. Centralized design Build a system that is comprised of agents - should provide good performance Advantages may arise from: possibility to develop each agent as an expert incorporation of non-local expertise rather simple to have multiple developers working concurrently Example: a system within an organization

MAS - Two Approaches 2. Open MAS Usually, the system has no prior static design, only single agents within Agents seek others to provide services, without knowing in advance who they are There is a need for agent finding mechanism Other agent may be non-cooperative or untrusted or malicious Example: markets, Internet

Centrally Designed MAS Advantages: Distributed loads and expertise Simplicity and predictability, since components are known interaction language and protocols are agreed upon agents can be (and usually are) cooperative agents share architecture - software reuse Costly maintenance (adding new agents may necessitate system re-design) May be less fault tolerant. Rigid Difficult to inter-operate with others Not reflecting real-world requirements (not realistic) Disadvantages:

Open MAS Advantages: Disadvantages: X Single agent or groups are designed separately (modular) Flexible, fault tolerant Evolutionary design Easier to maintain Dynamic, open society Overall behavior of the system not predictable Communication protocols, languages, ontologies may vary across agent types Self-interest and malicious behavior difficult to avoid Require more careful agent and interaction protocols design X Disadvantages:

Design and Architecture - Outline Design philosophies Information processing and needs Reactive architectures Deliberative architectures Layered architectures Belief, Desire, Intention (BDI) Concurrent architecture (RETSINA)

Agent Design Philosophies Agents reside in the environment: the world and other agents The environment can be characterized by a set of states S={s1,s2,...} Agents execute actions A={a1,a2,…} An action is a function action: SS, that affects the environment via state change So, in general, an agent is a set of actions that receive input from the world and manipulate the state of the world

Agent - Environment Interaction Output Sensor Input Environment

Architecture Design A map of the internal structures of the agent, includes: data structures operation that can be performed on them control and data flow between the structures Starts from a high-level definition and traverses through refinements Design decisions result in adding details, getting closer to code level, reducing generality

Information and Processing Needs System architecture design needs knowledge of: what the expected inputs are? what the required/expected outputs are? what processing can provide this relation between input and output? For agents, in particular: what are the possible/expected states of the world? how should the world state be perceived? how should the agent’s reasoning be affected by the world state? how should agent reasoning result in agent action?

Agent Information Needs Is the “state of the world” sufficient? Yes. Usually it is too much: some or most of it may be inaccessible dynamic and possibly non-deterministic includes other agents, users, internet, etc Perception filters and reduces amount of info. Design question: what is the minimal set of data and how to filter/extract it?

Agent Architectures Reactive architectures Deliberative architectures Layered architectures Belief, Desire, Intention (BDI) Concurrent architecture (RETSINA)

Agent Architectures (cont) Reactive vs. deliberative Reactive agents: upon input from the environment, they react with action execution Deliberative agents: upon input, a reasoning process is invoked. Action is based on the results of the reasoning Agents may populate the whole spectrum between purely reactive and maximally deliberative (rational)

Agent Processing Needs Reactive agents only need to map between world states and actions Deliberative agents need to reason for action. May include: taking into account historical states of world creating and maintaining an internal state reasoning about world and states planning, re-planning for current/future action learning

Agent Processing Needs (continued) Collaborative (social) agents need, in addition: maintain models of other agents and the society reason about others plan collaborative activity reason about interaction: communication, coordination, collaboration

Required Agent Attributes Perception - a function perceive: SP, where P={p1,p2,…} a set of percepts: required for both reactive and deliberative agents may be provided via sensor or any other input Internal state I (records history) not necessary for reactive agents deliberative agents need to maintain information regarding past activity to allow for deliberation Reasoning: performed mainly by deliberative agents, but may be useful for reactive, too Learning: only in deliberative agents

Reactive Architecture Agent Perception Action Environment

Agents without State (Reactive) Perception is a function perceive: SP Action is a function action: SS, Action selection is a function as: PA The world state results in a percept via perception, the percept results in an action selection, and the action transforms the state of the world

Example: Subsumption (Brooks) An agent decision making is performed by a set of task accomplishing behaviors (TAB) In Brooks’ implementation, each TAB is a finite state machine In other implementation, TABs are rules of the type situation  action, which maps percepts to actions In the subsumption architecture, multiple behaviors can be activated simultaneously Action selection is based on a subsumption hierarchy, behaviors arranged in layers, which are at different layers of abstraction (layered architecture)

Reactive: Pros and Cons simplicity, economy computational tractability fault tolerance overall behavior emerges from component interaction Cons: without a model of the environment, agents need sufficient info. to determine action agents are “short-sighted”, may limit decision quality relationship between components not clear.

Agents with State = Deliberative Perception Action Reasoning State Environment

Agents with State (Deliberative) Perception is still a function perceive: S  P Action is still action: SS But action selection (was as: PA), is now the function as: IA In addition, update: P×II is a function that update the internal state based on percepts (may include complex reasoning)

Agents with State: Refinement Perception Action Reasoning State planning learning inference Environment

Layered Agent Architectures Usually, but not always, deliberative architectures Decision making is performed via separation to several software layers Each layer reasons at a different level of abstraction. Layers interact Two major types: vertical layers: perception input and action output are dealt with by a single layer each horizontal layers: each layer directly connects to perception input and action output

Layers’ design Typically, at least two layers, one for reactive behavior and one for proactive No reason not to have multiple layers Typology: information and control flow between the layers, e.g.: Agent Perception Action

Information and Control Flow Action output Layer n Layer 2 Layer 1 Layer n Layer 2 Layer 1 Layer n Layer 2 Layer 1 Perceptual input Action output Perceptual input Perceptual input Action output Horizontal Vertical (one pass) Vertical (two pass)

Layers Pros and Cons Horizontal Vertical each layer acts like an agent - provides independency, simplicity for n different behaviors we implement n layers competition between layers can cause incoherence need for mediation between layers: exponentially complex, a control bottleneck Vertical Low complexity, no control bottleneck Less flexible and not fault tolerant: one decision needs all layers

TOURINGMACHINES Three layers produce suggestions for action: reactive: implements situation-action rules as in Brooks’ subsumption architecture planning: achieves proactiveness via plans based on a library of schemas modeling: model of world, other agents, self, predicts conflicts, generates goals to resolve them Domain of implementation: multiple vehicles

Example: TOURINGMACHINES Perception input Modeling layer Perception subsystem Action subsystem Planning layer Reactive layer Action output Control subsystem

INTERRAP A vertically layered two pass architecture Layers have similar purposes as in TOURINGMACHINES Each layer is associated with a knowledge-base Layers interact with each other: bottom-up: activation top-down: execution

Example: INTERRAP Cooperation layer Social knowledge Plan layer Planning knowledge Behavior layer World model World interface Perception input Action output

Belief-Desire-Intention Architecture Based on practical reasoning and decision on actions. Involves: decision on what goals we want to achieve: deliberation decision on how to achieve these goals: means-ends reasoning Choosing some options creates intentions. These: usually lead to action should persist: once adopted, an agent should persist with the intention, attempt to achieve it the intention should be dropped if it is clearly non-achievable it was already achieved the reason for the intention is not there anymore are related to beliefs about the future

BDI Architecture Components A set of current beliefs about the environment A belief revision function (brf) - updates current beliefs based on perception An option generation function - determines available options (desires) based on beliefs and intentions A set of desires (current options) - possible courses of action available A set of current intentions - the options the agents is committed to trying to perform A filter function (deliberation) - determines new intentions based on current beliefs, desires, intentions An action selection function - selects actions based on intentions

Schematic BDI Architecture Perception input brf beliefs Generate options desires filter intentions action Action output

BDI Pros and Cons Key problem: difficult to balance between mental activities Example: dropping intentions requires reconsideration, which is costly but needed Rate of environment change helps set re-consideration Questionable: what advantage do mental states provide? Intuitive, provides functional decomposition Easy to define formally, using logic, and convert to code

Concurrent Architectures (RETSINA: Sycara & al.) Include multiple functional and knowledge modules that work concurrently Coherence between the functional modules is achieved via shared databases Typical functional separation: communication and collaboration planning and reasoning action scheduling execution and monitoring

Example: RETSINA Agent Architecture

Functional Components Communicator: handles incoming and outgoing messages in an ACL. Converts requests into goals/objectives Planner: takes objectives and devises detailed plans to achieve them. Creates tasks, actions and new objectives. Uses plan fragments from libraries Scheduler: schedules actions for execution Execution monitor: executes actions and monitors Coordination/collaboration: reasons for such activities, may be internal to planner or to communicator Self-awareness: maintains self model: load, state, etc

Planning By incremental instantiation of plan fragments Conditional planning mechanisms Interleaving planning, information gathering, and execution Declarative description of information flow and control flow requirements 11

Task decomposition

Knowledge Components Objective DB: holds the agent’s objectives Task DB: holds the agent’s tasks and actions, before they are scheduled for execution Schedule: holds scheduled actions Task reduction library: includes a set of possible task decompositions Task schema library: includes plan fragments, each provides details on how to perform a task Beliefs DB: holds the beliefs of the agent regarding information relevant to its activity

Architecture Attributes Functional components do not directly interface or synchronize with each other Knowledge components do not directly interface or synchronize with each other Functional components work concurrently These provide: reusability and substitutability of components efficient utilization of computational resources timely task performance reduced development effort

RETSINA Agent Functionality Interacts with humans and other agents Anticipates and satisfy human information needs Provides decision support Integrates planning, information gathering and execution Acquires, use and disseminate timely and relevant information Adapts to user, task and situation 10