Katia SycaraAAAI'971 James Bond and Michael Ovitz The Secret Life of Agents Katia Sycara The Robotics Institute Carnegie Mellon University Pittsburgh, PA (412) Project Homepage:

Katia SycaraAAAI'972 Liren Chen Somesh Jha Rande Shern Dajun Zeng Keith Decker Anadeep Pannu Vandana Verma Team Members CMU Prasad Chalasani Kostya Domashnev Onn Shehory

Katia SycaraAAAI'973 Talk Outline Introduction to Agents Motivations and advantages of distributed agent technology The Retsina Approachª Retsina Agent Architecture Middle Agents Multi-agent interaction protocols (negotiation, contingent contracting) Retsina Applications Concluding Remarks ________________________________ ª Retsina stands for “Reusable Task Structured Intelligent Networked Agents.”

Katia SycaraAAAI'974 The (Re)-Emergence of Agents: The Marriage of Two Holy Grails Goal-directed Adaptive Knowledge-based Reusability Robustness Flexibility AISE Ubiquitous Networked Information Access Intelligent Software Agents!

Katia SycaraAAAI'975 What is an Agent? A computational system that –has goals, sensors and effectors –is autonomous –is adaptive –is long lived –lives in a networked infrastructure –interacts with other agents

Katia SycaraAAAI'976 Agent vs. Agent James BondMichael Ovitz

Katia SycaraAAAI'977 Next Generation of Agent Technology Currently, agent technology is mostly single agent focusing on information retrieval and filtering according to user profile Multi-Agent Systems that interact with humans and each other Integrate information management and decision support Enable real-time synchronization of the tasks and actions of humans in teams and organizations Acquire and disseminate timely and relevant information Anticipate and satisfy human information and problem solving needs Notify about changes in the environment Adapt to user, task and situation

Katia SycaraAAAI'978 Motivation for Multi Agent Systems (MAS) Technology Global Information & Markets Increasingly networked world Vast quantities of unorganized information Diverse information sources Inability for human to manage information access process--- information overloading Moving from locating documents to making decisions

Katia SycaraAAAI'979 Features of MAS Multiple agents connected through communication networks Coordination - no agent has sufficient information or capabilities to solve problem aloneª Decentralized control - no master agent Decentralized data - no global data storage Agent Coupling - balancing computation and communication Asynchronous - multiple activities operating in parallel _______________________ ª Agents could be cooperative or self-interested.

Katia SycaraAAAI'9710 MAS Basic Questions (Bond and Gasser, 88) Coherence in coordinated decision making Recognition and reconciliation of disparate viewpoints or conflicting intentions Graceful performance degradation in face of missing information and resources Satisfaction of system-wide criteria (e.g., optimality of solution, hard real-time deadlines, etc.) How to recognize workload imbalances and appropriately redistribute activities and responsibilities among agents Modeling other agents Synthesizing different views and results

Katia SycaraAAAI'9711 MAS Concepts and Tools Distributed Constraint Satisfaction and Optimization (Yokoo, et al,~1991; Sycara, et al. 1991) Distributed Truth Maintenance and Multi-Agent Search (Huhns et al. 1991, Ishida 1997) Organizational Structuring (Lesser and Corkill 1991, Gasser 1993, Decker 1995) Multiagent Planning (Georgeff 1983, 1984, 1995; Durfee and Lesser 1991, Jennings 1995, Grosz et al. 1996) Contracting (Smith 1980, Mueller 1993, Sandholm 1997) Negotiation (Sycara 1990; Kraus 1991, Zlotkin 1996)

Katia SycaraAAAI'9712 MAS Concepts and Tools (Contd.) Economic and Game Theoretic Techniques (Rosenschein 1995; Gmytrasiewicz et al. 1991, Wellman 1993) Open systems (Hewitt 1991, Gasser 91) Multiagent Logics and Ecological Approaches (Cohen and Levesque 1987; Ferber 1990, Shoham 1993; Huberman et al. 1996) Social Laws and Norms (Tenenholtz 1991, Castelfranchi 1993) Multi-Agent Learning (Sen 1993, Durfee 1994, Sycara 1996)

Katia SycaraAAAI'9713 Retsina Approach Architecture that includes data- and knowledge-bases and a distributed collection of intelligent agentsª Reusable and composable agent components (agent editor, agent operating system) Operates in an open world –agents, network links, and information sources appear/disappear –uncertainty Dynamic agent team formation on-demand ____________________________ ª K.Sycara, K.Decker, A.Pannu, M.Williamson, and D.Zeng. Distributed Intelligent Agents. IEEE Expert, Dec-96

Katia SycaraAAAI'9714 Retsina Functional Organization

Katia SycaraAAAI'9715 Main System Issues in MAS Single agent architecture –Retsina agent architecture –Agent Self-cloning Finding other agents –Middle agents –Matchmaking and brokering Agent interaction protocols –Negotiation –Contingent contracting

Katia SycaraAAAI'9716 Retsina Agent Architecture Planning –hierarchical task network-based formalism –library of task reduction schemas Salternative task reductions Scontingent plans, loops –incremental task reduction, interleaved with execution  information gathered during execution directs future planning Scheduling –fully expanded leaf nodes = executable basic actions –enabled actions (all parameters and provisions in place) –adjust periodic tasks with missed deadlines

Katia SycaraAAAI'9717 Communication and Coordination –processes incoming and outgoing messages –creates new goals/objectives –determines coordination interactions –addresses security issues Execution Monitoring setup execution context (parameters and provisions) action monitoring Sdeadlines/timeouts Sdata collection for decisions (e.g. cloning) complete execution (provide results to appropriate downstream actions)

Katia SycaraAAAI'9718 Retsina Agent Architecture

Katia SycaraAAAI'9719 A task Structure (Advertisement Task Structure)

Katia SycaraAAAI'9720 Reusable Behaviors Advertising –send agent capability model to middle-agent(s) –shared query behavior for other agents Polling for messages Answering queries: one-shot and periodic Monitoring for changes and notification Self-cloning

Katia SycaraAAAI'9721 Self-Cloning Process Agents that perceive an overload look for other agents to pass tasks to (simple model to predict idle time using learned estimation of task durations) When other agents not found: –locate a host with resources for cloning –create a clone on the host –partition tasks and transfer to clone –the ``old'' agent updates its advertisement; the ``clone'' agent advertises When clone is idle -- consider self-extinction, and shut down if necessary.

Katia SycaraAAAI'9722 Cloning Experimental Setting Number of agents: 10 to 20. Number of clones allowed: up to 10. Number of tasks arriving at the system: up to Task distribution with respect to the required capabilities for execution: normal distribution, where 10% of the tasks are beyond the capabilities of the agents. Agent capabilities: an agent can perform up to 20 average tasks simultaneously.

Katia SycaraAAAI'9723 Task Completion w/wo Cloning

Katia SycaraAAAI'9724 Middle Agents An agent needs to have some task/service performed. How can it find agents able to perform that task? In an open system: –agents generally don't have knowledge of all other agents –service providers are liable to come and go over time A solution: middle agents that specialize in making connections between agentsª _________________________ ª K.Decker, K.Sycara, M. Williamson. Middle-Agents for the Internet. IJCAI-97

Katia SycaraAAAI'9725 Middle Agent Types Capabilities Initially Known By

Katia SycaraAAAI'9726 Matchmaking: Agent Yellow Page Services

Katia SycaraAAAI'9727 Matchmaking in Agent Coordination When an agent A advertises its capability, it Intends toª perform any task that fits the specification of that capability. –In the Retsina system an agent A advertises a relational schema S A, i.e., agent A intends to answer any query on its schema. If an agent B finds another agent A with a certain capability through matchmaking, B believes that agent A can successfully perform the task. Matchmaking gives operational semantics to predicates such as Intend.to, Bel. _______________________________ ª Grosz and Kraus 1996

Katia SycaraAAAI'9728 Performance of Match-made System

Katia SycaraAAAI'9729 Performance of Brokered System

Katia SycaraAAAI'9730 Agents in Electronic Commerce

Katia SycaraAAAI'9731 Adaptive Negotiation (the Bazaar Model) Aims at modeling multi-issue negotiation processesª Combines the strategic modeling aspects of game-theoretic models and single agent sequential decision making models Supports an open world model Addresses heterogeneous multi-agent learning utilizing the iterative nature of sequential decision making and the explicit representation of beliefs about other agents ______________________________ ª D.Zeng and K.Sycara. “Benefits of Learning in Negotiation.” Proceedings of AAAI-97.

Katia SycaraAAAI'9732 Utility of Learning: Experimental Design The set of players N is comprised of one buyer and one supplier who make alternative proposals. For simplicity, the range of possible prices is from 0 to 100 units and this is public information The set of possible actions (proposed prices by either the buyer or the supplier) A equals to {0, 1, 2,…, 100} Reservation prices are private information. Each player's utility is linear to the final price ( a number between 0 and 100) accepted by both players Normalized Nash product as joint utility (the optimal joint utility when full information is available is 0.25)

Katia SycaraAAAI'9733 Average Performance of Three Experimental Configurations in Bazaar A non-learning agent makes decisions based solely on his own reservation price A learning agents makes decisions based on both the agent's own and the opponent's reservation price

Katia SycaraAAAI'9734 Contingent Contracts and Options Most multi-agent systems don't handle uncertainty effectively –rigid task delegation mechanism (contracts are binding rather than contingent –no explicit modeling of stochastic events –no explicit mechanism for controlling agent performance variability We are exploring the use of option pricing to address the above issues

Katia SycaraAAAI'9735 Evaluation of Contingent vs Binding Contracts In the experiments, we only had two kinds of agents: –Interface Agents: Accept queries from the user. –Information Agents: Answer queries given by the Interface agents. In each cycle a new information agent with a load randomly distributed between L and 0.9 appears with probability . When a new information agent comes up, interface agents have the option to abort the query on the old information agent and restart it on the new one. Interface agents can only switch a bounded number of queries to the new agent. This is indicated as Bound in the graphs. In the experiments the average delay in answering the queries was measured. This is indicated as Delay in the graphs.

Katia SycaraAAAI'9736 Contingent Contracts L

Katia SycaraAAAI'9737 Contingent Contracts Bound

Katia SycaraAAAI'9738 Contingent Contracting to Handle Unreliability of Information Sources Uncertain waiting time in response to queries –random network congestion –uncertain serve congestion/breakdown

Katia SycaraAAAI'9739 The Query Restart Problem Agent A sends query to Agent B. Agent B can complete the query in time X, where TX = 1 with probability p. TX = c (c > 1) with probability 1 - p. Expectation: E X = p + (1 - p) c If not done by time 1, should agent A abort and restart, or wait? Can restarting reduce expectation? The variance? Both? Does it help to repeatedly restart k times?

Katia SycaraAAAI'9740 Strategy: restart just after time 1, if not done by then. Let X i = completion time of i'th query, i = 1,2. X 1, X 2 are independent, identically distributed. New completion time is Y: Y = New expectation E Y = p + (1 - p)(1 + E X 2 )(X 1, X 2 indep.) = 1 + p (1 - p) + (1 - p)  c If (and only if) c > / p, E Y < X 1 ! A Simple Scenario: Single restart { 1 if X 1 = 1, 1 + X 2 if X 1 = c.

Katia SycaraAAAI'9741 A Simple Scenario: k Restarts Number of Restarts k

Katia SycaraAAAI'9742 Applications Visitor Hoster (PLEIADES) Satellite Visibility (THALES) Portfolio Management (WARREN)

Katia SycaraAAAI'9743 Characteristics of Retsina Open System Adaptivity at the agent and organization level provides robustness Service-based, economic coordination of agents Reusable and extensible domain-independent computational infrastructure Integrates information gathering and execution monitoring with decision making Framework for addressing uncertainty and strategic interactions

Katia SycaraAAAI'9744 Future of Software Agents Agent-based software development is an emerging paradigm Agent society that parallels human society Implication of the emergence of agent society for human workplaces, institutions, and social relations Agent society as a unit of intelligence Opportunities and Challenges –The WEB is a vast knowledge base presenting novel opportunities for AI –Overall system (human and software agent) predictability –Security, privacy, trust issues –Integration of legacy systems

Katia SycaraAAAI'9745 Overall Issues in Open MAS Overall agent organization Single agent architecture –Retsina agent architecture TAgent Self-cloning Finding other agents –Middle agents TMatchmaking and brokering Agent interaction protocols –Negotiation –Contingent contracting

Katia SycaraAAAI'9746 Overall Issues in Open MAS Overall agent organization Single agent architecture –Retsina agent architecture TAgent Self-cloning Finding other agents –Middle agents TMatchmaking and brokering Agent interaction protocols –Negotiation –Contingent contracting

Katia SycaraAAAI'9747 Overall Issues in Open MAS Overall agent organization Single agent architecture –Retsina agent architecture TAgent Self-cloning Finding other agents –Middle agents TMatchmaking and brokering Agent interaction protocols –Negotiation –Contingent contracting