1. Intelligent Agents - Lecture 3 Katia Sycara The Robotics Institute katia@cs.cmu.edu www.cs.cmu.edu/~softagents

2. Course Policies The course is based on lectures, lecture notes, and additional materials provided either electronically or in hard copy There will be no exams. Instead: Grading will be based on two projects – mid-term project (40%): a survey on a class-related topic development of an agent business case for agent technology in an area –bigger final project (60%)

3. Example Projects: A. Projects involving programming  Write part of a Retsina information agent. The other, standard parts will be provided. NOTE: requires moderate knowledge and programming experience in PERL, or Webl or JAVA)  Implement agents that participate in an auction (e.g. sealed bid first price auction) NOTE: requires advanced programming skills in JAVA or C++  Given the Retsina matchmaker code, implement other types of middle agents. NOTE: requires advanced programming skills in JAVA.

4. B. Example Midterm Projects not involving programming  Comparative review of auctions (pros and cons)  Comparative survey of advanced e-commerce sites (i.e. not just simply listing a product or set of products)  Comparative survey of software tools for building agents, and what is missing.  Value added uses of agents in e-commerce.  Comparative survey of shopbots, issues raised by shopbots.  How would commerce change if everyone had personal agents available? Which types of personal agents would be useful?  Models of marketing on the Web, and use of agents in this endeavor.  Identify and document uses of agent technology in at least 25 top sites. Discuss trends. File sharing programs (e.g. gnutella)

5. Agent Architectures Reactive architectures Deliberative architectures Layered architectures Belief, Desire, Intention (BDI) Concurrent architecture (RETSINA)

6. Concurrent Architectures (RETSINA: Sycara & al.) Include multiple functional and knowledge modules that work concurrently Coherence between the functional modules is achieved via shared databases Typical functional separation: –communication and collaboration –planning and reasoning –action scheduling –execution and monitoring

7. Example: RETSINA Agent Architecture

8. Functional Components Communicator: handles incoming and outgoing messages in an ACL. Converts requests into goals/objectives Planner: takes objectives and devises detailed plans to achieve them. Creates tasks, actions and new objectives. Uses plan fragments from libraries Scheduler: schedules actions for execution Execution monitor: executes actions and monitors Coordination/collaboration: reasons for such activities, may be internal to planner or to communicator Self-awareness: maintains self model: load, state, etc

9. Knowledge Components Objective DB: holds the agent’s objectives Task DB: holds the agent’s tasks and actions, before they are scheduled for execution Schedule: holds scheduled actions Task reduction library: includes a set of possible task decompositions Task schema library: includes plan fragments, each provides details on how to perform a task Beliefs DB: holds the beliefs of the agent regarding information relevant to its activity

10. Architecture Attributes Functional components do not directly interface or synchronize with each other Knowledge components do not directly interface or synchronize with each other Functional components work concurrently These provide: –reusability and substitutability of components –efficient utilization of computational resources –timely task performance –reduced development effort

11. 11 Planning By incremental instantiation of plan fragments Conditional planning mechanisms Interleaving planning, information gathering, and execution Declarative description of information flow and control flow requirements

12. Planning by Reduction T T1T2 Reduce T1 Reduce T2 T T1 T2 A A A1A2

13. Task decomposition

14. 10 RETSINA Agent Functionality Interact with humans and each other Anticipate and satisfy human information and problem solving needs Acquire and disseminate timely and relevant information Provide decision support Adapt to user, task and situation Integrate information management and decision support

15. Multi-Agent Organization Centrally designed/closed organization Open organization: RETSINA Flat Modular Hierarchical Heterarchical Subsumed/compositional

16. RETSINA - REusable Task-based System of Intelligent Network Agents An adaptive, self-organizing collection of autonomous agents that interact with humans and each other: –integrate decision support and information management –anticipate and satisfy human information processing and problem solving needs –perform real-time synchronization of domain activities –notify users and each other about significant changes in the environment –adapt to user, task and situation

17. A distributed collection of intelligent agents Agents collaborative and asynchronous Dynamic, open organization –agents may enter and leave at any time –heterogeneous agent capabilities –alternating user goals and preferences –dynamic information environment RETSINA: System Attributes

18. 8 Agent Types: Interface Agents Task Agents Information Agents Middle Agents All share same basic architecture

19. The RETSINA Multi-Agent Organization User 1User 2User u Info Source 1 Info Source 1 Interface Agent 1 Interface Agent 2 Interface Agent i Task Agent 1 Task Agent 2 Task Agent t MiddleAgent 2 Info Agent n Info Source 2 Info Source 2 Info Source m Info Source m Goal and Task Specifications Results SolutionsTasks Info & Service Requests Information Integration Conflict Resolution Replies Advertisements Info Agent 1 Queries Answers distributed adaptive collections of information agents that coordinate to retrieve, filter and fuse information relevant to the user, task and situation, as well as anticipate user's information needs.

20. RETSINA: Design Goals The agents and the system should: –gather information from distributed, multi- modal sources, provide decision support –access, filter, integrate info. and make decisions – run across different machines –use models of users, tasks, agents to deal with tasks (decompose, pass, etc) –allow code reusability and modularity –be flexible and robust

21. Locating Other Agents Open, dynamic MAS - agents appear and disappear Agents must be able to locate others Broadcast on WWW - inappropriate Solution - middle agents - advertising Remaining problem - whom to choose to maximize system performance

22. Middle Agents Specialize in making connection between agents Store, maintain and provide connection information Provide translation and interoperation services Some middle agent types: –Matchmakers match advertised capabilities –Blackboard agents collect requests –Broker agents process both

23. * Brokering: Requester Provider Broker Request-for-Service Advertise/Unadvertise-Services Reply-Result-of-Service Request-for-Service Requester * Matchmaking: Provider Matchmaker Request-for-Service Advertise/Unadvertise-Services Request-for-Service Reply-Provider-Agents-Names Reply-Result-of-Service Service Brokering vs. Matchmaking in Multi-Agent Systems

24. Locating Agents Agents advertise capabilities to middle agents Use functionally-specific vocabulary Agents ask middle agents for services Middle agents finds providers, notify requesting agents (or broker the service) Requesting agents contact providers when contact info. is given

25. Advertisement Agent contact information Service/capability description Ontology Service availability Cost Advertisement –agent’s interface to the society –expresses agent commitment

26. Some RETSINA Applications Team aiding in joint tactical planning (using a simulated battlefield) Agent-aided aircraft maintenance E-commerce in wholesale markets Robot teams for de-mining Team Rescue Scenario (NEO) Warren system for financial portfolio management

27. Joint Tactical Mission Scenario

29. Air-Force Aircraft Maintenance Aircraft (e.g. F-15) arrive at maintenance airbase Periodic maintenance must be completed in 60 days (varies for different aircraft types) Mechanics inspect aircraft. For a discrepancy: –if unknown to mechanic: consults: experienced mechanics, manuals. –for all discrepancy: a 202A form is filled (by hand). Sent to expert engineers reply (202B) includes repair instructions, approval –mechanic repairs and files a report (and the 202)

30. Aircraft-service Air-Force base 3 Hangar computer RETSINA-based solution Aircraft-service Air-Force base 2 Hangar computer Form agent Aircraft-service Air-Force base 1 Hangar computer History agent History agent History agent Form agent Form 202A 202 DB 202 DB 202 DB Manuals agent Manuals agent Manuals agent Request relevant historical repairs Request relevant manual pages Engineer A’s workstation Engineer B’s workstation Request relevant historical repairs Form 202B Form 202A Sent by mechanic Sent by mechanic Form 202B Sent to mechanic To mechanic

32. Warren Agent Collaboration in –Information Gathering –Information integration –Information filtering –One shot & Persistent queries –Decision Support –a rapidly changing environment

33. Warren (cont) Services to user Decision support –Evaluation –Suggestion Intelligent anticipation –Models & suggests based on allocation –Monitors for news/events

34. Warren Demo Each user has a portfolio When user wants to buy (or sell) a stock –Stock price agent is invoked When a new stock is added to portfolio –Risk allocation critic is invoked User can –Monitor a stock –Retrieve information about finances of company

37. Coalitions for E-Commerce In wholesale markets, purchasing more of the same product reduces price per unit Sellers benefit from selling more and spending less on marketing/distribution Buyers usually do not need large quantities Can form coalitions of buyers, but: –difficult to find other buyers –need agreement on profit and product distribution Solution is simpler via buyers’ agents : –find others, form coalitions, negotiate with sellers

38. The Virtual Market Monitor Coalition Matcher Negotiation Matcher seller 1 seller 2 Catalog Coalition buyer 2 buyer 1 buyer 3

39. De-mining Teams of Robots Multiple robots for de-mining: –sophisticated searchers find mines –inexpensive exploders detonate mines –coordinators support team behavior and interaction with human operator Requires coordination, team behavior –how to form and maintain a team? –when a team member fails, what to do? –how to translate human requests to team activity?

40. Open MAS Demonstration A MAS comprised of RETSINA agents, OAA agents and agentified application are presented We demonstrate: –human/agent, agent/agent interaction –agents collaborate to solve a real-world problem –perform information gathering and filtering –MAS interoperability via middle agents –robustness/fault-tolerance of the system via service/agent substitutability –automation of information-rich and time-critical planning, decision making and execution monitoring

41. A NEO TIE Non-combatant Evacuation Operation: –requires information from multiple, multi-modal sources, may change dynamically, be unreliable –requires time-critical (re-) planning under uncertainty –collaboration among distributed humans, machines –plan execution monitoring Agent Technology Integration Experiment: –demonstrating agent support for NEO activity by exploiting agent technology for information gathering, user interfacing and collaboration automating and speeding up critical planning and monitoring –interoperation with legacy components (e.g. air lift planner such as CAMPS) for NEO

42. The NEO Problem Domain Location: Kuwait city. Time: 2005 Iraq not a threat, reduced military presence in mideast Congress in Kuwait city on ecology & oil conservation Unrest: –night explosion at the conference center - no casualties –terrorist group issues threat against attendees, American, etc. US activity: –ambassador orders evacuations (smaller numbers) –Air Mobility Commander plans large evacuation –Joint Force Commander involved (plan, secure evacuation routes)

43. NEO Scenario Ambassador, JFC discuss evacuation via RETSINA UI Messenger (UIM) and OAA MMM JFC’s UIM monitors content, requests relevant info WebMate provides text news, Maestro - video RETSINA flights agent begins providing schedules OAA flights agent takes over when RETSINA’s fails UIM receives flights’ schedule and displays OAA weather agent begins providing weather RETSINA weather agent takes over when OAA’s fails RETSINA route planner plans evacuation to KWI

44. NEO Scenario (continued) OAA phone agent provides info. about roadblocks Route planner re-plans to KWI airport AMC rep. uses a UIM to request CAMPS to provide airlift plan from KWI RETSINA Visual Sensor Agent (VSA) reports an explosion near KWI, displays it on MMM JFC designates alternative abandoned airfield Route planner re-plans to new destination AMC via UIM requests a new plan from CAMPS CAMPS returns a plan, presented by UIM

45. DemoDisplay Visualization Agent

46. NEO TIE: the contribution The NEO TIE –uses existing agents, MAS, services, legacy systems –makes them inter-operate: resolve conflicts, build interoperability agents –enhance human/MAS collaboration/interfacing –integrate agent and legacy (re-) planning –demonstrate robustness via agent substitutability The agents in the NEO TIE do not –take the critical decisions (but may advise) –evacuate (but may plan for, monitor and display it)

47. 1. Coherent communication of meaningful content between two heterogeneous MAS 2. Functional substitutability of agents 3. “Agentification” of legacy systems 4. Adaptation at different levels: interfaces, MAS organization, single agent 5. Collaboration:human-human, human-agent, MAS Scientific and Technical Contribution

48. Lessons Learned? Agents and MAS may be a promising technology when: –the amount of information and pace of change are overwhelming –users, information and resources are distributed –expertise and activity are distributed –there is no central control –there is a need for flexibility and robustness in face of uncertainty and partial failures –incorporation of pre-existing systems is advantageous

49. Characteristics of RETSINA Open system / Organization robust & adaptive Service-based collaboration Agents are honest Agents “passively” volunteer Matchmaking preserves privacy

50. RETSINA - Conclusion The RETSINA multi-agent approach: –highly dynamic, open, robust –agents actively seek out information –specialized agents collaborate –interleaving planning and execution –reusable infrastructure and architecture –several domain specific implementations