1. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents1

2. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents2 Objectives List and describe the dimensions of a multi-agent system: heterogeneity, control regime, cooperation, and goals List and describe the axes for describing a MAS task (time, subject of action, movement, dependency) List and describe the axes for describing a MAS collective (composition, size, communications, reconfigurability) Compute the social entropy of a team. Describe the use of social rules and internal motivation for emergent social behavior.

3. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents3 The Study of Multiple Robots Distributed Artificial Intelligence Distributed Problem Solving Multi- Agent Systems

4. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents4 The Study of Agency (after Stone and Veloso 2002) Distributed Artificial Intelligence Distributed Problem Solving Multi- Agent Systems How to solve problems Or meet goals by “divide and conquer” Single computer: How to decompose task? How to synthesize solutions? Divide among agents: Who to subcontract to? How do they cooperate?

5. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents5 4 Dimensions of a Multi-agent System Heterogeneity –Same (homogeneous) vs. different (heterogeneous) –Can be different on either software or hardware Control Regime –Centralized vs. Distributed Cooperation –Active (acknowledge each other) vs. Non-active (cooperation emerges, not explicit) –Communicating or non-communicating Goals –Common goal (same, explicit) vs. Individual goal

6. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents6 The Ecological Niche of a Multi- Agent System Remember…. Single Robot –Task –Environment –Agent

7. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents7 The Ecological Niche of a Multi- Agent System Multi-agent system –Task –Environment –Individual Agent –Collective emphasis

8. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents8 MAS Ecological Niche: Task (after Balch 02) There are 4 axes of a MAS Task –Time –Subject of Action –Movement –Dependency

9. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents9 4 Categories of Time 1.Fixed time –ex. Collect as many cans in 10 minutes 2.Minimum time –ex. Visit all rooms as fast as possible (minimize the time) 3.Unlimited time –ex. Patrol the building 4.Synchronization required –ex. Push two buttons at the same time

10. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents10 Class Question Recall the “Call a Conference” task on the Scientific American Frontiers’ Robots Alive! In that task robot(s) had to find an empty conference room, then let professors know where and when the meeting was This task falls into what category of time? –Fixed –Minimum –Unlimited –Synchronized

11. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents11 Class Question Consider the task of humanitarian demining– clearing a complex terrain of land mines- with robots. This task falls into what category of time? –Fixed –Minimum –Unlimited –Synchronized

12. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents12 2 Categories of Subject of Action Subject of Action: –Object-based robots place a single object- ex. soccer –Robot-based robots place themselves- ex. mapping

13. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents13 Collaborative Mapping

14. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents14 Class Question Recall the “Call a Conference” task on the Scientific American Frontiers’ Robots Alive! In that task robot(s) had to find an empty conference room, then let professors know where and when the meeting was This task falls into what category of subject of action? –Object-based –Robot-based

15. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents15 Class Question Consider the task of humanitarian demining– clearing a complex terrain of land mines- with robots. This task falls into what category of subject of action? –Object-based –Robot-based

16. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents16 4 Categories of Movement 1.Coverage –Spread out to cover as much as possible 2.Convergence –Robots meet from different start positions 3.Movement-to –Going to a single location 4.Movement-while –Formation control

17. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents17 Class Question Recall the “Call a Conference” task on the Scientific Amercian Frontiers’ Robots Alive! In that task robot(s) had to find an empty conference room, then let professors know where and when the meeting was This task falls into what category of movement? –Coverage –Convergence –Movement-to –Movement-with

18. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents18 Class Question Consider the task of humanitarian demining– clearing a complex terrain of land mines- with robots. This task falls into what category of movement? –Coverage –Convergence –Movement-to –Movement-with

19. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents19 3 Categories of Dependency 1.Independent –Robots don’t have to work directly or be aware of others 2.Dependent –Must work together for efficiency ex. Box pushing 3.Interdependent –Cyclic dependency ex. resupply

20. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents20 Box-Pushing

21. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents21 MAS Task Summary Time –Fixed time task (ex. Collect as many cans in 10 minutes) –Minimum time (ex. Visit all rooms as fast as possible) –Unlimited time (ex. Patrol the building) –Synchronization required (ex. Push two buttons at same time) Subject of Action –Object-based (e.g., robots place a single object- soccer) –Robot-based (e.g., robots place themselves- mapping) Movement –Coverage (ex. Spread out to cover as much as possible) –Convergence (ex. Robots meet from different start positions) –Movement-to (ex. Going to a single location) –Movement-while (ex. Formation control) Dependency –Independent (ex. Doesn’t require agents to know about others) –Dependent (ex. Task requires multiple agents) –Interdependent (ex. Agents depend on each other cyclically)

22. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents22 Class Question Consider the task of search and rescue, where multiple robots are to be used to search a collapsed building. Describe the task in terms of the 4 axes of a collective task –Time –Subject of action –Movement –Dependency

23. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents23 Class Question Consider the task of forensic sampling, where robots are to enter the floor a building where a crime has been committed, and then photograph and scan the entire floor as accurately as possible. Describe the task in terms of the 4 axes of a collective task –Time –Subject of action –Movement –Dependency

24. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents24 MAS Ecological Niche: Collective (after Dudek, Jenkin, and Milios 02) There are 4 axes of a collective: –Composition –Size of the collective –Communication –Collective reconfigurability

25. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents25 2 Categories of Composition Composition –Homogeneous –Heterogeneous

26. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents26 Case Studies Georgia Tech 1994 AAAI Mobile Robot Competition team Each robot hardware and software homogeneous Reactive behaviors –Wander-for-goal –Move-to-goal –Avoid –Avoid-other-robots –Grab-trash –Drop-trash Affordances –Orange=goal –Green=robot –Blue=trashcan Dimensional score: Homogeneous Distributed control Active cooperation (though minimal) Individual goal

27. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents27 Example of Heterogeneous Team USF USAR team Robot had different hardware, software Currently teleoperated navigation with autonomous reactive victim detection Single goal, active cooperation –Confirm a victim with distributed sensors –Open door, “spotting” for navigation in confined spaces Dimensional score: Heterogeneous Distributed control (could be central.) Active cooperation Single goal

28. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents28 Social Entropy Way to measure heterogeneity of a collective (go to board-> 4 identical, 4 marsupial)

29. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents29 Example of Heterogeneous Team USC UAV/UGV team autonomous helicopter transporting small robot Currently teleoperated Single goal, active cooperation

30. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents30 4 Categories of Size Size of the collective –Alone –Pair –Limited n<<than size of task or environment –Infinite n>>than size of task or environment)

31. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents31 3 Categories of Reconfigurability Collective reconfigurability –Static The organization doesn’t change, no matter what –Communicated Coordinated rearrangement - Ex. Ordered to change formation –Dynamically Changes arbitrarily (esp. due to failure) Ex. A robot fails

32. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents32 Box Pushing: Dynamic Reconfigurability

33. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents33 Ex. Dynamic Reconfigurability

34. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents34 Ex. Dynamic Reconfigurability

35. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents35 Physically Reconfigurable Robots

36. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents36 Categories of Communication One robot causes an external change in world that can be observed by another robot) Can minimize interference Communication Range - None: robots cannot communicate with other robots directly - Near: robots can only communicate with other robots that are sufficiently near (due to the decaying power of communication signal with distance), - Infinite: robots can communicate with any other robot.

37. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents37 Communication Bandwidth –Infinite comms are free –Motion costs as much to communicate as it would to move –ex. Box pushing (if other robot can feel the box, it’s comms) –Low comms costs more than moving from one location to another –Zero no communication between agents

38. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents38 Communication Topology Broadcast Address Tree Graph

39. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents39 What Do Robots Say to Each Other? How do they “talk”? –Implicit: signaling, postures, smell –Explicit: language Who does the talking? –“the boss” - Centralized control –Everybody - Distributed control

40. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents40 What do Robots Say? (after Jung and Zelinsky 02) Communication without meaning preservation –Emitter can’t interpret its own signal –Receiver reacts in a specific way (stimulus-response) –Ex. Mating displays, bacteria emit chemicals Communication with meaning preservation –Shared common representation –Ex. Ant leaves pheromone trail to food, itself & peers can follow –Ex. Wolves leave scent markings

41. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents41 Class Exercise Consider the case of resupply, where many multiple vehicles are in the field and a lesser number of smaller vehicles exist to carry fuel to them, return to base, and then carry more fuel out on demand. A field vehicle emits a message that it needs to be refueled. The message intensity increases inversely proportional to the amount of remaining fuel. Describe the MAS task. Describe the MAS collective.

42. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents42 In the end…most popular Homogeneous Non-communicating agents Heterogeneous Non-communicating agents Homogeneous communicating agents Heterogeneous communicating agents

43. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents43 Class Exercise Design a multi-agent team for USAR in terms of –Heterogeneity –Control –Cooperation –Goals

44. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents44 How to Get “Right” Emergent Behavior Societal Rules vs. behaviors –Nerd Herd, Maja Mataric What if homogeneous, individual goals operating in the same area?: example-- traffic and traffic jams Motivation –ALLIANCE, Lynn Parker What if have single goal, divided among homogeneous agents and one robot breaks?: example—cleaning up a nuclear spill

45. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents45 Mataric’s Nerd Herd and Social Rules

46. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents46 Explicit Social Rules vs. Behaviors Societal Rules –Ignorant Coexistence Basic reactive approach, except robots couldn’t recognize other robots High degree of task interference –Informed Coexistence Recognize each other PLUS simple social rule: if detect robot, stop and wait for time P; if still there, turn left then resume move to goal Better –Intelligent Coexistence Recognize each other PLUS behavior: repulsed by other robots concurrent with attraction to move in same direction as the majority Best

47. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents47 Motivation: ALLIANCE Divide and conquer works until a robot fails - then what about the failed robot’s area? Robot A fails: –It may realize that it is not doing a good job: becomes increasingly FRUSTRATED and change behavior (give up) - it is called ACQUIESCENCE Allows other robots to help without task interference Robot B is finished with its task –Sees that it is waiting for Robot A and becomes increasingly FRUSTRATED until it decides to help - IMPATIENCE –Goes and helps

48. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents48 Emotional Waiters USF heterogeneous team –Robot Server, serve food, count treat removal –Robot Refiller –Problem in 1999: refiller blocked, Server stuck Each robot has an emotional state generator –Emotions result from observed progress on task (and personality, motivation) Waiter calls for refill, if refiller takes long time, gets impatient, begins to move towards the refiller (intercept), eventually goes to refill station

49. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents49 Emotions: Waiters Wait Get Intercept Go Home Hurry Refill Refiller Serve Intercept Go Serve Impatient Request Waiter

50. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents50 Example: Comms using ACL Wait Get Intercept Hurry Refill Refille r Serve Intercept Impatient Request Waiter (achieve :sender butler :receiver leguin :reply-with 1234 :language plain :ontology waiters :content refill )

51. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents51 Cont. Wait Get Intercept Hurry Refill Refiller Serve Intercept Impatient Request Waiter (achieve :sender butler :receiver leguin :reply-with 1234 :language plain :ontology waiters :content hurry )

52. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents52 Cont. Wait Get Intercept Hurry Refill Refiller Serve Intercept Impatient Request Waiter (achieve :sender butler :receiver leguin :reply-with 1234 :language plain :ontology waiters :content meet me half way )

53. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents53 Summary Many, cheap robots are often better than single, expensive robot Multi-agents are generally at least reactive, sometimes hybrid deliberative/reactive Dimensions for categorizing: –Heterogeneity, control, cooperation, and goals –(may change dynamically) Interference is a big problem –Social rules –Emotions, Motivation Social entropy can be used to measure heterogeneity

54. 8 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 8: Multi-agents54 Review Questions What are the dimensions of a multi-agent system? –Heterogeneity, control regime, cooperation, goals What are the four axes of a task in a collective? – time, subject of action, movement, dependency What are the four axes of a collective? –composition, size, communications, reconfigurability Which is more likely to fail to in the field? –a team R with 1 member of caste 1 and 5 members of caste 2 –A team R with 6 members of caste 1