1. Topic 2: Multi-Agent Systems a practical example categories of MAS examples definitions: agents and MAS conclusion

2. Think about this … a practical exercise: “re-arranging entities in a constrained world”  problem domain: Constrained world Entities  Position  Mobile Formation - new position

4. Think about this … a practical exercise: “re-arranging entities in a constrained world”  problem domain: Constrained world Entities  Position  Mobile Formation - new position  solution ?

5. Think about this … (cont.) 1. a centralized solution  how ?  one “overall planner”  everyone gives his information (initial and target position) to the planner  the planner calculates a plan  the planner: instructs individual entities (you move to position X, …) or distributes individual plans  advantages ?  simple  easy to maintain  planning quite well-known  disadvantages ?  bottleneck  single-point-of-failure  tractable?  who is the planner?  what global knowledge does planner need? #entities, start of protocol, … collect plan re-distribute

6. Think about this … (cont.) 2. a distributed solution  how ?  different tasks in re-arranging e.g. per group of entities  one “planner” per task/group  one overall task manager (coordinates tasks)  collection + redistribution (per task)  …  advantages ?  more scalable (a bit)  easy distribution  disadvantages ?  bottleneck / scalability ?  many-points-of-failure  how to designate the planners?  what global knowledge does this require the sorters to have ? #entities, groups, start of protocol, … ? … …

7. Think about this … (cont.) 3. a distributed, functional decomposition solution  how ?  assign experts collector planner distributor  advantages ?  more scalable (a bit)  easy distribution  clear roles / responsibilities  disadvantages ?  bottleneck / scalability?  many-points-of-failure  how to designate the experts?  what global knowledge do experts need? #entities, start of protocol, …? the experts collector planner distributor

8. Think about this … (cont.)  ok, but what if … #entities is not known …? entities may enter or leave the system at all times…? target positions can be changed during execution? frequency of change > time to (re)plan? we do this for 10.000s of entities? what if a movement did not happen the way it was supposed to? in general, what if it is not a “single shot application”, but a “going concern” ? … it’s a tough world …

9. Think about this … (cont.) 4. an enhanced distributed, functional decomposition solution  how ?  make the experts more intelligent  monitor the system for change  use knowledge to tackle change efficiently (collecting/planning/distributing)  extra advantages ?  more flexible to change  awareness of failures  disadvantages ?  scalability  more complex experts  global knowledge the experts collector planner distributor

10. Think about this … (cont.) 5. a decentralized solution  how ?  every entity tries to force its way to target position  local conflict resolution to avoid deadlocks/starvation  advantages ?  simple (if robots: low cost)  easy distribution  scalable  no single-point of failure  only local actions  only local knowledge (neighbours, order)  disadvantages ?  no central view ==> not for one-shot applications  no hard guarantees?

11. Think about this … (cont.)  ok, but what if … #entities is not known …?OK entities may enter or leave the system at all times…?OK target positions can be changed during execution?OK frequency of change > time to (re)plan?OK we do this for 10.000s of entities?OK in general, what if it is not a “single shot application”, but a “going concern” ?OK … it’s a tough world …

12. What’s the point …? Problems may have many solutions…  solutions have qualities  pick the appropriate solution for the set of required qualities This course is about…  distributed problems in a dynamic environment  with requirements for flexibility adaptability scalability …

13. What is a MAS ? “in essence, a MAS is a philosophy to model systems”  indicating how to solve a problem …  in a complex world …  with autonomous entities … solutions of example centralized distributed distributed experts enhanced distributed experts decentralized multi-agent systems

14. What is a MAS ? Categories of MAS … 1. distributed “experts”  enhanced distributed entities  cooperating entities  mainly functional decomposition  focus on detailed modeling of individual agent  mental states  knowledge representation  planning 2. decentralized systems  collective behaviour  local actions/interactions only  focus on the collection and the environment

15. Application domains distributed control applications  e.g. AGVs, robots, virtual entities, trafic, logistics simulation pure algorithms …

16. Example: a distributed control system - 1 Control system for automated warehouse management  Egemin N.V.  AGVs (automated guided vehicles)  control system  the world: large dynamic  tasks  AGV failure  batteries

17.  e.g. Vredestein tyres (NL)  centralized solution ?  towards a distributed solution in a tough world distributed  enhanced distributed system(cat. 1) distributed  decentralized system(cat. 2)

18. Example: a distributed control system - 2 Reactive Bubbles  moving entities  constraint environment  what if … massively distributed !? no global knowledge !? new bubbles / bubbles disappear !? environment changes !?  is it efficient !? efficient ? NO ! but fascinating! and a solution of our problem in a tough world …

19. Example: Collective Robotics Collective robotics cooperative parts of one robot  e.g. several servos for a single robot arm multiple cooperating robots  coordinate actions  to accomplish common task a a a

20. Example: MAS – simulation Goal: study phenomena in physics ecology biology chemistry social sciences geography … Approaches mathematical:relationships of variables  (differential) equations, transition matrices, … MAS  model individual entities / environment / …

21. MAS – simulation (cont.)  MAS simulations  model individuals’  behaviour  actions  interactions  …  model environment  advantages: miniature laboratory  simulation model close to real-world entities  allows to study consequence of individual behaviour  reasoning process can be included  …

22. Example: algorithms edge detection / constraint satisfaction problems / data mining

23. Why use MAS for such systems ? clean software abstraction for modelling such systems autonomy cooperation agents are (should be) critical by nature  do not rely on “anything”  should be “designed to be flexible” agents are (should be) adaptive by nature  change their behaviour in this highly dynamic world … MAS solves problems! but also introduces new challenges…  communication  coherent behaviour  control MAS is NOT a holy grail … !! MAS is NOT suitable for all applications … !!

24. A multi-agent system is a system consisting of multiple autonomous entities, called agents, which are situated in an environment that the agents can partially observe and in which they can act and cooperate to achieve system objectives. MAS: definition …

25. … requires a definition of … agents  actions  production/consumption/manipulation of objects  perception an environment objects in the environment objectives MAS: definition …

26. MAS environment properties  accessible vs. inaccessible  accessible = can provide complete, up-to-date information about the (entire) environment state  the more “accessible”, the easier the system  most real-world systems: inaccessible state  deterministic vs. non-deterministic  determinism w.r.t. result of agent actions  e.g. up (B,C)  ??  most real-world systems: non-deterministic environment  agents do not have full control  actions can fail  …

27. MAS environment (cont.)  …  static vs. dynamic  static: does not change between two actions (e.g. planning algorithms)  most real-world systems: dynamic environment  under constant change objects in the environment change environment changes concurrent / autonomous agents  concurrent actions  e.g. software environment real-world environment:  discrete vs. continuous  as in “number of states”

28. Agents  An agent is physical or virtual entity capable of perceiving its environment capable of acting in an environment (not just reasoning) capable of communicating driven by goals possessing resources having partial representation of the environment having particular capabilities (skills and services) acts towards its objectives J. Ferber environment action observation agent

29. an agent  properties reactivity reacts to stimuli (changes in env., communication, …) autonomy does not require user interaction proactive-ness aims to achieve its own goals, therefore initiates appropriate actions social ability cooperates / coordinates / communicates / … embodied situated in the environment mobile moves around network sites learning learn from past experiences … essential extra

30. MAS: agent actions what is an agent action ? “an attempt … … to bring about a state of affairs … … in the environment … … or another agent” non-deterministic  did anything happen ?  did happen what agent expected ?  did something else happen ?  did the message arrive ?  … this is not a limitation to make things hard … it’s a fact of life … ==> agents need to be flexible / adaptive / …

31. MAS: objects in environment objects in the environment passive objects  e.g. pallets, files, … dynamic objects  e.g. rolling ball, stones cooling down agents

32. MAS objectives / behaviour objectives system objective  e.g. find a solution to a constraint problem  e.g. “ongoing concern”  e.g. continuous schedule of product manufacturing  e.g. ant colony survival  e.g. network management individual agent objective  directly related to system objective  e.g. efficient expert service  emergent behaviour  system objective “emerges” from achievements of individual objectives  e.g. ant foraging

33. A multi-agent system is a system consisting of multiple autonomous entities, called agents, which are situated in an environment that the agents can partially observe and in which they can act and cooperate to achieve system objectives. MAS: definition …

34. Key challenge of MAS imagine that you have to develop software which will work in an environment …  that you can only partially observe  where the results of what you do are not guaranteed  where the environment constantly changes by other activities  which can achieve to perform its objective communicate cooperate coordinate adapt own behaviour! negotiate …

35. Conclusion MAS: Issues and Challenges How do we decompose problems into behaviour for individual agents ? How do we ensure agents act coherently in making decision or taking action ?  do local actions have harmful global effects  avoiding unstable system behaviour How to enable agents to communicate and interact ?  communication languages and protocols  interoperation of heterogeneous agents  finding useful existing agents in open environments How does agent decide what to do ?  action selection mechanisms How do we build agents ?  actions, plans, and knowledge  coordination actions How do we compromise different views and conflicting goals of agents trying to coordinate ? How do we engineer multiagent systems ?

36. Conclusion MAS is about  software engineering distributed applications engineering  “synergy” of different techniques/philosophies/applications/… distributed systems software engineering robot control languages for programming MAS AI …  AI is a subfield of MAS “Agents are 99% computer science, and 1% AI.”