Lecture VI: Collective Behavior of Multi-Agent Systems II: Intervention Zhixin Liu Complex Systems Research Center, Academy of Mathematics and Systems Sciences, CAS

In the last lecture, we talked about Collective Behavior of Multi-Agent Systems I: Analysis

In the last lecture, we talked about Introduction Model: Vicsek model

Multi-Agent System (MAS) Autonomy: capable of autonomous action Interactions: capable of interacting with other agents MAS Many agents Local interactions between agents Collective behavior in the population level More is different.---Philp Anderson, 1972 e.g., small-world, swarm intelligence, panic, phase transition, coordination, synchronization, consensus, clustering, aggregation, …… Examples: Physical systems Biological systems Social and economic systems Engineering systems … …

Vicsek Model (T. Vicsek et al. , PRL, 1995) http://angel.elte.hu/~vicsek/ r A bird’s neighborhood Alignment: steer towards the average heading of neighbors xi(t) : position of agent i in the plane v: the constant speed of birds r: radius of neighborhood : heading of agent i

Vicsek Model Neighbors: r Position: Heading: A bird’s neighborhood Neighbors: Position: Alignment: steer towards the average heading of neighbors Heading: Synchronization: There exists θ, such that http://angel.elte.hu/~vicsek/

In the last lecture, we talked about Introduction Model Theoretical analysis Concluding remarks

The Linearized Vicsek Model A. Jadbabaie , J. Lin, and S. Morse, IEEE Trans. Auto. Control, 2003.

Synchronization of the linearized Vicsek model Theorem 2 (Jadbabaie et al. , 2003) Joint connectivity of the neighbor graphs on each time interval [th, (t+1)h] with h >0 Synchronization of the linearized Vicsek model Related result: J.N.Tsitsiklis, et al., IEEE TAC, 1984

Random Framework Random initial states: 1) The initial positions of all agents are uniformly and independently distributed in the unit square; 2) The initial headings of all agents are uniformly and independently distributed in [-+ε, -ε] with ε∈ (0, ). The initial headings and positions are independent.

Theorem 7 High Density Implies Synchronization For any given system parameters and when the number of agnets n is large, the Vicsek model will synchronize almost surely. This theorem is consistent with the simulation result.

Theorem 8 High density with short distance interaction Let and the velocity satisfy Then for large population, the MAS will synchronize almost surely.

Three Categories of Research on Collective Behavior

Three Categories of Research on Collective Behavior Analysis Given the local rules of the agents, what is the collective behavior of the overall system ? (Bottom Up) Design Given the desired collective behavior, what are the local rules for agents ? (Top Down) Intervention Given the local rule of the agents, how we intervene the collective behavior? J.Han, M.Li, L.guo, JSSC,2006

Example 1: Synchronization A bird’s Neighborhood Simulation Result Alignment: steer towards the average heading of neighbors Q: Under what conditions such a system can reach consensus?

Example 2: Escape Panic D. Helbing, et al., Nature, Vol. 407, 2000 Normal, no panic Fire, panic

Three Categories of Research on Collective Behaviors Analysis Given the local rules of the agents, what is the collective behavior of the overall system ? (Bottom Up) Design Given the desired collective behavior, what are the local rules for agents ? (Top Down) Intervention Given the local rule of the agents, how we intervene the collective behavior? J.Han, M.Li, L.guo, JSSC,2006

Example 1: Formation control How we design the control law of each plane to maintain the form ?

Example 2: Swarm Intelligence (Marco Dorigo et al., 2001-2004) Using their gripper (red in the photos), they can connect together. Then they can, for instance, pass over gap and steps where a single robot would have failed. Using their integrated force sensor, they can coordinate to retrieve an object to a certain location without the use of explicit communication. This is the way ants bring preys to the nest. S-bots can connect to other s-bots to create a bigger structure known as the swarm-bot. The project, that lasted 42 months, was successfully completed on March 31, 2005. It has been selected as one of the success stories of the Future and Emerging Technologies (FET) program of the European Commission. S-bot mobile robot. The s-bot is a small (15 cm) mobile robot developed at the LIS (Laboratory of Intelligent Systems [1]) at the EPFL in Lausanne, Switzerland between 2001 and 2004. Targeted to swarm robotics, a field of artificial intelligence, it was developed within the Swarm-bots project, a Future and Emerging Technologies project coordinated by Prof. Marco Dorigo. Built by a small team of engineers (Francesco Mondada, André Guignard, Michael Bonani and Wikipedia editor Stéphane Magnenat (nct)) of the group of Prof. Dario Floreano and with the help of student projects, it is considered at the time of completion as one of the most complex and featured robots ever for its size. Purpose and use of the s-bot This is a research robot, aimed at studying team work and inter-robot communication. To do this, the s-bots have several special abilities: Of course, all other sensors and actuators, also found on other robots, can be used to do team work such as food foraging. Technical details Closeup of s-bot fixed gripper, used to hold and lift another robot. General 12 cm diameter 15 cm height 660 g 2 LiIon batteries 1 hour autonomy moving Control 400 MHz custom XScale CPU board, 64 MB of RAM, 32 MB of flash memory 12 distributed PIC microcontroller for low-level handling Custom Linux port running Familiar Wifi Image of the s-bot mobile robot climbing a step in the swarm-bot configuration. Actuators 2 treels turret rotation rigid gripper elevation rigid gripper 3 axis side arm side arm gripper Sensors 15 infrared sensors around the turret 4 infrared sensors below the robot position sensors on all degrees of freedom except gripper force and speed sensors on all major degrees of freedom 2 humiditiy sensors 2 temperature sensors 8 ambient light sensors around the turret 4 accelerometers, which allow three-dimensional orientation 1 640x480 camera sensor. Custom optic based on spherical mirror provides omnidirectional vision 4 microphones 2 axis structure deformation sensors optical barrier in grippers S-bot by night showing color ring. LEDS 8 x RGB Light-emitting diodes around the turret red Light-emitting diodes in grippers Several s-bots in swarm-bot configuration passing over a gap. Special abilities S-bots can connect to other s-bots to create a bigger structure known as the swarm-bot. To do so, they attach together using their rigid gripper and ring. An s-bot has sufficient force to lift another one. Integrated software The s-bot features a custom Linux port running the Familiar GNU/Linux distribution. All sensors and actuators are easily accessible through a simple C API. External links Swarm-bots project homepage - The project in which the s-bot was developed. Future and Emerging Technologies - The IST program in which the swarm-bots project takes place. Information Society Technologies - The European Union research activity in which the FET program takes place. LIS - The lab where the s-bot was developed. Swarmbots page at LIS - Pictures and video of the s-bot References Mondada, F., Guignard, A., Bonani, M., Bär, D., Lauria, M. and Floreano, D. (2003) SWARM-BOT: From Concept to Implementation. In Proceedings of the International Conference on Intelligent Robots and Systems 2003, IEEE Press. pp. 1626-1631. PDF BibTeX Mondada, F., Pettinaro, G. C., Guignard, A., Kwee, I., Floreano, D., Deneubourg, J.-L., Nolfi, S. and Gambardella, L.M., Dorigo, M. (2004) SWARM-BOT: a New Distributed Robotic Concept. Autonomous Robots, special Issue on Swarm Robotics, Volume 17, Issue 2-3, September - November 2004, Pages 193 - 221. PDF BibTeX Marco Dorigo, V. Trianni, E. Sahin, T. H. Labella, R. Gross, G. Baldassarre, S. Nolfi, J.-L. Deneubourg, F. Mondada, D. Floreano & L. M. Gambardella (2004). Evolving Self-Organizing Behaviors for a Swarm-bot. Autonomous Robots, 17 (2–3): 223–245. PDF BibTex This entry is from Wikipedia, the leading user-contributed encyclopedia. It may not have been reviewed by professional editors (see full disclaimer) Donate to Wikimedia www.answers.com/topic/s-bot-mobile-robot

Example 3: Distributed Control in Boid Model Each agent is described by a double integrator (Newton's second law of motion ): where xi, vi and ui represent the position, velocity and the control input of the agent i. Goal: 1) Avoid collision 2) Alignment 3) Cohension What information can be used to design the controller? The position and velocity of neighbors R. Olfati-Saber, IEEE Trans. Auto. Control ,2006.

Algorithm Controller design: Theorem 1: where A=[aij(q)] is the adjacency matrix, (·) is the action function, isσ-norm, and Neighbor graph Theorem 1: If the neighbor graphs are connected at each time instant. Then The group will form cohesion. All agents asymptotically move with the same velocity. No interagent collisions occur.

Three Categories of Research on Collective Behaviors Analysis Given the local rules of the agents, what is the collective behavior of the overall system ? (Bottom Up) Design Given the desired collective behavior, what are the local rules for agents ? (Top Down) Intervention Given the local rule of the agents, how we intervene the collective behavior? J.Han, M.Li, L.guo, JSSC,2006

Intervention Example 1: Can we guide the birds’ flight if we know how they fly ?

Example 2: Leadership by Numbers Couzin, et al., Nature, Vol. 433, 2005 The larger the group is, the smaller the leaders are needed.

Example 3: Cockroach J.Halloy, et al., Science, November 2007

III. Intervention Given the local rule of the agents, how we intervene the collective behavior? The current control theory can not be applied directly, because It is a many-body self-organized system. The purpose of control aims to collective behavior. Not allowed to change the local rules of the existing agents; Distributed Control: special task of formation, Pinning Control: Networked system, imposed controllers on selected nodes

Intervention Via Soft Control

Soft Control The multi-agent system: Many agents Each agent follows the local rules Autonomous, distributed Agents are connected, the local effect will affect the whole. From Jing Han’s PPT

Soft Control y(t) u(t) an associate of a person selling goods or services or a political group, who pretends no association to the seller/group and assumes the air of an enthusiastic customer. The “Control”: No global parameter to adjust Not to change the local rule of the existing agents; Put a few “shill” agents to guide (seduce) Shill: is controlled by us, not following the local rules, is treated as an ordinary agent by other ordinary agents The power of shill seems limited The ‘control’ is soft and seems weak From Jing Han’s PPT

Soft Control Key points: U(t) y(t) Different from distributed control approach. Intervention to the distributed system Not to change the local rule of the existing agents Add one (or a few) special agent – called “shill” based on the system state information, to intervene the collective behavior; The “ shill” is controlled by us, but is treated as an ordinary agent by all other agents. Shill is not leader, not leader-follower type. Feedback intervention by shill(s). This page is very important! From Jing Han’s PPT

There Are Lots of Questions … What is the purpose/task of control here? Synchronization/consensus Group connected / Dissolve a group Turning (Minimal Circling) Lead to a destination (in a shortest time) Avoid hitting an object Tracking … In what degree we can control the shill? (heading, position, speed, …) How much information the shill can observe ? (positions, headings, …) From Jing Han’s PPT

A Case Study Problem statement: Assumptions: System: A group of n agents with initial headings i(0)[0, ); Goal: all agents move to the direction of  eventually. Soft control: Design one shill agent based on the agents’ state information. Assumptions: The local rule about the ordinary agents is known The position x0(t) and heading 0(t) of the spy can be controlled at any time step t The state information (headings and positions) of all ordinary agents are observable at any time step From Jing Han’s PPT

Vicsek Model Neighbors: r Heading: Position: A bird’s Neighborhood Neighbors: Heading: Alignment: steer towards the average heading of neighbors Position: Synchronization: There exists θ, such that http://angel.elte.hu/~vicsek/

A Case Study Problem statement: Assumptions: System: A group of n agents with initial headings i(0)[0, ); Goal: all agents move to the direction of  eventually. Soft control: Design one shill agent based on the agents’ state information. Assumptions: The local rule about the ordinary agents is known The position x0(t) and heading 0(t) of the shill can be controlled at any time step t The state information (headings and positions) of all ordinary agents are observable at any time step From Jing Han’s PPT

Control the Shill agent The Control Law u From Jing Han’s PPT

Control the Shill agent Theorem 4: For any initial headings and positions  i(0)[0, ), xi(0)R2, 1 i  n, the update rule and the control law uβ will lead to the asymptotic synchronization of the group. It is possible to control the collective behavior of a group of agents by a shill. J.Han, M.Li, L.guo, JSSC,2006

Simulation

An Alternative Control Law otherwise where Result: The control law ut will also lead to asymptotic synchronization of the group.

Switching between u and ur Simulations Switching between u and ur Control Law u

Remarks on Soft Control It is not just for the above model Can be applied to other MAS ,e.g., Panic in Crowd Evolution of Language Multi-player Game …… “Add the special agent(s)” is just one way Should be other ways for different systems: Remove agents Put obstacle … … We need a theory for Soft Control ! From Jing Han’s PPT

Leader-Follower Model (LFM) Intervention Via Leader-Follower Model (LFM)

Example 1: Leadership by Numbers Couzin, et al., Nature, Vol. 433, 2005 The larger the group is, the smaller the leaders are needed.

Leader-Follower Model Problem statement: System: A group of n agents; Goal: All agents move with the expected direction eventually. Intervention by leaders: Add some information agents-called “leaders”, which move with the expected direction.

Leader-Follower Model Ordinary agents Information agents Key points: Not to change the local rule of the existing agents. Add some (usually not very few) “information” agents – called “leaders”, to control or intervene the MAS; But the existing agents treated them as ordinary agents. The proportion of the leaders is controlled by us (If the number of leaders is small, then connectivity may not be guaranteed). Open-loop intervention by leaders.

Mathematical Model Ordinary agents (labeled by 1,2,…,n): Neighbors: Position: Heading: Heading: Position: Leader agents (labeled by ):

Simulation Example N=1000

Q: How many leaders are required for consensus/synchronization?

Assumption on the initial states Random Framework Assumption on the initial states 1) The initial positions of all agents are independently and uniformly distributed in the unit square. 2) The initial headings of the agents are uniformly and independently distributed in [-π, π), and the initial headings of the leaders are . The headings and the positions are mutually independent.

Some Notations Degree matrix: If i ~ j Adjacency matrix: Degree: Otherwise Weighted adjacency matrix: Weighted degree matrix: Leader degree matrix: Average matrix: Weighted average matrix:

Some Notations (cont.) Laplacian : L(0)=D(0) – A(0) “Normalized Laplacian” : Spectrum : “Spectral gap”: where

Key Steps in the Analysis of the LFM Analysis of the system dynamics Estimation of the rate of consensus Dealing with the matrices with increasing dimension Dealing with the inherent nonlinearity

Analysis of the System Dynamics Evolution of the distance Lemma 1: For any two agents i and j, their distance satisfy the following inequality: where is important for the evolution of the distance!

Analysis of the System Dynamics Evolution of the headings Step 1: Projection

Analysis of the System Dynamics Step 2: Analyze the stability of where Step 3: Dealing with the changing neighbor graphs

Estimation of Consensus Rate The consensus rate depends on 1) A key lemma: For any vector f=[f1,f2,…,fn]τ, we have 2)

Dealing with the Matrices with Increasing Dimension Estimation of multi-array martingales where Moreover, if then we have

Dealing with the Matrices with Increasing Dimension Using the above corollary, we have for large n where

The Degree of The Initial Graph Lemma: For initial graph G0, we have for large n

The Degree of The Initial Graph Corollary:

Dealing with the Inherent Nonlinearity Proposition 1 For any positive v and r, we have for large n where

Main Result Theorem 5 Let the velocity v > 0 and radius r > 0 be positive constants. If the proportion of the leaders satisfies where C is a constant depending on v and r, then the headings of all agents will converge to almost surely when the population size n is large enough.

Concluding Remarks In this talk, we talked about intervention to the multi-agent systems: Soft control Design the control law of the “shill” Leader-follower model Control the number of the leaders

Concluding Remarks These two lectures mainly focus on the collective behavior of the MAS. In the next lecture, we will talk about game theory.

Thank you!