1. Lecture VI: Collective Behavior of Multi-Agent Systems II: Intervention
Zhixin Liu
Complex Systems Research Center,
Academy of Mathematics and Systems Sciences, CAS

2. In the last lecture, we talked about
Collective Behavior of Multi-Agent Systems I: Analysis

3. In the last lecture, we talked about
Introduction
Model: Vicsek model

4. Multi-Agent System (MAS)
Autonomy: capable of autonomous action
Interactions: capable of interacting with other agents
MAS
Many agents
Local interactions between agents
Collective behavior in the population level
More is different.---Philp Anderson, 1972
e.g., small-world, swarm intelligence, panic, phase transition, coordination, synchronization, consensus, clustering, aggregation, ……
Examples:
Physical systems
Biological systems
Social and economic systems
Engineering systems
… …

5. Vicsek Model (T. Vicsek et al. , PRL, 1995) r
A bird’s neighborhood
Alignment: steer towards the average heading of neighbors
xi(t) : position of agent i in the plane
v: the constant speed of birds
r: radius of neighborhood
: heading of agent i

6. Vicsek Model Neighbors: r Position: Heading:
A bird’s neighborhood
Neighbors:
Position:
Alignment: steer towards the average heading of neighbors
Heading:
Synchronization: There exists θ, such that

7. In the last lecture, we talked about
Introduction
Model
Theoretical analysis
Concluding remarks

8. The Linearized Vicsek Model
A. Jadbabaie , J. Lin, and S. Morse, IEEE Trans. Auto. Control, 2003.

9. Synchronization of the linearized Vicsek model
Theorem 2 (Jadbabaie et al. , 2003)
Joint connectivity of the neighbor graphs on each time interval [th, (t+1)h] with h >0
Synchronization of the linearized Vicsek model
Related result: J.N.Tsitsiklis, et al., IEEE TAC, 1984

10. Random Framework Random initial states:
1) The initial positions of all agents are uniformly and independently distributed in the unit square;
2) The initial headings of all agents are uniformly and independently distributed in [-+ε, -ε] with ε∈ (0, ). The initial headings and positions are independent.

11. Theorem 7 High Density Implies Synchronization
For any given system parameters
and when the number of agnets n
is large, the Vicsek model will synchronize almost surely.
This theorem is consistent with the simulation result.

12. Theorem 8
High density with short distance interaction
Let and the velocity
satisfy
Then for large population, the MAS will synchronize almost surely.

13. Three Categories of Research on Collective Behavior

14. Three Categories of Research on Collective Behavior
Analysis
Given the local rules of the agents, what is the collective behavior of the overall system ? (Bottom Up)
Design
Given the desired collective behavior, what are the local rules for agents ? (Top Down)
Intervention
Given the local rule of the agents, how we intervene the collective behavior?
J.Han, M.Li, L.guo, JSSC,2006

15. Example 1: Synchronization
A bird’s Neighborhood
Simulation Result
Alignment: steer towards the average heading of neighbors
Q: Under what conditions such a system can reach consensus?

16. Example 2: Escape Panic D. Helbing, et al., Nature, Vol. 407, 2000
Normal, no panic
Fire, panic

17. Three Categories of Research on Collective Behaviors
Analysis
Given the local rules of the agents, what is the collective behavior of the overall system ? (Bottom Up)
Design
Given the desired collective behavior, what are the local rules for agents ? (Top Down)
Intervention
Given the local rule of the agents, how we intervene the collective behavior?
J.Han, M.Li, L.guo, JSSC,2006

18. Example 1: Formation control
How we design the control law of each plane to maintain the form ?

19. Example 2: Swarm Intelligence (Marco Dorigo et al., 2001-2004)
Using their gripper (red in the photos), they can connect together. Then they can, for instance, pass over gap and steps where a single robot would have failed.
Using their integrated force sensor, they can coordinate to retrieve an object to a certain location without the use of explicit communication. This is the way ants bring preys to the nest.
S-bots can connect to other s-bots to create a bigger structure known as the swarm-bot.
The project, that lasted 42 months, was successfully completed on March 31, It has been selected as one of the success stories of the Future and Emerging Technologies (FET) program of the European Commission.
S-bot mobile robot.
The s-bot is a small (15 cm) mobile robot developed at the LIS (Laboratory of Intelligent Systems [1]) at the EPFL in Lausanne, Switzerland between 2001 and Targeted to swarm robotics, a field of artificial intelligence, it was developed within the Swarm-bots project, a Future and Emerging Technologies project coordinated by Prof. Marco Dorigo. Built by a small team of engineers (Francesco Mondada, André Guignard, Michael Bonani and Wikipedia editor Stéphane Magnenat (nct)) of the group of Prof. Dario Floreano and with the help of student projects, it is considered at the time of completion as one of the most complex and featured robots ever for its size.
Purpose and use of the s-bot
This is a research robot, aimed at studying team work and inter-robot communication. To do this, the s-bots have several special abilities:
Of course, all other sensors and actuators, also found on other robots, can be used to do team work such as food foraging.
Technical details
Closeup of s-bot fixed gripper, used to hold and lift another robot.
General
12 cm diameter
15 cm height
660 g
2 LiIon batteries
1 hour autonomy moving
Control
400 MHz custom XScale CPU board, 64 MB of RAM, 32 MB of flash memory
12 distributed PIC microcontroller for low-level handling
Custom Linux port running Familiar
Wifi
Image of the s-bot mobile robot climbing a step in the swarm-bot configuration.
Actuators
2 treels
turret rotation
rigid gripper elevation
rigid gripper
3 axis side arm
side arm gripper
Sensors
15 infrared sensors around the turret
4 infrared sensors below the robot
position sensors on all degrees of freedom except gripper
force and speed sensors on all major degrees of freedom
2 humiditiy sensors
2 temperature sensors
8 ambient light sensors around the turret
4 accelerometers, which allow three-dimensional orientation
1 640x480 camera sensor. Custom optic based on spherical mirror provides omnidirectional vision
4 microphones
2 axis structure deformation sensors
optical barrier in grippers
S-bot by night showing color ring.
LEDS
8 x RGB Light-emitting diodes around the turret
red Light-emitting diodes in grippers
Several s-bots in swarm-bot configuration passing over a gap.
Special abilities
S-bots can connect to other s-bots to create a bigger structure known as the swarm-bot. To do so, they attach together using their rigid gripper and ring. An s-bot has sufficient force to lift another one.
Integrated software
The s-bot features a custom Linux port running the Familiar GNU/Linux distribution. All sensors and actuators are easily accessible through a simple C API.
External links
Swarm-bots project homepage - The project in which the s-bot was developed.
Future and Emerging Technologies - The IST program in which the swarm-bots project takes place.
Information Society Technologies - The European Union research activity in which the FET program takes place.
LIS - The lab where the s-bot was developed.
Swarmbots page at LIS - Pictures and video of the s-bot
References
Mondada, F., Guignard, A., Bonani, M., Bär, D., Lauria, M. and Floreano, D. (2003) SWARM-BOT: From Concept to Implementation. In Proceedings of the International Conference on Intelligent Robots and Systems 2003, IEEE Press. pp PDF BibTeX
Mondada, F., Pettinaro, G. C., Guignard, A., Kwee, I., Floreano, D., Deneubourg, J.-L., Nolfi, S. and Gambardella, L.M., Dorigo, M. (2004) SWARM-BOT: a New Distributed Robotic Concept. Autonomous Robots, special Issue on Swarm Robotics, Volume 17, Issue 2-3, September - November 2004, Pages PDF BibTeX
Marco Dorigo, V. Trianni, E. Sahin, T. H. Labella, R. Gross, G. Baldassarre, S. Nolfi, J.-L. Deneubourg, F. Mondada, D. Floreano & L. M. Gambardella (2004). Evolving Self-Organizing Behaviors for a Swarm-bot. Autonomous Robots, 17 (2–3): 223–245. PDF BibTex
This entry is from Wikipedia, the leading user-contributed encyclopedia. It may not have been reviewed by professional editors (see full disclaimer)
Donate to Wikimedia

20. Example 3: Distributed Control in Boid Model
Each agent is described by a double integrator (Newton's second law of motion ):
where xi, vi and ui represent the position, velocity and the control input of the agent i.
Goal: 1) Avoid collision
2) Alignment
3) Cohension
What information can be used to design the controller?
The position and velocity of neighbors
R. Olfati-Saber, IEEE Trans. Auto. Control ,2006.

21. Algorithm Controller design: Theorem 1:
where A=[aij(q)] is the adjacency matrix,
(·) is the action function,
isσ-norm, and
Neighbor graph
Theorem 1:
If the neighbor graphs are connected at each time instant. Then
The group will form cohesion.
All agents asymptotically move with the same velocity.
No interagent collisions occur.

22. Three Categories of Research on Collective Behaviors
Analysis
Given the local rules of the agents, what is the collective behavior of the overall system ? (Bottom Up)
Design
Given the desired collective behavior, what are the local rules for agents ? (Top Down)
Intervention
Given the local rule of the agents, how we intervene the collective behavior?
J.Han, M.Li, L.guo, JSSC,2006

23. Intervention
Example 1: Can we guide the birds’ flight if we know how they fly ?

24. Example 2: Leadership by Numbers
Couzin, et al., Nature, Vol. 433, 2005
The larger the group is, the smaller the leaders are needed.

25. Example 3: Cockroach J.Halloy, et al., Science, November 2007

26. III. Intervention Given the local rule of the agents, how we intervene the collective behavior?
The current control theory can not be applied directly, because
It is a many-body self-organized system.
The purpose of control aims to collective behavior.
Not allowed to change the local rules of the existing agents;
Distributed Control: special task of formation,
Pinning Control: Networked system, imposed controllers on selected nodes

27. Intervention Via
Soft Control

28. Soft Control The multi-agent system: Many agents
Each agent follows the local rules Autonomous, distributed
Agents are connected, the local effect will affect the whole.
From Jing Han’s PPT

29. Soft Control
y(t)
u(t)
an associate of a person selling goods or services or a political group, who pretends no association to the seller/group and assumes the air of an enthusiastic customer.
The “Control”:
No global parameter to adjust
Not to change the local rule of the existing agents;
Put a few “shill” agents to guide (seduce)
Shill: is controlled by us, not following the local rules,
is treated as an ordinary agent by other ordinary agents
The power of shill seems limited The ‘control’ is soft and seems weak
From Jing Han’s PPT

30. Soft Control Key points: U(t) y(t)
Different from distributed control approach. Intervention to the distributed system
Not to change the local rule of the existing agents
Add one (or a few) special agent – called “shill” based on the system state information, to intervene the collective behavior;
The “ shill” is controlled by us, but is treated as an ordinary agent by all other agents.
Shill is not leader, not leader-follower type.
Feedback intervention by shill(s).
This page is very important!
From Jing Han’s PPT

31. There Are Lots of Questions …
What is the purpose/task of control here?
Synchronization/consensus
Group connected / Dissolve a group
Turning (Minimal Circling)
Lead to a destination (in a shortest time)
Avoid hitting an object
Tracking …
In what degree we can control the shill? (heading, position, speed, …)
How much information the shill can observe ? (positions, headings, …)
From Jing Han’s PPT

32. A Case Study Problem statement: Assumptions:
System: A group of n agents with initial headings i(0)[0, );
Goal: all agents move to the direction of  eventually.
Soft control:
Design one shill agent based on the agents’ state information.
Assumptions:
The local rule about the ordinary agents is known
The position x0(t) and heading 0(t) of the spy can be controlled at any time step t
The state information (headings and positions) of all ordinary agents are observable at any time step
From Jing Han’s PPT

33. Vicsek Model Neighbors: r Heading: Position:
A bird’s Neighborhood
Neighbors:
Heading:
Alignment: steer towards the average heading of neighbors
Position:
Synchronization: There exists θ, such that

34. A Case Study Problem statement: Assumptions:
System: A group of n agents with initial headings i(0)[0, );
Goal: all agents move to the direction of  eventually.
Soft control:
Design one shill agent based on the agents’ state information.
Assumptions:
The local rule about the ordinary agents is known
The position x0(t) and heading 0(t) of the shill can be controlled at any time step t
The state information (headings and positions) of all ordinary agents are observable at any time step
From Jing Han’s PPT

35. Control the Shill agent
The Control Law u
From Jing Han’s PPT

36. Control the Shill agent
Theorem 4:
For any initial headings and positions  i(0)[0, ), xi(0)R2, 1 i  n, the update rule and the control law uβ will lead to the asymptotic synchronization of the group.
It is possible to control the collective behavior of a group of agents by a shill.
J.Han, M.Li, L.guo, JSSC,2006

37. Simulation

38. An Alternative Control Law
otherwise
where
Result: The control law ut will also lead to asymptotic synchronization of the group.

39. Switching between u and ur
Simulations
Switching between u and ur
Control Law u

40. Remarks on Soft Control
It is not just for the above model
Can be applied to other MAS ,e.g.,
Panic in Crowd
Evolution of Language
Multi-player Game ……
“Add the special agent(s)” is just one way Should be other ways for different systems:
Remove agents
Put obstacle
… …
We need a theory for Soft Control !
From Jing Han’s PPT

41. Leader-Follower Model (LFM)
Intervention Via
Leader-Follower Model (LFM)

42. Example 1: Leadership by Numbers
Couzin, et al., Nature, Vol. 433, 2005
The larger the group is, the smaller the leaders are needed.

43. Leader-Follower Model
Problem statement:
System:
A group of n agents;
Goal:
All agents move with the expected direction eventually.
Intervention by leaders:
Add some information agents-called “leaders”, which move with the expected direction.

44. Leader-Follower Model
Ordinary agents
Information agents
Key points:
Not to change the local rule of the existing agents.
Add some (usually not very few) “information” agents – called “leaders”, to control or intervene the MAS; But the existing agents treated them as ordinary agents.
The proportion of the leaders is controlled by us (If the number of leaders is small, then connectivity may not be guaranteed).
Open-loop intervention by leaders.

45. Mathematical Model Ordinary agents (labeled by 1,2,…,n):
Neighbors:
Position:
Heading:
Heading:
Position:
Leader agents (labeled by ):

46. Simulation Example
N=1000

47. Q: How many leaders are required for consensus/synchronization?

48. Assumption on the initial states
Random Framework
Assumption on the initial states
1) The initial positions of all agents are independently and uniformly distributed in the unit square.
2) The initial headings of the agents are uniformly and independently distributed in [-π, π), and the initial headings of the leaders are . The headings and the positions are mutually independent.

49. Some Notations Degree matrix: If i ~ j Adjacency matrix: Degree:
Otherwise
Weighted adjacency matrix:
Weighted degree matrix:
Leader degree matrix:
Average matrix:
Weighted average matrix:

50. Some Notations (cont.) Laplacian : L(0)=D(0) – A(0)
“Normalized Laplacian” :
Spectrum :
“Spectral gap”:
where

51. Key Steps in the Analysis of the LFM
Analysis of the system dynamics
Estimation of the rate of consensus
Dealing with the matrices with increasing dimension
Dealing with the inherent nonlinearity

52. Analysis of the System Dynamics
Evolution of the distance
Lemma 1: For any two agents i and j, their distance satisfy the following inequality:
where
is important for the evolution of the distance!

53. Analysis of the System Dynamics
Evolution of the headings
Step 1: Projection

54. Analysis of the System Dynamics
Step 2: Analyze the stability of
where
Step 3: Dealing with the changing neighbor graphs

55. Estimation of Consensus Rate
The consensus rate depends on
1) A key lemma: For any vector f=[f1,f2,…,fn]τ, we have 2)

56. Dealing with the Matrices with Increasing Dimension
Estimation of multi-array martingales
where
Moreover, if
then we have

57. Dealing with the Matrices with Increasing Dimension
Using the above corollary, we have for large n
where

58. The Degree of The Initial Graph
Lemma: For initial graph G0, we have for large n

59. The Degree of The Initial Graph
Corollary:

60. Dealing with the Inherent Nonlinearity
Proposition 1
For any positive v and r, we have for large n
where

61. Main Result
Theorem 5
Let the velocity v > 0 and radius r > 0 be positive constants. If the proportion of the leaders satisfies
where C is a constant depending on v and r, then the headings of all agents will converge to almost surely when the population size n is large enough.

62. Concluding Remarks
In this talk, we talked about intervention to the multi-agent systems:
Soft control
Design the control law of the “shill”
Leader-follower model
Control the number of the leaders

63. Concluding Remarks
These two lectures mainly focus on the collective behavior of the MAS.
In the next lecture, we will talk about game theory.

64. Thank you!