1. UCLA March 2, 2006 IPAM Workshop on Swarming by Nature and by Design thanks to the organizers: A. Bertozzi D. Grunbaum P. S. Krishnaprasad I. Schwartz

2. M. Cao & B. D. O. Anderson UCLA March 2, 2006 Reaching a Consensus in a Dynamically Changing Environment A. S. Morse Yale University IPAM Workshop on Swarming by Nature and by Design

4. CRAIG REYNOLDS - 1987 leader - follower queue

5. neighborhood separationcohesion alignment CRAIG REYNOLDS - 1987

6. Motivated by simulation results reported in the paper:

7. Each agent’s heading is updated at the same time as the rest using a local rule based on the average of its own current heading plus the headings of its “neighbors.”  i = heading ii s = speed s Vicsek et al. simulated a flock of n agents {particles} all moving in the plane at the same speed s, but with different headings  1,  2, ….  n Vicsek’s simulations demonstrated that these nearest neighbor rules can cause all agents to eventually move in the same direction despite the absence of a leader and/or centralized coordination and despite the fact that each agent’s set of neighbors changes with time. A theoretical explanation for this observed behavior can be found in Jadbabaie, Lin & Morse, IEEE TAC, June 2003

8. L. Moreau W. Ren and R. Beard D. Angeli and P. A. Bliman V. D. Blondell, J. M. Hendrichx, A. Olshevsky, and J. N. Tsitsiklis

9. riri agent i neighbors of agent i each agent is a neighbor of itself

10.  i = heading ii s = speed s N i (t) = set of indices of agents i’s “neighbors” at time t n i (t) = number of indices in N i (t) HEADING UPDATE EQUATIONS Average at time t of headings of neighbors of agent i.

11. N (t) = { V, A (t)} - a directed graph with vertex set V and arc set A (t) V = agent index set ={1, 2, …, n} Neighbor Graphs 7 4 1 3 5 2 6 (1,2) All vertices have self-arcs G sa = set of all directed graphs with vertex set V and self-arcs at all vertices (j, i) 2 A (t) if j is a neighbor of i in N i (t) j i

12. adjacency matrix A =[a ij ] n £ n a ij = 1 if i is a neighbor of j a ij = 0 otherwise d i = in-degree of vertex i Matrix Representation of N = { V, A } 2 G sa 7 4 1 3 5 2 6 (1,2) in-degree = 1 in-degree = 2 D = diagonal {d 1, d 2, …., d n } n £ n

13. 7 4 1 3 5 2 6 (1,2) State Space Equation adjacency matrix A =[a ij ] n £ n a ij = 1 if i is a neighbor of j a ij = 0 otherwise D = diagonal {d 1, d 2, …., d n } n £ n D(t) and A(t) are respectively the in-degree matrix and adjacency matrix of N (t).

15. A switched linear system 1. non-negative entries 2. row sums all = 1 stochastic F1 = 1 1 Induced infinity norm of n £ n nonnegative matrix Q, written ||Q||, is largest of Q’s row sums ||F|| = 1

17. and so where For if this is so, then Problem reduces to determining conditions under which where

18. All n £ n flocking matrices F are in S S is closed under multiplication. Therefore its sufficient to determine conditions on sequence of matrices S 1, S 2,.... in S for S = set of all n £ n stochastic matrices with positive diagonal elements. Problem reduces to determining conditions under which

19. S = set of all n £ n stochastic matrices. Therefore its sufficient to determine conditions on sequence of matrices S 1, S 2,.... in S for

20. S = set of all n £ n stochastic matrices. Therefore its sufficient to determine conditions on sequence of matrices S 1, S 2,.... in S for This is a well studied problem in the theory of non-homogeneous Markov Chains

21. One can also write S = 1 d S e - w S x where d S e is the smallest row vector for c which 1c – S is non-negative and w S x = 1 d S e – S. It can be shown for any pair of n £ n stochastic matrices S 1 and S 2 that [S2S1] · (S2)[S1][S2S1] · (S2)[S1] S = 1 b S c + u S v where b S c = the largest row vector c 1 £ n for which S - 1c is non-negative and S u S v = S - 1 b S c For any S 2 S one can write [S] = u S v + w S x Let Then uSv · [S]uSv · [S] scrambling constant

22. It can be shown for any pair of n £ n stochastic matrices S 1 and S 2 that uSv · [S]uSv · [S] [S2S1] · (S2)[S1][S2S1] · (S2)[S1] S = 1 b S c + u S v where b S c = the largest row vector c 1 £ n for which S - 1c is non-negative and S u S v = S - 1 b S c For any S 2 S one can write  S j  S 2 S 1 = 1 b  S j  S 2 S 1 c + u  S j  S 2 S 1 v Thus for any sequence of matrices S 1,S 2,.... in S [S j  S 2 S 1 ] ·  (S j )  (S j - 1 )   (S 2 )[S 1 ] u S j  S 2 S 1 v ·  (S j )  (S j - 1 )   (S 2 )[S 1 ] || u S j  S 2 S 1 v || ·  (S j )  (S j - 1 )   (S 2 )||[S 1 ]||

23.  S j  S 2 S 1 = 1 b  S j  S 2 S 1 c + u  S j  S 2 S 1 v Thus for any sequence of matrices S 1,S 2,.... in S if and as fast as the product  (S j )  (S j - 1 )   (S 2 ) converges to 0. Therefore the product  S j  S 1 converges to 1c where || u S j  S 2 S 1 v || ·  (S j )  (S j - 1 )   (S 2 )||[S 1 ]||

24. RE-CAP The state  of converges to  SS 1 as t ! 1 as fast as the product  F(t))  (F(t -1))   (F(0)) converges to 0 where for any n £ n flocking matrix F = [f ij ],  (F) is the scrambling constant

25. For S 2 S, graph(S) is that graph whose adjacency matrix is the transpose of the matrix which results when all non-zero entries in S are replaced by 1s. In general, graph(F) = N, N 2 G sa Transitioning from Matrices to Graphs Convergence {and worst case convergence rates} can be decided completely in terms of graphs without any reference at all to stochastic matrices, linearity or the like. This is especially important because it allows one to consider flocking {consensus} problems which are nonlinear – e.g., the work of Tanner, Jadbabaie & Pappas which explains Reynold’s academy award winning animation algorithms.

26. 2 1 4 3 2 and 4 share 4 1 and 2 share 2 1 and 3 share 2 1 and 4 share 4 3 and 4 share 1 2 and 3 share 2 Neighbor-Shared Directed Graph A neighbor shared graph is a directed graph in which each pair of distinct vertices share a common neighbor

27. Worst Case Convergence Rate for Products of Flocking Matrices Since all d i · n, all non-zero f ij satisfy d i = in-degree of vertex i D = diagonal {d 1, d 2, …, d n } n £ n a ij = 1 if i is a neighbor of j a ij = 0 otherwise A = [a ij ] graph(F) = N F = D ¡ 1 A 0

28. Worst Case Convergence Rate for Products of Flocking Matrices Since all d i · n, all non-zero f ij satisfy d i = in-degree of vertex i D = diagonal {d 1, d 2, …, d n } n £ n a ij = 1 if i is a neighbor of j a ij = 0 otherwise A = [a ij ] graph(F) = N F = D ¡ 1 A 0 Fix distinct i and j and let k be a shared neighbor. Then f ik  0  f jk. Suppose N neighbor shared

29. Vertex 1 has only itself as a neighbor Vertex 2 has every vertex as a neighbor n ¸ 3 For i > 2, vertex i has only itself and vertex 1 as neighbors 2 1 4 3 Worst Case Convergence Rate for Products of Flocking Matrices graph(F) = N F = D ¡ 1 A 0 Suppose N neighbor shared How tight is this bound?

30. There exist infinite product of n £ n flocking matrices with neighbor-shared graphs which actually converge to a rank-one matrix product 1c at this rate. Neighbor-Shared Graphs Summary Every infinite product of n £ n flocking matrices with neighbor-shared graphs converges to a rank-one matrix product 1c at a rate no slower than

31. Rooted Graph A root of a directed graph is a vertex i for which, for each other vertex j in the graph, there is a directed path from i to j. 2 1 4 3 A rooted graph is a directed graph with at least one root. Any neighbor-shared graph is rooted. By the composition of a graph G 1 with a graph G 2, written G 2 ± G 1, is meant that graph which has an arc from i to j just in case there is a vertex k for which G 1 has an arc from i to k and G 2 has an arc from k to j. graph(S 2 S 1 ) = graph(S 2 ) ± graph(S 1 ) The composition of n – 1 or more rooted graphs in G sa is neighbor shared.

32. Need Some Facts About Stochastic Matrices  (S) = smallest nonzero entry in S  S 2 S 1 ) ¸  (S 2 )  (S 1 ) If graph(S) = neighbor - shared  S) · 1 -  (S) If S 1, S 2, …. S m are stochastic matrices for which  (S i ) ¸ b, i 2 {1, 2, …, m}, then  (S m S m-1  S 1 ) ¸ b m. The composition of n – 1 or more rooted graphs in G sa is neighbor shared. If m ¸ n – 1 and each S i has a rooted graph, then the graph of S m S m-1  S 1 is neighbor - shared. ¹ ( S m S m ¡ 1 ¢¢¢ S 1 ) · 1 ¡ b m Thus if m ¸ n – 1 and each S i is rooted and satisfies  (S i ) ¸ b, then

33. ¹ ( S m S m ¡ 1 ¢¢¢ S 1 ) · 1 ¡ b m n 1 ¡ ¡ 1 n ¢ n ¡ 1 o 1 n ¡ 1 This implies that an infinite product of n £ n flocking matrices whose graphs are all rooted must converge at a rate no slower than Thus if F 1, F 2, …, F m are m = n – 1 flocking matrices with rooted graphs N 1, N 2, …, N m respectively, then N m ± N m-1 ±  ± N 1 is neighbor – shared and ¹  F m F m ¡  ¢¢¢ F   ·  ¡ ³  n ´ n ¡            F                 Á  F  ¸  n

34. While there no not exist infinite products of n £ n flocking matrices with rooted graphs which actually converge at this rate, there do exist infinite products of n £ n stochastic matrices with rooted graphs which do. Summary Rooted Graphs Every infinite product of n £ n flocking matrices with rooted graphs converges to a rank-one matrix product 1c at a rate no slower than n 1 ¡ ¡ 1 n ¢ n ¡ 1 o 1 n ¡ 1

35. While there no not exist infinite products of n £ n flocking matrices with strongly connected graphs which actually converge at this rate, there do exist infinite products of n £ n stochastic matrices with strongly connected graphs which do. Strongly Connected Graphs Main Results A strongly connected graph is a graph for which for each distinct pair of vertices i and j, there is a directed path from i to j. Every infinite product of n £ n flocking matrices with strongly connected graphs converges to a rank-one matrix product 1c at a rate no slower than © 1 ¡ ¡ 1 n ¢ m ª 1 m where m is the unique integer quotient of n divided by 2. A strongly connected graph is a rooted graph in which every vertex is a root.

36. Agreeing Asynchronously

37. Each agent’s heading is updated at the same time as the rest using a local rule based on the average of its own current heading plus the headings of its “neighbors.”  i = heading ii s = speed s Vicsek et al. simulated a flock of n agents {particles} all moving in the plane at the same speed s, but with different headings  1,  2, ….  n at the same time as the rest What happens if each agent updates independent of the rest according to its own clock? How might one go about analyzing such an asynchronous system?

38. Update Rule for Agent i’s Heading  i Agent i’s event times = 0, t i1, t i2, t i3, … assumed to satisfy At t ik agent i computes its ith way-point and updates its heading on (t ik, t i(k+1 )] Event times not nec. evenly spaced or synchronized with other agents’ event times. labels of agent i’s neighbors at t ik number of labels in N i (t ik )  i (t ik )  i (t i(k+1) ) = w i (t ik )

40. ii ii 0 1 ti1ti1 ti2ti2 ti3ti3 ti4ti4 ti5ti5 waypoint

41. Can combine agent i’s two update equations to get the familiar update equation This formula tells how  i evolves only on agent i’s event time set. But to use this formula we need to know values of the  j at agent i’s event times In the synchronous case where event times are the same for all agents, the t ik are independent of i, and the preceding update equations are sufficient. For the asynchronous case a common time scale is needed …..

42. A Common Time Scale T = set of all event times t ik of all n agents Re-label the elements of T as t 0, t 1, t 2, … where t 0 = 0 and t  < t  +1 for  2 {0, 1, 2, …}

43. t 11 t 12 t 15 t 16 t 13 t 14 t 26 t 25 t 24 t 21 t 27 t 23 t 22 agent 2 agent 1 interacting

44. agent 2 agent 1 t 11 t 12 t 15 t 16 t 13 t 14 t 26 t 25 t 24 t 21 t 27 t 23 t 22 t3t3 t5t5 t 10 t 13 t7t7 t8t8 t 11 t9t9 t6t6 t1t1 t 12 t4t4 t2t2 T =

45. A Common Time Scale T = set of all event times t ik of all n agents Relabel the elements of T as t 0, t 1, t 2, … where t 0 = 0 and t  < t  +1 for  2 {0, 1, 2, …} T i = set of t  2 T for which t  is an event time of agent i Define Agent i’s Neighbor Between its Event times

46. N (  ) = { V, A (  )} - a directed graph with vertex set V and arc set A (  ) V = agent index set ={1, 2, …, n} Neighbor Graphs 7 4 1 3 5 2 6 (1,2) All vertices have self-arcs (j, i) 2 A (  ) if j is a neighbor of i in

47. A Few Thing About Directed Graph A directed graph is rooted if it is rooted at least one vertex. A directed graph G is strongly rooted at vertex i if for each other vertex j in the graph, there is a directed arc from i to j. A directed graph is strongly rooted if it is strongly rooted at least one vertex. 3 roots strongly rooted at A directed graph G is rooted at vertex i if for each other vertex j in the graph, there is a directed path from i to j.

48. Compositions of Directed Graph A finite sequence of graphs G 1, G 2, …., G m is jointly rooted if G m  ±  ±  G 1 is a rooted graph. An infinite sequence of graphs G 1, G 2, …. is repeatedly jointly rooted if there is a finite integer m such that each sequence G 1+mk, G 2 +mk, …, G (m+1)k, k ¸ 0, is jointly rooted. Thus the set of all directed graphs on n vertices with self arcs at each vertex, namely G sa, is closed under composition. Two more definitions… It is easy to show that every arc in either of two given graphs G 1 and G 2 is in their composition provided G 1 and G 2 have self-arcs at all vertices.

49. In the synchronous case event times t ik are independent of i so setting t k = t ik gives The Update Equations Again Combining the two gives

50. along which the sequence of neighbor graphs N (0), N (1), …. is repeatedly jointly rooted, there is a constant  ss to which each agent’s heading  i converges exponentially fast. For any trajectory of the synchronous system along which the sequence of neighbor graphs N (0), N (1), …. is repeatedly jointly rooted, there is a constant  ss to which each agent’s heading  i converges exponentially fast. For any trajectory of the asynchronous system i 2 {1, 2, …,n} Asynchronous Case: Synchronous Case: CONVERGENCE

51. The n mutually unsynchronized processes below, P 1, P 2, … P n together constitute the asynchronous system to be analyzed via “analytic synchronization.” Merge all event time sequences into a single ordered sequence T. Analytic Synchronization At times in T between two successive event times in T i, define the state of P i to be constant at the same value as at the first of these two event times. Define the “synchronized state” of P i at event times t  2 to be the original unsynchronized state of P i at these times plus possibly some additional variables. Analyze the synchronous system S comprised of the n synchronized P i Define the neighbor of each agent to be itself between its event times i 2 {1, 2, …,n}

52. Synchronizing P i For each event time t q 2 T i, define where t q 0 is the first event time in T i after t q Can show that these variable evolve on all of T as where for t  2 T i

54. Defining the Synchronous System S Comprised of the n Synchronized P i stochastic matrix

55. R = set of all lists of n real numbers r = {r 1, r 2, …., r n } where r i 2 [0, 1] B = set of all lists of n integers b = {b 1, b 2, …., b n } where b i 2 {0, 1} S(  ) =  ( N (  ),  (  ), b(  ))  : G sa £ R £ B ! set of all 2n £ 2n stochastic matrices where It is possible to construct a function, which is continuous on R, such that Asynchronous Flocking Matrices Note that the set of all asynchronous flocking matrices, namely image of  is compact because R is closed.

56. To prove that all  i converge to a common heading  ss can be shown to be equivalent to proving that call 2n entries x i in x converge to  ss. Thus as before the problem reduces to determining conditions under which

57. Basic Convergence Result But the product of m stochastic matrices will have a strongly rooted graph iff the composition of the graphs of the m matrices in the product is rooted. So when is the composition of m graphs rooted? This result can be applied to the consensus problem under consideration if we can find an integer m for which the graphs of the matrices are all strongly rooted. This is because Any product of stochastic matrices from a closed set converges exponentially fast to a matrix of the form 1c provided all matrices in the set have strongly rooted graphs.

58. 1 3 5 4 6 8 7 2 The graph of a typical asynchronous flocking matrix for a 4 agent system might look like this: This graph would be encountered along a trajectory at a time  coinciding with event times of agents 1 and 3, but not agents 2 or 4. Note that vertices 1 and 3 do not have self-arcs. Let G 1, G 2,…, G m be a finite sequence of rooted graphs with the same vertex set. If each vertex of each graph has a self arc and m ¸ (n - 1) 2, then G m ± G m-1 ±  ± G 1 is strongly rooted. Thus the above result is not directly applicable to the asynchronous problem

59. 1 3 5 4 6 8 7 2

60. 1 3 5 4 6 8 7 2 1 3 4 2 quotient graph N S8£8S8£8 = N

61. Vicsek Model Bifurcation Red Leader Leader Follower