Analytic Synchronization A concept for understanding emergent behaviors in asynchronous multi-agent systems A. S. Morse Yale University Napa Valley August 2, 2005 www.swarms.org

Research Objectives SWARMS To study multi-agent motion control problems with the objective of better understanding the process of devising provably correct local control strategies which can coordinate and otherwise induce desired emergent {group-wide} behaviors. Formation control: With the objective of understanding how to form, maintain, and otherwise manage a formation of mobile autonomous agents, we are actively studying a variety of algorithms based on the classical concept of graph rigidity. Much of this research has been done with Brian Anderson who will be giving a talk at the workshop on this subject. Sensor localization: As a direct spin-off from our original work formation control, we are also actively studying how global graph rigidity can be used to advantage to deal with long-standing localization problems which arise in sensor networks. Distributed control: We are heavily involved in the study of high-level distributed control strategies of all types with the goal of understanding how to analyze and synthesize them. Earlier work with Ali Jadbabaie on heading synchronization as well as more recent work on rendezvousing are representative directions. SWARMS

We will talk about two problems: We have been specifically interested in the development of techniques for analyzing asynchronous multi-agent systems. One such technique, called analytic synchronization, will be outlined later in the talk. We will talk about two problems: The Asynchronous Rendezvous Problem The Asynchronous Heading Synchronization Problem SWARMS

The Asynchronous Rendezvous Problem SWARMS

Unsynchronized Strategies ! The Asynchronous Rendezvous Problem Consider a set of n mobile autonomous agents which can all move in the plane. Agents Each agent is able to continuously sense the relative positions of all other agents in its “sensing region” where by agent i’s sensing region is meant a closed disk of radius r centered at agent i’s current position. agent i r sensing region Problem: Devise local control strategies, one for each agent, which without active communication between agents, cause all members of the group to eventually rendezvous at a single unspecified point. Synchronized strategies proposed in Unsynchronized Strategies ! H. Ando, Y. Oasa, I. Suzuki, M. Yamashita, October 1999 IEEE Trans. Rob. & Auto. Lin, Morse, Anderson, CDC 2003 SWARMS

Stop and Go Maneuvers {of Agent i} A stop and go maneuver takes place within a time interval consisting of two successive subintervals: fixed positive numbers sensing period maneuvering period Each agent is stationary during each of its sensing periods. During a maneuvering period, an agent moves from its current rest position to its next “way-point” and again comes to rest. Successive way-points for each agent are constrained to be within r units of each other. is an upper bound on the time it takes for agent i to move from one way-point to the next. Details of maneuvering to way-points not considered. SWARMS

successive subintervals: Stop and Go Maneuvers {of Agent i} A stop and go maneuver takes place within a time interval consisting of two successive subintervals: sensing period maneuvering period tik kth Agent i’s event times: ti1, ti2, …. Not synchronized with other agents’ event times A design parameter called a sensing time S is chosen to satisfy Registered neighbors of agent i at event time tik are those agents which during agent i’s kth sensing period are stationary for at least S seconds at positions within agent i’s sensing region. It is assumed that no agent stops for as long as S seconds during any of its maneuvering periods. SWARMS

during which agent j is stationary. sensing period maneuvering period tik kth Registered neighbor is a symmetric relation: If agent j is a registered neighbor of agent i at tik and q is the sensing/maneuvering period of agent j during which registration takes place, then agent i is a registered neighbor of agent j at tjq. Registered position of neighbor j at tik is the position of agent j during the last interval of at length at least S within agent i’s kth sensing period, during which agent j is stationary. Registered neighbors of agent i at event time tik are those agents which during agent i’s kth sensing period are stationary for at least S seconds at positions within agent i’s sensing region. SWARMS

j i SWARMS Retaining Neighbors Agent i is said to satisfy the motion constraint induced by neighbor j at time tik if the point to which agent i moves at the end of its kth maneuvering period is within a closed disk of diameter r centered at the mean of its position at tik and the registered position of neighbor j at tik. i j Neighbor Retention: Suppose that agents i and j satisfy the motion constraints induced by their registered neighbors. If agent j is a registered neighbor of agent i at tik, the agent j is also a registered neighbor of agent i at ti(k+1) SWARMS

SWARMS Retaining Neighbors Agents is said to satisfy the motion constraint induced by neighbor j at time tik if the point to which agent i moves at the end of its kth maneuvering period is within a closed disk of diameter r centered at the mean of its position at tik and the registered position of neighbor j at tik. Cooperation Assumption: Each agent satisfies the motion constrains induced by each of its registered neighbors. Neighbor Retention: Suppose that agents i and j satisfy the motion constraints induced by their registered neighbors. If agent j is a registered neighbor of agent i at tik, the agent j is also a registered neighbor of agent i at ti(k+1) Implies that each agent will retain all of its current registered neighbors forever. SWARMS

SWARMS sensing region constraint disc Agent A neighbors Agent A’s Motion Constraints Induced by its Neighbors sensing region constraint disc Agent A neighbors constraint disc constraint set constraint disc constraint disc Either contains just agent A or is strictly convex with nonempty interior SWARMS

SWARMS Way Point Selection Rules By the kth local convex hull of agent i is meant the convex hull of the set consisting of agent i’s position at time tik and the registered positions of agent i’s neighbors at time tik. Agent i’s kth way point 1. must be the same as its position at time tik if agent i has no neighbors at time tik. 2. must be within its kth local convex hull. 3. must not be a corner of its kth local convex hull unless the local convex hull is a single point. 4. must be within the motion constraint sets induced by its neighbors. SWARMS

SWARMS convex hull target constraint set 0 is in the Interior of

SWARMS convex hull target  constraint set 0 is on Boundary of

MAIN RESULT Despite the lack of synchronization it is possible to define a simple undirected graph at each event time tik of each agent which characterizes which agents are in which other agents’ sensing regions at time tik. It can be shown that if there is ever an event time of any one agent at which this neighbor graph is connected, then the use of the aforementioned type of strategy by all n agents will cause them to eventually rendezvous at one point. Proving that this is so is challenging because the n agent system under consideration is asynchronous. SWARMS

Rendezvousing Asynchronously Each agent’s behavior can be modeled as a system Ai evolving on its event time sequence ti1, ti2,…. where, as we said before, tik is the time at which agent i’s kth maneuvering period begins. The n interacting Ai constitute the asynchronous multi-agent system of interest. Since event time sequences of different agents are not synchronized, standard dynamical systems analysis tools cannot be directly applied. ANALYTIC SYNCHRONIZATION The n agent asynchronous system is analyzed by “embedding” all the Ai into a single synchronous system S evolving on the time set T which results when the n individual agent event time sequences are merged into a single ordered set. This is accomplished by appropriately extending the domain of definition of each agent subsystem Ai from the event time sequence ti1, ti2,… to all of T. Convergence is then established using familiar ideas applicable to S. SWARMS

We will talk about two problems: The Asynchronous Rendezvous Problem The Asynchronous Heading Synchronization Problem SWARMS

The Asynchronous Heading Synchronization Problem SWARMS

Motivated by simulation results reported in the paper: The Asynchronous Heading Synchronization Problem Motivated by simulation results reported in the paper: SWARMS

A theoretical explanation for this observed behavior can be found in Vicsek et al. simulated a flock of n agents {particles} all moving in the plane at the same speed s, but with different headings 1, 2, …. n i = heading qi s = speed s Each agent’s heading is updated using a local rule based on the average of its own current heading plus the headings of its “neighbors.” Vicsek’s simulations demonstrated that these nearest neighbor rules can cause all agents to eventually move in the same direction despite the absence of a leader and/or centralized coordination and despite the fact that each agent’s set of neighbors changes with time. A theoretical explanation for this observed behavior can be found in Jadbabaie, Lin & Morse, IEEE TAC, June 2003 SWARMS

ri neighbors of agent i agent i SWARMS each agent is a neighbor of itself neighbors of agent i ri agent i SWARMS

HEADING UPDATE EQUATIONS s = speed s  i = heading  i Average at time t of headings of neighbors of agent i. Ni (t) = set of indices of agents i’s “neighbors” at time t ni(t) = number of indices in Ni(t) SWARMS

SWARMS Jadbabaie, Lin & Morse, IEEE TAC, June 2003 Interesting property: Neighbor graphs change with time Feature of the original Vicsek model considered in the above paper No leader All ri = r. Synchronous operation No delays in sensing of headings Main technical tools exploited Simple {undirected} graphs to describe neighbor relationships Algebraic graph theory A theorem of J. Wolfowitz in Proc. AMS, 1963 which gives conditions for an infinite product of left stochastic matrices to converge to a rank one matrix. SWARMS

Subsequent research by B. Francis, L. Moreau, V. Blondell, J Subsequent research by B. Francis, L. Moreau, V. Blondell, J. Tsitsiklis, D. Angeli, D. Spielman, M. Cao, B. Anderson, G. Tanner, G. Pappas, R. Beard, and others. No leader convergence rates ri  r All ri = r. sensing delays Synchronous operation No delays in sensing of headings asynchronous operation Additional technical tools exploited include directed graphs scrambling matrices, Sarymsakov matrices, and random walks a special partial Lyapunov function tailored for Markov Chains – Doob {1953}, Senta {1981}, Tsisiklis {1984}, Bertsekas & Tsisiklis {1989}. SWARMS

The Asynchronous Vicsek Flocking Problem A modified version of the Vicsek flocking problem in which each agent updates its heading at times determined by its own clock. Groups’ clocks are not assumed to be synchronized together. The times at which any one agent updates its heading are not assumed to be evenly spaced. SWARMS

The Asynchronous Vicsek Model {Here t denotes real continuous time} There are n agents labeled 1 through n. Each agent i maintains a constant heading i(t) for ti(k -1) < t · tik, k ¸ 1 where ti0 = 0 and tik is agent i’s kth update event time. Event time sequences are not assumed to be synchronized qi = heading qi s = speed s Ni (tik) = set of indices of agents i’s neighbors including itself, at time tik ni(tik) = number of indices in Ni(tik) SWARMS

Flocking Asynchronously Each agent’s behavior can be modeled as a system Ai evolving on its event time sequence ti1, ti2,…. where, as we said before, tik is the time at which agent i instantaneously changes its heading from one value to the next. The n interacting Ai constitute the asynchronous multi-agent system of interest. Since event time sequences of different agents are not synchronized, standard dynamical systems analysis tools cannot be directly applied. ANALYTIC SYNCHRONIZATION The n agent asynchronous system is analyzed by “embedding” all the Ai into a single synchronous system S evolving on the index set of the time set T which results when the n individual agent event time sequences are merged into a single ordered set. This is accomplished by appropriately extending the domain of definition of each agent subsystem Ai from the event time sequence ti1, ti2,… to all of T. Convergence is then established using familiar ideas applicable to S. SWARMS

SWARMS agent 1 t11 t12 t15 t16 t13 t14 interacting agent 2 t26 t25 t24

SWARMS agent 1 T = t3 t5 t10 t13 t7 t8 t11 t9 t6 t1 t12 t4 t2 t11 t12

SWARMS Analytic Synchronization Let T = set of all event times tik of all n agents. Re-label elements of T as t0, t1, t2, … so that t + 1 > t  = 0, 1, 2,... Define Since it must be true for any value of  for which t is an event time of agent i, that where  0 is such that t 0 = next event time of agent i after t . because i(t) is constant for t < t · t 0 . But Therefore SWARMS

SWARMS At this point: if t is an event time of agent i But i(t) is constant between event times so if t is not an event time of agent i So if we define if t is not an event time of agent i then if t is not an event time of agent i Therefore all  ¸ 0 SWARMS

SWARMS Summary Let T = set of all event times tik of all n agents. Re-label elements of T as t0, t1, t2, … so that t + 1 > t  = 0, 1, 2,... At its event times, agent i has the same neighbors as before embedding if t is an event time of agent i Between its event times, agent i has only itself as a neighbor if t is not an event time of agent i all  ¸ 0 SWARMS

P = index set of all possible neighbor configurations. V = agent index set ={1, 2, …, n} For each p 2 P Gp = {V, Ap} - a directed graph with vertex set V and arc set Ap (i, j) 2 A p if agent i is a neighbor of agent j in configuration p All vertices have self-arcs 7 4 1 3 5 2 6 (1,2) G = set of all directed graphs with vertex set V and self-arcs at all vertices Since agent i has no neighbors other than itself between its event times, at such times vertex i has only one incoming arc. SWARMS

Matrix Representation of Gp = {V, Ap } adjacency matrix Ap =[aij]n£ n aij(p) = 1 if i is a neighbor of j aij(p) = 0 otherwise Dp = diagonal {d1(p), d2(p), …., dn(p)}n£n di(p) = in-degree of vertex i 7 4 1 3 5 2 6 (1,2) in-degree = 1 in-degree = 2 SWARMS

adjacency matrix Ap =[aij]n £ n State Space Equation adjacency matrix Ap =[aij]n £ n aij(p) = 1 if i is a neighbor of j aij(p) = 0 otherwise Dp = diagonal {d1(p), d2(p), …., dn(p)}n £ n 7 4 1 3 5 2 6 (1,2) s() = index in P of neighbor configuration at time . SWARMS

SWARMS Switching Among Neighbor Graphs if t is an event time of agent i if t is not an event time of agent i Constraints induced by analytic synchronization If t is not an event time of agent i, then vertex i in G() cannot have any incoming arcs other than its own. SWARMS

SWARMS Some Graph Concepts Call a graph G 2 G rooted if there is at least one vertex v for which, for each vertex i, there is a directed path from v to i. Rooted graphs are important because if the graphs in the sequence G(0), G(0) ... encountered along a system trajectory are rooted, then convergence to a common heading occurs. Although rooted graphs could easily arise along a trajectory in the synchronous flocking problem, in the unsynchronized version of the problem under consideration here, this can never occur except possibly in the highly unusual situation when different agents have some of their event times occurring at exactly the same instant. By the composition of a graph G1 with a graph G2, written G2 ± G1, is meant that graph which has an arc from i to j just in case there is a vertex k for which G1 has an arc from i to k and G2 has an arc from k to j. Call a finite sequence of graphs G1, G2,….Gm in G jointly rooted, if the composition Gm± ± G1 is a rooted graph. Call an infinite sequence of graphs G1,G2,….Gj, … in G repeatedly jointly rooted, if there is a finite integer m such that each successive subsequence Gm(k-1) +1, G(m(k-1)+2), … Gmk, k ¸ 1, is jointly rooted. SWARMS

Reaching a Common Heading Asynchronously If the sequence of graphs G (0), G(1),... encountered along a trajectory is repeatedly jointly rooted, then at an exponentially fast convergence rate. The hypothesis above is not vacuous: Despite the constraints imposed on the neighbor sets by analytic synchronization, there are in fact plenty of trajectories of the asynchronous system of interest which generate sequences of graphs which are repeatedly jointly rooted. SWARMS

Observations New data structures, models, etc. are needed to represent large groups of mobile autonomous agents and sensors at various degrees of granularity, for purposes of simulations, management, analysis and control. Such representations will exploit tools from both graph theory and from the theory of dynamical systems At least initially, individual agent/sensor descriptions using simple kinematic and dynamic models will suffice. System complexity will stem more from the number of agent/sensor models being studied than from the detailed properties of the individual agent models. SWARMS

SWARMS