Dimitrios Hristu-Varsakelis Mechanical Engineering and Institute for Systems Research University of Maryland, College Park Joint work with: M. Egerstedt, S. B. Andersson, C. Shao. P.R. Kumar, P. S. Krishnaprasad, Cooperative Optimization and Navigation Problems

Outline Ensembles of autonomous vehicles operating on “expansive” terrain. Bio-inspired trajectory optimization Language-based navigation Report on Progress – Event-driven communication

Examples from biology (bees, ants, fish etc.) Ensembles can accomplish tasks that are impossible for an individual. Coordination requires thinking about control/communication interactions. Ensembles of Autonomous Systems

V1V1 VnVn vehicle obstacle control station target start Trajectory optimization without a map A group of vehicles traveling between a fixed pair of locations Terrain is unknown - no “global” map. On-board sensing provides local information about vehicle’s immediate surroundings PROBLEM: Given an initial path between a pair of “start” and “target” locations, find the optimal path connecting that pair, using “local” interactions between vehicles. V n-1

V1V1 VnVn vehicle obstacle control station target start Trajectory optimization without a map A group of vehicles traveling between a fixed pair of locations Terrain is unknown - no “global” map. On-board sensing provides local information about vehicle’s immediate surroundings PROBLEM: Given an initial path between a pair of “start” and “target” locations, find the optimal path connecting that pair, using “local” interactions between vehicles. V n-1

Local pursuit: A biologically-inspired algorithm Theorem (on ): The iterated paths converge to a straight line as... K+2 k+1 k Start Target : Initial path : path followed by the k-th vehicle,... (on a smooth manifold M): If vehicle separation is sufficiently small, then the iterated paths converge to a geodesic. [Bruckstein, 92]

Experimental results: with Euclidean metric Initial path length ~7m Vehicle separation ~1.5m A collection of mobile robots with: Wireless communication between neighbors Sonar and odometry sensors TARGET START

: Minimum-length geodesic connecting to M : location of k-th vehicle Local Pursuit Idea: Find optimal trajectory to leader and follow it momentarily.

M : Minimum-length geodesic connecting a to b : location of k-th vehicle Pursuit decreases vehicle separation

The k-th vehicle moves as follows: Wait at until t=Δ(κ+1) At time t, “follow the optimal trajectory” from to Local pursuit for more general optimal control problems Let Given an initial trajectory with converge to a local min. for Assumptions: uniqueness, smoothness Find that minimizes s.t. As, iterated trajectories

Simulation: pursuit on 5m trajectory 0.7m separation

A sub-Riemannian example 5m trajectory 1.5m separation fixed

Pursuit in vehicles with drift (minimum time problem)

Summary and Work in Progress A biologically-inspired trajectory optimization algorithm - local pursuit forms a “string” of vehicles - each vehicle uses local information and communicates with its closest neighbors Target state and optimal trajectory are unknown Local convergence Experiments Escaping local minima Comparison with gradient descent methods

Control in a reasonably complex world The problem of specifying control tasks (e.g. “go to the refrigerator and get the milk”) Solving motion control problems of adequate complexity Many interesting systems evolve in environments that are not smooth, simply connected, etc. Using language primitives to navigate: Specify control policies Represent the environment (what parts do we ignore?)

Def: MDLe is the formal language defined by the context free grammar with production rules: Motion Description Languages Evaluate Evolve under until Concatenate, encapsulate atoms to form complex strings (plans), e.g. N: nonterminals T: terminals S: start symbol ε: empty string Fact: MDLe is context free but not regular Atom:

Symbolic Navigation Keep only “interesting” details about how to navigate the world Landmark: L = (M,x)M: map “patch”, x: coordinates Sensor signature: L = L i if s(t) = s i (t) for t in [t 0,T] Navigation –Local navigation: on a given landmark L i –Global navigation: between landmarks M x World

A directed graph representation of a map Represent only “interesting parts” of the world. G = {L,E} L i : landmarks E ij : {i,j,  ij } Γ ij : an MDLe program E ij  E ji Idea: Replace details locally by a feedback program

Experiment: indoor navigation Lab 1 Lab 2 Office Partial floor plan of 2 nd floor A.V. Williams

Experiment, cont’d Goal: Navigate between three landmarks Front of labRear of labOffice

Experiment: Example MDLe plans {Lab2toLab1Plan (bumper) (Atom (atIsection 0100) (goAvoid )) (Atom (atIsection 0010) (go )) (Atom (wait ) (align 7 9)) (Atom (atIsection 1000) (goAvoid )) (Atom (atIsection 0100) (go )) (Atom (wait ) (align 3 5)) (Atom (wait 7) (goAvoid )) (Atom (atIsection 1000) (goAvoid )) } {Lab1toOfficePlan (bumper) (Atom (atIsection 1001) (goAvoid )) (Atom (atIsection 0011) (go )) (Atom (wait ) (align 11 13)) (Atom (atIsection 0100) (goAvoid )) (Atom (wait 10) (rotate -90)) }

Experiment: A typical run

Controllers (and MDLe plans) are not always successful. Environmental factors (moving obstacles) System uncertainty (e.g. actuator noise) Associate a probability density function with an MDLe plan Enumerate the MDLe strings associated with an environment graph G = {L,E}, Define Prob. of arriving at by executing from Assumptions: G is a “good” description of the world Sensor model: Incorporating Uncertainty

A prototype navigation problem How do I get to a given landmark ? Prob. density at time k, given observations up to time k. Probability after evaluating plan and making a new observation: Maximize probability of arriving at a desired landmark in N “steps” Maximize probability of arriving at desired landmark in N “steps” Information at “time” k Maximize prob. of arrival at a desired landmark with minimum of “steps”

A navigation example

Example - data Example: L2 to L3 (syntax: (ξ,u)) with N(0,0.01) actuator noise

Example: steer to a landmark in N “steps” X 0 =L 1, X F =L 2, N=3 P 0|0 =[1/3, 1/3, 1/3] Evolution of probability density on G Desired success probability set to 95%

True and observed landmarks Example: steer to a landmark in N “steps”

Executed Plans Example: steer to a landmark in N “steps”

Summary and Ongoing Work Language-based Control The motion description language MDLe “Landmark+instruction”-based descriptions of the world Optimal navigation via dynamic-programming Obtaining “nominal” densities for navigation Software

References: S. Andersson and D. Hristu-Varsakelis, “Stochastic Language-based Motion Control”, to appear, CDC D. Hristu-Varsakelis, M. Egerstedt, P. S. Krishnaprasad, “On the Structural Complexity of the Motion Description Language MDLe”, to appear, CDC D. Hristu-Varsakelis and P. R. Kumar, “Interrupt-based feedback control over a shared communication medium”, IEEE CDC M. Egerstedt and D. Hristu-Varsakelis, “Observability and Policy Optimization for Mobile Robots”, CDC 2002

This page intentionally left blank

Event-Based Stabilization of Ensembles-Users of a Shared Network

Dynamical systems as users of a shared “network” Control of collections of systems with limited communication A prototype problem in “divided attention”. N=number of systems in the ensemble n=max. number of feedback loops that can be closed at any time How much communication time must be devoted to each system to guarantee that the collection remains stable? Can the ensemble be stabilized? G 1 (s ) … K1K1 G 2 (s) K2K2 G N (s) KNKN controller plant shared medium

A feedback communication policy We would like to avoid having to specify the communication policy in advance (thus the need for memory, clocks) How much information is needed to implement an event-driven policy? Let’s define a simple rule for deciding which system(s) should be allowed to use the “network”. Idea: Close loops corresponding to states that are “furthest” from the origin 0... Ex.: N=3, n=2

0... A feedback communication policy Ex.: N=3, n=2 (feedback loop closed) (feedback loop open) Let’s define a simple rule for deciding which system(s) should be allowed to use the “network”. Idea: Close loops corresponding to states that are “furthest” from the origin Definition: An ensemble is δ-captured if for all i after some time For each system, find a Lyapunov function V( ) such that:

Some possibilities for interrupt-based communication (special case n=1) Policy: (sampled CLS-  ): 2’. When, set. Policy: (CLS-  ): Let 1. At time, close the loop of the system, 2. When, set. 3. Policy: (open loop CLS): 1’’. At time, close the loop of the system, 2’’. When, set. feedback

A test run 

A “least conservative” feedback communication policy Policy: (MACLS  1. At time t, close the loop of the system where 2. When, repeat from step 1. Theorem: the ensemble is captured using CLS-  for  large enough, if where otherwise there exists a choice of dynamics with the same for which there is no stabilizing communication sequence. Policy: (Control Zone  Pick  1. At time t, close the loop of the system where 2. When, repeat from step 1. An alternative communication policy:

A simulation example (N=3, n=1,  =1,  =0.2)

Experiment: stabilizing a pair of pendulums Lengths: 20cm, 45cm Communication: 115Kbps ρ=0.8

Experiment: stabilizing a pair of pendulums

Event-based feedback control - Summary and Work in Progress A class of feedback communication policies - sampled Lyapunov functions - continuously monitored Lyapunov functions - continuously monitored state norms Sufficient condition for stability Stochasticity Performance analysis Effects of delays in the feedback loop