Optimal Control and Reachability with Competing Inputs Ian Mitchell Department of Computer Science The University of British Columbia research supported by National Science and Engineering Research Council of Canada Good afternoon. I’ll be talking to you today about level set methods for analyzing, verifying and controlling continuous and hybrid systems. So obviously, I’ll have to explain what I mean by reachability, and what is a hybrid system. I’ll show how reachability can be used to analyze, verify and create controllers for various types of systems. This is joint work with Professor Claire Tomlin in Aero/Astro, Professor Ron Fedkiw in Computer Science, and with students in their labs at Stanford.

Ian Mitchell (UBC Computer Science) Competing Inputs What do we do when there are multiple parameters, some of which we can choose but some of which have unknown and uncontrolled value “Control” input denoted by u (or a) “Disturbance” input denoted by d (or b) Choose control input as before to optimize trajectory or achieve safety Due to disturbance input, system remains nondeterministic even if control signal is fixed First, let’s look at how I represent sets. The implicit surface representation has attracted lots of attention over the past few decades in computer science and engineering, because it handles general sets so gracefully. We just define a scalar function J such that J is negative inside the set and positive outside. Constructive geometry is easily accomplished on such sets; for example, the union of two sets is described implicitly by the taking minimum of their two implicit functions at each point in the state space. I give an example of an oddly shaped set and one possible implicit surface representation of that set at the bottom of the slide. Implicit surface functions like J are not unique, but if we place the additional restriction on J that its gradient have magnitude one, then we can deduce two additional bits of information from J at each point in the state space: the distance and direction of the nearest point on the boundary of the set. This information proves useful in the context of synthesizing safe control signals, such as the video game that I showed earlier. March 2008 Ian Mitchell (UBC Computer Science)

Two Treaments of Disturbance Stochastic perturbations d(t) ~ D Discrete state Poisson processes Markov decision processes Stochastic differential equations Bounded value inputs d(t) 2 D Robust reach sets Two player zero sum games First, let’s look at how I represent sets. The implicit surface representation has attracted lots of attention over the past few decades in computer science and engineering, because it handles general sets so gracefully. We just define a scalar function J such that J is negative inside the set and positive outside. Constructive geometry is easily accomplished on such sets; for example, the union of two sets is described implicitly by the taking minimum of their two implicit functions at each point in the state space. I give an example of an oddly shaped set and one possible implicit surface representation of that set at the bottom of the slide. Implicit surface functions like J are not unique, but if we place the additional restriction on J that its gradient have magnitude one, then we can deduce two additional bits of information from J at each point in the state space: the distance and direction of the nearest point on the boundary of the set. This information proves useful in the context of synthesizing safe control signals, such as the video game that I showed earlier. March 2008 Ian Mitchell (UBC Computer Science)

Markov Decision Process Discrete time, discrete state model with probabilistic transitions Typically specified by Alternatively specified by x(t+1) = d(t) where d(t) is drawn from the Bernoulli Scheme with For discrete time systems, many distributions are supported First, let’s look at how I represent sets. The implicit surface representation has attracted lots of attention over the past few decades in computer science and engineering, because it handles general sets so gracefully. We just define a scalar function J such that J is negative inside the set and positive outside. Constructive geometry is easily accomplished on such sets; for example, the union of two sets is described implicitly by the taking minimum of their two implicit functions at each point in the state space. I give an example of an oddly shaped set and one possible implicit surface representation of that set at the bottom of the slide. Implicit surface functions like J are not unique, but if we place the additional restriction on J that its gradient have magnitude one, then we can deduce two additional bits of information from J at each point in the state space: the distance and direction of the nearest point on the boundary of the set. This information proves useful in the context of synthesizing safe control signals, such as the video game that I showed earlier. March 2008 Ian Mitchell (UBC Computer Science)

Stochastic Differential Equations (SDEs) Two mathematical frameworks: Ito and Stratonovich Conversions exist between the two SDE is ODE with a Brownian motion (Wiener process) perturbation Restrictive class of distributions eg: cannot guarantee bound on stochastic term Equivalent of Hamilton-Jacobi equation for SDEs is the Fokker-Planck equation First, let’s look at how I represent sets. The implicit surface representation has attracted lots of attention over the past few decades in computer science and engineering, because it handles general sets so gracefully. We just define a scalar function J such that J is negative inside the set and positive outside. Constructive geometry is easily accomplished on such sets; for example, the union of two sets is described implicitly by the taking minimum of their two implicit functions at each point in the state space. I give an example of an oddly shaped set and one possible implicit surface representation of that set at the bottom of the slide. Implicit surface functions like J are not unique, but if we place the additional restriction on J that its gradient have magnitude one, then we can deduce two additional bits of information from J at each point in the state space: the distance and direction of the nearest point on the boundary of the set. This information proves useful in the context of synthesizing safe control signals, such as the video game that I showed earlier. March 2008 Ian Mitchell (UBC Computer Science)

Continuous Backward Reachable Tubes Set of all states from which trajectories can reach some given target state For example, what states can reach G(0)? Continuous System Dynamics Target Set G(0) Backward Reachable Set G(t) March 2008 Ian Mitchell (UBC Computer Science)

Reachable Tubes (controlled input) For most of our examples, target set is unsafe If we can control the input, choose it to avoid the target set Backward reachable set is unsafe no matter what we do “Minimal” backward reach tube Continuous System Dynamics March 2008 Ian Mitchell (UBC Computer Science)

Reachable Tubes (uncontrolled input) Sometimes we have no control over input signal noise, actions of other agents, unknown system parameters It is safest to assume the worst case “Maximal” backward reach tube Continuous System Dynamics March 2008 Ian Mitchell (UBC Computer Science)

Ian Mitchell (UBC Computer Science) Two Competing Inputs For some systems there are two classes of inputs  = (u,d) Controllable inputs u  U Uncontrollable (disturbance) inputs d  D Equivalent to a zero sum differential game formulation If there is an advantage to input ordering, give it to disturbances Continuous System Dynamics March 2008 Ian Mitchell (UBC Computer Science)

Ian Mitchell (UBC Computer Science) Objective Function Extends in obvious way to the additional input eg: discrete time discounted with fixed finite horizon tf eg: continuous time no discount with target set T First, let’s look at how I represent sets. The implicit surface representation has attracted lots of attention over the past few decades in computer science and engineering, because it handles general sets so gracefully. We just define a scalar function J such that J is negative inside the set and positive outside. Constructive geometry is easily accomplished on such sets; for example, the union of two sets is described implicitly by the taking minimum of their two implicit functions at each point in the state space. I give an example of an oddly shaped set and one possible implicit surface representation of that set at the bottom of the slide. Implicit surface functions like J are not unique, but if we place the additional restriction on J that its gradient have magnitude one, then we can deduce two additional bits of information from J at each point in the state space: the distance and direction of the nearest point on the boundary of the set. This information proves useful in the context of synthesizing safe control signals, such as the video game that I showed earlier. March 2008 Ian Mitchell (UBC Computer Science)

Ian Mitchell (UBC Computer Science) Who Goes First? One input is chosen to maximize and the other to minimize the objective But what knowledge is available when choosing an input? Current state? Other input? “Non-anticipative strategies” One player gets to know the other player’s input value (as well as current state) However, that player must declare their strategy (reaction to every input) in advance First, let’s look at how I represent sets. The implicit surface representation has attracted lots of attention over the past few decades in computer science and engineering, because it handles general sets so gracefully. We just define a scalar function J such that J is negative inside the set and positive outside. Constructive geometry is easily accomplished on such sets; for example, the union of two sets is described implicitly by the taking minimum of their two implicit functions at each point in the state space. I give an example of an oddly shaped set and one possible implicit surface representation of that set at the bottom of the slide. Implicit surface functions like J are not unique, but if we place the additional restriction on J that its gradient have magnitude one, then we can deduce two additional bits of information from J at each point in the state space: the distance and direction of the nearest point on the boundary of the set. This information proves useful in the context of synthesizing safe control signals, such as the video game that I showed earlier. March 2008 Ian Mitchell (UBC Computer Science)

Zero Sum Game Value Function Value function is then defined as optimization over appropriate strategy and input signal pair “Lower” value function, since disturbance (minimizer) has the advantage Parallel “upper” value function can be defined If inputs are independent, optimal strategy will ignore additional information about the other input Upper and lower value functions will be equal First, let’s look at how I represent sets. The implicit surface representation has attracted lots of attention over the past few decades in computer science and engineering, because it handles general sets so gracefully. We just define a scalar function J such that J is negative inside the set and positive outside. Constructive geometry is easily accomplished on such sets; for example, the union of two sets is described implicitly by the taking minimum of their two implicit functions at each point in the state space. I give an example of an oddly shaped set and one possible implicit surface representation of that set at the bottom of the slide. Implicit surface functions like J are not unique, but if we place the additional restriction on J that its gradient have magnitude one, then we can deduce two additional bits of information from J at each point in the state space: the distance and direction of the nearest point on the boundary of the set. This information proves useful in the context of synthesizing safe control signals, such as the video game that I showed earlier. March 2008 Ian Mitchell (UBC Computer Science)

Competing Inputs: Final Comments Feedback control is more realistic implementation If order of input decision is irrelevant (upper and lower value functions are equal), then nonanticipative strategy results will be equivalent to feedback results For robustness, give advantage (eg strategy) to the disturbance input if it matters (potentially pessimistic) Input signals still drawn from set of measureable functions Two player concepts have been extended to viability theory and set-valued analysis First, let’s look at how I represent sets. The implicit surface representation has attracted lots of attention over the past few decades in computer science and engineering, because it handles general sets so gracefully. We just define a scalar function J such that J is negative inside the set and positive outside. Constructive geometry is easily accomplished on such sets; for example, the union of two sets is described implicitly by the taking minimum of their two implicit functions at each point in the state space. I give an example of an oddly shaped set and one possible implicit surface representation of that set at the bottom of the slide. Implicit surface functions like J are not unique, but if we place the additional restriction on J that its gradient have magnitude one, then we can deduce two additional bits of information from J at each point in the state space: the distance and direction of the nearest point on the boundary of the set. This information proves useful in the context of synthesizing safe control signals, such as the video game that I showed earlier. March 2008 Ian Mitchell (UBC Computer Science)

Optimal Control and Reachability with Competing Inputs For more information contact Ian Mitchell Department of Computer Science The University of British Columbia mitchell@cs.ubc.ca http://www.cs.ubc.ca/~mitchell Good afternoon. I’ll be talking to you today about level set methods for analyzing, verifying and controlling continuous and hybrid systems. So obviously, I’ll have to explain what I mean by reachability, and what is a hybrid system. I’ll show how reachability can be used to analyze, verify and create controllers for various types of systems. This is joint work with Professor Claire Tomlin in Aero/Astro, Professor Ron Fedkiw in Computer Science, and with students in their labs at Stanford.