1. Optimal Control and Reachability with Competing Inputs
Ian Mitchell
Department of Computer Science
The University of British Columbia
research supported by
National Science and Engineering Research Council of Canada
Good afternoon. I’ll be talking to you today about level set methods for analyzing, verifying and controlling continuous and hybrid systems. So obviously, I’ll have to explain what I mean by reachability, and what is a hybrid system. I’ll show how reachability can be used to analyze, verify and create controllers for various types of systems.
This is joint work with Professor Claire Tomlin in Aero/Astro, Professor Ron Fedkiw in Computer Science, and with students in their labs at Stanford.

2. Ian Mitchell (UBC Computer Science)
Competing Inputs
What do we do when there are multiple parameters, some of which we can choose but some of which have unknown and uncontrolled value
“Control” input denoted by u (or a)
“Disturbance” input denoted by d (or b)
Choose control input as before to optimize trajectory or achieve safety
Due to disturbance input, system remains nondeterministic even if control signal is fixed
First, let’s look at how I represent sets. The implicit surface representation has attracted lots of attention over the past few decades in computer science and engineering, because it handles general sets so gracefully. We just define a scalar function J such that J is negative inside the set and positive outside. Constructive geometry is easily accomplished on such sets; for example, the union of two sets is described implicitly by the taking minimum of their two implicit functions at each point in the state space. I give an example of an oddly shaped set and one possible implicit surface representation of that set at the bottom of the slide.
Implicit surface functions like J are not unique, but if we place the additional restriction on J that its gradient have magnitude one, then we can deduce two additional bits of information from J at each point in the state space: the distance and direction of the nearest point on the boundary of the set. This information proves useful in the context of synthesizing safe control signals, such as the video game that I showed earlier.
March 2008
Ian Mitchell (UBC Computer Science)

3. Two Treaments of Disturbance
Stochastic perturbations d(t) ~ D
Discrete state Poisson processes
Markov decision processes
Stochastic differential equations
Bounded value inputs d(t) 2 D
Robust reach sets
Two player zero sum games
First, let’s look at how I represent sets. The implicit surface representation has attracted lots of attention over the past few decades in computer science and engineering, because it handles general sets so gracefully. We just define a scalar function J such that J is negative inside the set and positive outside. Constructive geometry is easily accomplished on such sets; for example, the union of two sets is described implicitly by the taking minimum of their two implicit functions at each point in the state space. I give an example of an oddly shaped set and one possible implicit surface representation of that set at the bottom of the slide.
Implicit surface functions like J are not unique, but if we place the additional restriction on J that its gradient have magnitude one, then we can deduce two additional bits of information from J at each point in the state space: the distance and direction of the nearest point on the boundary of the set. This information proves useful in the context of synthesizing safe control signals, such as the video game that I showed earlier.
March 2008
Ian Mitchell (UBC Computer Science)

4. Markov Decision Process
Discrete time, discrete state model with probabilistic transitions
Typically specified by
Alternatively specified by x(t+1) = d(t)
where d(t) is drawn from the Bernoulli Scheme with
For discrete time systems, many distributions are supported
First, let’s look at how I represent sets. The implicit surface representation has attracted lots of attention over the past few decades in computer science and engineering, because it handles general sets so gracefully. We just define a scalar function J such that J is negative inside the set and positive outside. Constructive geometry is easily accomplished on such sets; for example, the union of two sets is described implicitly by the taking minimum of their two implicit functions at each point in the state space. I give an example of an oddly shaped set and one possible implicit surface representation of that set at the bottom of the slide.
Implicit surface functions like J are not unique, but if we place the additional restriction on J that its gradient have magnitude one, then we can deduce two additional bits of information from J at each point in the state space: the distance and direction of the nearest point on the boundary of the set. This information proves useful in the context of synthesizing safe control signals, such as the video game that I showed earlier.
March 2008
Ian Mitchell (UBC Computer Science)

5. Stochastic Differential Equations (SDEs)
Two mathematical frameworks: Ito and Stratonovich
Conversions exist between the two
SDE is ODE with a Brownian motion (Wiener process) perturbation
Restrictive class of distributions
eg: cannot guarantee bound on stochastic term
Equivalent of Hamilton-Jacobi equation for SDEs is the Fokker-Planck equation
First, let’s look at how I represent sets. The implicit surface representation has attracted lots of attention over the past few decades in computer science and engineering, because it handles general sets so gracefully. We just define a scalar function J such that J is negative inside the set and positive outside. Constructive geometry is easily accomplished on such sets; for example, the union of two sets is described implicitly by the taking minimum of their two implicit functions at each point in the state space. I give an example of an oddly shaped set and one possible implicit surface representation of that set at the bottom of the slide.
Implicit surface functions like J are not unique, but if we place the additional restriction on J that its gradient have magnitude one, then we can deduce two additional bits of information from J at each point in the state space: the distance and direction of the nearest point on the boundary of the set. This information proves useful in the context of synthesizing safe control signals, such as the video game that I showed earlier.
March 2008
Ian Mitchell (UBC Computer Science)

6. Continuous Backward Reachable Tubes
Set of all states from which trajectories can reach some given target state
For example, what states can reach G(0)?
Continuous System Dynamics
Target Set G(0)
Backward Reachable Set G(t)
March 2008
Ian Mitchell (UBC Computer Science)

7. Reachable Tubes (controlled input)
For most of our examples, target set is unsafe
If we can control the input, choose it to avoid the target set
Backward reachable set is unsafe no matter what we do
“Minimal” backward reach tube
Continuous System Dynamics
March 2008
Ian Mitchell (UBC Computer Science)

8. Reachable Tubes (uncontrolled input)
Sometimes we have no control over input signal
noise, actions of other agents, unknown system parameters
It is safest to assume the worst case
“Maximal” backward reach tube
Continuous System Dynamics
March 2008
Ian Mitchell (UBC Computer Science)

9. Ian Mitchell (UBC Computer Science)
Two Competing Inputs
For some systems there are two classes of inputs  = (u,d)
Controllable inputs u  U
Uncontrollable (disturbance) inputs d  D
Equivalent to a zero sum differential game formulation
If there is an advantage to input ordering, give it to disturbances
Continuous System Dynamics
March 2008
Ian Mitchell (UBC Computer Science)

10. Ian Mitchell (UBC Computer Science)
Objective Function
Extends in obvious way to the additional input
eg: discrete time discounted with fixed finite horizon tf
eg: continuous time no discount with target set T
First, let’s look at how I represent sets. The implicit surface representation has attracted lots of attention over the past few decades in computer science and engineering, because it handles general sets so gracefully. We just define a scalar function J such that J is negative inside the set and positive outside. Constructive geometry is easily accomplished on such sets; for example, the union of two sets is described implicitly by the taking minimum of their two implicit functions at each point in the state space. I give an example of an oddly shaped set and one possible implicit surface representation of that set at the bottom of the slide.
Implicit surface functions like J are not unique, but if we place the additional restriction on J that its gradient have magnitude one, then we can deduce two additional bits of information from J at each point in the state space: the distance and direction of the nearest point on the boundary of the set. This information proves useful in the context of synthesizing safe control signals, such as the video game that I showed earlier.
March 2008
Ian Mitchell (UBC Computer Science)

11. Ian Mitchell (UBC Computer Science)
Who Goes First?
One input is chosen to maximize and the other to minimize the objective
But what knowledge is available when choosing an input?
Current state? Other input?
“Non-anticipative strategies”
One player gets to know the other player’s input value (as well as current state)
However, that player must declare their strategy (reaction to every input) in advance
First, let’s look at how I represent sets. The implicit surface representation has attracted lots of attention over the past few decades in computer science and engineering, because it handles general sets so gracefully. We just define a scalar function J such that J is negative inside the set and positive outside. Constructive geometry is easily accomplished on such sets; for example, the union of two sets is described implicitly by the taking minimum of their two implicit functions at each point in the state space. I give an example of an oddly shaped set and one possible implicit surface representation of that set at the bottom of the slide.
Implicit surface functions like J are not unique, but if we place the additional restriction on J that its gradient have magnitude one, then we can deduce two additional bits of information from J at each point in the state space: the distance and direction of the nearest point on the boundary of the set. This information proves useful in the context of synthesizing safe control signals, such as the video game that I showed earlier.
March 2008
Ian Mitchell (UBC Computer Science)

12. Zero Sum Game Value Function
Value function is then defined as optimization over appropriate strategy and input signal pair
“Lower” value function, since disturbance (minimizer) has the advantage
Parallel “upper” value function can be defined
If inputs are independent, optimal strategy will ignore additional information about the other input
Upper and lower value functions will be equal
First, let’s look at how I represent sets. The implicit surface representation has attracted lots of attention over the past few decades in computer science and engineering, because it handles general sets so gracefully. We just define a scalar function J such that J is negative inside the set and positive outside. Constructive geometry is easily accomplished on such sets; for example, the union of two sets is described implicitly by the taking minimum of their two implicit functions at each point in the state space. I give an example of an oddly shaped set and one possible implicit surface representation of that set at the bottom of the slide.
Implicit surface functions like J are not unique, but if we place the additional restriction on J that its gradient have magnitude one, then we can deduce two additional bits of information from J at each point in the state space: the distance and direction of the nearest point on the boundary of the set. This information proves useful in the context of synthesizing safe control signals, such as the video game that I showed earlier.
March 2008
Ian Mitchell (UBC Computer Science)

13. Competing Inputs: Final Comments
Feedback control is more realistic implementation
If order of input decision is irrelevant (upper and lower value functions are equal), then nonanticipative strategy results will be equivalent to feedback results
For robustness, give advantage (eg strategy) to the disturbance input if it matters (potentially pessimistic)
Input signals still drawn from set of measureable functions
Two player concepts have been extended to viability theory and set-valued analysis
First, let’s look at how I represent sets. The implicit surface representation has attracted lots of attention over the past few decades in computer science and engineering, because it handles general sets so gracefully. We just define a scalar function J such that J is negative inside the set and positive outside. Constructive geometry is easily accomplished on such sets; for example, the union of two sets is described implicitly by the taking minimum of their two implicit functions at each point in the state space. I give an example of an oddly shaped set and one possible implicit surface representation of that set at the bottom of the slide.
Implicit surface functions like J are not unique, but if we place the additional restriction on J that its gradient have magnitude one, then we can deduce two additional bits of information from J at each point in the state space: the distance and direction of the nearest point on the boundary of the set. This information proves useful in the context of synthesizing safe control signals, such as the video game that I showed earlier.
March 2008
Ian Mitchell (UBC Computer Science)

14. Optimal Control and Reachability with Competing Inputs
For more information contact
Ian Mitchell
Department of Computer Science
The University of British Columbia
Good afternoon. I’ll be talking to you today about level set methods for analyzing, verifying and controlling continuous and hybrid systems. So obviously, I’ll have to explain what I mean by reachability, and what is a hybrid system. I’ll show how reachability can be used to analyze, verify and create controllers for various types of systems.
This is joint work with Professor Claire Tomlin in Aero/Astro, Professor Ron Fedkiw in Computer Science, and with students in their labs at Stanford.