An Introduction to the Soft Walls Project Adam Cataldo Prof. Edward Lee University of Pennsylvania Dec 18, 2003 Philadelphia, PA

Outline The Soft Walls system Objections Control system design Current Research Conclusions

Introduction On-board database with “no-fly-zones” Enforce no-fly zones using on-board avionics Non-networked, non-hackable

Autonomous control Pilot Aircraft Autonomous controller

Softwalls is not autonomous control Pilot Aircraft Softwalls + bias pilot control

Relation to Unmanned Aircraft Not an unmanned strategy –pilot authority Collision avoidance

A deadly weapon? Project started September 11, 2001

Design Objectives Maximize Pilot Authority!

Unsaturated Control No-fly zone Control applied Pilot lets off controls Pilot turns away from no-fly zone Pilot tries to fly into no-fly zone

Objections Reducing pilot control is dangerous –reduces ability to respond to emergencies

There is No Emergency That Justifies Landing Here

Objections Reducing pilot control is dangerous –reduces ability to respond to emergencies There is no override –switch in the cockpit

Hardwall

Objections Reducing pilot control is dangerous –reduces ability to respond to emergencies There is no override –switch in the cockpit Localization technology could fail –GPS can be jammed

Localization Backup Inertial navigation Integrator drift limits accuracy range

Objections Reducing pilot control is dangerous –reduces ability to respond to emergencies There is no override –switch in the cockpit Localization technology could fail –GPS can be jammed Deployment could be costly –Software certification? Retrofit older aircraft?

Deployment Fly-by-wire aircraft –a software change Older aircraft –autopilot level Phase in –prioritize airports

Objections Reducing pilot control is dangerous –reduces ability to respond to emergencies There is no override –switch in the cockpit Localization technology could fail –GPS can be jammed Deployment could be costly –how to retrofit older aircraft? Complexity –software certification

Not Like Air Traffic Control Much Simpler No need for air traffic controller certification

Objections Reducing pilot control is dangerous –reduces ability to respond to emergencies There is no override –switch in the cockpit Localization technology could fail –GPS can be jammed Deployment could be costly –how to retrofit older aircraft? Deployment could take too long –software certification Fully automatic flight control is possible –throw a switch on the ground, take over plane

Potential Problems with Ground Control Human-in-the-loop delay on the ground –authorization for takeover –delay recognizing the threat Security problem on the ground –hijacking from the ground? –takeover of entire fleet at once? –coup d’etat? Requires radio communication –hackable –jammable

Here’s How It Works

What We Want to Compute No-fly zone Backwards reachable set States that can reach the no-fly zone even with Soft Walls controlller Can prevent aircraft from entering no-fly zone

What We Create The terminal payoff function l:X -> Reals The further from the no-fly zone, the higher the terminal payoff payoff northward position eastward position no-fly zone (constant over heading angle) - +

What We Compute No-fly zone terminal payoff Backwards Reachable Set optimal payoff

Our Control Input Backwards Reachable Set optimal control at boundary dampen optimal control away from boundary State Space optimal payoff function

How we computing the optimal payoff (analytically) We solve this equation for J*: Reals^n x (- ∞, 0] -> Reals J* is the viscosity solution of this equation J* converges pointwise to the optimal payoff as T-> ∞ (Tomlin, Lygeros, Pappas, Sastry) dynamicsspatial gradient time derivative terminal payoff

(Mitchell) Computationally intensive: n states  O(2^n) How we computing the optimal payoff (numerically) northward position eastward position heading angle no-fly zone time 01M

Current Research Model predictive controller –Linearize the system –Discretize time –Compute an optimal control input for the next N steps –Use the optimal input at the current step –Recompute at the next step Feasible in real time

Current Research Discrete Abstraction –Discretize time –Discretize space into a finite set Feasible for Verification northward position eastward position heading angle

Conclusions Embedded control system challenge Control theory identified Future design challenges identified

Acknowledgements Ian Mitchell Xiaojun Liu Shankar Sastry Steve Neuendorffer Claire Tomlin

State space X, a subset of Reals^n Control space U and disturbance Space D, compact subsets of Reals^u and Reals^d Backup Slides—Problem Model northward position eastward position heading angle U D bank right - bank left + pilot Soft Walls

Actions in Continuous Time Pilot (Player 1) action d:Reals -> D, a Lebesgue measurable function The set of all pilot actions Dt time bank left (bank right)

Actions in Continuous Time Controller (Player 2) action u:Reals -> U, a Lebesgue measurable function The set of all controller actions Ut time bank left (bank right)

Dynamics The state trajectory x:Reals -> X The dynamics equation f:X x U x D -> Reals^n (d/dt)x(t) = f(x(t), u(t), d(t)) eastward position northward position heading angle trajectory

Cataldo Information Set At each time t, the controller knows pilot’s current action and the aircraft state We say s:Dt -> Ut is the controller’s nonanticipative strategy S is the set of nonanticipative strategies eastward position northward position heading angle trajectory time pilot bank left (bank right) controller knows

Information Set At each time t, the pilot knows pilot’s only the aircraft state We say the pilot uses a feedback strategy eastward position northward position heading angle trajectory pilot knows

What We Want to Compute No-fly zone Backwards reachable set States that can reach the no-fly zone even with Soft Walls controlller Can prevent aircraft from entering no-fly zone

The Terminal Payoff Function The terminal cost function l:X -> Reals The further from the no-fly zone, the higher the terminal payoff payoff northward position eastward position no-fly zone (constant over heading angle) - +

The Payoff Function Given the pilot action d and the control action u, the payoff is V: X x Ut X Dt -> Reals Aircraft state at time t is x(t) y is any state distance from no-fly zone time trajectories that terminate at y given u and d as inputs payoff at y given u and d

The Optimal Payoff At state y, the optimal payoff V*:X -> Reals comes from the control strategy s*:X ->S which maximizes the payoff for a disturbance which minimizes the payoff No-fly zone optimal control strategy optimal disturbance strategy

The Finite Duration Payoff We look back no more than T seconds to compute the finite duration payoff J: X x Ut x Dt x (- ∞, 0] -> Reals and the optimal finite duration payoff J*:X x (- ∞, 0] -> Reals the only difference

Computing V* Assuming J* is continuous, it is the viscosity solution of J* converges pointwise to V* as T-> ∞ (Tomlin, Lygeros, Pappas, Sastry) dynamicsspatial gradient finite duration optimal payoff

What Our Computation Gives Us No-fly zone terminal payoff Backwards Reachable Set optimal payoff

Control from Optimal Payoff Function Backwards Reachable Set optimal control at boundary dampen optimal control away from boundary State Space

(Mitchell) Computationally Intensive n states, O(2^n) Numerical Computation of V* northward position eastward position heading angle no-fly zone time 01M

Same state space X Same control space U and disturbance Space D What About Discrete-Time? northward position eastward position heading angle U D bank right - bank left + pilot Soft Walls

Actions in Discrete Time Pilot (Player 1) action d:Integers -> D Control (Player 2) action u:Integers -> U The set of all pilot actions Dk The set of all control actions Uk time bank left (bank right) time bank left (bank right)

Dynamics The state trajectory x:Integers -> X The dynamics equation f:X x U x D -> Reals^n x(k+1) = f(x(k), u(k), d(k)) eastward position northward position heading angle trajectory

Same Information Set eastward position northward position heading angle trajectory time pilot bank left (bank right) controller knows pilot knows We say s:Dk -> Uk is the controller’s nonanticipative strategy S is the set of nonanticipative strategies

Again We Compute No-fly zone Backwards reachable set Can prevent aircraft from entering no-fly zone

Same Terminal Payoff Function The terminal cost function l:X -> Reals The further from the no-fly zone, the higher the terminal payoff payoff northward position eastward position no-fly zone (constant over heading angle) - +

The Payoff Function Given the pilot action d and the control action u, the payoff is V: X x Ut X Dt -> Reals Aircraft state at time k is x(k) y is any state distance from no-fly zone time trajectories that terminate at y given u and d as inputs payoff at y given u and d

The Optimal Payoff At state y, the optimal payoff V*:X -> Reals comes from the control strategy s*:X ->S which maximizes the payoff for a disturbance which minimizes the payoff No-fly zone optimal control strategy optimal disturbance strategy

The Finite Duration Payoff We look back no more than K seconds to compute the finite duration payoff J: X x Ut x Dt x {…, -1, 0} -> Reals and the optimal finite duration payoff J*:X x {…, -1, 0} -> Reals the only difference

Computing V* J* is the solution of J* converges pointwise to V* as K-> ∞ dynamics finite duration optimal payoff

How Do We Solve This Numerically? No-fly zone terminal payoff Backwards Reachable Set optimal payoff