1. An Introduction to the Soft Walls Project Adam Cataldo Prof. Edward Lee University of Pennsylvania Dec 18, 2003 Philadelphia, PA

2. Outline The Soft Walls system Objections Control system design Current Research Conclusions

3. Introduction On-board database with “no-fly-zones” Enforce no-fly zones using on-board avionics Non-networked, non-hackable

4. Autonomous control Pilot Aircraft Autonomous controller

5. Softwalls is not autonomous control Pilot Aircraft Softwalls + bias pilot control

6. Relation to Unmanned Aircraft Not an unmanned strategy –pilot authority Collision avoidance

7. A deadly weapon? Project started September 11, 2001

8. Design Objectives Maximize Pilot Authority!

9. Unsaturated Control No-fly zone Control applied Pilot lets off controls Pilot turns away from no-fly zone Pilot tries to fly into no-fly zone

10. Objections Reducing pilot control is dangerous –reduces ability to respond to emergencies

11. There is No Emergency That Justifies Landing Here

12. Objections Reducing pilot control is dangerous –reduces ability to respond to emergencies There is no override –switch in the cockpit

13. Hardwall

14. Objections Reducing pilot control is dangerous –reduces ability to respond to emergencies There is no override –switch in the cockpit Localization technology could fail –GPS can be jammed

15. Localization Backup Inertial navigation Integrator drift limits accuracy range

16. Objections Reducing pilot control is dangerous –reduces ability to respond to emergencies There is no override –switch in the cockpit Localization technology could fail –GPS can be jammed Deployment could be costly –Software certification? Retrofit older aircraft?

17. Deployment Fly-by-wire aircraft –a software change Older aircraft –autopilot level Phase in –prioritize airports

18. Objections Reducing pilot control is dangerous –reduces ability to respond to emergencies There is no override –switch in the cockpit Localization technology could fail –GPS can be jammed Deployment could be costly –how to retrofit older aircraft? Complexity –software certification

19. Not Like Air Traffic Control Much Simpler No need for air traffic controller certification

20. Objections Reducing pilot control is dangerous –reduces ability to respond to emergencies There is no override –switch in the cockpit Localization technology could fail –GPS can be jammed Deployment could be costly –how to retrofit older aircraft? Deployment could take too long –software certification Fully automatic flight control is possible –throw a switch on the ground, take over plane

21. Potential Problems with Ground Control Human-in-the-loop delay on the ground –authorization for takeover –delay recognizing the threat Security problem on the ground –hijacking from the ground? –takeover of entire fleet at once? –coup d’etat? Requires radio communication –hackable –jammable

22. Here’s How It Works

23. What We Want to Compute No-fly zone Backwards reachable set States that can reach the no-fly zone even with Soft Walls controlller Can prevent aircraft from entering no-fly zone

24. What We Create The terminal payoff function l:X -> Reals The further from the no-fly zone, the higher the terminal payoff payoff northward position eastward position no-fly zone (constant over heading angle) - +

25. What We Compute No-fly zone terminal payoff Backwards Reachable Set optimal payoff

26. Our Control Input Backwards Reachable Set optimal control at boundary dampen optimal control away from boundary State Space optimal payoff function

27. How we computing the optimal payoff (analytically) We solve this equation for J*: Reals^n x (- ∞, 0] -> Reals J* is the viscosity solution of this equation J* converges pointwise to the optimal payoff as T-> ∞ (Tomlin, Lygeros, Pappas, Sastry) dynamicsspatial gradient time derivative terminal payoff

28. (Mitchell) Computationally intensive: n states  O(2^n) How we computing the optimal payoff (numerically) northward position eastward position heading angle no-fly zone time 01M

29. Current Research Model predictive controller –Linearize the system –Discretize time –Compute an optimal control input for the next N steps –Use the optimal input at the current step –Recompute at the next step Feasible in real time

30. Current Research Discrete Abstraction –Discretize time –Discretize space into a finite set Feasible for Verification northward position eastward position heading angle 1 2 3 4

31. Conclusions Embedded control system challenge Control theory identified Future design challenges identified

32. Acknowledgements Ian Mitchell Xiaojun Liu Shankar Sastry Steve Neuendorffer Claire Tomlin

33. State space X, a subset of Reals^n Control space U and disturbance Space D, compact subsets of Reals^u and Reals^d Backup Slides—Problem Model northward position eastward position heading angle U D bank right - bank left + pilot Soft Walls

34. Actions in Continuous Time Pilot (Player 1) action d:Reals -> D, a Lebesgue measurable function The set of all pilot actions Dt time bank left (bank right)

35. Actions in Continuous Time Controller (Player 2) action u:Reals -> U, a Lebesgue measurable function The set of all controller actions Ut time bank left (bank right)

36. Dynamics The state trajectory x:Reals -> X The dynamics equation f:X x U x D -> Reals^n (d/dt)x(t) = f(x(t), u(t), d(t)) eastward position northward position heading angle trajectory

37. Cataldo Information Set At each time t, the controller knows pilot’s current action and the aircraft state We say s:Dt -> Ut is the controller’s nonanticipative strategy S is the set of nonanticipative strategies eastward position northward position heading angle trajectory time pilot bank left (bank right) controller knows

38. Information Set At each time t, the pilot knows pilot’s only the aircraft state We say the pilot uses a feedback strategy eastward position northward position heading angle trajectory pilot knows

39. What We Want to Compute No-fly zone Backwards reachable set States that can reach the no-fly zone even with Soft Walls controlller Can prevent aircraft from entering no-fly zone

40. The Terminal Payoff Function The terminal cost function l:X -> Reals The further from the no-fly zone, the higher the terminal payoff payoff northward position eastward position no-fly zone (constant over heading angle) - +

41. The Payoff Function Given the pilot action d and the control action u, the payoff is V: X x Ut X Dt -> Reals Aircraft state at time t is x(t) y is any state distance from no-fly zone time trajectories that terminate at y given u and d as inputs payoff at y given u and d

42. The Optimal Payoff At state y, the optimal payoff V*:X -> Reals comes from the control strategy s*:X ->S which maximizes the payoff for a disturbance which minimizes the payoff No-fly zone optimal control strategy optimal disturbance strategy

43. The Finite Duration Payoff We look back no more than T seconds to compute the finite duration payoff J: X x Ut x Dt x (- ∞, 0] -> Reals and the optimal finite duration payoff J*:X x (- ∞, 0] -> Reals the only difference

44. Computing V* Assuming J* is continuous, it is the viscosity solution of J* converges pointwise to V* as T-> ∞ (Tomlin, Lygeros, Pappas, Sastry) dynamicsspatial gradient finite duration optimal payoff

45. What Our Computation Gives Us No-fly zone terminal payoff Backwards Reachable Set optimal payoff

46. Control from Optimal Payoff Function Backwards Reachable Set optimal control at boundary dampen optimal control away from boundary State Space

47. (Mitchell) Computationally Intensive n states, O(2^n) Numerical Computation of V* northward position eastward position heading angle no-fly zone time 01M

48. Same state space X Same control space U and disturbance Space D What About Discrete-Time? northward position eastward position heading angle U D bank right - bank left + pilot Soft Walls

49. Actions in Discrete Time Pilot (Player 1) action d:Integers -> D Control (Player 2) action u:Integers -> U The set of all pilot actions Dk The set of all control actions Uk time bank left (bank right) time bank left (bank right)

50. Dynamics The state trajectory x:Integers -> X The dynamics equation f:X x U x D -> Reals^n x(k+1) = f(x(k), u(k), d(k)) eastward position northward position heading angle trajectory

51. Same Information Set eastward position northward position heading angle trajectory time pilot bank left (bank right) controller knows pilot knows We say s:Dk -> Uk is the controller’s nonanticipative strategy S is the set of nonanticipative strategies

52. Again We Compute No-fly zone Backwards reachable set Can prevent aircraft from entering no-fly zone

53. Same Terminal Payoff Function The terminal cost function l:X -> Reals The further from the no-fly zone, the higher the terminal payoff payoff northward position eastward position no-fly zone (constant over heading angle) - +

54. The Payoff Function Given the pilot action d and the control action u, the payoff is V: X x Ut X Dt -> Reals Aircraft state at time k is x(k) y is any state distance from no-fly zone time trajectories that terminate at y given u and d as inputs payoff at y given u and d

55. The Optimal Payoff At state y, the optimal payoff V*:X -> Reals comes from the control strategy s*:X ->S which maximizes the payoff for a disturbance which minimizes the payoff No-fly zone optimal control strategy optimal disturbance strategy

56. The Finite Duration Payoff We look back no more than K seconds to compute the finite duration payoff J: X x Ut x Dt x {…, -1, 0} -> Reals and the optimal finite duration payoff J*:X x {…, -1, 0} -> Reals the only difference

57. Computing V* J* is the solution of J* converges pointwise to V* as K-> ∞ dynamics finite duration optimal payoff

58. How Do We Solve This Numerically? No-fly zone terminal payoff Backwards Reachable Set optimal payoff