1. Reachability-based Controller Design for Switched Nonlinear Systems EE 291E / ME 290Q Jerry Ding 4/18/2012

2. Hierarchical Control Designs To manage complexity, design of modern control systems commonly done in hierarchical fashion e.g. aircraft, automobiles, industrial machinery Low level control tend to use continuous abstractions and design methods ODE model Stability, trajectory tracking Linear/Nonlinear control methods High level control tend to use discrete abstractions and design methods Finite state automata, discrete event systems Logic specifications of qualitative behaviors: e.g. LTL Model checking, supervisory control 2

3. Challenges of Interfacing Layers of Control Problem becomes more difficult at interface: Closed loop behavior results from composition of discrete and continuous designs Discrete behaviors may not be implemented exactly by continuous controllers Continuous designs may be unaware of high level specifications In safety-critical control applications, specifications often involves stringent requirements on closed-loop behavior Current design approaches involve a mixture of heuristics and extensive verification and validation 3

4. Hybrid Systems Approach Capture closed-loop system behavior through hybrid system abstraction 4

5. Hybrid Systems Approach Formulate design methods within the framework of hybrid system theory 5 Challenges: Nonlinear dynamics, possibly with disturbances Controlled switching: switching times, switching sequence, switching policy Autonomous switching: discontinuous vector fields, state resets

6. Reachability-Based Design for Switched Systems Consider subclass of hybrid systems with: Controlled switches, no state resets –Fixed mode sequence –Variable mode sequence Nonlinear continuous dynamics, subject to bounded disturbances Design controllers to satisfy reachability specifications Reach-avoid problem: Given target set R, avoid set A, design a controller to reach R while avoiding A Methods based upon game theoretic framework for general hybrid controller design [Lygeros, et al., Automatica, 1999] [Tomlin, et al., Proceedings of the IEEE, 2000] 6

7. Outline Switched Systems with Fixed Mode Sequences: Design of Safe Maneuver Sequence for Automated Aerial Refueling (AAR) Switched Systems with Variable Mode Sequences: Sampled-data switched systems Controller synthesis algorithm for reach-avoid problem Application example: STARMAC quadrotor experiments 7

8. 8 Automated Aerial Refueling Procedures

9. 9 Discrete Transitions Detach 1 PrecontactContactPostcontact Detach 2 Rejoin Start End High Level Objective: Visit waypoint sets R i, i = 1,…,6, in sequence

10. 10 Continuous Dynamics Relative States: x1, x2 = planar coordinates of tanker in UAV reference frame x3 = heading of tanker relative to UAV Controlled inputs: u1 = translational speed of UAV u2 = turn rate of UAV Disturbance inputs: d1 = translational speed of Tanker d2 = turn rate of Tanker Low Level Objective: Avoid protected zone A around tanker aircraft

11. 11 Maneuver Sequence Design Problem Given waypoint sets R i, protected zone A, design continuous control laws K i (x) and switching policies F i (x) such that 1) The hybrid state trajectory (q, x) passes through the waypoint sets q i × R i in sequence 2) The hybrid state trajectory (q, x) avoids the protected zones q i × A at all times Design approach: Select switching policy as follows: in maneuver q i, switch to next maneuver if waypoint R i is reached Use reachable sets as design tool for ensuring –safety and target attainability objectives for each maneuver –compatibility conditions for switching between maneuvers

12. 12 Capture sets and Unsafe sets

13. 13 Computation of Reachable Sets Unsafe set computation (Mitchell, et al. 2005): Let be the viscosity solution of Use terminal condition to encode avoid set Then Capture set computation similar Numerical toolbox for MATLAB is available to approximate solution [Ian Mitchell, http://www.cs.ubc.ca/~mitchell/ToolboxLS/, 2007]

14. Maneuver Design Using Reachability Analysis 14 For mode q N 1) Design a control law to drive R N -1 to R N 2) Compute capture set to first time instant  N such that

15. Maneuver Design Using Reachability Analysis 15 For mode q N 3) Compute unsafe set, and verify safety condition Modify control law design as necessary

16. Maneuver Design Using Reachability Analysis 16 For modes q k, k < N 3) Iterate procedures 1-3 recursively For q 1, R 0 = X 0, where X 0 is the initial condition set

17. Properties of Control Law 17 Continuous control laws designed in this manner satisfy a reach-avoid specification for each maneuver: Reach waypoint set R i at some time, while avoiding protected zone A at all times Furthermore, they satisfy a compatibility condition between maneuvers This ensures that whenever a discrete switch take place, the specifications of next maneuver is feasible Execution time of refueling sequence is upper bounded by

18. 18 Specifications for Aerial Refueling Procedure Target Sets of the form Avoid sets of the form Control laws of the form

19. 19 Capture Set and Unsafe Set Computation Result Precontact (Mode q 2 ) Time Horizon

20. 20 Simulation of Refueling Sequence Input bounds Target Set Radius Collision Set Radius Collision Zone Unsafe Set For Detach 1 Target Set Capture Set For Detach 1

21. Accounting for Disturbances 21 Capture sets and unsafe sets can be modified to account for fluctuations in tanker velocity using disturbance set Collision Zone In UAV Coordinates Unsafe set for contact maneuver without disturbances Unsafe set for contact maneuver with 10% velocity deviation Reachable set slice at relative angle 0 Rescaled coordinates: distance units in tens of meters

22. Outline Switched Systems with Fixed Mode Sequences: Design of Safe Maneuver Sequence for Automated Aerial Refueling (AAR) Switched Systems with Variable Mode Sequences: Sampled-data switched systems Controller synthesis algorithm for reach-avoid problem Application example: STARMAC quadrotor experiments 22

23. Switched System Model – Dynamics 23 Continuous Dynamics Continuous State Space Discrete State Space Reset Relations

24. Switched System Model – Inputs 24 0T2T3T4T5T Piece-wise constant Time- Varying Sampled-data system for practical implementation Quantized input for finite representation of control policy Switching Signal Continuous Input Disturbance

25. Switched System Model – Control and Disturbance Policies On sampling interval [kT, (k+1)T], define 25 One step control policy One step disturbance strategy kT (k+1)T kT (k+1)T

26. Outline Switched Systems with Fixed Mode Sequences: Design of Safe Maneuver Sequence for Automated Aerial Refueling (AAR) Switched Systems with Variable Mode Sequences: Sampled-data switched systems Controller synthesis algorithm for reach-avoid problem Application example: STARMAC quadrotor experiments 26

27. Problem Formulation Given: Switched system dynamics; for simplicity, assume that Target set R Avoid set A 27 Target setAvoid set Mode

28. Problem Formulation Compute set of states (q, x) that can be controlled to target set while staying away from avoid set over finite horizon Call this reach-avoid set 28 Reach-avoid setTarget setAvoid set Mode

29. For any (q, x) in the reach-avoid set, automatically synthesize a feedback policy that achieves the specifications Problem Formulation 29 Reach-avoid setTarget setAvoid set Mode

30. One Step Capture and Unsafe sets 30 For each, compute one step capture and unsafe sets assuming over one sampling interval where is solution ofon One step capture set One step unsafe set

31. Reach-avoid Set Computation – Step 1 31 For each, compute one step reach-avoid set using set difference Mode For sets represented by level set functions The set differenceis represented by

32. Reach-avoid Set Computation – Step 2 Compute feasible set for one step reach-avoid problem, by taking union over 32 Mode For sets represented by level set functions The set union is represented by

33. Reach-avoid Set Computation – Iteration Iterate to compute the reach-avoid set over [0,NT] By induction, can show that Initialization: forto end Return: 33

34. Reach-avoid control law synthesis 34 At time k < N Step 2: Find minimum time to reach Step 1: Obtain state measurement

35. Reach-avoid control law synthesis 35 At time k < N Step 3: Find a control input such that Step 4: Apply input and iterate steps 1-3

36. Explicit Form of Control Laws 36 Explicit control laws given by Number of reachable sets required is given by Length of time horizon Number of discrete modes Number of quantization levels in mode q i where for

37. Outline Switched Systems with Fixed Mode Sequences: Design of Safe Maneuver Sequence for Automated Aerial Refueling (AAR) Switched Systems with Variable Mode Sequences: Sampled-data switched systems Controller synthesis algorithm for reach-avoid problem Application example: STARMAC quadrotor experiments 37

38. STARMAC Quadrotor Platform 38 Ultrasonic Ranger Senscomp Mini-AE Ultrasonic Ranger Senscomp Mini-AE Inertial Meas. Unit Microstrain 3DM-GX1 Inertial Meas. Unit Microstrain 3DM-GX1 GPS Novatel Superstar II GPS Novatel Superstar II Low Level Control Atmega128 Low Level Control Atmega128 Carbon Fiber Tubing Fiberglass Honeycomb Sensorless Brushless DC Motors Axi 2208/26 Sensorless Brushless DC Motors Axi 2208/26 Electronic Speed Controllers Castle Creations Phoenix-25 Electronic Speed Controllers Castle Creations Phoenix-25 Battery Lithium Polymer Battery Lithium Polymer High Level Control Gumstix PXA270, or ADL PC104 High Level Control Gumstix PXA270, or ADL PC104

39. Experiment Setup 39 Objectives: Drive a quadrotor to a neighborhood of 2D location in finite time, while satisfying velocity bounds Disturbances: model uncertainty, actuator noise System model

40. Reach-avoid Problem Set-Up Target Set: +/- 0.2 m for position, +/- 0.2 m/s for velocity Avoid Set: +/- 1 m/s for velocity Time Step: 0.1 seconds, 25 time steps Pitch and roll commands: Disturbance bounds: 40

41. Reach-avoid Set - Plots 41

42. Reach-avoid Set - Plots 42 Reach-avoid at Time Step 1 for All Inputs

43. Reach-avoid Set - Plots 43

44. Experimental Results 44

45. Experimental Results 45

46. Experimental Results 46 Moving car experiment

47. References John Lygeros, Claire Tomlin, and S. Shankar Sastry. Controllers for reachability specifications for hybrid systems. Automatica, 35(3):349 – 370, 1999. Claire J. Tomlin, John Lygeros, and S. Shankar Sastry. A game theoretic approach to controller design for hybrid systems. Proceedings of the IEEE, 88(7):949–970, July 2000. Jerry Ding, Jonathan Sprinkle, S. Shankar Sastry, and Claire J. Tomlin. Reachability calculations for automated aerial refueling. In 47th IEEE Conference on Decision and Control, pages 3706–3712, Dec. 2008. Jerry Ding, Jonathan Sprinkle, Claire Tomlin, S. Shankar Sastry, and James L. Paunicka. Reachability calculations for vehicle safety during manned/unmanned vehicle interaction. AIAA Journal of Guidance, Control, and Dynamics, 35(1):138–152, 2012. 47

48. References Jerry Ding and Claire J. Tomlin. Robust reach-avoid controller synthesis for switched nonlinear systems. In 49th IEEE Conference on Decision and Control (CDC), pages 6481–6486, Dec. 2010. Jerry Ding, Eugene Li, Haomiao Huang, and Claire J. Tomlin. Reachability-based synthesis of feedback policies for motion planning under bounded disturbances. In IEEE International Conference on Robotics and Automation (ICRA), pages 2160 – 2165, May 2011. 48