1. Hybrid Systems Controller Synthesis Examples EE291E Tomlin

2. Backwards Reachable Set All states for which, for all possible control actions, there is a disturbance action which can drive the system state into a region G(0) in time t Backwards Reachable Set Reachability as game: disturbance attempts to force system into unsafe region, control attempts to stay safe

3. Reachable Set Propagation [Mitchell, Bayen, Tomlin 2005] Theorem [Computing ]: where is the unique Crandall-Evans-Lions viscosity solution to:

4. Backwards Reachable Set: Safety unsafe Backwards Reachable Set On boundary, apply control to stay out of red In red, system may become unsafe In blue, system will stay safe Safety Property can be encoded as a condition on the system’s reachable set of states

5. Example 1: Aircraft Collision Avoidance Two identical aircraft at fixed altitude & speed: ‘evader’ (control) ‘pursuer’ (disturbance) x y u v  d v

6. Continuous Reachable Set  x y

7. Collision Avoidance Filter Simple demonstration –Pursuer: turn to head toward evader –Evader: turn to head right pursuer safety filter’s input modification pursuer’s inputevader’s desired input evader evader’s actual input unsafe set collision set Movies…

8. Collision Avoidance Control http://www.cs.ubc.ca/~mitchell/ToolboxLS/

9. Overapproximating Reachable Sets [Khrustalev, Varaiya, Kurzhanski] Overapproximative reachable set: Exact: Approximate: ~1 sec on 700MHz Pentium III (vs 4 minutes for exact) Polytopic overapproximations for nonlinear games Subsystem level set functions “Norm-like” functions with identical strategies to exact [Hwang, Stipanović, Tomlin]

11. 1 23K modes 1 2 3 n iterations unsafe safe Computing Reach Sets for Hybrid Systems

12. Reach Sets: uncontrollable predecessor 1 23K modes 1 2 3 n iterations uncontrolled transition unsafe “safe”

13. Reach Sets: controllable predecessor 1 23K modes 1 2 3 n iterations safe controlled transition “safe”

14. Reach Sets: Variational Inequality 1 23K modes 1 2 3 n iterations States which reach G without hitting E first: where subject to

15. Reach Sets: Iterate 1 23K modes 1 2 3 n iterations

16. Can separation assurance be automated? Requires provably safe protocols for aircraft interaction Must take into account: Uncertainties in sensed information, in actions of the other vehicle Potential loss of communication Intent, or non-intent

17. unsafe set with choice to maneuver or not? Example 2: Protocol design unsafe set with maneuver unsafe set without maneuver ? unsafe safe

18. Protocol Safety Analysis Ability to choose maneuver start time further reduces unsafe set safe without switch unsafe to switch safe with switch unsafe with or without switch

19. Implementation: a finite automaton It can be easier to analyze discrete systems than continuous: use reachable set information to abstract away continuous details q1q1 safe at present will become unsafe unsafe to  1 q5q5 safe at present always safe safe to  1 q3q3 safe at present will become unsafe safe to  1 q4q4 safe at present always safe unsafe to  1 q2q2 unsafe at present will become unsafe unsafe to  1 qsqs SAFE ququ UNSAFE forced transition controlled transition (  1 ) q1q1 q5q5 q3q3 ququ q4q4 q2q2

20. San Francisco Airport 750 ft separation Example 2: Closely Spaced Parallel Approaches

21. Example 3: Closely Spaced Approaches evader EEM Maneuver 1: accelerate EEM Maneuver 2: turn 45 deg, accelerate EEM Maneuver 3: turn 60 deg [Rodney Teo]

22. Sample Trajectories Segment 1 Segment 2 Segment 3

23. Dragonfly 3Dragonfly 2 Ground Station Tested on the Stanford DragonFly UAVs

24. EEM alert Separation distance (m) North (m) East (m) time (s) Above threshold Accelerate and turn EEM Put video here Tested at Moffett Federal Airfield

25. EEM alert Separation distance (m) North (m) East (m) time (s) Above threshold Put video here Coast and turn EEM Tested at Moffett Federal Airfield

26. Tested at Edwards Air Force Base T-33 Cockpit [DARPA/Boeing SEC Final Demonstration: F-15 (blunderer), T-33 (evader)]

27. Photo courtesy of Sharon Houck; Tests conducted with Chad Jennings

28. Implementation: Display design courtesy of Chad Jennings, Andy Barrows, David Powell R. Teo’s Blunder Zone is shown by the yellow contour Red Zone in the green tunnel is the intersection of the BZ with approach path. The Red Zone corresponds to an assumed 2 second pilot delay. The Yellow Zone corresponds to an 8 second pilot delay

29. R. Teo’s Blunder Zone is shown by the yellow contour Red Zone in the green tunnel is the intersection of the BZ with approach path. The Red Zone corresponds to an assumed 2 second pilot delay. The Yellow Zone corresponds to an 8 second pilot delay

30. Map View showing a blunder The BZ calculations are performed in real time (40Hz) so that the contour is updated with each video frame.

31. Map View with Color Strips The pilots only need to know which portion of their tunnel is off limits. The color strips are more efficient method of communicating the relevant extent of the Blunder zone

32. Experimental Platform: STARMAC The Stanford Testbed of Autonomous Rotorcraft for Multi-Agent Control

33. Example 4: Collision Avoidance Pilots instructed to attempt to collide vehicles

34. Aircraft must stay within safe flight envelope during landing: –Bounds on velocity ( ), flight path angle (  ), height ( ) –Control over engine thrust ( ), angle of attack (  ), flap settings –Model flap settings as discrete modes –Terms in continuous dynamics depend on flap setting Example 5: Aircraft Autolander inertial frame wind frame body frame

36. Autolander: Synthesizing Control For states at the boundary of the safe set, results of reach-avoid computation determine –What continuous inputs (if any) maintain safety –What discrete jumps (if any) are safe to perform –Level set values and gradients provide all relevant data

37. Application to Autoland Interface Controllable flight envelopes for landing and Take Off / Go Around (TOGA) maneuvers may not be the same Pilot’s cockpit display may not contain sufficient information to distinguish whether TOGA can be initiated flare flaps extended minimum thrust rollout flaps extended reverse thrust slow TOGA flaps extended maximum thrust TOGA flaps retracted maximum thrust flare flaps extended minimum thrust rollout flaps extended reverse thrust TOGA flaps retracted maximum thrust revised interface existing interface controllable flare envelope controllable TOGA envelope intersection

38. Aircraft Simulator Tests Setup –Commercial flight simulator, B767 pilot –Digital video of primary flight display Maneuver –Go-around at low speed, high descent rate Goal –Determine whether problematic behavior predicted by our model is possible in aircraft flight simulator

39. Aircraft Simulator Results Produced unexpected behavior Non-standard procedure; Unable to duplicate Validated types of problems addressed by this method

40. Backwards Reachable Set: Safety unsafe Backwards Reachable Set On boundary, apply control to stay out of red In red, system may become unsafe In blue, system will stay safe Safety Property can be encoded as a condition on the system’s reachable set of states

41. Backwards Reachable Set: Capture desired Backwards Reachable Set Capture property can also be encoded as a condition on the system’s reachable set of states

42. Maneuver sequencing, “Reachavoid” Target Set Maneuver sequencing is accomplished by stringing together capture sets, starting from the target set and working backwards Avoid sets can be combined with capture sets to guarantee safety Unsafe Set

43. Example 5: Quadrotor Back-Flip Divide flip into three modes Difficult problem: –Hitting some target sets while avoiding some unsafe sets Solution: –Analyze rotational dynamics and vertical dynamics separately Impulse DriftRecovery

44. Back-flip: Method (1) Recovery Drift Impulse Identify target region in rotational state space for each mode Use reachable sets to calculate capture basin for each target –Dynamic game formulation accounts for worst-case disturbances Verify that target of each mode is contained by capture basin of next mode

45. Back-flip: Method (2) Identify unsafe region in vertical state space for final mode Use reachable sets to propagate unsafe set for each mode –Dynamic game formulation accounts for worst-case disturbances Verify that control keeps state out of unsafe set

47. Back-Flip: Results

48. Assumptions Validated Safety Guaranteed Reachability Demonstrated

49. Example 6: Automated aerial refueling Desired Target Set

50. Capture Set and Unsafe Set Computation Result

51. Example 7: Teaming up humans and robots http://www.goforyourlife.vic.gov.au/hav/articles.nsf/pages/Capture_the_Flag Multiple players Adversarial game Limited Information Multiple objectives

52. Flag Capture Only Flag Return Only Full Game “Capture the Flag”

53. Action Support For Human Agents Undergraduate Team Scott Hoag Andrew Sy The computed solution can be used to guide and assist human agents. attacker defender

54. Supporting Complex Actions Reachable sets also assist and enable more complex actions and strategic decision making. attacker defender In this case reachability information helps the attacker mislead the defender to win from a losing initial configuration.

55. Reachability-Guided UAV Search UAV Visibility Attacker Defender Attacker Goal Defender Winning Region Attacker Winning Region Attacker Visibility Possible Defender Locations