1. Safe Control Strategies for Hopping Over Uneven Terrain Part I Brian Howley RiSE Group Meeting October 9, 2006

2. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 2 Outline Motivation and Approach Vertical Hopping Control Synthesis Problem Reach Optimization Application to Vertical Hopper Hopper Ruggedness Issues –Singularity Avoidance –Control dependent phase transitions

3. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 3 Motivation Biomimetic principles for legged robots have been demonstrated experimentally, but quantifiable design approaches have not been established Motivating questions include: –At what speeds and over what range of surface irregularities can a robot successfully traverse? –How will changes in control, mechanical properties, or leg morphology affect performance? –Under what conditions and to what extent will additional sensors be advantageous? Goal: develop ‘first principles’ insight into motion over uneven terrain Goal: develop ‘first principles’ insight into motion over uneven and uncertain terrain

4. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 4 Approach Start with the simplest problem of interest: single-legged vertical hopping – Well studied for level terrain Beuhler & Koditschek prove large stable operating regime (1988) Ringrose proposes ‘self-stabilizing’ open loop control (1997) –Topic largely unexplored for variable terrain Use game theoretic approach to robot-environment interaction –Determine worst case disturbances and optimize control –Quantify performance with respect to safe operating limits Develop general analysis tools –Adapt/refine synthesis approaches for hybrid dynamical systems –Numerical tool: Maximal Invariant Safe Subset (MISS) Algorithm Extend approach to more complex problems

5. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 5 Why Game Theory? Two fundamental ways to deal with uncertainty: –Random process approach Requires assumptions about statistical properties (or a whole lot of data). Can be difficult to interpret if usual assumptions about normal gaussian statistics don’t apply. –Worst case analysis Provides a conservative answer Justified for low probability/high cost events: e.g. safety Game Theory provides the worst case analysis for a dynamic system by assuming continuous and discrete disturbances are applied in the worst possible way

6. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 6 Vertical Hopping Control hopping height – 1 st problem in running –Well studied problem on level terrain Variable terrain while hopping in place –Safety constraints  Stumbling  Joint constraints Hopping as a game –Terrain height changes within +/- limits while hopper is at apex of flight Hopper ‘wins’ by maintaining safe operation Environment ‘wins’ otherwise –Determine optimal control and disturbance strategies –Determine optimal mechanical properties

7. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 7 Vertical Hopping Control hopping height – 1 st problem in running –Well studied problem on level terrain Variable terrain while hopping in place –Safety constraints  Stumbling  Joint constraints Hopping as a game –Terrain height changes within +/- limits while hopper is at apex of flight Hopper ‘wins’ by maintaining safe operation Environment ‘wins’ otherwise –Determine optimal control and disturbance strategies –Determine optimal mechanical properties

8. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 8 Vertical Hopping Control hopping height – 1 st problem in running –Well studied problem on level terrain Variable terrain while hopping in place –Safety constraints  Stumbling  Joint constraints Hopping as a game –Terrain height changes within +/- limits while hopper is at apex of flight Hopper ‘wins’ by maintaining safe operation Environment ‘wins’ otherwise –Determine optimal control and disturbance strategies –Determine optimal mechanical properties

9. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 9 Vertical Hopping Control hopping height – 1 st problem in running –Well studied problem on level terrain Variable terrain while hopping in place –Safety constraints  Stumbling  Joint constraints Hopping as a game –Terrain height changes within +/- limits while hopper is at apex of flight Hopper ‘wins’ by maintaining safe operation Environment ‘wins’ otherwise –Determine optimal control and disturbance strategies –Determine optimal mechanical properties ?

10. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 10 Hopper Dynamics Hybrid system –Discrete events change dynamics Flight phase –Ascent, Step, Descent phases accommodate changes in terrain height Contact phase –Thrusting may occur any time in contact –Liftoff transition (not shown) if ground reaction forces less than zero

11. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 11 Safety The safe set, F, is the subset of the discrete and continuous state space considered to be safe: The unsafe set, G, is the complement of F: Vertical Position Vertical Velocity Safe States, F Unsafe States, G (minimum leg length constraint) 0-0.5 0 Vertical Velocity Safe States, F 0-0.5 0 Vertical Position Unsafe States, G (‘Stumble’ Condition)

12. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 12 Control Synthesis Problem Given a safe set, F, determine (i) the maximal controlled invariant set contained in F, and (ii) the controller which renders this set invariant General Algorithm: Start with the safe set, F, and iterate backwards in time eliminating those states that can be rendered unsafe Reference: Tomlin, Claire, and Lygeros, John, and Sastry, Shankar, “A Game Theoretic Approach to Controller Design for Hybrid Systems”, Proceedings of the IEEE, Vol. 88, No. 7, July 2000.

13. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 13 Reach Operation Reach(G,E): the set of states that can be driven to the set of unsafe states, G, prior to reaching a (safe) escape set, E. –The Reach operation is formulated like a pair of pursuit-evasion games System dynamics: Two objective functions: q=‘Contact’ Vertical Velocity Vertical Position G  G (x)<0 E  E (x)<0 0-0.5 ( x >0, non ‘contact’ region)

14. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 14 Optimization-Reach Use optimal control theory to determine ‘least restrictive’ control and worst case disturbance inputs. –Hamilton-Jacobi equations: Since objective functions, J x, depends only on the terminal state, x(t f ), the optimal control and disturbance inputs, u*, d*, are such that the gradient of the objective function is perpendicular f q (x,u*,d*).  Don’t have to integrate the Hamilton-Jacobi equations!!

15. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 15 Optimization-Reach Vertical Position Vertical Velocity -0.50 E G Consider J G and J E optimization separately and combine results: let t f G < t f E J G Optimization at t=t f G, J G =  G q=‘Contact’

16. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 16 Optimization-Reach Vertical Position Vertical Velocity -0.50 E G Consider J G and J E optimization separately and combine results: let t f G < t f E q=‘Contact’ J G Optimization at t=t f G, J G =  G Gradient non-zero and perpendicular to boundary

17. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 17 Optimization-Reach Vertical Position Vertical Velocity -0.50 E G Consider J G and J E optimization separately and combine results: let t f G < t f E q=‘Contact’ Extrema point, x* J G Optimization at t=t f G, J G =  G Gradient non-zero and perpendicular to boundary Find extrema points, x*:

18. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 18 Optimization-Reach Vertical Position Vertical Velocity -0.50 E G Consider J G and J E optimization separately and combine results: let t f G < t f E q=‘Contact’ J G Optimization at t=t f G, J G =  G Gradient non-zero and perpendicular to boundary Find extrema points, x* Backward propagation Find u*, d* at each time step u min u max Initial Condition Boundary

19. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 19 Optimization-Reach Vertical Position Vertical Velocity -0.50 E G Consider J G and J E optimization separately and combine results: let t f G < t f E q=‘Contact’ J G Optimization at t=t f G, J G =  G Gradient non-zero and perpendicular to boundary Find extrema points, x* Backward propagation Find u*, d* at each time step u min u max Initial Condition Boundary

20. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 20 J G Optimization at t=t f G, J G =  G Gradient non-zero and perpendicular to boundary Find extrema points, x* Backward propagation Find u*, d* at each time step Forward propagation Find u*, d* at each time step  Optimization reverses sign Optimization-Reach Vertical Position Vertical Velocity -0.50 G Consider J G and J E optimization separately and combine results: let t f G < t f E q=‘Contact’ u max u min Exit Condition Boundary

21. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 21 J G Optimization at t=t f G, J G =  G Gradient non-zero and perpendicular to boundary Find extrema points, x* Backward propagation Find u*, d* at each time step Forward propagation Find u*, d* at each time step  Optimization reverses sign Optimization-Reach Vertical Position Vertical Velocity -0.50 G Consider J G and J E optimization separately and combine results: let t f G < t f E q=‘Contact’ Exit Condition Boundary

22. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 22 J E Optimization E is subset of Exit Conditions ( EC ) boundary Optimization-Reach Vertical Position Vertical Velocity -0.50 E G Consider J G and J E optimization separately and combine results: let t f G < t f E q=‘Contact’ Exit Condition (EC) boundary

23. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 23 J E Optimization E is subset of Exit Conditions ( EC ) boundary At t=t f E, J E =  E, x  EC Optimization-Reach Vertical Position Vertical Velocity -0.50 E G Consider J G and J E optimization separately and combine results: let t f G < t f E q=‘Contact’

24. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 24 J E Optimization E is subset of Exit Conditions ( EC ) boundary At t=t f E, J E =  E, x  EC Gradient non-zero at border Extrema points, x*, at border Optimization-Reach Vertical Position Vertical Velocity -0.50 E G Consider J G and J E optimization separately and combine results: let t f G < t f E q=‘Contact’ Extrema points, x*

25. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 25 J E Optimization E is subset of Exit Conditions ( EC ) boundary At t=t f E, J E =  E, x  EC Gradient non-zero at border Extrema points, x*, at border Backward propagation Optimization-Reach Vertical Position Vertical Velocity -0.50 E G Consider J G and J E optimization separately and combine results: let t f G < t f E q=‘Contact’ u min u max u min u max u min u max

26. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 26 J E Optimization E is subset of Exit Conditions ( EC ) boundary At t=t f E, J E =  E, x  EC Gradient non-zero at border Extrema points, x*, at border Backward propagation Find u*, d* at each time step Optimization-Reach Vertical Position Vertical Velocity -0.50 E G Consider J G and J E optimization separately and combine results: let t f G < t f E q=‘Contact’

27. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 27 Combined Optimization Find intersection points Optimization-Reach Vertical Position Vertical Velocity -0.50 E G Consider J G and J E optimization separately and combine results: let t f G < t f E q=‘Contact’

28. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 28 Combined Optimization Find intersection points Intersect state spaces Optimization-Reach Vertical Position Vertical Velocity -0.50 E G Consider J G and J E optimization separately and combine results: let t f G < t f E q=‘Contact’

29. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 29 Combined Optimization Find intersection points Intersect state spaces Optimization-Reach Vertical Position Vertical Velocity -0.50 E Consider J G and J E optimization separately and combine results: let t f G < t f E q=‘Contact’ Reach(G,E) Safe Subset Reach(G,E) The boundaries of the safe subset constitute the least restrictive control. Within the subset any control action u min <u<u max is safe until the state reaches the boundary. u min u max

30. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 30 Combined Optimization Find intersection points Intersect state spaces Backwards Chaining The new initial conditions modify the escape set E for predecessor modes Optimization-Reach Vertical Position Vertical Velocity -0.50 E Consider J G and J E optimization separately and combine results: let t f G < t f E q=‘Contact’ Reach(G,E) Safe Subset u min u max New set of safe Initial Conditions

31. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 31 Application to Vertical Hopper Contact Phase Descent Phase Step Phase Ascent Phase Hopper Phase Plane Position vs Velocity

32. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 32 Application to Vertical Hopper Contact Phase Extrema point where f(x,u,d) is tangent to constraint surface Joint Compression Constraint at -0.5

33. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 33 Application to Vertical Hopper Backward time propagation at u = u max To Contact phase Initial Condition Forward time propagation at u = u max To Contact phase Exit Condition Contact Phase Propagation

34. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 34 Application to Vertical Hopper Descent Phase backward propagation from Initial to Exit conditions Descent Phase backward propagation from Exit to Initial conditions Descent Phase Propagation

35. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 35 Application to Vertical Hopper Shift for Step Up in backward time Shift for Step Down in backward time Step Phase

36. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 36 Application to Vertical Hopper Ascent Phase Propagation

37. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 37 Application to Vertical Hopper Contact Phase Propagation u=u min u=u max Descent Phase Propagation

38. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 38 Application to Vertical Hopper 2 nd Iteration through Contact and Descent Phases

39. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 39 Application to Vertical Hopper 3rd Iteration through Contact and Descent Phases

40. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 40 Application to Vertical Hopper 4th Iteration through Contact and Descent Phases

41. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 41 Application to Vertical Hopper Final Boundary for Safe States u=u min u=u max u=u min Safe Subset Boundaries of The Safe Subset Determine the Least Restrictive Control

42. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 42 Vertical Hopper Results Safe Subset Safe Subset Environment can force crash or stumble Environment can force stumble Low damping Moderate damping

43. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 43 Ruggedness Ruggedness is a measure of terrain variability the hopper can tolerate safely. For moderate to high damping ruggedness is maximized with soft leg springs. These results support biomimetic design principles: Control should be divided between both active and passive elements

44. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 44 Issues Singularites –At singular points ( f(x,u,d)=0 ) backward time propagation is undefined –General approach is to define a boundary or new phase that prescribes u, d so that singularities are avoided The implementation will be domain specific and I don’t know how to “program” a general solution What if settling to a singular point is “happy”? Control/Disturbance dependent phase transitions –Example: transition from sliding to non-sliding contact depends on control force, then changing values of control induces thrashing between modes –General approach is to define intermediate phases which fix control inputs to avoid thrashing This same restrictions probably can’t be imposed on disturbance inputs.

45. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 45 Singularity Avoidance Singular points for 0 ≤ u ≤ u max Approach: If f(x*,u*) trajectory reaches singularity avoidance boundary then set u=u rtn (magenta) or u=0 (blue). Control is least restrictive in the sense that initial/exit subboundaries are unchanged. Alternative: Could set u=u max to ‘blow through’ any singularities

46. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 46 Control Dependent Transitions Transition from contact to ascent when ground reaction force is zero: –GRF = ky+by’+u Define a liftoff phase where u=u max in transition from contact to ascent and u=0 in transition from ascent to contact. Forward in time from A1, Propagation is bound by Points at A2 and A2’. Backward in time from B1, propagation is bound By points at B2 and B2’

47. 10/09/06 BJH Safe Control Strategies for Hopping over Uneven Terrain 47 Conclusions At least for simple systems, a game theoretic approach can be applied to gain useful insight to system limitations and behaviors in environment. –More complicated systems require more complicated analysis tools (topic for part II). –Theoretically, the approach generates a control law but implementation issues (delays, etc.) have not been addressed here. The approach has difficulty with singularities and control/disturbance dependent phase transitions which can be addressed with some thought and some restrictions.