1. Simulating Balance Recovery Responses to Trips Based on Biomechanical Principles Takaaki Shiratori 1,2 Rakié Cham 3 Brooke Coley 3 Jessica K. Hodgins 1,2 31 2

2. Physical Simulation for Human Characters Steady-state behaviors. Reactive responses required. Yin et al., 2007 Muico et al., 2009 1 http://lh6.ggpht.com/_UAku2WOHdSE/SlP6lTodsMI/AAAAAAAADOU/BFQRfrvySDM/ Interaction 1

3. Clear obstacle. Reactive Response to Trips Collision with obstacle.Recover balance. Obstacle Motion capture dataSimulation Controller Biomechanical Principles

4. Synthesize reactive responses. Prior Work Kudoh et al., 2002Zordan et al., 2002 Simulation-based method Ye et al., 2008 Biomechanics-based method Not applicable to tripping. Not for human characters. Macchietto et al., 2009Komura et al., 2004 Trip and slip for bipedal robots. Boone et al., 1997

5. Strategies Clearance with tripped leg Clearance with non-tripped leg Flight phase or double support Flight phase or double support Touchdown Push-off reaction Push-off reaction Strategy selection Collision in late swing (40-75% of entire swing) Collision in early swing (5-50% of entire swing) Elevating strategy Lowering strategy [Eng et al. 1994, Schillings et al. 2000, Pijnappels et al. 2004, 2005] Leg swap

6. Ground reaction force vector passes anteriorly to the COM. Push-off Reaction [Pijnappels et al. 2005] Reduce forward angular velocity.

7. Increase moment of inertia to reduce angular acceleration. Arm ipsilateral to tripped leg moves in sideward direction. Arm contralateral to tripped leg moves in forward direction. Protect head/chest. Arm Motions [Roos et al. 2008, Pijnappels et al. 2008]

8. Subjects must not know when/where tripping occurs. Capturing Tripping Motion Trip machine Harness Semi-rigid shoe Trip slide Look here

9. Elev.-DSElev.-FLLower.-DSLower.-FL # subjects4343 Speed [ m/s ] (SD) 1.15 (0.146) 1.44 (0.0751) 0.942 (0.191) 1.44 (0.232) Motion Capture Dataset (DS: double support FL: flight phase) ElevatingLowering Faster walking speeds tend to lead to Flight Phase. Elev.-DSElev.-FLLower.-DSLower.-FL # subjects4343 Speed [ m/s ] (SD) 1.15 (0.146) 1.44 (0.0751) 0.942 (0.191) 1.44 (0.232)

10. Human Model Create a 3D skin model from ~400 optical markers.  Calculate mass and moment of inertia from volume. [Park and Hodgins 2006] 42 DOFs in total 96 contact points per foot.

11. Finite state machine with Proportional Derivative (PD) servo. Controller Overview Yes Collision Passive Reaction Clearance Flight Single Support After Trip Double Support Fall Muscle activities start. Leading leg contacts ground. Leading leg contacts ground. Trailing leg leaves ground. No Leg Swap Flight phase? Strategy? COM starts falling. Tripped leg touches ground. Elevating Lowering

12. Controller for Tripping Reactions Elevating Yes Collision Passive Reaction Clearance Flight Single Support After Trip Double Support Fall Muscle activities start. Leading leg contacts ground. Leading leg contacts ground. Trailing leg leaves ground. No Leg Swap Lowering Flight phase? Strategy? COM starts falling. Tripped leg touches ground. Baseline walking Playback of motion capture data. Simulation Initialized with tripping forces just before trip occurs. Simulation initialization

13. Controller for Tripping Reactions Yes Collision Passive Reaction Clearance Flight Single Support After Trip Double Support Fall Muscle activities start. Leading leg contacts ground. Leading leg contacts ground. Trailing leg leaves ground. No Leg Swap Flight phase? Strategy? COM starts falling. Tripped leg touches ground. Observed tripping forces. Vertical: sine function Fore-aft: Gaussian function [Pijnappels, et al., 2004] 100 50 0 -50 -100 -150 -200 00.020.040.060.080.10 Force [N] Time [sec] Fore-aft (x) Vertical (z) x z Elevating Lowering Simulation initialization

14. Controller for Tripping Reactions Yes Collision Clearance Flight Single Support After Trip Double Support Fall Muscle activities start. Leading leg contacts ground. Leading leg contacts ground. Trailing leg leaves ground. No Leg Swap Flight phase? Passive Reaction Strategy? COM starts falling. Tripped leg touches ground. Support leg Control attitude of upper body. Swing leg Moving forward like walking. Target angles: motion capture data of walking. Passive Reaction Elevating Lowering

15. Knee torque Controller for Tripping Reactions Yes Collision Clearance Flight Single Support After Trip Double Support Fall Muscle activities start. Leading leg contacts ground. Leading leg contacts ground. Trailing leg leaves ground. No Leg Swap Flight phase? Passive Reaction Strategy? COM starts falling. Tripped leg touches ground. Passive Reaction Muscle activities start. Elevating Vastus lateralis Rectus femoris Transit: touch sensor  brain  muscle Muscle recruitment (40 msec) [Schillings et al. 2000] [Pijnappels et al. 2005] [Ralston et al. 1976] Lowering Support knee 0: tripping instant

16. Controller for Elevating Strategy Yes Collision Flight Single Support After Trip Double Support Fall Muscle activities start. Leading leg contacts ground. Leading leg contacts ground. Trailing leg leaves ground. No Leg Swap Flight phase? Passive Reaction Elevating strategy? COM starts falling. Tripped leg touches ground. Passive Reaction Muscle activities start. Strategy? Clearance Support leg Push-off reaction: Extend all joints. Compensation torque to ankle for COM. Elevating Lowering

17. Controller for Elevating Strategy Yes Collision Flight Single Support After Trip Double Support Fall Muscle activities start. Leading leg contacts ground. Leading leg contacts ground. Trailing leg leaves ground. No Leg Swap Flight phase? Passive Reaction Elevating strategy? COM starts falling. Tripped leg touches ground. Muscle activities start. Strategy? Clearance Swing leg Clear the obstacle. Target angles: motion capture data. Motion captureSimulation Elevating Lowering

18. Controller for Elevating Strategy Yes Collision Flight Single Support After Trip Double Support Fall Muscle activities start. Leading leg contacts ground. Leading leg contacts ground. Trailing leg leaves ground. No Leg Swap Flight phase? Passive Reaction Strategy? COM starts falling. Tripped leg touches ground. Clearance Flight phase? Flight Leading leg Extended for touchdown. Target angles: motion capture data. Trailing leg Start flexion. Motion capture Simulation Elevating Lowering

19. Controller for Elevating Strategy Yes Collision Flight Single Support After Trip Double Support Fall Muscle activities start. Leading leg contacts ground. Leading leg contacts ground. Trailing leg leaves ground. No Leg Swap Flight phase? Passive Reaction Strategy? COM starts falling. Tripped leg touches ground. Clearance Flight Leading leg contacts ground. Single Support After Trip Leading leg Control attitude of upper body. Trailing leg Move forward for the next step. Elevating Lowering

20. Controller for Elevating Strategy Yes Collision Flight Single Support After Trip Double Support Fall Muscle activities start. Leading leg contacts ground. Leading leg contacts ground. Trailing leg leaves ground. No Leg Swap Flight phase? Passive Reaction Strategy? COM starts falling. Tripped leg touches ground. Clearance Flight phase? COM starts falling. Leading leg Extended for the next step Target angles: motion capture data. Trailing leg Control attitude of upper body. Keep extension. Motion capture Simulation Elevating Lowering

21. Controller for Elevating Strategy Yes Collision Flight Single Support After Trip Double Support Fall Muscle activities start. Leading leg contacts ground. Leading leg contacts ground. Trailing leg leaves ground. No Leg Swap Flight phase? Passive Reaction Strategy? COM starts falling. Tripped leg touches ground. Clearance Leading leg contacts ground. Double Support Leading leg Control attitude of upper body and extend knee. Trailing leg Control attitude of upper body. Plantar-flex ankle for the next step. Elevating Lowering

22. Controller for Elevating Strategy Yes Collision Flight Single Support After Trip Double Support Fall Muscle activities start. Leading leg contacts ground. Leading leg contacts ground. Trailing leg leaves ground. No Leg Swap Flight phase? Passive Reaction Strategy? COM starts falling. Tripped leg touches ground. Clearance Double Support Trailing leg leaves ground. Single Support After Trip Leading leg Control attitude of upper body. Trailing leg Move forward for the next step. Elevating Lowering

23. Controller for Lowering Strategy Yes Collision Flight Single Support After Trip Double Support Fall Muscle activities start. Leading leg contacts ground. Leading leg contacts ground. Trailing leg leaves ground. No Flight phase? Passive Reaction Elevating strategy? COM starts falling. Tripped leg touches ground. Passive Reaction Muscle activities start. Strategy? Clearance Leg Swap Swing leg (tripped) Extended for touchdown immediately. Elevating Lowering

24. Controller for Lowering Strategy Yes Collision Flight Single Support After Trip Double Support Fall Muscle activities start. Leading leg contacts ground. Leading leg contacts ground. Trailing leg leaves ground. No Flight phase? Passive Reaction Strategy? COM starts falling. Tripped leg touches ground. Clearance Leg Swap Swing leg (tripped) Extended for touchdown immediately. Support leg (non-tripped) Leaves ground after swing leg touchdown. Clear the obstacle. Tripped leg touches ground. Leg Swap Clearance Elevating Lowering

25. Start reaction. Timing: 100 msec Target angles: motion capture data. Control of Arm Motion Yes Collision Flight Single Support After Trip Double Support Fall Muscle activities start. Leading leg contacts ground. Leading leg contacts ground. Trailing leg leaves ground. No Flight phase? Passive Reaction Strategy? COM starts falling. Tripped leg touches ground. Clearance Leg Swap Clearance Motion capture Simulation [Pijnappels et al. 2008] Elevating Lowering

26. Control of Arm Motion Yes Collision Flight Single Support After Trip Double Support Fall Muscle activities start. Leading leg contacts ground. Leading leg contacts ground. Trailing leg leaves ground. No Leg Swap Flight phase? Passive Reaction Strategy? COM starts falling. Tripped leg touches ground. Clearance Single Support After Trip Back to motion in normal walking. Motion capture (walking) Simulation Elevating Lowering

27. Elevating strategy with double support. Input walking speed: 1.0 m/s Simulation Result ElevatingLowering DSFLDSFL

28. Elevating strategy with flight phase. Input walking speed: 1.4 m/s Simulation Result ElevatingLowering DSFLDSFL

29. Lowering strategy with double support. Input walking speed: 0.75 m/s Simulation Result ElevatingLowering DSFLDSFL

30. Lowering strategy with flight phase. Input walking speed: 1.1 m/s Simulation Result ElevatingLowering DSFLDSFL

31. Elevating strategy with flight phase. Comparison with Motion Capture Data -100 -80 -60 -40 -20 0 20 -0.5 0 0.5 1.0 Pitch [ deg ] Time [ sec ] -20 0 20 40 60 80 100 120 -0.5 0 0.5 1.0 Pitch [ deg ] Time [ sec ] HipKnee : tripping instant : simulation result : motion capture data

32. Elevating strategy with flight phase. Comparison with Motion Capture Data Foot trajectoryPelvis -10 -5 0 5 10 15 20 25 30 35 40 45 -0.5 0 0.5 1.0 Pitch [ deg ] Time [ sec ] 0 0.1 0.2 0.3 0.4 0.5 0 1.0 Height [ m ] Length [ m ] : tripping instant : simulation result : motion capture data

33. Root mean square errors Unit: [deg/frame] (frame rate = 120 Hz) Quantitative Comparison

34. Recovery with multiple steps. Discussion

35. Better contact model  Many force plates.  Larger marker set for feet.  More precise model of tripping forces. Discussion Push-off reactionTripping forces.

36. Controllers for strategies of balance recovery responses to trips. Graphics Integrate walking controllers. Other reactive responses. Biomechanics application Answer “what if” questions. Improve training and rehabilitation systems. Summary 1 2 3 2 http://www.yamakai.org/profiles/marriott.html 1 http://www.youtube.com/watch?v=LVStmLCoH30 3 http://www.treadmilladviser.com/landice-l7-rehabilitation-treadmill.html

37. Adam Bargteil for help with calculating mass and moment of inertia. Moshe Mahler for rendering animation. Justin Macey for the human model. Subjects for participation in the experiments. NSF -0540865 Quality of Life Technology Engineering Research Center F31 AG025684-03 NIH Ruth L. Kirschstein Award Autodesk for Maya donation. Acknowledgements