Can We Figure Out What Algorithms Are Used For Balance and Gait in Biological Systems? Chris Atkeson CMU

Goals What features can be used for standing balance and control? What experiments would distinguish between these approaches in humans, animals, and robots? Collaborative Research in Computational Neuroscience (CRCNS) program. Nov. 2. Humans and cats. Are there perturbations we can apply that can distinguish these approaches? Are there relevant motor illusions? How are quantities like COM position and velocity computed or perceived, if COM is used in control? Are there different learning effects? DARPA wants to know: how evaluate robot balance and walking?

Alternatives at CMU Map from muscle variables (length, velocity, force) to muscle activation, with minimal cross muscle activity (Geyer). Map from body parts (primarily torso position, orientation, angle, and angular velocity) to joint torque (Hodgins). Map from center of mass (COM) position and velocity to foot (and other contact) forces. Maybe regulate body angular momentum (Stephens). Map full state (all joint positions and velocities and root position, orientation, and corresponding velocities) to joint torques (Atkeson).

Neuromuscular Models (Geyer)

Local Feedback Am = g*Lm (g<0: springlike) Am = g*Fm (g>0: positive force feedback) Some coupling ATA = g*LTA – gFSOL AHFL = g*LHFL – g*LHAM

Issues Sparse interactions: Mostly homonymous feedback. Some antagonistic feedback. No synergistic feedback? Contralateral force feedback to initiate swing. Adapts to unknown slopes and steps. How launch, stop, turn (3D)? Relationship to CPG? What happens when feedback distorted or removed? Vestibular input: trunk uses vertical reference. Vision input? Foot placement? Anticipation? Planning?

A Possible Signature: Positive Force Feedback Can we test for sign of force feedback (GTO to extensor reflexes)? Need to disassociate from length or velocity feedback, joint feedback. What stabilizes positive force feedback?

Control at the level of body parts Raibert 3 part running control: Actual or virtual telescoping leg. Hopping height: more push off if low, … Attitude: Hip pitch torque on stance leg to level body (based on vestibular input) Τ = -g1*(θ – θd) – g2*θvel Foot placement: Cartesian step size based on desired velocity and velocity error: s = g3*Vd – g4*(V – Vd)

Hodgins Hodgins, Biped Gait Transitions ICRA 91 http://www.cs.cmu.edu/~cga/legs/hodgins_icra91.pdf

Wisse

Wisse

Modern Versions: Big Dog, Petman, Simbicon Pointer to Big Dog goes here Pointer to Petman, Atlas goes here Pointer to Simbicon goes here

Issues Global inputs: measuring height/energy, forward velocity, body angle with respect to vertical. Virtual model: telescoping leg, Cartesian foot placement. Adapts to unknown slopes and steps. How launch, stop, turn (3D)? Foot placement? Vision input? Relationship to CPG? What happens when feedback distorted or removed? Joint torques to muscle activations?

COM-based control Stephens, www.cs.cmu.edu/~bstephe1 19

Balance Recovery Strategies Ankle Strategy – single link inverted pendulum Hip Strategy – torso as flywheel Squat Strategy – change height of COM Step Strategy – change polygon of support Righting Reflex – arm as flywheel Grasp Strategy – change polygon of support ….

Approach Plan each strategy at COM level for N seconds ahead using trajectory optimization. Each different number of steps a distinct plan Multiple internal simulations. Pick best option. Do it for a fraction of a second. Repeat. Input is COM position and velocity. Outputs are contact forces and torques (COP)

COM-Level Stepping Optimization Choose step locations for particular feet. Timing, number of steps, initial stance foot fixed. Optimize closeness of COM and foot step locations to goals for these. Minimize COM velocity, COP velocity.

An Example Plan Double support Take step Stop Take step Push left Start falling forward

Full Body Control: Weighted Objective Optimization Objectives: Dynamics Task Objectives COM Acceleration Torque About COM Reference Pose Tracking Minimize Controls Constraints: Center of Pressure Friction Cone Joint Torque Limits Stephens M. de Lasa, I. Mordatch, and A. Hertzmann, “Feature-Based Locomotion Controllers,” ACM Transactions on Graphics, vol. 29, 2010.

Pushing: Ankle, Hip, and Stepping Strategies No Stepping Stepping Stephens

Pushing While Stepping In Place Stephens

Pushing: Moving Floor Stephens

Pushing While Doing A Task Stephens

Stephens

Human-like walking: Planning Plan for heel strike, push off on toe only. Plan for fully coupled swing leg and whole body dynamics. Only approach that does this now is trajectory optimization (trajectory library: Liu). Use torso and arms more effectively: pitch, roll, and yaw motions.

Human-like walking: Impacts Optimize contact initiation (impact). Very small Fx, Fy, Mx, My, Mz initially, low impedance in these directions. Avoid bounce, gradual increase in Fz, right vertical impedance. Stiffen after contact when know terrain height and slope.

Human-like walking: Foot forces and torques Optimize swing leg and whole body trajectories to: Min Fx/Fz, Fy/Fz for each foot to avoid slip Min Mx/Fz, My/Fz for each foot to avoid tipping Min Mz for each foot to avoid twist, and keep COP near center of foot (small Mx, My) Compliant ankles (small Mx, My, and Mz)

Human-like walking: Reflexes Limit Fx, Fy, Mx, My, and Mz based on Fz, COP on each foot in real time. Use grip like reflexes to detect and stop slipping, tripping, and twist on each foot.

Issues Body level inputs: COM XYZ location and velocity, body angle with respect to vertical. Body level outputs: Foot forces/COP COM is gateway/bottleneck. Expect synergies. Optimization allocates activation to muscles. Adapts to unknown slopes and steps. How launch, stop, turn (3D)? Foot placement easy. Relationship to CPG? What happens when feedback distorted or removed? Joint torques to muscle activations? More optimization.

Body-Level Control: Dynamic Programming Whitman

Full State Control: Trajectory Library Biped Walking Control Using a Trajectory Library Liu, Atkeson, Su; Robotica in press. www.cs.cmu.edu/~cga/papers/robotica.pdf Library for 10 dimensional walking Test set

Standing Balance Implementation

Pushing The Robot Push Size (N) Ankle Pos Hip Pos Ankle Torque Hip Torque

Push Estimation On Robot

Planar Walking Library (left) and Simulated Push Perturbations (right)

Simulated 16 Ns Impulse

Planar Walking Simulation “Instantaneous” and continuous pushes Pushes of different sizes, time varying pushes. Inclines Model perturbations (wrong mass) Ground model variations (ground stiffness)

Issues Memory intensive. Not clear how much memory load can be reduced by online optimization or learning. Can generate any behavior, so hard to explore experimentally. COM is NOT gateway/bottleneck. Synergies? DBFC allocates activation to muscles. Adapts to unknown slopes and steps. How launch, stop, turn (3D): Separate learned behaviors Foot placement easy. Relationship to CPG? What happens when feedback distorted or removed? Joint torques to muscle activations? More optimization.

Overall Issues Do these approaches have a “signature”? No perturbation – probably not. Perturbation? Need for feedback (CPG debate) Locality of feedback (Vibrate a tendon?) Synergies? (factor analysis) How does vision impact walking? (synergies again) How does vestibular system impact walking? (synergies again)

Detailed Numerical Modeling Match kinematics Match contact forces (GRF) Match inverse dynamics.This is really matching kinematics in a different coordinate system. Match EMG Match perturbation kinematics Match perturbation GRF Match perturbation EMG Match decision boundaries where strategies change

Perturbation Tests Moving terrain (“bus”) (Equitest platform, Stewart platform) Pushes (instantaneous and continuous) Slips (change terrain material properties) Trips (unexpected terrain obstacles) Unknown terrain shape (slopes, ripples, stairs, holes and poles). Varying terrain material properties (loose soil, sand, gravel, hard dirt, fixed and loose rocks, mud)

Control tests Foot placement targets Avoid no-go ground regions (at footstep scale and at vehicle scale) Follow velocity trajectories Follow desired paths (turning) Traverse known rough terrain

State Estimation Need to focus on sensor fusion Which way is up, angular velocity with respect to vertical (vestibular, optical, proprioception) Where is COM relative to COP and vertical? Body angular velocity Contact and ground conditions.

Other Approaches: ZMP