Motion Synthesis for Articulated Human Bodies

Contents

Human motion components Bone (skeleton): ~206 (motion invariant) Joint (articulation): limitation Tendon and ligament: elasticity / plasticity Muscle: ~639, works in group, limitation –uni-articulate and bi-articulate (passive inefficiency) –parallel or perpendicular shape Neuron-system: reflection speed Neuron  Muscle  Tedon  Bone

Human motion model (I) Pure articulated model –Simplified skeleton (hand / foot / tibia-fibula) –Directly control force and torque on joints –Convenient for high-level control

Human motion model (II) Deformable model –Skin mesh motored by skeleton or captured skin animation –Muscle shape representation

Human motion model (III) Tendons and muscles model –Musculoskeletal models and simulation –Muscle-tendon model (strand) Hand can have similar DoF with coarse whole body!

Human motion model (IV) Visible Human data set (realistic musculature and flesh) [Teran05]

Methods for motion synthesis (I) Data-driven based synthesis –Captured motion skeleton data –Captured deformable data (face / skin) –Motion dynamics and naturalness is trivial –Limited response to environment Blending, Editing, Style transfer; no guarantee for physics correctness –Captured data is sparse [pose, time] space is huge –Difficult to obtain data High dynamic motion Outdoor environment

Methods for motion synthesis (II) Pure generative-based synthesis –Physics constraint can be guaranteed –Easy to interact with dynamic environment –Hard to be natural Naturalness is a subset of physics correctness –Dynamics simulator complexity E.g. stable / dynamic frictions –Computational complexity and stableness High gain means high stiffness –Controller design Mutual controller is difficult to design and hard to generalize

Methods for motion synthesis (III) Pure motion planning –Can obtain solution in constrained environment –High-level path planning –Solution has no continuous or physics guarantee –Still time consuming High dimension of human skeleton

Combined methods Mocap data + physics: better controller –Controller for hand [Nancy05] –Controller for whole body [Yin08] Mocap data + motion planning: high level motion: –Manipulating [Katsu04] –Tangling [Edmond08] Dynamics + motion planning (for articulated model) [Russell07]

Other related Robotics [Kris06] –Primitive based motion planning use transition motion to balance natural motion and environment constraints. Biomechanics –Provides many principles for motion control: balance, Locomotion, neuroscience. [Alexandrov05] [Christine07]

Contents

Articulated body dynamics Ball-and-socket: 3 DoF Saddle: 2 DoF Hinge: 1 Dof Each joint i is 1 DoF Ball-and-socket and saddle can be represented by some 1 DoF

A special case of multi-body dynamics Yin’s work gives an example Joint makes impulse and penalty difficult

Parameters for articulated body State of root (position, rotation, linear / angular velocity) Configuration of joints (q,, ) Inertia matrix (for articulated body, inertia matrix is an equivalent inertia matrix: give a test force, get an acceleration  inertia) Lecture in COMP Lecture

Inverse dynamics Known: q,, Unknown: f, τ Simple: recursive [Featherstone] –Compute v and a; compute net force on a link, similar to f = ma (downwards) –Compute force and torque on a joint (upwards) –Root’s p, v, a –End effector’s force and torque

Forward dynamics Known: q,, f, τ Unknown: More difficult –3 loops [Evangelos04]

Limitation of basic algorithms Many things are simplified in algorithms –Joint limitation –Unilateral constraint (ground constraint) –Friction (kinetic, static, rolling and spinning) –Non-interpenetrate constraint –Collision response Modeling contact and constraint is a well- studied problem for rigid body simulation. –See David Baraff’s papers and note.

Linear complementarity problem (LCP) [Andreas] One of the standard methods to handle contact and constraints.

Solution to LCP: pivot method Basic idea: if q is positive or zero, solution is trivial: (w = q, z = 0) Pivot q and w to make it true where

Solution to LCP: iterative method LCP can be represented by QP, so can be solved by iterative methods, like Gauss- Seidel, Newton

Pros and Cons Pivot method –Convergence is guaranteed after limited steps –suffer from numerical problem, especially for large-scale and/or ill-conditioned problems Iterative method –easier to implement and –numerically robust –convergence is proven only for a limited class of M matrix

LCP model for constraints Unilateral constraint ( force exists only when contacting ) Joint Limit LCP guarantees zero virtual work for contact forces w = Mz + q?

Extended LCP model for frictions Coulomb cone Dynamic friction Static friction Solved by extending iterative LCP solver

Other possible solutions Recent there are many new algorithms other than LCP to deal with unilateral constraints and friction –Nonconvex Rigid Bodies with Stacking [Eran03] Extension to LCP –Staggered Projections for Frictional Contact in Multibody Systems [Danny08] –Velocity based shock propagation [Kenny09] –Implicit Contact Handling for Deformable Objects [Miguel09]

Impact constraints Non-interpenetrate constraint Collision reaction Impulse or force modeling See David Baraff’s papers and note.

Simulation loops LCP solver forward dynamics Ode solver Collision detector Contact response

Contents

Optimization based motion synthesis [Sumit09]

System overview

Lagrange mechanics Lagrange mechanics for rigid body Lagrange mechanics for articulated body Lagrange function Generalize d force Gravity, contact force, external force

Complete force modeling [Liu05]

Actuated and non actuated joint Root joint: non (passive) actuated –It’s configuration (position and rotation) is actuated by joint constraints. Other joints: (active) actuated –It’s configuration is actuated by muscle energy Muscle force

Passive controller Muscle controller –Limited torque / force –Limited torque / force change rate Contact controller –Static friction –Dynamic friction –Non-penetration

Optimization with passive controller Why (minimize q variation)? –Avoid trivial solution, like sliding or break-off Zero virtual work guarantee –Roll back

User controller specification Balance controller Climb controller Swing controller Multiple controller composition controller example

Controller protocol Finite State Machine Each state has its own constraints (in objective form) State transition happens when feasible solution can not find or contact breaks.

Balance controller balance takeStep relaxFoot p: target position qv: slow spline: perpendicular to ground cp: CoM balance cf: support feet position c: friction

Climb controller allSupport moveHand relaxHand moveFoot relaxFoot com: change CoM

Swing controller passiveSwing trySwing

Visual sensor Reachable objects search

Contents

Human motion control model

Feedback based balance control Balance control under small perturbation [Yin03] Balance control under large perturbation [Yin08] Similar strategy: feed-back + feed-forward, large perturbation  step strategy

System description The dynamics system does not use generalized coordinate Instead, use full-coordinate constrained form for dynamics Traditional style (based on Barraf 1996 paper) Control is a hybrid of generalized- coordinate form and full-coordinate form

Full-coordinate constrained matrix form (I) Constraint i between body a and b Constraint system (each item in J is 3*3 matrix) Here f is dim-12 vector, every 3 sub-vector is force for body (a, b, c, or d), J is 9*12

Full-coordinate constrained matrix form (II) For torque, similar row number of H is number of freedom, its column number is 3 * object number. Each term is joint i’s torque on all objects

Balance control under small perturbation Preprocess: use inverse dynamics to compute force for original motion In each step of dynamic simulation –Forward dynamics: current state –Feedback control –Feedforward control –Net control

Simulation result Perturbation by a ball

Balance control under large perturbation Two differences: –Motion is modeled by finite state machine –Under large perturbation, a pre-computed feed-forward input is not suitable (using adaptive control instead)

Motion FSM Difference with FSM in [Sumit09] –For control torque heuristics –More robust (precision, static)

Feed-forward: Feed-back error learning Cyclic motion ( function of phase) Learn inverse model dynamically (system identification)

Feed-back control (I) PD control Contact control (support leg) Low gain Follow mocap data: high gain

Feed-back control (II) COM feedback + v + v + d<0 v=0 + d>0 v=0 COM velocity COM position v>0 or d > 0  forward step quickly

Feed-back control (III) CoM feedback Basic controller (default target) Continuous feedback General form for multiple joints (matrix form)

Control summary 2D to 3D: –sagittal and coronal planes Feedback and feedforward torques

Simulation result demo Overview Downhill Drunk Limp Spin Boxes Different friction

Reference [Edmond08] Planning tangling motions for humanoids [Nancy05] Physically Based Grasping Control from Example [Yin08] SIMBICON: Simple Biped Locomotion Control [Katsu04] Synthesizing Animations of Human Manipulation Tasks [Russell07] Efficient Motion Planning of Highly Articulated Chains using Physics- based sampling [Kris06] Using motion primitives in probabilistic sample-based planning for humanoid robots [Teran05] Creating and Simulating Skeletal Muscle from the Visible Human Data Set [Alexandrov05] Feedback equilibrium control during human standing [Christine07] Bipedal locomotion: toward unified concepts in robotics and neuroscience [Evangelos04] Practical Physics for Articulated Characters [Shin’schiro07] Constraint-based dynamics simulator for humanoid robots with shock absorbing mechanics [Featherstone] Robot Dynamics Algorithms [Andreas] Practical Optimization [Eran03] Nonconvex Rigid Bodies with Stacking [Danny08] Staggered Projections for Frictional Contact in Multibody Systems

[Sumit09] Optimization-based interactive motion synthesis [Liu05] Towards a generative model of natural motion [Baraff89] Analytical methods for dynamics simulation of non-penetrating rigid bodies [Yin03] Motion Perturbation Based on Simple Neuromotor Control Models [Kenny09] Velocity-based shock progagation for multibody dynamics animation [Miguel09] Implicit Contact Handling for Deformable Objects