Motion Editing and Retargetting Jinxiang Chai

Outline Motion editing [video, click here]here Motion retargeting [video, click here]here

Required Readings Comparing Constraint-Based Motion Editing Methods Comparing Constraint-Based Motion Editing Methods Retargeting Motion to New Characters

Kinematic Motion Editing Goal: Edit an input motion sequence to achieve new goals specified by the user

Kinematic Motion Editing Goal: Edit an input motion sequence to satisfy achieve new goals specified by the user e.g., edit the motion to meet new constraints

Kinematic Motion Editing Goal: Edit an input motion sequence to satisfy achieve new goals specified by the user e.g., edit the motion to meet new constraints - could be multiple constraints

Toy Examples Edit 1D function to match user constraints Old motion M={m(t)| t=1,…,T} t m(t)

Toy Examples Edit 1D function to match user constraints t m(t) Old motion M={m(t)| t=1,…,T}

Toy Examples Edit 1D function to match user constraints - could be multiple constraints t m(t) Old motion M={m(t)| t=1,…,T}

Toy Examples Edit 1D function to match user constraints - could be multiple constraints Old motion m(t), t=1,…,T t m(t) So how can we generate a new motion M’ to satisfy new constraints c?

Toy Examples: Idea #1 Modify the motion m(t), t=1,…,T using new constraints C t M(t) Old motion m(t), t=1,…,T

Toy Examples: Idea #1 Modify the motion m(t), t=1,…,T using new constraints C t m’(t) Does this work?

Toy Examples: Idea #1 Modify the motion m(t), t=1,…,T using new constraints C t m’(t) Does this work? - probably not, the new motion is not smooth!

Toy Examples: Idea #1 Modify the motion m(t), t=1,…,T using new constraints C t M’(t) Does this work? - probably not, the new motion is not smooth! - filter the motion!

Toy Examples: Idea #1 Modify the motion m(t), t=1,…,T using new constraints C Filter the edited motion M’ t M’(t) Does this work? - probably no, the new motion is not smooth! - filter the motion! - but the new motion now does not meet cons!

Toy Examples: Idea #1 Modify the motion m(t), t=1,…,T using new constraints C Filter the edited motion M’ t M’(t) Does this work? - probably no, the new motion is not smooth! - filter the motion! - repeat modification and filtering process!

Idea #1: Modification & Filter

Another Example: Three Constraints

Idea #1: Summary Pros - easy to implement - computationally efficient Cons - does not handle complex constraints, e.g., constraint functions involved more than one frames Old motion M={m(t)| t=1,…,T} t m(t) t1t1 t2t2 e.g., m’(t 1 )-m’(t 2 )=5

Idea #1: Summary Pros - easy to implement - computationally efficient Cons - does not handle complex constraints, e.g., constraint functions involved more than one frames How to address this limitation?

Idea #2: Constrained Optimization Modify the motion M to satisfy new constraints C Old motion m(t), t=1,…,T t m(t)

Idea #2: Constrained Optimization Modify the motion M to satisfy new constraints C Old motion m(t), t=1,…,T t m(t) But this is an ill-posed problem - There are tons of solutions that satisfy the new constraints C

Idea #2: Constrained Optimization Modify the motion M to satisfy new constraints C Old motion m(t), t=1,…,T t m(t) But this is an ill-posed problem - There are tons of solutions that satisfy the new constraints C - Remove the ambiguity by picking the solution with a minimal change across an entire motion

Idea #2: Constrained Optimization Modify the motion M to satisfy new constraints C Old motion m(t), t=1,…,T t m(t)

Idea #2: Constrained Optimization Modify the motion M to satisfy new constraints C Old motion m(t), t=1,…,T t m(t) Minimize the changes Satisfy new constraints

Idea #2: Constrained Optimization We can also formulate this as an unconstrained optimization problem Old motion m(t), t=1,…,T t m(t)

Idea #2: Constrained Optimization This is also called as spacetime optimization or trajectory optimization Simultaneously compute the entire motion Old motion m(t), t=1,…,T t m(t)

Idea #2: Constrained Optimization This is also called as spacetime optimization or trajectory optimization Simultaneously compute the entire motion Old motion m(t), t=1,…,T t m(t)

Now Back to Human Motion Editing!

A sequence of poses: q 1,q 2,…q T Each pose is represented as a high-dimensional vector q t : R n Motion trajectoriesPose q t Motion q 1,…q T Human motion representation

Human Motion Editing Human motion is a high-dimensional function Constraints could be any kinematic constraints

User Constraints Any kinematic constraints throughout the motion Position cons. Orientation cons. Distance cons. Joint angle cons.

Solution #1: Per-frame IK+Filtering Step 1: Per-frame inverse kinematics to modify poses in each frame with new constraints. - this might be ill-posed - remove ambiguity by minimizing the difference between old poses and new poses Step 2: motion filtering. Repeat step 1 and 2 until the solution converges

Solution #2: Spacetime Motion Editing Consider all constraints simultaneously Use optimization to solve the entire motion

Solution #2: Spacetime Motion Editing Consider all constraints simultaneously Use optimization to solve the entire motion - Currently, minimizing joint angle poses across the entire sequence - A better criterion is possible!

Solution #2: Spacetime Editing Better Criterion? - Geometric constraints such as contact constraints are often more important

Solution #2: Spacetime Editing Better Criterion? - Frequency Content or details

Solution #2: Spacetime Motion Editing Implementation details: - use Spline representation to reduce the solution space - often computationally expensive - optimization over sub windows when working on a long motion sequence

Motion Retargeting One motion, a cast of characters Focus on similar structure

Formulated as motion editing problem Motion Retargeting: Key Idea Skeleton of new characters

Step 1: Extract constraints/properties C from source motion Motion Retargeting: Key Idea Extracted constraints

Step 1: Extract constraints/properties C from source motion Motion Retargeting: Key Idea Extracted constraints Retargeted motionSource motion

Step 1: Extract constraints/properties C from source motion Motion Retargeting: Key Idea Extracted constraints Retargeted motionSource motion Skeleton of new characters

Step 2: Apply to new characters Motion Retargeting: Key Idea

Step 3: Approximate answer Motion Retargeting: Key Idea How to initialize m’(t)?

Step 4: Solve constraints using spacetime optimization Motion Retargeting: Key Idea

Motion Retargeting Youtube video (click here)here