1. Motion Editing and Retargetting Jinxiang Chai

2. Outline Motion editing [video, click here]here Motion retargeting [video, click here]here

3. Required Readings Comparing Constraint-Based Motion Editing Methods Comparing Constraint-Based Motion Editing Methods Retargeting Motion to New Characters

4. Kinematic Motion Editing Goal: Edit an input motion sequence to achieve new goals specified by the user

5. Kinematic Motion Editing Goal: Edit an input motion sequence to satisfy achieve new goals specified by the user e.g., edit the motion to meet new constraints

6. Kinematic Motion Editing Goal: Edit an input motion sequence to satisfy achieve new goals specified by the user e.g., edit the motion to meet new constraints - could be multiple constraints

7. Toy Examples Edit 1D function to match user constraints Old motion M={m(t)| t=1,…,T} t m(t)

8. Toy Examples Edit 1D function to match user constraints t m(t) Old motion M={m(t)| t=1,…,T}

9. Toy Examples Edit 1D function to match user constraints - could be multiple constraints t m(t) Old motion M={m(t)| t=1,…,T}

10. Toy Examples Edit 1D function to match user constraints - could be multiple constraints Old motion m(t), t=1,…,T t m(t) So how can we generate a new motion M’ to satisfy new constraints c?

11. Toy Examples: Idea #1 Modify the motion m(t), t=1,…,T using new constraints C t M(t) Old motion m(t), t=1,…,T

12. Toy Examples: Idea #1 Modify the motion m(t), t=1,…,T using new constraints C t m’(t) Does this work?

13. Toy Examples: Idea #1 Modify the motion m(t), t=1,…,T using new constraints C t m’(t) Does this work? - probably not, the new motion is not smooth!

14. Toy Examples: Idea #1 Modify the motion m(t), t=1,…,T using new constraints C t M’(t) Does this work? - probably not, the new motion is not smooth! - filter the motion!

15. Toy Examples: Idea #1 Modify the motion m(t), t=1,…,T using new constraints C Filter the edited motion M’ t M’(t) Does this work? - probably no, the new motion is not smooth! - filter the motion! - but the new motion now does not meet cons!

16. Toy Examples: Idea #1 Modify the motion m(t), t=1,…,T using new constraints C Filter the edited motion M’ t M’(t) Does this work? - probably no, the new motion is not smooth! - filter the motion! - repeat modification and filtering process!

17. Idea #1: Modification & Filter

18. Another Example: Three Constraints

20. Idea #1: Summary Pros - easy to implement - computationally efficient Cons - does not handle complex constraints, e.g., constraint functions involved more than one frames Old motion M={m(t)| t=1,…,T} t m(t) t1t1 t2t2 e.g., m’(t 1 )-m’(t 2 )=5

21. Idea #1: Summary Pros - easy to implement - computationally efficient Cons - does not handle complex constraints, e.g., constraint functions involved more than one frames How to address this limitation?

22. Idea #2: Constrained Optimization Modify the motion M to satisfy new constraints C Old motion m(t), t=1,…,T t m(t)

23. Idea #2: Constrained Optimization Modify the motion M to satisfy new constraints C Old motion m(t), t=1,…,T t m(t) But this is an ill-posed problem - There are tons of solutions that satisfy the new constraints C

24. Idea #2: Constrained Optimization Modify the motion M to satisfy new constraints C Old motion m(t), t=1,…,T t m(t) But this is an ill-posed problem - There are tons of solutions that satisfy the new constraints C - Remove the ambiguity by picking the solution with a minimal change across an entire motion

25. Idea #2: Constrained Optimization Modify the motion M to satisfy new constraints C Old motion m(t), t=1,…,T t m(t)

26. Idea #2: Constrained Optimization Modify the motion M to satisfy new constraints C Old motion m(t), t=1,…,T t m(t) Minimize the changes Satisfy new constraints

27. Idea #2: Constrained Optimization We can also formulate this as an unconstrained optimization problem Old motion m(t), t=1,…,T t m(t)

28. Idea #2: Constrained Optimization This is also called as spacetime optimization or trajectory optimization Simultaneously compute the entire motion Old motion m(t), t=1,…,T t m(t)

29. Idea #2: Constrained Optimization This is also called as spacetime optimization or trajectory optimization Simultaneously compute the entire motion Old motion m(t), t=1,…,T t m(t)

30. Now Back to Human Motion Editing!

31. A sequence of poses: q 1,q 2,…q T Each pose is represented as a high-dimensional vector q t : R n Motion trajectoriesPose q t Motion q 1,…q T Human motion representation

32. Human Motion Editing Human motion is a high-dimensional function Constraints could be any kinematic constraints

33. User Constraints Any kinematic constraints throughout the motion Position cons. Orientation cons. Distance cons. Joint angle cons.

34. Solution #1: Per-frame IK+Filtering Step 1: Per-frame inverse kinematics to modify poses in each frame with new constraints. - this might be ill-posed - remove ambiguity by minimizing the difference between old poses and new poses Step 2: motion filtering. Repeat step 1 and 2 until the solution converges

35. Solution #2: Spacetime Motion Editing Consider all constraints simultaneously Use optimization to solve the entire motion

36. Solution #2: Spacetime Motion Editing Consider all constraints simultaneously Use optimization to solve the entire motion - Currently, minimizing joint angle poses across the entire sequence - A better criterion is possible!

37. Solution #2: Spacetime Editing Better Criterion? - Geometric constraints such as contact constraints are often more important

38. Solution #2: Spacetime Editing Better Criterion? - Frequency Content or details

39. Solution #2: Spacetime Motion Editing Implementation details: - use Spline representation to reduce the solution space - often computationally expensive - optimization over sub windows when working on a long motion sequence

40. Motion Retargeting One motion, a cast of characters Focus on similar structure

41. Formulated as motion editing problem Motion Retargeting: Key Idea Skeleton of new characters

42. Step 1: Extract constraints/properties C from source motion Motion Retargeting: Key Idea Extracted constraints

43. Step 1: Extract constraints/properties C from source motion Motion Retargeting: Key Idea Extracted constraints Retargeted motionSource motion

44. Step 1: Extract constraints/properties C from source motion Motion Retargeting: Key Idea Extracted constraints Retargeted motionSource motion Skeleton of new characters

45. Step 2: Apply to new characters Motion Retargeting: Key Idea

46. Step 3: Approximate answer Motion Retargeting: Key Idea How to initialize m’(t)?

47. Step 4: Solve constraints using spacetime optimization Motion Retargeting: Key Idea

48. Motion Retargeting Youtube video (click here)here