1. Real-Time Enveloping with Rotational Regression Robert Wang Kari Pulli Jovan Popović

2. Enveloped (skinned) characters are pervasive. Skeletons are often used to control meshes. skeletonmesh

3. Physically based modeling provides realistic deformations.  Realistic deformations – Finite-element based[Teran et al. 2005] – Anatomy based[Scheepers et al. 1997] – Elastically deformable[Capell et al. 2002, 2005] – Used in movie production – Off-the-shelf commercial tools  Slow evaluation [Teran et al. 2005] [Absolute Character Tools 1.6]

4. We learn a fast model from exported examples. Exported Examples (skeleton-mesh pairs) Fast Model Our method Black Box Simulation

5. Artists can still use existing modeling tools or scanned data. Exported Examples (skeleton-mesh pairs) Fast Model 3-D Scan Data

6. This is analogous to mesh simplification. High-resolution mesh Low-resolution mesh mesh simplification  Higher quality  Used in movie production  Faster to render  Optimized for interactive applications Physical simulation Rotational Regression Enveloping learning

7. How do we map a skeleton to a mesh? ? What parameters should we learn? How to model muscle deformations for fast evaluation?

8. Linear blend skinning linearly maps joint rotations to vertex positions.  Most popular enveloping technique for games  Coarse modeling parameters (but very simple)  Not very expressive (but very fast) + y y Figure from [Wang and Phillips 2002]

9. Linear blend skinning has many names.  Also known as, – Single-Weight Enveloping – Skeletal Subspace Deformation (SSD) – Or just, “Skinning”  We will use “Linear Blend Skinning” or “SSD.”

10. The two steps of our work are deformation gradients prediction and mesh reconstruction. Mesh reconstruction Deformation gradients prediction (Rotational Regression)

11. We present a replacement for linear blend skinning.  Coarse modeling parameters.  Can’t handle certain types of deformations.  Fast  Lets you use your existing modeling tool.  Good for muscle bulges.  Fast Whenever you have an existing model, you should use our technique instead of linear blend skinning. +

12.  Rigid components move with the bone rotation  Other surfaces rotate in the opposite direction Our model is inspired by the behavior of a flexing bicep. Surface rotation Bone rotation

13. Angle is scaled by u. Axis is offset by rotation W. source rotation (bone) target rotation (surface)

14. We map a sequence of bone rotations to a sequence of surface rotations. source rotation sequence (bone) target rotation sequence (surface) ……

15. We fit parameters u and W by regression. Skeleton rotations Surface rotations u’,W’ Best-fit parameters

16. Rotational regression is good at capturing muscle bulges.

17. Mesh reconstruction stitches deformation gradients together. Deformation gradients prediction Mesh reconstruction

18.  Least-squares problem equivalent to linear system.  Computation is matrix- multiplication. Mesh reconstruction solved with least-squares. deformation gradients vertex positions Least –squares C D(q)D(q)

19. Near-rigid vertices help eliminate low-frequency errors at extremities.  Low-frequency errors can accumulate at extremities of mesh  We fix a set of near-rigid vertices to their SSD predictions  Still a least squares problem

20. We build upon existing mesh reconstruction work.  Mesh IK[Sumner et al. 2005], [Der et al. 2006]  SCAPE[Anguelov et al. 2005]  Similar formulation, faster evaluation. [Anguelov et al. 2005]

21. Here’s a review of what we’ve covered. Rotational Regression Deformation Gradients Prediction Mesh Reconstruction Least-squares problem C Dk(q)Dk(q)

22. Model reduction lowers the dimensionality of problem.  Large multiplication on CPU  Smaller multiplications on GPU Dk(q)Dk(q) C  C’C’  Dl(q)Dl(q) SSD

23. Model reduction uses greedy clustering.  Vertices clustered into proxy-bones.  Per-triangle deformation gradients clustered into “key” deformation gradients. P = 45022511050 25

24. Mesh reconstruction reduced to the following matrix-multiplications. C’C’  Dl(q)Dl(q) SSD weights “key” deformation gradients Map from “key” deformation gradients to proxy-bones All on GPU:  Computation in fragment program

25. Skinning Mesh Animations is a an alternative approach to model reduction.  The method from Skinning Mesh Animations uses mean-shift clustering and is more robust to errors. [James and Twigg 2005]  Our method minimizes vertex error and is faster

26. Deformation gradients prediction is now on “key” deformation sequences.  Fewer deformation gradient sequences to predict rotational regression.

27. Mesh reconstruction step reduced to matrix-multiplications on GPU.  Smaller matrix- multiplications  Supported on graphics hardware C’C’  Dl(q)Dl(q)

28. Our Technical Contributions: Rotational Regression Accurate and GPU-Ready Poisson Reconstruction Model Reduction

29. Results

32. Our work approximates the training examples better than SSD and also generalizes well.

33. Our model is suitable for interactive techniques.  Evaluation speed within a factor of two of SSD  Off-line training preprocess is usually less than half an hour

34. How does our work fit with previous work?

35. Our work is complementary to displacement correcting techniques.  Previous work provide corrective displacements. – Pose space deformation[Lewis et al. 2000], – Shape by example[Sloan et al. 2001], – Eigenskin[Kry et al. 2002]  Our work provides better approximation of rotations.  Our work complements approaches that build upon SSD. Figure from [Kry et al. 2001]

36. Displacement correcting approaches fail when SSD is very wrong.

37. Our work builds upon previous ideas on enriching the SSD model.  Multi-weight enveloping[Wang and Phillips 2002]  Additional joints[Mohr and Gleicher 2003]  Our technique has more parameters than SSD and generalizes the additional-bones model.

38. A more expressive model is useful here.

39. Our model doesn’t do a perfect job.  Not perfect reproduction – Inspired by muscle bulging and twisting. Other behaviors empirically validated. – Displacement correcting technique can be used for exact reproduction of examples.

40. Conclusion: Fast and accurate enveloping.  Fast evaluation of physical simulations through learning. – Within a factor of two of SSD on most models  Accurate reproduction of details – Better approximation and generalization – Complementary to previous work  A replacement for linear blend skinning

41. Acknowledgements  Funding – Nokia Research Center – National Science Foundation – Pixar Animation Studios  Hardware/Software – NVIDIA Corporation – Autodesk  Data – Drago Anguelov – Joel Anderson – Michael Comet, Comet Digital, LLC – Mark Snoswell, CG Character – Joseph Teran, Ron Fedkiw  MIT Graphics Group – Ilya Baran – Jiawen Chen – Sylvain Paris

42. Questions?  Thank you for coming to our talk!

43. Learning tasks trade expressiveness and simplicity. More Expressive: Captures more types of deformation. Simpler: Easier to fit Fewer training examples needed. Less likely to overfit. Rotational Regression

44. Linear blend skinning (SSD) is a rough and ready map from joint rotation matrices to vertex positions.  Most popular enveloping technique for games  Coarse modeling parameters (but very simple)  Not expressive enough (but very fast) desired deformation SSD deformation

45. Model Reduction  True optimization not as tractable  We approximate it with a greedy algorithm inspired by mesh simplification. difficult to solve simultaneously discrete optimization

46. Our work builds upon previous ideas on  Additional joints[Mohr and Gleicher 2003]  Multi-weight enveloping[Wang and Phillips 2002]  Our technique generalize the additional- bones model  We evaluate cross-validation error to test for overfitting [Wang and Phillips 2002]

47. Rotational regression is good at capturing muscle bulges.