1. Efficient Dynamic Skinning with Low-Rank Helper Bone Controllers Tomohiko Mukai, Tokai University Shigeru Kuriyama, Toyohashi University of Technology 1

2. Motivation 2

3. Wish List from Game Developers Robustness Simplicity –Compatibility with existing workflow Performance –Fast & Predictable execution time –Small memory footprint Quality –Physically-valid natural skin deformation 3 [Hecker 2011] –Plausible dynamic deformation × Details (e.g. wrinkles) × External forces (e.g. gravity) × Physical validity

4. Linear Blend Skinning (LBS) Robust Simple –Supported by most engines –Established tools High performance –Efficient & Predictable Dynamic deformation 4 [Thalmann 1988]

5. Helper Bone Rig 5 [Mohr 2003, Parks 2005, Kim 2011] Procedural control on CPU （ e.g. driven-key/expression in Maya ） Primary bone Helper bone LBS on GPU

6. Example-based Helper Bone Control 6 ・ State-space model ・ System identification ・ Nuclear norm optimization Skin weights & Helper bone transformation Example skin & skeleton animation

7. Helper Bone Controller 7 SSM of LTI Polynomial function [Mukai 2015]

8. Related Dynamic Skinning Methods 8 Dyna [Pons-Moll 2015] DMPL [Loper 2015] Position-based dynamics [Rumann 2015] Skeleton-driven elastic simulation [Park 2006] Pose-space subspace dynamics [Xu 2016] Kinodynamic skinning [Angelidis 2007] Elastic material param- eter learning [Shi 2008]

9. Building Helper Bone Controller 9

10. Input –Example skeleton animation –Example shape animation –Number of helper bones Output –Helper bone controller –Skinning weight Approximation criterion –Squared reconstruction error of vertex position Authoring System 10 ・・・

11. Example-based Controller Building - Static controller (muscle bulging) - Dynamic controller (jiggling) 11 Skinning decomposition [Mukai 2015, Le 2012] & [Mukai 2015] Skin weights & Helper bone transformation Example skin & skeleton animation

12. Dynamic Control with LTI System 12 Linear Time- Invariant system （ unknown ） Output (helper bone motion) Input (skeleton motion) Internal state （ unknown ） e.g. kinetic energy, inertia force

13. State-Space Model of LTI System 13 Helper bone motion Internal state at next time step Internal state Skeleton motion System matrix LTI

14. System Identification 14 Hankel matrix of helper bone motion Null-space projection of Hankel matrix of skeleton motion Truncated singular value decomposition Internal state & System matrix Cancel the linear effect of skeleton motion from helper bone motion

15. Dimensionality Reduction of Internal State 15 : Nuclear norm

16. N2SID: N uclear N orm S ystem ID entification Relaxing rank reduction problem into nuclear norm minimization problem User-specified precision of helper bone control 16 Example helper bone motion Helper bone motion minimizing nuclear norm [Liu 2013]

17. Algorithm Summary 17 SSM + N2SID Polynomial function [Mukai 2015]

18. Experimental Results 18

19. Monster’s Leg Model Maya muscle Height = 200 cm # of vertices = 663 # of muscles = 11 Joint DOF = 5 Training sequence of 4,000 frames 19

20. Sampling of Training Data 20 Spline interpolation Freeze interpolation

21. Runtime Rig with Four Helper Bones 21

22. Performance Evaluation 22 Method RMSE (cm) Execution time (μs/frame) avg. dim(z) Data size (kB) N4SID (w/o NNO) 3.0722.59.1751.0 2.5019.33.21 7.3 2.4819.23.00 7.1 2.4920.44.04 9.0

23. Autoregressive Model-based Control 23 [de Aguiar 2010, Pons-Moll 2015, Loper 2015]

24. Approximation of Human-like Muscle Rig http://www.behind-universe.org/ 24

25. Exaggeration of Dynamic Deformation 25 x 0.5x 1.0 x 2.0x 4.0 x 6.0

26. Level of Deformation Details 26 Dynamic + Static （ ~20 μs ） Static only (~5 μs) No control

27. Level of Deformation Details 27

28. Pros and Cons Pros ○Plausible dynamic skinning ○Simple implementation of state-space model ○Efficient computation via nuclear norm optimization Cons ×Detailed deformation (e.g. wrinkles) ×External force (gravity, contact, self-collision) ×Physical validity (volume preservation) 28

29. Conclusion Helper bone rig –Fully compatible with standard workflow State-space model –Simple and Robust （ via Eigen analysis ） Nuclear norm optimization –Efficient computation （ ~20 μs ） –Small memory footprint (~10 kB) Future work –Locally-adaptive rigging –Non-linear skinning models 29 Acknowledgements SIGGRAPH reviewers TAISO, Renpoo (www.behind-universe.org/) JSPS-KAKENHI 15K16110, 15H02704