Real-time Skeletal Skinning with Optimized Centers of Rotation Dionysi Alexandridis 5523532 Kevin Makatipu 4297024

Contents Introduction Related Work Linear Blend Skinning Method Results Limitations Conclusion Discussion

Contents Introduction Related Work Linear Blend Skinning Method Results Limitations Conclusion Discussion

Introduction https://doc.cgal.org/latest/Surface_mesh_deformation/index.html

Introduction Geometric skinning Physical based methods Skeletal skinning Physical based methods Collisions Jiggling Data-driven methods Blend shapes

Contents Introduction Related Work Linear Blend Skinning Method Results Limitations Conclusion Discussion

Related Work Log Matrix Skinning (LMS) Spherical Blend Skinning (SBS) Linear Blend Skinning (LBS) Dual Quaternion Skinning (DQS)

Contents Introduction Related Work Linear Blend Skinning Method Results Limitations Conclusion Discussion

Linear Blend Skinning Early ideas appeared in: Exact description in: BADLER, N. I., AND MORRIS, M. 1982. Modelling flexible articulated objects. In Proc. Computer Graphics’ 82, Online Conf, 305–314. MAGNENAT-THALMANN, N., LAPERRIÈRE, R., AND THALMANN, D. 1988. Joint-dependent local deformations for hand animation and object grasping. In Graphics Interface, 26–33.\ Exact description in: LEWIS, J. P., CORDNER, M., AND FONG, N. 2000. Pose space deformation: a unified approach to shape interpolation and skeleton-driven deformation. In Proceedings of ACM SIGGRAPH, 165–172. Kavan, L. 2014. SIGGRAPH Course 2014 - Skinning: Real-time Shape Deformation, Part I: Direct Skinning Methods and Deformation Primitives in ACM SIGGRAPH 2014 Courses

Linear Blend Skinning Input data: Rest pose shape: e.g. polygon mesh Bone transformations Skinning weights Output: Deformed mesh

Linear Blend Skinning 𝑣 𝑖 ′ = 𝑗=1 𝑚 𝑤 𝑖𝑗 𝑇 𝑗 𝑣 𝑖 𝑣 𝑖 ′ = 𝑗=1 𝑚 𝑤 𝑖𝑗 𝑇 𝑗 𝑣 𝑖 𝑤 𝑖𝑗 Skinning weight for vertex 𝑖 for bone 𝑗 𝑇 𝑗 Transformation for bone 𝑗 𝑣 𝑖 Vertex 𝑖

Linear Blend Skinning Interpolate according to skinning weights 𝑣 𝑖 ′ = 𝑗=1 𝑚 𝑤 𝑖𝑗 𝑇 𝑗 𝑣 𝑖

Linear Blend Skinning Interpolate according to skinning weights 𝑣 𝑖 ′ = 𝑗=1 𝑚 𝑤 𝑖𝑗 𝑇 𝑗 𝑣 𝑖

Linear Blend Skinning Interpolate according to skinning weights 𝑣 𝑖 ′ = 𝑗=1 𝑚 𝑤 𝑖𝑗 𝑇 𝑗 𝑣 𝑖

Contents Introduction Related Work Linear Blend Skinning Method Results Limitations Conclusion Discussion

Method – Requirements & Assumptions Backward compatible with existing animation pipelines Rest pose Skeleton Skinning weights Local transformations rigid Similar skinning weights? Similar transformations I’ll now start explaining our paper’s method of skinning. Most important requirement: backward compatible Two key assumptions

Method – Transformation For every vertex 𝒑 of rest pose, transform with 𝑅 𝒑 𝑡 𝒑 Get 𝑅 𝒑 with QLERP So how are we going to transform a rest-pose vertex? We want for every vertex p a transformation matrix. The rotation we can get with LERP, or more specifically…

Method – Transformation For every vertex 𝒑 of rest pose, transform with 𝑅 𝒑 𝑡 𝒑 Get 𝑅 𝒑 with QLERP And tp? …QLERP, which can interpolate in circles instead of straight lines. Won’t go into detail. But what about tp?

Method – Transformation Translation A vertex 𝒗 with similar skinning weights to 𝒑 has similar transformations 𝒑 and all similar vertices 𝒗 undergo rigid transformation ⇩ Try to rigidly translate, match 𝒑 and all 𝒗 to LBS-deformed pose as much as possible Use that translation as 𝑡 𝒑 ! Taking back our two assumptions. Try to translate entire region to match LBS as much as possible. LBS isn’t rigid, but we try to rigidly translate p and {v} to LBS. We use that translation purely for p.

Method – Transformation Translation How do we get all vertices 𝒗 similar to 𝒑? How do we match 𝒑 and all 𝒗 to LBS-deformed pose rigidly? Two problems remain: How do we get similar vertices? How do we match the regions with LBS? We’ll first tackle the first problem.

Method – Transformation Translation – Similarity How do we get all vertices 𝒗 similar to 𝒑? We want a similarity function 𝑠 𝑤 𝒑 , 𝑤 𝒗 Based on: Contribution of bones Some “distance” between weights How do we get {v}? We want a similarity function s(…) that gives the similarity. Based on these two things.

Method – Transformation Translation – Similarity We want a similarity function 𝑠 𝑤 𝒑 , 𝑤 𝒗 Based on: Contribution of bones Some “distance” between weights Compare effect of all pairs of bones 𝑗,𝑘 on 𝒑,𝒗: Contribution of bones: 𝑤 𝒑𝑗 𝑤 𝒑𝑘 𝑤 𝒗𝑗 𝑤 𝒗𝑘 (higher = better) Distance between weights: 𝑤 𝒑𝑗 : 𝑤 𝒑𝑘 ≈ 𝑤 𝒗𝑗 : 𝑤 𝒗𝑘 𝑤 𝒑𝑗 𝑤 𝒗𝑘 − 𝑤 𝒑𝑘 𝑤 𝒗𝑗 ≈0 p j v k Take our example arm back. For vertex p, we want to find similarity with a vertex v. Compare effect of all possible pairs of bones j and k on vertices. Contribution of bones is represented by this. Distance between weights by this. Ratio should be equal, shuffle things around and get this. We’re not going too in-depth on this.

Method – Transformation Translation – Similarity Compare effect of all pairs of bones 𝑗,𝑘 on 𝒑,𝒗: Contribution of bones: 𝑤 𝒑𝑗 𝑤 𝒑𝑘 𝑤 𝒗𝑗 𝑤 𝒗𝑘 (higher = better) Distance between weights: 𝑤 𝒑𝑗 : 𝑤 𝒑𝑘 ≈ 𝑤 𝒗𝑗 : 𝑤 𝒗𝑘 𝑤 𝒑𝑗 𝑤 𝒗𝑘 − 𝑤 𝒑𝑘 𝑤 𝒗𝑗 ≈0 𝑠 𝑤 𝒑 , 𝑤 𝒗 = ∀𝑗≠𝑘 𝑤 𝒑𝑗 𝑤 𝒑𝑘 𝑤 𝒗𝑗 𝑤 𝒗𝑘 𝑒 − 𝑤 𝒑𝑗 𝑤 𝒗𝑘 − 𝑤 𝒑𝑘 𝑤 𝒗𝑗 2 𝜎 2 So having this, our similarity function becomes this. Yellow sums over all pairs. Green is contribution of bones. Cyan is distance between weights, with the exponential making it so that higher = more similar. Now we have our similarity function.

Method – Transformation Translation How do we get all vertices 𝒗 similar to 𝒑? How do we match 𝒑 and all 𝒗 to LBS-deformed pose rigidly? Going back to the two problems we needed to solve to get tp, we now have solved the first one.

Method – Transformation Translation How do we get all vertices 𝒗 similar to 𝒑? ⇒ 𝑠 𝑤 𝒑 , 𝑤 𝒗 How do we match 𝒑 and all 𝒗 to LBS-deformed pose rigidly? We have our similarity function, but the second problem still remains.

Method – Transformation Translation How do we match 𝒑 and all 𝒗 to LBS-deformed pose rigidly? 𝑡 𝒑 = arg min 𝑡 𝒗∈Ω 𝑠 𝑤 𝒑 , 𝑤 𝒗 𝑅 𝒑 𝒗+𝑡− 𝒗 𝟐 ⅆ𝒗 𝒗 = 𝑗=1 𝑚 𝑤 𝒑𝑗 𝑅 𝑗 𝒗+ 𝑡 𝑗 = LBS-deformed 𝒗 So how do we deform p and similar vertices {v}? With this function. Again, I won’t go in-depth. Basically: For every vertex in the pose: Rotate vertices Try out different t and translate Subtract by LBS-deformed v To get distance and thus error Multiply by similarity Get the t that minimizes this

Method – Transformation Translation 𝒑 𝑳𝑩𝑺 ∗ How do we match 𝒑 and all 𝒗 to LBS-deformed pose rigidly? 𝑡 𝒑 = arg min 𝑡 𝒗∈Ω 𝑠 𝑤 𝒑 , 𝑤 𝒗 𝑅 𝒑 𝒗+𝑡− 𝒗 𝟐 ⅆ𝒗 = 𝑗=1 𝑚 𝑤 𝒑𝑗 𝑅 𝑗 𝒑 ∗ + 𝑡 𝑗 − 𝑅 𝒑 𝒑 ∗ 𝑡 𝒑 ∗ 𝒑 𝒓𝒆𝒔𝒕 ∗ 𝒑 ∗ independent of bone transformations! Pre-compute it 𝒑 ∗ plays role of center of rotation (CoR) of rest-pose 𝒑 Yellow = LBS applied to 𝒑 ∗ = CoR of transformed 𝒑 𝒑 ∗ = 𝒗∈Ω 𝑠 𝑤 𝒑 , 𝑤 𝒗 𝒗ⅆ𝒗 𝒗∈Ω 𝑠 𝑤 𝒑 , 𝑤 𝒗 ⅆ𝒗 Math magic rewrites the function to this. Minimization problem is removed. p* is substitution, weird integral that takes long to compute. a to b = area triangle, height = similarity. Anyhow, observations the researchers made. p* can be pre-computed. p* = CoR. Yellow = LBS p*.

Method – Transformation Translation How do we get all vertices 𝒗 similar to 𝒑? ⇒ 𝑠 𝑤 𝒑 , 𝑤 𝒗 How do we match 𝒑 and all 𝒗 to LBS-deformed pose rigidly? Know we finally know how to match p and {v} to LBS.

Method – Transformation Translation How do we get all vertices 𝒗 similar to 𝒑? ⇒ 𝑠 𝑤 𝒑 , 𝑤 𝒗 How do we match 𝒑 and all 𝒗 to LBS-deformed pose rigidly? ⇒ 𝑗=1 𝑚 𝑤 𝒑𝑗 𝑅 𝑗 𝒑 ∗ + 𝑡 𝑗 − 𝑅 𝒑 𝒑 ∗ With this function, pre-computing p*.

Method – Transformation For every vertex 𝒑 of rest pose, transform with 𝑅 𝒑 𝑡 𝒑 Get 𝑅 𝒑 with QLERP And tp? So now we have both the rotation and the translation of our transformation!

Method – Transformation For every vertex 𝒑 of rest pose, transform with 𝑅 𝒑 𝑡 𝒑 Get 𝑅 𝒑 with QLERP And tp with 𝑗=1 𝑚 𝑤 𝒑𝑗 𝑅 𝑗 𝒑 ∗ + 𝑡 𝑗 − 𝑅 𝒑 𝒑 ∗ Here.

Contents Introduction Related Work Linear Blend Skinning Method Results Limitations Conclusion Discussion

Results https://www.youtube.com/watch?v=DfIfcQiC2oA&t=164

Results Method is not sensitive to parameter selection

Results Method is not sensitive to parameter selection Low pre-computational time

Results Method is not sensitive to parameter selection Low pre-computational time Tackles Artifacts produced by LBS and DQS Volume Loss Collapsing elbow Bulging

Contents Introduction Related Work Linear Blend Skinning Method Algorithm Results Limitations Conclusion Discussion

Limitations Limitations of direct skinning methods: Intersections Jiggling Still produces volume loss For 3D-CoR bodies Determining skinning weights Real-time visualization of CoRs

Contents Introduction Related Work Linear Blend Skinning Method Algorithm Results Limitations Conclusion Discussion

Conclusion Addresses common artifacts of LBS and DQS Can still produce volume loss at smaller rate Drop-in for many existing applications Compute CoRs from only rest-pose Performs in real-time

Contents Introduction Related Work Linear Blend Skinning Method Algorithm Results Limitations Conclusion Discussion

Discussion The authors assume that “local transformations are rigid”, however distance between vertices is never taken into account by the provided equations. Do you think that this assumption is necessary? We expect that the authors assume that vertices with similar skinning weights are close to each other, since the distance between vertices is not taken into account in their method. Their method also tries to transform the vertices with similar skinning weights rigidly, which together with their assumption implies that local transformations are also rigid. We also expect that vertices with similar skinning weights, that also have a large distance to each other, are uncommon.

Questions?