Spherical Skinning with Dual-Quaternions and QTangents

Slides:



Advertisements
Similar presentations
Stupid OpenGL Shader Tricks
Advertisements

Low-Complexity Transform and Quantization in H.264/AVC
Today Composing transformations 3D Transformations
Points, Vectors, Lines, Spheres and Matrices
16.1 Si23_03 SI23 Introduction to Computer Graphics Lecture 16 – Some Special Rendering Effects.
Animation in Video Games presented by Jason Gregory
Common Variable Types in Elasticity
Linear Transformations and Matrices Jim Van Verth Lars M. Bishop
Vectors, Points, Lines and Planes Jim Van Verth Lars M. Bishop
Epipolar Geometry.
5.1 Real Vector Spaces.
6.4 Best Approximation; Least Squares
Common Variable Types in Elasticity
CS123 | INTRODUCTION TO COMPUTER GRAPHICS Andries van Dam © 1/16 Deferred Lighting Deferred Lighting – 11/18/2014.
Normal Map Compression with ATI 3Dc™ Jonathan Zarge ATI Research Inc.
Computer Graphics Lecture 4 Geometry & Transformations.
Technische Universität München Fakultät für Informatik Computer Graphics SS 2014 Transformations Rüdiger Westermann Lehrstuhl für Computer Graphik und.
Dual Numbers: Simple Math, Easy C++ Coding, and Lots of Tricks
Real-time Geometry Caches
The Programmable Graphics Hardware Pipeline Doug James Asst. Professor CS & Robotics.
Chapter 4.1 Mathematical Concepts. 2 Applied Trigonometry Trigonometric functions Defined using right triangle  x y h.
CSCE 590E Spring 2007 Basic Math By Jijun Tang. Applied Trigonometry Trigonometric functions  Defined using right triangle  x y h.
3D orientation.
CSCE 689: Computer Animation Rotation Representation and Interpolation
Midterm Review CSE169: Computer Animation Instructor: Steve Rotenberg UCSD, Winter 2004.
University of Texas at Austin CS 378 – Game Technology Don Fussell CS 378: Computer Game Technology Beyond Meshes Spring 2012.
Week 4 - Monday.  What did we talk about last time?  Vectors.
09/18/03CS679 - Fall Copyright Univ. of Wisconsin Last Time Bump Mapping Multi-pass algorithms.
CS 450: COMPUTER GRAPHICS 3D TRANSFORMATIONS SPRING 2015 DR. MICHAEL J. REALE.
Velocities and Static Force
Computer Graphics Mirror and Shadows
Mathematical Fundamentals
Computer Graphics: Programming, Problem Solving, and Visual Communication Steve Cunningham California State University Stanislaus and Grinnell College.
Polygon Shading. Assigning color to a shape to make graphical scenes look realistic, or artistic, or whatever effect we’re attempting to achieve But first.
CS 450: Computer Graphics REVIEW: OVERVIEW OF POLYGONS
Mathematical Topics Review of some concepts:  trigonometry  aliasing  coordinate systems  homogeneous coordinates  matrices, quaternions.
Geometric Transformation. So far…. We have been discussing the basic elements of geometric programming. We have discussed points, vectors and their operations.
CS 450: COMPUTER GRAPHICS QUATERNIONS SPRING 2015 DR. MICHAEL J. REALE.
Computer Graphics Bing-Yu Chen National Taiwan University.
Elementary Linear Algebra Anton & Rorres, 9th Edition
Buffers Textures and more Rendering Paul Taylor & Barry La Trobe University 2009.
09/09/03CS679 - Fall Copyright Univ. of Wisconsin Last Time Event management Lag Group assignment has happened, like it or not.
Week 5 - Wednesday.  What did we talk about last time?  Project 2  Normal transforms  Euler angles  Quaternions.
Maths and Technologies for Games Quaternions CO3303 Week 1.
视觉的三维运动理解 刘允才 上海交通大学 2002 年 11 月 16 日 Understanding 3D Motion from Images Yuncai Liu Shanghai Jiao Tong University November 16, 2002.
CMPS 1371 Introduction to Computing for Engineers MATRICES.
Advanced Computer Graphics Advanced Shaders CO2409 Computer Graphics Week 16.
Maths & Technologies for Games Animation: Practicalities CO3303 Week 3.
Matthew Christian. About Me Introduction to Linear Algebra Vectors Matrices Quaternions Links.
CS559: Computer Graphics Lecture 8: 3D Transforms Li Zhang Spring 2008 Most Slides from Stephen Chenney.
09/16/03CS679 - Fall Copyright Univ. of Wisconsin Last Time Environment mapping Light mapping Project Goals for Stage 1.
Review on Graphics Basics. Outline Polygon rendering pipeline Affine transformations Projective transformations Lighting and shading From vertices to.
Affine Geometry.
COMPUTER GRAPHICS CS 482 – FALL 2015 SEPTEMBER 29, 2015 RENDERING RASTERIZATION RAY CASTING PROGRAMMABLE SHADERS.
Computer Graphics Matrices
3D Game Engine Design 1 3D Game Engine Design Ch D MAP LAB.
Mesh Skinning Sébastien Dominé. Agenda Introduction to Mesh Skinning 2 matrix skinning 4 matrix skinning with lighting Complex skinning for character.
Basic Theory (for curve 01). 1.1 Points and Vectors  Real life methods for constructing curves and surfaces often start with points and vectors, which.
Honors Geometry.  We learned how to set up a polygon / vertex matrix  We learned how to add matrices  We learned how to multiply matrices.
Week 5 - Monday.  What did we talk about last time?  Lines and planes  Trigonometry  Transforms  Affine transforms  Rotation  Scaling  Shearing.
Week 4 - Monday CS361.
Deferred Lighting.
Joshua Barczak CMSC 435 UMBC
Computer Animation and Visualisation Lecture 4. Skinning
UMBC Graphics for Games
with Applications in Computer Graphics
Lecture 8: 3D Transforms Li Zhang Spring 2008
UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN
Game Programming Algorithms and Techniques
Presentation transcript:

Spherical Skinning with Dual-Quaternions and QTangents Ivo Zoltan Frey Crytek R&D

#2 Reduce the memory foot-print of skinned geometry Goals #1 Improve performance by reducing the shader constant requirements for joint transformations 30% shader constants reduction #2 Reduce the memory foot-print of skinned geometry 22% vertex memory reduction 29% for static geometry

Skinned Geometry

Goal #1 Improve performance by reducing the shader constant requirements for joint transformations Skinned geometry requires multiple passes Motion Blur requires twice the transformations The amount of required shader constants affects the performance of a single pass

Skinning with Quaternions ~30% less shader constants consumption compared to 4x3 packed matrices Quaternion Linear Skinning Accumulated transformations don’t work for positions Explosion of vertex instructions Quaternion Spherical Skinning [HEIJL04] Extra vertex attribute required Doesn’t handle well more than 2 influences per vertex Dual-Quaternion Skinning [KCO06] [KCZO08] Increase in vertex instructions [HEIJL04] also requires careful re-rigging of the skinned geometry.

Dual-Quaternion Skinning [KSO06] [KCZO08] Compared to Linear Skinning with matrices Accumulation of transformations is faster Applying the transformation is slower With enough influences per vertex it becomes overall faster The reduction of shader constants was a win over the extra vertex instructions cost We offload antipodality handling on the CPU by making sure the child is in the same hemisphere as the parent when we generate the dual-quaternion transformations. for (int i=0; i<jointCount; ++i) { int p = GetJointParentIndex(i); if (p < 0) continue; DualQuat& dq = GetJointTransformation(i); const DualQuat& pdq = GetJointTransformation(p); if (dot(dq, pdq) < 0.0f) SetJointTransformation(i, -dq); }

From Linear to Spherical Geometry needs to be rigged differently And you will still need your helper joints Riggers and Animators need to get used to it Some will love it, others will hate it Most will keep changing their mind You might have to write skinning plug-ins for third party authoring software Some recent authoring packages have adopted Dual-Quaternion Skinning out of the box Although Spherical Skinning techniques claim to reduce the need for deformation helper joints, we found that for our production geometry we still require them to reach our target quality. Given enough processing power the ideal solution would be to arbitrary chose whether to use Linear or Spherical Skinning on a per joint basis.

Reduce the memory foot-print of skinned geometry Goal #2 Reduce the memory foot-print of skinned geometry We are now developing on consoles, every byte counts! More compact vertex format will also lead to better performance Do not sacrifice quality in the process!

In further optimizing them we need to ensure that Tangent Frames Tangent Frames were the biggest vertex attribute after our trivial memory optimizations In further optimizing them we need to ensure that They keep begin efficiently transformed by Dual-Quaternions All our Normal Maps keep working as they are Our trivial memory optimizations mostly involved moving from 32bit floating-point formats to 16bit floating-point and integer formats.

Please make them orthogonal! About Tangent Frames Please make them orthogonal! If they are not, you are introducing skewing You can’t use a transpose to invert the frame matrix You need a full matrix inversion This will also prevent you from using some compression techniques! Most geometry in modern productions are made of unique uv atlases, at times shared through the use of mirroring to mitigate memory constraints. For performance reasons the welding of vertices is also encouraged, which at times sacrifices smoothing accuracy across vertex normal. As long as the exact inverse of the transformation used to bring object space normals into tangent space is used to bring them back into object space, the correctness of the frame as far as exactly following the texture space is not required. Geometry where more accurately following the texture space is desirable turns out to mostly already yield to a orthogonal frame.

Compressed Matrix Format Vertex attributes contain two of the frame’s vectors and a reflection scalar The third frame’s vector is rebuild from a cross product of the given vectors and a multiplication with the reflection scalar normal = cross(tangent, biTangent) * s Tangent BiTangent Reflection x y z s Industry standard for storing a compressed form of Tangent Frames. Note that if skewing is present on the normal (i.e. resulting form vertex welding for performance optimization), this compression technique won’t yield to the correct results!

Tangent Frames With Quaternions Quaternion to Matrix conversion t = transform(q, vec3(1, 0, 0)) b = transform(q, vec3(0, 1, 0)) n = transform(q, vec3(0, 0, 1)) Quaternions don’t natively contain reflection information

Bringing Reflection Into the Equation Similarly to the compressed matrix format, we can introduce reflection with a scalar value t = transform(q, vec3(1, 0, 0)) b = transform(q, vec3(0, 1, 0)) n = transform(q, vec3(0, 0, 1)) * s

Tangent Frame Format Memory Comparison Compressed Matrix Tangent BiTangent Reflection x y z s Quaternion Reflection x y z w s

Our Quaternion Properties They are normalized length(q) == 1 And they are sign invariant q == -q

Quaternion Compression We can compress a Quaternion down to three elements by making sure one of the them is greater than or equal to zero if (q.w < 0) q = -q We can then rebuild the missing element with q.w = sqrt(1 – dot(q.xyz, q.xyz))

Tangent Frame Format Memory Comparison Compressed Matrix Tangent BiTangent Reflection x y z s Quaternion Reflection x y z w s Compressed Quaternion Quaternion Reflection x y z s Note that if 16bit formats are used the Compressed Matrix format will require 8 elements while the Quaternion format will require 6 elements.

Instruction Cost Quaternion decompression 5 mov, dp3, add, rsq, rcp Quaternion to Tangent and BiTangent 6 add, mul, mad, mad, mad, mad Normal and Reflection computation 3 mul, mad, mul Total 11 for Tangent, BiTangent and Reflection 14 for full Tangent Frame

Avoiding Quaternion Compression Isn't there a way to encode the reflection scalar in the Quaternion, instead of compressing it? Remember, Quaternions are sign invariant q == -q We can arbitrarily decide whether one of its elements has to be negative or positive!

Encoding Reflection First we initialize the Quaternion by making sure q.w is always positive if (q.w < 0) q = -q If then we require reflection, we make q.w negative by negating the entire Quaternion if (reflection < 0) q = -q

Decoding Reflection All we have to do in order to decode our reflection scalar is to check for the sign of q.w reflection = q.w < 0 ? -1 : +1 As for the Quaternion itself, we can use it as it is! q = q Note that the comparison with “< 0” will return false even in floating-point values that are “-0”. On CPU the sign bit needs to be explicitly set and checked for to avoid the need for the bias described in the upcoming slides. This is purposely left wrong in order to lead into the upcoming singularity handling slides.

Instruction Cost Reflection decoding 2 slt, mad Quaternion to Tangent and BiTangent 6 add, mul, mad, mad, mad, mad Normal and Reflection computation 3 mul, mad, mul Total 8 for Tangent, BiTangent and Reflection 11 for full Tangent Frame

Tangent Frame Transformation with Dual-Quaternion Quaternion-Vector transformation | | float3x3 frame; 6 | frame[0] = transform_quat_vec( | skinningQuat, vertex.tangent.xyz); | 6 | frame[1] = transform_quat_vec( | skinningQuat, vertex.biTangent.xyz); | 2 | frame[2] = cross(frame[0], frame[1]); 1 | frame[2] *= vertex.tangent.w; | 15 instructions Quaternion-Quaternion transformation | 5 | float4 q = transform_quat_quat( | skinningQuat, vertex.qTangent) | 8 | float3x3 frame = quat_to_mat(q); | 3 | frame[2] *= vertex.qTangent.w < 0 ? -1 : +1; | | | | 16 instructions Although the Quaternion-Quaternion transformation is 1 instruction longer, it ends up being faster since it requires 1 less vertex attribute. float3 transform_quat_vec(const float4 quat, const float3 vec) { // mad + mul + mad + mul + mad + mad return vec + cross(quat.yzw, cross(quat.yzw, vec) + quat.w * vec) * 2.0; } float4 transform_quat_quat(const float4 q, const float4 p) float4 c, r; // mul + mad : 2 c.xyz = cross(q.xyz, p.xyz); // dot : 1 c.w = -dot(q.xyz, p.xyz); // mad : 1 r = p * q.w + c; r.xyz = q * p.w + r; return r; float3x3 quat_to_mat(const float4 quat) // add : 1 alu float4 q2 = quat + quat; // mul : 1 alu float3 r2 = q2.w * float3(-1.0, 0.0, 1.0); // mad : 1 alu float3 r0 = quat.wzy * r2.zxz + float3(-1.0, 0.0, 0.0); r0 = quat.x * q2.xyz + r0; float3 r1 = quat.zwx * r2.zzx + float3(0.0, -1.0, 0.0); r1 = quat.y * q2.xyz + r1; // mul + mad : 2 alu r2 = cross(r0, r1); return float3x3(r0, r1, r2);

QTangent Definition A Quaternion of which the sign of the scalar element encodes the Reflection

Stress-Testing QTangents By making sure we throw at it our most complex geometry!

Singularity Found!

Singularity Found! At times the most complex cases pass, while the simplest fail!

Singularity Our singularities manifest themselves when the Quaternion’s scalar element is equal to zero Matrix -1, 0, 0 Quaternion 0, -1, 0 0, 0, 1, 0 0, 0, 1 This means the Tangent Frame’s surface is perpendicular to one of the identity’s axis A box aligned with the identity’s axis and requiring reflection will have 3 of its 6 faces manifest our singularity.

Floating-Point Standards So what happens when the Quaternion’s scalar element is 0? The IEEE Standard for Floating-Point Arithmetic does differentiate between -0 and +0, so we should be fine! However GPUs don’t exactly always comply to this standard, at times for good reasons

GPUs Floating-Point “Standards” GPUs allow vertex attributes to be specified as integers representing normalized unit scalars They are then resolved into Floating-Point values Integers don’t differentiate between -0 and +0, thus this information is lost in the process Behavior on how GPUs handle floating-points -0/+0 can also vastly vary from hardware to hardware.

Handling Singularities In order to use integers to encode reflection, we need to ensure that q.w is never zero When we find q.w to be zero, we need to apply a bias

Defining Our Bias Constant We define our bias constant as the smallest value that will satisfy q.w != 0 If we are using an integer format, this value is given by bias = 1 / (2BITS-1 – 1)

Applying the Bias Constant We need to apply our bias for each Quaternion satisfying q.w < bias, and while doing so we make sure our Quaternion stays normalized if (q.w < bias) { q.xyz *= sqrt(1 - bias*bias) q.w = bias }

QTangents with Skinned Geometry Position 4 float16 8 bytes TexCoord 2 float16 4 bytes Tangent 4 int16 BiTangent SkinIndices 4 uint8 SkinWeights From 36 bytes to 28 bytes per vertex ~22% memory saved No overhead with Dual-Quaternion Skinning ~8 instruction overhead with Linear Skinning

QTangents with Static Geometry Position 4 float16 8 bytes TexCoord 2 float16 4 bytes Tangent 4 int16 BiTangent From 28 bytes to 20 bytes per vertex ~29% memory saved ~8 instruction overhead

Future Developments Quaternions across polygons Interpolating Quaternions across polygons and making use of them at the pixel level Quaternions in G-Buffers Encoding the whole Tangent Frame instead of just Normals Can open doors to more Deferred techniques Anisotropic Shading Directional blur along Tangents

Special Thanks Ivo Herzeg, Michael Kopietz, Sven Van Soom, Tiago Sousa, Ury Zhilinsky Chris Kay, Andreas Kessissoglou, Mathias Lindner, Helder Pinto, Peter Söderbaum Crytek

References [HEIJL04] Heijl, J., "Hardware Skinning with Quaternions", Game Programming Gems 4, 2004 [KCO06] Kavan, V., Collins, S., O'Sullivan, C., "Dual Quaternions for Rigid Transformation Blending", Technical report TCD-CS-2006-46, 2006 [KCZO08] Kavan, V., Collins, S., Zara, J., O'Sullivan, C., "Geometric Skinning with Approximate Dual Quaternion Blending", ACM Trans. Graph, 2008

Questions? ivof@crytek.com