Large Mesh Deformation Using the Volumetric Graph Laplacian Kun Zhou Jin Huang* John Snyder^ Xinguo Liu Hujun Bao* Baining Guo Heung-Yeung Shum Microsoft Research Asia *Zhejiang University ^Microsoft Research Good afternoon. My name is Kun Zhou. In this work, we propose a technique for large mesh deformation.

Mesh Deformation Smooth geometry Detailed geometry Freeform deformation [Barr84, Singh98, Bendels03] Energy minimization [Welch94, Taubin95, Botsch04] Detailed geometry Multi-resolution editing [Zorin97, Kobbelt98, Guskov99] Differential domain methods: Poisson mesh editing [Yu04] Laplacian surface editing [Sorkine04, Lipman05] Mesh deformation has long been an active research area in computer graphics. Traditional techniques, like freeform deformation and energy minimization, usually deal with smooth surfaces. For detailed geometry, such as that acquired from scanning devices, multi-resolution methods provide a better solution. A model is first decomposed into a smooth base shape and a set of geometric details such as displacement vectors. After modifying the base shape with some freeform deformation, the details can be re-inserted. Recent differential domain editing methods, like Poisson mesh editing and Laplacian surface editing, also encode surface details using local derivatives or differences. Unlike multi-resolution methods which add details back directly, these methods reconstruct details by optimization, so errors are distributed more uniformly over the entire surface.

Large Mesh Deformation Challenge to existing techniques Local self-intersection, unnatural volume change Large deformations remain challenging today. Existing techniques often produce implausible results with self-intersections or unnatural volume changes. The bending on the left produces local pinching and self-intersection. The twisting on the right leads to undesirable shrinkage. Bending Twisting

Large Mesh Deformation Challenge to existing techniques Local self-intersection, unnatural volume change Compared to the previous technique of Poisson mesh editing, our approach eliminates this self-intersection artifact caused by a large bend. Poisson Mesh Editing VGL

Large Mesh Deformation Challenge to existing techniques Local self-intersection, unnatural volume change And our technique better preserves volume in this large twist example. Poisson Mesh Editing VGL

Large Deformation: Why Difficult? Differential domain methods [Yu04, Sorkine04]  Uniform error distribution using least-squares optimization  Only surface details, volume ignored Displacement volumes [Botsch03]  Volumetric constraints  Iterative relaxation produces artifacts Why is large deformation difficult? Recent differential domain methods distribute errors uniformly over the whole mesh by using least-squares optimization. However they preserve only surface details and ignore volume, leading to artifacts under large deformations. Some researchers realized the importance of volumetric constraints and proposed “displacement volumes”. Unfortunately, the use of iterative relaxation tends to lose detail in highly deformed regions. Thus, the key to large deformation is a good way to enforce volumetric constraints, which should be solved more efficiently by applying least-squares optimization. Solution: Volumetric Constraints & Least-Squares Optimization

Poisson Mesh Editing F F’ Step 1: Specify control curve Our technique builds upon our last year’s work “Poisson mesh editing”. I will review it here and explain why it is not good for large deformation. Then I will describe our new method. Poisson mesh editing consists of 4 steps. First, users specify a control curve. Each vertex on the control curve has a local frame F. Secondly, users perform editing operations specifying where the control curve deforms to. This leads to a new local frame on the deformed control curve. Step 1: Specify control curve Step 2: Edit control curve

Poisson Mesh Editing Step 3: Propagate local frame transformations Thirdly, the local frame changes on the control curve are propagated to triangles along the geodesic paths on the mesh. Each triangle vertex is moved by the propagated transform, disconnecting the the triangles. Finally, we solve Poisson equation to smoothly reconnect these triangles and obtain the deformed result. Step 3: Propagate local frame transformations Step 4: Solve Poisson equation

Surface Details & Laplacian Poisson mesh represents surface detail using surface Laplacian. In the following, we will use this 2D cat shape for description clarity. For example, given this 2D cat shape, the Laplacian coordinate of point Pi is computed as the difference between it and the average of its neighbors.

Surface Details & Laplacian Now assume a user coarsely specifies some deformation, such as pulling the cat’s front foot forward. Minimizing the sum of squared changes to the Laplacian coordinates after the deformation automatically preserves surface details.

Surface Details & Laplacian If the deformation gets too large, pinching artifacts like this can easily happen. They arise because the surface deforms without regard for preserving interior volume.

Volumetric Details & Laplacian Our solution is to introduce the idea of volumetric details. First we build a volumetric graph inside the mesh. Then for each point in the graph, the difference between the point and its neighbors is computed. We call this difference “volumetric details” or “volumetric graph Laplacian” of the point.

Volumetric Details & Laplacian Preserving volumetric details during deformation improves results by eliminating those pinching artifacts.

Quadric Energy Function Surface Details Position Constraints Volumetric Details Applying these same ideas to 3D surfaces, we present a general energy function for large mesh deformation. The first part corresponds to surface details preserving. The second part is position constraints posed by users. The third part is volumetric details. The whole energy is a quadric function. Minimizing this quadric energy functions turns out to be a sparse linear system. We can simply view it as Ax equals b. Notice that the matrix A is dependent only on the graph before deformation while b is also dependent on the new Laplacian coordinates and position constraints. Thus, we precompute the inverse of A using LU decomposition and dynamically execute the back substitution step to obtain x. Since A is fixed during deformation, this solution is very efficient and allows the user to change the deformation target interactively.

Deformation Using VGL Construct the volumetric graph Compute Laplacian coordinates Compute and apply local transformation Solve the sparse linear system Now we summarize the 4 steps of VGL deformation. First, construct a volumetric graph for the input mesh. Second, compute the Laplacian coordinates for each point in the graph. Third, compute the local transform for each point and apply the local transform to get the new Laplacian coordinates. Finally, solve the sparse linear system to reconstruct the graph. Steps 2 and 4 have been described earlier. In step 3, local transformations are computed using an approach similar to that from Poisson mesh editing. The basic idea is to propagate the local frame changes from the control curves to the whole mesh. In the following, we will explain step 1: how to construct the volumetric graph.

Volumetric Graph Construction Given a triangular mesh,

Volumetric Graph Construction Construct an inner shell We first construct an inner shell for the mesh by offsetting each vertex a distance in the direction opposite its normal. The purpose of this shell is to ensure that inner points are inserted even within thin features, like the cat’s tail.

Volumetric Graph Construction Embed both the mesh and shell in a lattice Then embed the mesh and the shell in a lattice and remove lattice nodes outside the shell.

Volumetric Graph Construction Build edge connections Thirdly, we build edge connections among the mesh, shell and the lattice nodes.

Volumetric Graph Construction Simplify and smooth the graph Finally, simplify the graph using edge collapse and smooth the graph. Note that our volumetric graph does not form a tetrahedral meshing of the mesh interior. Solidly meshing a complex object is notoriously difficult. Available packages also remesh geometry and disturb its connectivity, violating a common requirement in mesh deformation. [We avoid these problems and also trivially support the addition of a thin outer graph layer which helps to prevent self-intersection.] [JMS: might want to add one more slide showing outer shell and how it prevents self-intersection.] Not Tetrahedral Mesh

Deformation Comparison Now let’s compare Laplacian surface deformation, Poisson deformation and VGL deformation. You can see how well the sharp features are preserved as well as the volume using VGL. Laplacian surface [Sorkine04] Poisson mesh [Yu04] VGL

Deformation Comparison Here is another comparison. VGL is able to avoid local self-intersections. Laplacian surface [Sorkine04] Poisson mesh [Yu04] VGL

Deformation Comparison This is a more complex example. Note that Poisson mesh editing causes unnatural shrinkage especially in the model’s right thigh. Original model Poisson mesh VGL

Deformation Interface 3D space manipulation [Yu04] Tedious and require artistic skill 2D sketch-based interface Modeling: Teddy [Igarashi99] Editing: [Zelinka04, Kho05, Nealen05] Recall that we need to edit the control curve to deform the whole mesh. Previous methods like Poisson mesh editing manipulate 3D vertices directly, which is tedious and requires artistic skill. Some methods modify 3D objects by 2D curve editing, such as the sketch-based modeling system “Teddy” and several recent works. Another contribution of our work is an intuitive, “Teddy-like” deformation system based on 2D curve manipulation. “Teddy-like” deformation: intuitive and easy to use

2D Sketch-based Deformation Here we show a simple example. The user deforms the mesh by manipulating the 2D curve. This can be done by sketching a new curve. Or by editing the existing curve’s B-spline control points.

Deformation Retargeting A novel application of our system is to transfer exaggerated deformations from 2D cartoon characters to 3D models.

Deformation Retargeting Here we demonstrate use of a cartoon sequence to deform the dinosaur model. First, we specify three 3D control curves on the mesh along the spine and legs. Then, based the cartoon image sequence, we rotate the model to a proper view to define the projection plane for each control curve. After that, we set the strength field for each control curve. To deform the spine of the dinosaur, we go to the third frame of the cartoon sequence and specify the spine curve of the cat. Then the projected curve is deformed to match the spine curve of the cat, and this 2D deformation is automatically transferred to the mesh. Similarly we can deform the right leg.

Results Now we show some results produced using our system. Here’s a kicking dinosaur.

Results We can easily apply different cartoon deformation to the same 3D model. Here is the same dinosaur dancing.

Results Here is a walking cat.

Results Here is a running lioness.

Results The final example is the Stanford armadillo throwing a baseball.

Conclusion Volumetric graph Laplacian (VGL) Volumetric constraints Least squares minimization No tetrahedral mesh construction 2D sketch-based deformation system “Teddy-like” deformation system Cartoon deformation retargeting To conclude, we have presented a differential operator, the volumetric graph Laplacian, to enforce volumetric constraints for large mesh deformations. Such volumetric constraints can be represented by a quadric energy function. Thus least squares minimization yields the global optimal solution. We also demonstrated that VGL can be applied to an easy-to-build volumetric graph without meshing surface interiors with tetrahedra. Another contribution of our work is a 2D sketch-based deformation system, which just like “Teddy”. Using this system, we successfully transferred cartoon motions to complicated 3D models.

Future Work Anchor-based deformation Dynamic connectivity Automatic contour tracking For future work, supporting anchor-based deformation should be a complement to our system. We currently fix graph connectivity during deformation. Adaptive connectivity is necessary for very large deformations and represents an area of future work. Another enhancement would be to automatically track curves in cartoon sequences.

Acknowledgement Cartoons from Disney Feature Animation and Dongyu Cao 3D models from Stanford, MIT, Cyberware Xin Sun, Jianwei Han Steve Lin, Bo Zhang NSFC and 973 Program of China We would like to thank Disney for giving us the permission to use some of the famous animations. Some cartoon drawings are provided by Dongyu. Thanks to Stanford, Cyberware and MIT for providing the 3D models and many people for their help on this project.

Thank You !

Poisson Mesh Editing Mesh Geometry Guidance Field Boundary Condition Let’s look at the Poisson equation for mesh geometry. F corresponds to vertex positions of the mesh we want to solve. Boundary condition is the vertex positions of the control curve. V the guidance filed, generated by propagating the local frame transformations in the boundary condition to each triangle of the mesh. Delta is the mesh Laplacian. Div is the divergence operator. Mesh Geometry Guidance Field Boundary Condition

Quadric Energy Function Surface Details Position Constraints Volumetric Details : unknown positions : control points : mesh Laplacian : new mesh Laplacian coords : volumetric graph Laplacian : new graph Laplacian coords : control parameters With surface details and volumetric details in mind, we present a general energy function for large mesh deformation. Here P is the unknown point positions we want to solve. Q is the control point positons. L_M is the mesh Laplace operator. \epislon is the new mesh Laplacian coordinates. L_G is the volumetric graph Laplacian. \delta is the new graph Laplacian coodinates. \alpha and \beta are the control parameters This first part corresponds to surface details preserving. The second part is position constraints posed by users. The third part is volumetric details. The whole energy is a quadric function.

Compute Local Transformation Geodesic distance fields-based blending Compute the local transform for each control point Blend the transforms of all control points Blend the transform with the identity