1. Lapped textures Emil Praun Adam Finkelstein Hugues Hoppe
2000/07/28
Lapped Textures
Emil Praun
Adam Finkelstein
Hugues Hoppe
Princeton University
Microsoft Research
SIGGRAPH 2000

2. ? Goal mesh geometry textured surface “example” image Lapped textures
2000/07/28
Goal
mesh geometry ?
The goal of our project is to texture 3D objects by example: given a triangle mesh and a 2D image [animation], we want to cover the object with texture that looks like the provided example.
textured surface
“example” image
SIGGRAPH 2000

3. Goal Little user effort No apparent seams No obvious periodicity
Lapped textures
2000/07/28
Goal
Little user effort
No apparent seams
No obvious periodicity
Low distortion
Local texture control
Anisotropy
There are a number of properties that such a texture creation method should have: it should require relatively little user effort, the resulting object should be seamlessly covered, there should be no obvious periodicity, as one notices when tiling the plane with a given image,
The mapping of the texture onto curved regions should be achieved without large distortion.
Finally it would be nice if the user could locally control the texture appearance, like relative size and, for anisotropic textures, the orientation.
SIGGRAPH 2000

4. Previous 2D Texture Synthesis
Lapped textures
2000/07/28
Previous 2D Texture Synthesis
Histogram equalization [Heeger ’96]
Laplacian block shuffling [de Bonet ’97]
Pixel template matching [Efros ’99] [Wei ’00]
Most work in synthesizing 2D textures by example has utilized randomness combined with constrained matching. These methods are usually fairly slow to compute and while they may be extended to 3D, they would be even slower.
SIGGRAPH 2000

5. Previous 2D Texture Synthesis
Lapped textures
2000/07/28
Previous 2D Texture Synthesis
Histogram equalization [Heeger ’96]
Laplacian block shuffling [de Bonet ’97]
Pixel template matching [Efros ’99] [Wei ’00]
Random pasting of image blocks [Xu ’00]
Our work is largely inspired by that of Xu et all, whose method I will describe briefly. They start with a small toroidal image, which they tile to produce an image of the desired size.[animation]
They select rectangular blocks of texture which they copy and paste across the image with a small amount of blending at the edges.
[animation] Here is an example of a box that was moved.
[animation] And here is another example.
SIGGRAPH 2000

6. Previous 3D Texturing Volumetric textures:
Lapped textures
2000/07/28
Previous 3D Texturing
Volumetric textures:
Noise functions [Perlin ’85, Worley ’96]
Solid textures by example [Ghazanfarpour ’96]
Synthesizing texture on a surface:
Reaction-diffusion [Turk ’91, Witkin ’91]
Cellular textures [Fleischer ’95]
Covering surface with triangular tiles [Neyret ’99]
Volumetric textures are good for many applications, but not all surface textures can be modeled with solid textures. In particular, it is hard to model anisotropy, when the texture preferred direction is supposed to change across a surface: it is hard to model the bark on a very twisted tree w/ solid textures.
Reaction-diffusion and cellular texture methods work directly on surface of arbitrary topology in 3D. However, they are slow, and being parametric, it is hard to control the final appearance of the texture.
Neyret & Cani’s scheme, presented last year, does not handle anisotropic textures.
SIGGRAPH 2000

7. Approach texture patch surface Lapped textures 2000/07/28
Our method is very similar to the work of Xu et al in 2D: it works by cutting regions of the input image, that we call texture patches,
texture patch
surface
SIGGRAPH 2000

8. Key Idea: Patch Pasting
Lapped textures
2000/07/28
Key Idea: Patch Pasting
And repeatedly pasting them onto the 3D surface,[animation] until we cover it.[animation] Note that we do not produce a tiling of the surface, since we place the patches in overlapping fashion. This is where the name of our method comes from.
texture patch
“lapped textures”
surface
SIGGRAPH 2000

9. PROCESS Lapped textures 2000/07/28
I am now going to present the process.
SIGGRAPH 2000

10. Algorithm texture patch surface Lapped textures 2000/07/28
The overview of the algorithm is as follows:
We start by cutting the texture patches. Quite often, a single patch suffices.
For anisotropic textures only we specify a direction field over the mesh.
{Next Slides:}
We select an uncovered mesh point,
Grow a surface patch around it,
Parametrize it over the texture,
And using this parametrization, we paste a copy of the texture patch onto the surface.
We update face coverage information, and then repeat the process starting with another uncovered mesh point.
We stop when we have covered the mesh.
texture patch
surface
SIGGRAPH 2000

11. Algorithm texture patch surface Lapped textures 2000/07/28
We select an uncovered mesh point,
Grow a surface patch around it,
Parametrize it over the texture,
And using this parametrization, we paste a copy of the texture patch onto the surface.
texture patch
surface
SIGGRAPH 2000

12. Algorithm texture patch surface Lapped textures 2000/07/28
We update face coverage information, and then repeat the process starting with another uncovered mesh point.
texture patch
surface
SIGGRAPH 2000

13. Algorithm texture patch surface Lapped textures 2000/07/28
We stop when we have covered the mesh.
texture patch
surface
SIGGRAPH 2000

14. Issues Texture patch creation Specifying direction field
Lapped textures
2000/07/28
Issues
Texture patch creation
Specifying direction field
Surface patch growth
Patch parametrization
Face coverage estimation
Texture storage and rendering
In order to implement this algorithm we had to address these six issues.
SIGGRAPH 2000

15. Issues Texture patch creation Specifying direction field
Lapped textures
2000/07/28
Issues
Texture patch creation
Specifying direction field
Surface patch growth
Patch parametrization
Face coverage estimation
Texture storage and rendering
The first step in our process is the texture patch creation.
SIGGRAPH 2000

16. Texture Patch Creation
Lapped textures
2000/07/28
Texture Patch Creation
For very structured textures, the final result looks better if the patch boundaries do not cut across semantic features in the input texture, like bricks, or strands in this woven mat image. Since semantic classification of textures is not a solved problem, we let the user cut the patches.
The patch consists of the input image together with a transparency mask: the patch is opaque at its center but has a gradual blend region near its boundaries. This helps hide the seams between different patches on the textured mesh.
SIGGRAPH 2000

17. Less Structure  Splotch
Lapped textures
2000/07/28
Less Structure  Splotch
For less structured images, the patch boundaries do not have to align with specific texture features, so we can use a generic shape, such as an irregular blob.[animation]
We prefer wavy boundaries since the human eye is better at detecting straight-line discontinuities rather than wavy ones.
SIGGRAPH 2000

18. Issues Texture patch creation Specifying direction field
Lapped textures
2000/07/28
Issues
Texture patch creation
Specifying direction field
Surface patch growth
Patch parametrization
Face coverage estimation
Texture storage and rendering
The next step is to establish a direction field over the mesh.
SIGGRAPH 2000

19. Direction Field: User-specified
Lapped textures
2000/07/28
Direction Field: User-specified
This is very important for anisotropic textures.
Quite a number of real-world textures are anisotropic: they don’t look the same if you rotate them. Our algorithm needs to know how to align these texture with respect to the surface. For this purpose, the user specifies the desired direction for a small number of faces. The direction is interpolated for the remaining faces.
SIGGRAPH 2000

20. Direction Field: Local to Patch
Lapped textures
2000/07/28
Direction Field: Local to Patch
For anisotropic textures, there is no need for a globally consistent direction field, however we need local ones in order to compute the parametrization for each patch. A random tangential direction is chosen for the center of the patch and subsequently propagated to all the faces that are added to the patch.
SIGGRAPH 2000

21. Issues Texture patch creation Specifying direction field
Lapped textures
2000/07/28
Issues
Texture patch creation
Specifying direction field
Surface patch growth
Patch parametrization
Face coverage estimation
Texture storage and rendering
This brings us to the next point, surface patch growth.
SIGGRAPH 2000

22. Patch Growth Lapped textures 2000/07/28
We start with a seed point on the mesh and the face containing it. The seed point is mapped to the texture patch center.
{Next slides:}
As we add faces to the patch, we make an initial guess for their parametrization.
We need this initial guess in order to know how large we should grow the patch.
When growing the patch we avoid introducing faces that would cause unacceptable texture distortion.
Also, the patch is kept homeomorphic to a disk, so we do not allow any holes or cylindrical patches.
SIGGRAPH 2000

23. Patch Growth Lapped textures 2000/07/28
As we add faces to the patch, we make an initial guess for their parametrization.
SIGGRAPH 2000

24. Patch Growth

25. Patch Growth Lapped textures 2000/07/28
We need this initial guess in order to know how large we should grow the patch.
SIGGRAPH 2000

26. Patch Growth Lapped textures 2000/07/28
When growing the patch we avoid introducing faces that would cause unacceptable texture distortion.
SIGGRAPH 2000

27. Lapped textures
2000/07/28
Patch Growth
SIGGRAPH 2000

28. Patch Growth Lapped textures 2000/07/28
Also, the patch is kept homeomorphic to a disk, so we do not allow any holes or cylindrical patches.
SIGGRAPH 2000

29. Issues Texture patch creation Specifying direction field
Lapped textures
2000/07/28
Issues
Texture patch creation
Specifying direction field
Surface patch growth
Patch parametrization
Face coverage estimation
Texture storage and rendering
Once we have a surface patch, we optimize its parametrization.
SIGGRAPH 2000

30. Align Patch to Direction Field
Lapped textures
2000/07/28
Align Patch to Direction Field
The goal is to align the texture principal direction, such as the grooves in a tree bark texture, with the specified direction field.
texture patch
surface
SIGGRAPH 2000

31. Tangential Vector Field
Lapped textures
2000/07/28
Tangential Vector Field
Every triangle has a local coordinate frame, composed of the given direction field, the triangle normal, and their cross product.
The red vector represent the local “right” direction for every triangle while the green one is the local “up” direction.
SIGGRAPH 2000

32. Optimizing the Parametrization
Lapped textures
2000/07/28
Optimizing the Parametrization
The optimization tries to match these direction to the corresponding axes unit vectors in texture space, in a least squares sense. We construct a sparse linear system and solve for the texture coordinates at the vertices. Our only constraints are that all the red vectors should match the unit OX vector and all the green vectors should match the unit OY vector. There is no explicit fairness functional, the fairness comes from the underlying direction field on the mesh.
Least squares best match to unit axes
Sparse linear system. No explicit fairness functional
SIGGRAPH 2000

33. Result of Optimization
Lapped textures
2000/07/28
Result of Optimization
Here you can see the result of the optimization. Note that since the method locally warps the texture everywhere to align to the globally consistent direction field, derivative continuity at patch boundaries is very good. This would not happen with methods that use a singe direction constraint for a whole patch (for instance at its center). If you look at the texture squares, they are aligned everywhere and they follow the direction filed streamlines. Such a streamline is shown in yellow here.
SIGGRAPH 2000

34. Issues Texture patch creation Specifying direction field
Lapped textures
2000/07/28
Issues
Texture patch creation
Specifying direction field
Surface patch growth
Patch parametrization
Face coverage estimation
Texture storage and rendering
In order to know when to stop pasting we need to keep track of what parts of the surface have been covered.
SIGGRAPH 2000

35. Coverage estimation Render patch triangles Flag covered triangles
Lapped textures
2000/07/28
Coverage estimation
Render patch triangles
Flag covered triangles
Remember 1 pixel
per uncovered triangle
After each paste operation, we update the coverage information for the affected faces. We render these faces in an off-screen buffer.
We composite all the previous paste operations that affect them.
off-screen buffer
SIGGRAPH 2000

36. Coverage estimation Render patch triangles Flag covered triangles
Remember 1 pixel
per uncovered triangle
off-screen buffer

37. Coverage estimation Render patch triangles Flag covered triangles
Remember 1 pixel
per uncovered triangle
off-screen buffer

38. Issues Texture patch creation Specifying direction field
Lapped textures
2000/07/28
Issues
Texture patch creation
Specifying direction field
Surface patch growth
Patch parametrization
Face coverage estimation
Texture storage and rendering
The final issue is texture storage and rendering.
SIGGRAPH 2000

39. Texture Storage and Rendering
Lapped textures
2000/07/28
Texture Storage and Rendering
Method 1: Texture Atlas
Pre-composite into a global texture map.
-- OR --
Method 2: Runtime pasting
Composite at run-time using hardware
Once we have the patch parameterizations, we can store and create the texture in two ways:
The traditional representation, that people normally use, is to create a global texture map for the whole mesh and pre-composite all the paste operations in it. Such a global texture map is called a texture atlas.
We introduce an alternate method which consists of just storing the computed parameterizations and compositing all the patches at runtime.
SIGGRAPH 2000

40. Method 1: Texture Atlas Patches of triangles with similar normals
Lapped textures
2000/07/28
Method 1: Texture Atlas
In order to create a texture atlas,we group faces with similar normals together to form patches which we relax using the described optimization. To have a single unified texture for the whole mesh we need to arrange the 2D images of these patches inside a unit square.
Finding the optimum arrangement is not tractable for a few hundred arbitrary polygons. There exist a few heuristics but they don’t perform as well as a trained human. In our prototype implementation we let the user arrange the pieces by hand. Of course, the atlas is independent of the texture so this only needs to happen once for every mesh that need to be textured.
Patches of triangles with similar normals
2D packing problem for arbitrary polygons
SIGGRAPH 2000

41. Method 2: Runtime Pasting
Lapped textures
2000/07/28
Method 2: Runtime Pasting
Store vertex coordinates for each patch
Composite at run-time using hardware
May render triangles several times
For the runtime pasting approach, we store the texture coordinates at the vertices for every paste operation. We then draw all the patches, compositing them using the frame buffer. Since triangles may be covered by different patches, they may be drawn more than once.
This problem may be alleviated using multitexturing hardware.
SIGGRAPH 2000

42. Atlas vs. Runtime Pasting
Lapped textures
2000/07/28
Atlas vs. Runtime Pasting
Atlas
Faster rendering, more portable
Easy to paint unique details (eyes, nose on bunny)
Sampling artifacts; user effort
Pasting
Increases model complexity ( 1.6 –3)
Huge effective resolution
Reuse splotch parameterization for many textures
Here are the pros and cons of the atlas approach versus runtime pasting.
Meshes textured with an atlas render slightly faster and may be more portable. The suffer from resampling artifacts since there is an extra resampling step: pixels move from the original texture to the texture atlas and then to the screen. Also, since the atlas image has to represent the whole surface, its resolution may not be enough to capture all the detail present in the original image.
User effort may be needed to create a good texture atlas.
Runtime pasting has the drawback of slightly increasing the model complexity.
However, it offers a huge effective resolution by storing a small texture and displaying it many times.
For textures that accommodate a generic patch boundary, the computed parametrization can be reused allowing the user to browse through several textures.
All the results that we present use runtime pasting.
SIGGRAPH 2000

43. RESULTS

44. (completely automatic: no direction field)
Lapped textures
2000/07/28
Results: Splotches
These results were produced in a completely automatic fashion, since these textures are isotropic and also don’t require customized patch boundaries.
(completely automatic: no direction field)
SIGGRAPH 2000

45. Results: Anisotropic Lapped textures 2000/07/28
Here are some examples of structured textures with user-defined patch boundaries.
SIGGRAPH 2000

46. Controlling Direction and Scale
Lapped textures
2000/07/28
Controlling Direction and Scale
The stripes on the body of the bunny differ in orientation and size, while the ones on the head and on the ears are roughly the same.
SIGGRAPH 2000

47. 256 x 256 texture 15,000 faces 25 frames per sec! Lapped textures
2000/07/28
256 x 256 texture
(282 times)
15,000 faces
The rock bunny from the proceedings cover stores only a 256 by 256 texture image, instanced 282 times, to texture a model with faces. It renders at 25 frames per second at a resolution of 1024 squared.
25 frames
per sec!
SIGGRAPH 2000

48. direction field singularities
Lapped textures
2000/07/28
Limitations
Like many texturing algorithms, our method has some limitations. It does not work well with textures that have strong low frequency components, since the patch overlap might juxtapose image regions with different overall color, revealing the patch boundaries.
Also, since we place patches in random location, the texture features at the boundaries might not line up exactly with the features of texture that has already been placed. In this brick example we see fragmented bricks between the two patches.
In covering an arbitrary topology mesh with a consistent direction field, we may need to introduce singularity points or discontinuity lines.
low-frequency
components
boundary
mismatches
direction field singularities
SIGGRAPH 2000

49. Video

50. Interactive Paint Demo

51. Timings Texture patch creation: 1 min Human effort
Lapped textures
2000/07/28
Timings
Texture patch creation: min
Specifying direction field: 15 min
Surface patch growth
Patch parameterization
Face coverage estimation
Rendering: 10242
Human effort
Preprocessing:
20sec – 6 min
Here are the timings for the different stages of the process.
Note that human effort is only needed for some textures: stochastic textures don’t need customized patches while isotropic textures don’t need a globally consistent direction field.
Preprocessing took between 20 sec and 6 min, but it was in most cases under 1 min.
The models render at 25 frames per second at full screen resolution on a GeForce graphics card.
Pentium III 733MHz, GeForce graphics
SIGGRAPH 2000

52. Conclusions Effective texture-by-example through:
Lapped textures
2000/07/28
Conclusions
Effective texture-by-example through:
 Overlapping texture patches
 Minimal edge blending
Aligning to direction field  fast optimization
Runtime pasting  high effective resolution
In conclusion, we have shown how to do effective texturing of meshes using the concept of placing overlapping texture patches. Hiding the patch seams is done by alpha blending over a few pixels region near the boundaries.
The point of this talk is: even though it may seem surprising, since one would expect to see these boundaries, THE METHOD WORKS. THE FINAL RESULTS LOOK VERY GOOD FOR A LARGE VARIETY OF TEXTURES AND MESHES.
We have also shown how the parametrization based on direction field alignment is fast, since it only requires solving a sparse linear system.
It is also robust and does not use an explicit fairness functional, the fairness coming from the underlying direction field.
Finally, runtime pasting is an efficient method for storing lapped textures, offering high effective resolution, and opening the way for many interesting applications. This brings us to future work:
SIGGRAPH 2000

53. Future Work Other texture types: Greater automation
Lapped textures
2000/07/28
Future Work
Other texture types:
Animated
“Thick” (volumetric) textures  fur
NPR rendering
Greater automation
Fine-tuning patch placement
We are investigating the use of the lapped textures paradigm for:
Animated (video) textures: instead of browsing through a set of texture images, we could run a small movie inside the texture patch.
Volumetric textures, that represent the mesh as a series of concentric shells, are useful for rendering such phenomena as fur.
We are also investigating the use of lapped textures for the parametrization of the strokes that are drawn in a non-photorealistic rendering style.
A possible direction for future work might be to alleviate the problem of boundary feature mismatch, as in the brick case: by aligning a patch to the previously-placed texture we might improve the match.
The method may also benefit from an automated patch boundary selection mechanism for structured textures. Such a method may use snakes or intelligent scissoring. The direction field may also be created automatically, using curvature information from the mesh.
SIGGRAPH 2000