Lapped Textures Emil Praun Adam Finkelstein Hugues Hoppe Emil Praun Adam Finkelstein Hugues Hoppe Princeton University Microsoft Research Princeton University Microsoft Research

Goal “example” image textured surface mesh geometry ?

Goal Little user effort No apparent seams No obvious periodicity Low distortion Local texture control Anisotropy Little user effort No apparent seams No obvious periodicity Low distortion Local texture control Anisotropy

Previous 2D Texture Synthesis Histogram equalization [Heeger ’96] Laplacian block shuffling [de Bonet ’97] Pixel template matching [Efros ’99] [Wei ’00] Histogram equalization [Heeger ’96] Laplacian block shuffling [de Bonet ’97] Pixel template matching [Efros ’99] [Wei ’00]

Histogram equalization [Heeger ’96] Laplacian block shuffling [de Bonet ’97] Pixel template matching [Efros ’99] [Wei ’00] Random pasting of image blocks [Xu ’00] Histogram equalization [Heeger ’96] Laplacian block shuffling [de Bonet ’97] Pixel template matching [Efros ’99] [Wei ’00] Random pasting of image blocks [Xu ’00] Previous 2D Texture Synthesis

Previous 3D Texturing Volumetric textures: Noise functions [Perlin ’85, Worley ’96] Noise functions [Perlin ’85, Worley ’96] Solid textures by example [Ghazanfarpour ’96] Solid textures by example [Ghazanfarpour ’96] Synthesizing texture on a surface: Reaction-diffusion [Turk ’91, Witkin ’91] Reaction-diffusion [Turk ’91, Witkin ’91] Cellular textures [Fleischer ’95] Cellular textures [Fleischer ’95] Covering surface with triangular tiles [Neyret ’99] Covering surface with triangular tiles [Neyret ’99] Volumetric textures: Noise functions [Perlin ’85, Worley ’96] Noise functions [Perlin ’85, Worley ’96] Solid textures by example [Ghazanfarpour ’96] Solid textures by example [Ghazanfarpour ’96] Synthesizing texture on a surface: Reaction-diffusion [Turk ’91, Witkin ’91] Reaction-diffusion [Turk ’91, Witkin ’91] Cellular textures [Fleischer ’95] Cellular textures [Fleischer ’95] Covering surface with triangular tiles [Neyret ’99] Covering surface with triangular tiles [Neyret ’99]

Approach texture patch surface

Key Idea: Patch Pasting texture patch surfacesurface “lapped textures”

PROCESS

Algorithm texture patch surface

Algorithm texture patch surface

Algorithm texture patch surface

Algorithm texture patch surface

Issues 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering

Issues 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering

Texture Patch Creation

Less Structure  Splotch

Issues 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering

Direction Field: User-specified

Direction Field: Local to Patch

Issues 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering

Patch Growth

Issues 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering

Align Patch to Direction Field texture patch surface

Tangential Vector Field

Optimizing the Parametrization Least squares best match to unit axes Sparse linear system. No explicit fairness functional Least squares best match to unit axes Sparse linear system. No explicit fairness functional

Result of Optimization

Issues 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering

Coverage estimation off-screen buffer Render patch triangles Flag covered triangles Remember 1 pixel per uncovered triangle per uncovered triangle Render patch triangles Flag covered triangles Remember 1 pixel per uncovered triangle per uncovered triangle

Coverage estimation off-screen buffer Render patch triangles Flag covered triangles Remember 1 pixel per uncovered triangle per uncovered triangle Render patch triangles Flag covered triangles Remember 1 pixel per uncovered triangle per uncovered triangle

Coverage estimation off-screen buffer Render patch triangles Flag covered triangles Remember 1 pixel per uncovered triangle per uncovered triangle Render patch triangles Flag covered triangles Remember 1 pixel per uncovered triangle per uncovered triangle

Issues 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering

Texture Storage and Rendering Method 1: Texture Atlas Pre-composite into a global texture map. Pre-composite into a global texture map. -- OR -- Method 2: Runtime pasting Composite at run-time using hardware Composite at run-time using hardware Method 1: Texture Atlas Pre-composite into a global texture map. Pre-composite into a global texture map. -- OR -- Method 2: Runtime pasting Composite at run-time using hardware Composite at run-time using hardware

Method 1: Texture Atlas Patches of triangles with similar normals 2D packing problem for arbitrary polygons Patches of triangles with similar normals 2D packing problem for arbitrary polygons

Method 2: Runtime Pasting Store vertex coordinates for each patch Composite at run-time using hardware May render triangles several times Store vertex coordinates for each patch Composite at run-time using hardware May render triangles several times

Atlas vs. Runtime Pasting Atlas + Faster rendering, more portable + Easy to paint unique details (eyes, nose on bunny) – Sampling artifacts; user effort Pasting – Increases model complexity (   1.6 –3) + Huge effective resolution + Reuse splotch parameterization for many textures Atlas + Faster rendering, more portable + Easy to paint unique details (eyes, nose on bunny) – Sampling artifacts; user effort Pasting – Increases model complexity (   1.6 –3) + Huge effective resolution + Reuse splotch parameterization for many textures

RESULTS

Results: Splotches (completely automatic: no direction field)

Results: Anisotropic

Controlling Direction and Scale

25 frames per sec! 256 x 256 texture (282 times) 256 x 256 texture (282 times) 15,000 faces

Limitations low-frequencycomponentsboundarymismatches direction field singularities

Video

Interactive Paint Demo

Timings Texture patch creation: 1 min Specifying direction field: 15 min Surface patch growth Patch parameterization Face coverage estimation Rendering: Texture patch creation: 1 min Specifying direction field: 15 min Surface patch growth Patch parameterization Face coverage estimation Rendering: Preprocessing: 20sec – 6 min Preprocessing: Pentium III 733MHz, GeForce graphics Human effort

Conclusions Effective texture-by-example through:  Overlapping texture patches  Minimal edge blending Aligning to direction field  fast optimization Runtime pasting  high effective resolution Effective texture-by-example through:  Overlapping texture patches  Minimal edge blending Aligning to direction field  fast optimization Runtime pasting  high effective resolution

Future Work Other texture types: Animated Animated “Thick” (volumetric) textures  fur “Thick” (volumetric) textures  fur NPR rendering NPR rendering Greater automation Fine-tuning patch placement Other texture types: Animated Animated “Thick” (volumetric) textures  fur “Thick” (volumetric) textures  fur NPR rendering NPR rendering Greater automation Fine-tuning patch placement

“X Covers Y”? Flattening covers texture patch Texture patch covers triangle Use inner / outer hulls of texture unit OR scan - convert (use hardware) Flattening covers texture patch Texture patch covers triangle Use inner / outer hulls of texture unit OR scan - convert (use hardware)

Polygonal Hulls Start w/ boundary pixels, edges between neighbors Conservatively simplify polygons Start w/ boundary pixels, edges between neighbors Conservatively simplify polygons

Algorithm Cut texture patches from input texture [Specify texture direction & scale on mesh] REPEAT Pick texture patch T, mesh point P Grow surface patch S around P to size(T) Flatten S over T; paste T on mesh Update mesh coverage info UNTIL mesh is covered Cut texture patches from input texture [Specify texture direction & scale on mesh] REPEAT Pick texture patch T, mesh point P Grow surface patch S around P to size(T) Flatten S over T; paste T on mesh Update mesh coverage info UNTIL mesh is covered

Patch Growth Guess initial parametrization Want to cover texture patch Overlaps ok Prevent excessive distortion Surface patch homeomorphic to a disc Guess initial parametrization Want to cover texture patch Overlaps ok Prevent excessive distortion Surface patch homeomorphic to a disc

Optimizing the Parametrization TT SS BB CC AA (T)(T)(T)(T) (T)(T)(T)(T) (S)(S)(S)(S) (S)(S)(S)(S) (A)(A)(A)(A) (A)(A)(A)(A) (B)(B)(B)(B) (B)(B)(B)(B) (C)(C)(C)(C) (C)(C)(C)(C)  tt^^ss ^^ face in 3D 2D texture space

Optimizing the Parametrization Scale control given by || S ||, || T || Solution unique up to a translation (fix patch center) No explicit fairness term No flipped triangle ambiguity as in methods using meshes of springs Patch may require further growth Scale control given by || S ||, || T || Solution unique up to a translation (fix patch center) No explicit fairness term No flipped triangle ambiguity as in methods using meshes of springs Patch may require further growth