1. Lapped Textures Emil Praun Adam Finkelstein Hugues Hoppe Emil Praun Adam Finkelstein Hugues Hoppe Princeton University Microsoft Research Princeton University Microsoft Research

2. Goal “ example ” image textured surface mesh geometry ?

3. Goal Little user effort No apparent seams No obvious periodicity Low distortion Local texture control Anisotropy Little user effort No apparent seams No obvious periodicity Low distortion Local texture control Anisotropy

4. Previous 2D Texture Synthesis Histogram equalization [Heeger ’ 96] Laplacian block shuffling [de Bonet ’ 97] Pixel template matching [Efros ’ 99] [Wei ’ 00] Histogram equalization [Heeger ’ 96] Laplacian block shuffling [de Bonet ’ 97] Pixel template matching [Efros ’ 99] [Wei ’ 00]

5. Histogram equalization [Heeger ’ 96] Laplacian block shuffling [de Bonet ’ 97] Pixel template matching [Efros ’ 99] [Wei ’ 00] Random pasting of image blocks [Xu ’ 00] Histogram equalization [Heeger ’ 96] Laplacian block shuffling [de Bonet ’ 97] Pixel template matching [Efros ’ 99] [Wei ’ 00] Random pasting of image blocks [Xu ’ 00] Previous 2D Texture Synthesis

6. Previous 3D Texturing Volumetric textures: Noise functions [Perlin ’85, Worley ’96] Noise functions [Perlin ’85, Worley ’96] Solid textures by example [Ghazanfarpour ’96] Solid textures by example [Ghazanfarpour ’96] Synthesizing texture on a surface: Reaction-diffusion [Turk ’91, Witkin ’91] Reaction-diffusion [Turk ’91, Witkin ’91] Cellular textures [Fleischer ’95] Cellular textures [Fleischer ’95] Covering surface with triangular tiles [Neyret ’99] Covering surface with triangular tiles [Neyret ’99] Volumetric textures: Noise functions [Perlin ’85, Worley ’96] Noise functions [Perlin ’85, Worley ’96] Solid textures by example [Ghazanfarpour ’96] Solid textures by example [Ghazanfarpour ’96] Synthesizing texture on a surface: Reaction-diffusion [Turk ’91, Witkin ’91] Reaction-diffusion [Turk ’91, Witkin ’91] Cellular textures [Fleischer ’95] Cellular textures [Fleischer ’95] Covering surface with triangular tiles [Neyret ’99] Covering surface with triangular tiles [Neyret ’99]

7. Approach texture patch surface

8. Key Idea: Patch Pasting texture patch surfacesurface “ lapped textures ”

9. Anisotropic vs. Isotropic images

10. PROCESS

11. Algorithm Cut texture patches from input texture [Specify texture direction & scale on mesh] REPEAT Pick texture patch T, mesh point P Grow surface patch S around P to size(T) Flatten S over T; paste T on mesh Update mesh coverage info UNTIL mesh is covered Cut texture patches from input texture [Specify texture direction & scale on mesh] REPEAT Pick texture patch T, mesh point P Grow surface patch S around P to size(T) Flatten S over T; paste T on mesh Update mesh coverage info UNTIL mesh is covered

12. Algorithm texture patch surface

13. Algorithm texture patch surface

14. Algorithm texture patch surface

15. Algorithm texture patch surface

16. Issues 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering

17. Issues 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering

18. Texture Patch Creation

19. Less Structure  Splotch

20. Issues 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering

21. Direction Field: User-specified

22. Angle - Only for anisotropic patches.Angle - Only for anisotropic patches. Can be set for only small number of faces – Gaussian radial basis function is used for interpolation.Can be set for only small number of faces – Gaussian radial basis function is used for interpolation. Since the normal lies within the plane of the face we get an orthogonal basis :Since the normal lies within the plane of the face we get an orthogonal basis : S = T x N. (N – normal to the plane). Angle - Only for anisotropic patches.Angle - Only for anisotropic patches. Can be set for only small number of faces – Gaussian radial basis function is used for interpolation.Can be set for only small number of faces – Gaussian radial basis function is used for interpolation. Since the normal lies within the plane of the face we get an orthogonal basis :Since the normal lies within the plane of the face we get an orthogonal basis : S = T x N. (N – normal to the plane).

23. Direction Field: Local to Patch

24. Issues 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering

25. Patch Growth 1. Choose a random point on a non fully covered triangle. 2. Add triangles in a BFS order to cover the texture patch. 3. Stop when the patch is fully coverd. Condintions: a.The surface patch homeomorphic to a disk. b.The distortion should not exceed a given limit. c.We add edge only if it is within the texture patch. 1. Choose a random point on a non fully covered triangle. 2. Add triangles in a BFS order to cover the texture patch. 3. Stop when the patch is fully coverd. Condintions: a.The surface patch homeomorphic to a disk. b.The distortion should not exceed a given limit. c.We add edge only if it is within the texture patch.

26. Patch Growth

34. Polygonal Hulls Start w/ boundary pixels, edges between neighbors Conservatively simplify polygons Start w/ boundary pixels, edges between neighbors Conservatively simplify polygons

35. Issues 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering

36. Align Patch to Direction Field texture patch surface

37. Tangential Vector Field

38. Optimizing the Parametrization Least squares best match to unit axes Sparse linear system. No explicit fairness functional Least squares best match to unit axes Sparse linear system. No explicit fairness functional

39. Optimizing the Parametrization TT SS BB CC AA (T)(T)(T)(T) (T)(T)(T)(T) (S)(S)(S)(S) (S)(S)(S)(S) (A)(A)(A)(A) (A)(A)(A)(A) (B)(B)(B)(B) (B)(B)(B)(B) (C)(C)(C)(C) (C)(C)(C)(C)  tt^^ss ^^ face in 3D 2D texture space

40. Optimizing the Parametrization For each mesh face f ={A,B,C} the vector T lies within the face plain. Barycentric coordinates: T =  A +  B +  C and  +  +  = 0. For each mesh face f ={A,B,C} the vector T lies within the face plain. Barycentric coordinates: T =  A +  B +  C and  +  +  = 0.

41. Optimizing the Parametrization TT SS BB CC AA (T)(T)(T)(T) (T)(T)(T)(T) (S)(S)(S)(S) (S)(S)(S)(S) (A)(A)(A)(A) (A)(A)(A)(A) (B)(B)(B)(B) (B)(B)(B)(B) (C)(C)(C)(C) (C)(C)(C)(C)  tt^^ss ^^ face in 3D 2D texture space d T =  (A) +  (B) +  (C) - t ^ We wish to minimize :

42. Result of Optimization

43. Issues 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering

44. Coverage estimation off-screen buffer Render patch triangles Flag covered triangles Remember 1 pixel per uncovered triangle per uncovered triangle Render patch triangles Flag covered triangles Remember 1 pixel per uncovered triangle per uncovered triangle

45. Coverage estimation off-screen buffer Render patch triangles Flag covered triangles Remember 1 pixel per uncovered triangle per uncovered triangle Render patch triangles Flag covered triangles Remember 1 pixel per uncovered triangle per uncovered triangle

46. Coverage estimation off-screen buffer Render patch triangles Flag covered triangles Remember 1 pixel per uncovered triangle per uncovered triangle Render patch triangles Flag covered triangles Remember 1 pixel per uncovered triangle per uncovered triangle

47. Issues 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering 1.Texture patch creation 2.Specifying direction field 3.Surface patch growth 4.Patch parametrization 5.Face coverage estimation 6.Texture storage and rendering

48. Texture Storage and Rendering Method 1: Texture Atlas Pre-composite into a global texture map. Pre-composite into a global texture map. -- OR -- Method 2: Runtime pasting Composite at run-time using hardware Composite at run-time using hardware Method 1: Texture Atlas Pre-composite into a global texture map. Pre-composite into a global texture map. -- OR -- Method 2: Runtime pasting Composite at run-time using hardware Composite at run-time using hardware

49. Texture Atlas Texture Atlas is a set of charts {  1,…,  k }. Each chart  j is a mapping from a subset U j of the mesh to the Euclidean plane (the texture plane). {U 1,…, U k } is a cover of the surface. If i  j then  j and  i don’t overlap except on the edges. Texture Atlas is a set of charts {  1,…,  k }. Each chart  j is a mapping from a subset U j of the mesh to the Euclidean plane (the texture plane). {U 1,…, U k } is a cover of the surface. If i  j then  j and  i don’t overlap except on the edges.

50. Texture Atlas Patches of triangles with similar normals 2D packing problem for arbitrary polygons Patches of triangles with similar normals 2D packing problem for arbitrary polygons

51. Runtime Pasting Store vertex coordinates for each patch Composite at run-time using hardware May render triangles several times Store vertex coordinates for each patch Composite at run-time using hardware May render triangles several times

52. Atlas vs. Runtime Pasting Atlas + Faster rendering, more portable + Easy to paint unique details (eyes, nose on bunny) – Sampling artifacts; user effort Pasting – Increases model complexity (   1.6 –3) + Huge effective resolution + Reuse splotch parameterization for many textures Atlas + Faster rendering, more portable + Easy to paint unique details (eyes, nose on bunny) – Sampling artifacts; user effort Pasting – Increases model complexity (   1.6 –3) + Huge effective resolution + Reuse splotch parameterization for many textures

53. RESULTS

54. Results: Splotches (completely automatic: no direction field)

55. Results: Anisotropic

56. Controlling Direction and Scale

57. Usage of more than 1 patch

58. 25 frames per sec! tiny texture footprint

59. Limitations low-frequencycomponentsboundarymismatches direction field singularities

60. Timings Texture patch creation: 1 min Specifying direction field: 15 min Surface patch growth Patch parameterization Face coverage estimation Rendering: 25fps @ 1024 2 Texture patch creation: 1 min Specifying direction field: 15 min Surface patch growth Patch parameterization Face coverage estimation Rendering: 25fps @ 1024 2 Preprocessing: 20sec – 6 min Preprocessing: Pentium III 733MHz, GeForce graphics Human effort

61. Conclusions Effective texture-by-example through:  Overlapping texture patches  Minimal edge blending Aligning to direction field  fast optimization Runtime compositing  high effective resolution Effective texture-by-example through:  Overlapping texture patches  Minimal edge blending Aligning to direction field  fast optimization Runtime compositing  high effective resolution

62. Future Work Other texture types: Animated Animated “Thick” (volumetric) textures  fur “Thick” (volumetric) textures  fur NPR rendering NPR rendering Fine-tuning patch placement Greater automation Other texture types: Animated Animated “Thick” (volumetric) textures  fur “Thick” (volumetric) textures  fur NPR rendering NPR rendering Fine-tuning patch placement Greater automation

63. “X Covers Y”? Flattening covers texture patch Texture patch covers triangle Use inner / outer hulls of texture unit OR scan - convert (use hardware) Flattening covers texture patch Texture patch covers triangle Use inner / outer hulls of texture unit OR scan - convert (use hardware)

64. Algorithm Cut texture patches from input texture [Specify texture direction & scale on mesh] REPEAT Pick texture patch T, mesh point P Grow surface patch S around P to size(T) Flatten S over T; paste T on mesh Update mesh coverage info UNTIL mesh is covered Cut texture patches from input texture [Specify texture direction & scale on mesh] REPEAT Pick texture patch T, mesh point P Grow surface patch S around P to size(T) Flatten S over T; paste T on mesh Update mesh coverage info UNTIL mesh is covered

65. Patch Growth Guess initial parametrization Want to cover texture patch Overlaps ok Prevent excessive distortion Surface patch homeomorphic to a disc Guess initial parametrization Want to cover texture patch Overlaps ok Prevent excessive distortion Surface patch homeomorphic to a disc

66. Optimizing the Parametrization Scale control given by || S ||, || T || Solution unique up to a translation (fix patch center) No explicit fairness term No flipped triangle ambiguity as in methods using meshes of springs Patch may require further growth Scale control given by || S ||, || T || Solution unique up to a translation (fix patch center) No explicit fairness term No flipped triangle ambiguity as in methods using meshes of springs Patch may require further growth

67. Video

68. Interactive Paint Demo