TextureAmendment Reducing Texture Distortion in Constrained Parameterizations Yu-Wing TaiNational University of Singapore Michael S. BrownNational University of Singapore Chi-Keung TangHong Kong University of Science and Technology Heung-Yueng ShumMicrosoft Research Asia

Texture mapping Standard approach to simulate fine detail and color on a 3D model. 1

Specifying a mapping for an image Distortion is unavoidable in highly textured regions. Constrained-parameterization 2

Specifying a mapping for an image 3 Low-distortion parameterization Little distortion, but not suitable for existing images.

Remove/reduce texture distortion from constrained texture mapping Our goal 4

Texture mapping with distorted textureDistortion removed Before After 4

Observation We can detect distorted regions using the input: - 3d model - texture image - constrained parameterization We may need simple user assistance - Segmentation markup - Texture orientation markup We can expand distorted regions via texture synthesis Where different texture is located How texture is oriented in the image 5

Texture Expand textured image regions to fit 3D model. Amendment Low-distortion parameterization provides the place for expansion. 6 Amended Texture

Related Work Parameterization Constrained [Lévy 2001] [Kraevoy et al. 2002] [Zhou et al. 2005] 7

[Desburn et al. 2002] [Sheffer et al. 2005] [Zhang et al. 2005] [Kraevoy et al. 2002] [Lévy et al. 2002] Parameterization Related Work Texture Synthesis [Heeger and Bergen 1995] [Wei and Levoy 2000] [Efros and Freeman 2001] Constrained Low-Distortion [Kwatra et al. 2003] [Lefebvre and Hoppe 2006] (more...) [Zhou et al. 2005] Graphite Software [Lévy 2001] 7

Differences Detail Preserving Image Warping Texture Amendment Related Work Detail Preserving Image Warping [Fang and Hart 2007] Goal is also to avoid texture (detail) distortion. -Final mesh can be broken - Synthesis follows flow + expanded regions - User provide texture flow, no feature lines - User provides feature line pairs - Synthesis follows feature lines -Topology does not change 8

Texture Amendment Process 9

TextureAmendment Procedure Input User markup Segmentation Texture Flow Amendment Procedure Distortion Detection Extraction Synthesis Segmentation Texture flow Texture Synthesis + Integration 10

Input 3) Constrained-Parameterization 1) Triangulated 3D mesh 2) Texture Image 11

Markup - Segmentation Initial automatic segmentation via JSEG [Deng and Manjunath 01] User refined segmentation. 12 Texture image is segmented into “texture pools”.

Markup – Texture Flow Orientation specified by user strokes. Resulting texture flow field. 13

Distortion Detection Geometric Distortion ‘Textureness’ High Geometric Distortion Low Geometric Distortion × High Textureness × Low Textureness × High Textureness → distortion → no distortion 14

Distortion Detection: User Parameters 1) Neighborhood size and 2) Threshold Neighborhood Size Threshold 15

Distortion Extraction Detected distorted regions “Flattened”using low-distortion parameterization. Copy orientation information. Copy non-distorted pixel boundary. Include additional non-distorted triangles at border 16

Texture Synthesis Synthesis via [Kwatra et al 03] 17 Synthesis is performed in gradient-domain. Modified to incorporate orientation and scale in source-target.

Color and synthesis integration Original texture warped to flatten region Bi-lateral filter to extract color Synthesis high-frequency detail Poisson Image Integration to combine color + synthesized details [Perez et al 03] 18

Results 19

Leopard Example 20

Leopard Example 20

Leopard Example 20

Comparison Original With amended texture 21 Original With amended texture

Van Gogh Example 22

Comparison Original With amended texture 23 Original With amended texture

Dot Examples 24 User refined distortion map.

Comparison Original With amended texture 25 Original With amended texture

Discussion and Limitations Amended results may still be unsatisfactory if there is no suitable texture for synthesis We do not address “structured features” Possible solutions “Structure Aware” Image Manipulation Avidan et al. SIGGRAPH 07, Wang et al. SIGGRAPH ASIA’08 Currently, our implementation works on only open meshes (i.e. not a closed surface) and single image To extend synthesis component needs to be modified. 26

Texture Amendment Summary Reduce texture distortion common in constrained texture mapping Combine advantages of both constrained parameterization and low-distortion parameterization Present an overall framework that is flexible 27

Thank You. Q&A. Acknowledgements Bruno Levy (and team) [Graphite Software] + Image Zhou Kun (Permission for use of images) [Zhou et al 05] John Hart (Permission for use of images) [Fang and Hart 07] 28