Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation

Topics Investigated Thus Far 2D Texture Mapping (CS318) 3D Textures (Hypertexture) 2D Texture Synthesis

+ Synthesize a texture on a surface by coloring mesh vertices Input TextureInput MeshResult Today’s Goal

Vertex Painting Advantages –Speed No additional transformation required –Simplicity –Uniformly distributed across geometry Disadvantages –Texture resolution dependent on mesh complexity

2 SIGGRAPH Papers: 1.Texture Synthesis on Surfaces, by Greg TurkTexture Synthesis on Surfaces 2.Texture Synthesis over Arbitrary Manifold Surfaces, by Li-Yi Wei and Marc LevoyTexture Synthesis over Arbitrary Manifold Surfaces

Desirable Properties Share advantages of 2D algorithm –Quality –Efficient –General –Easy to use Minimum distortion Minimum discontinuity

Differences between Images and Meshes image mesh Synthesis Order Pixels/Vertices Local Orientation (Vector field) ? ? scanline u v grid Neighborhood normal ? ? tangent bitangent

Wei/Levoy Solution image mesh Synthesis Order Pixels/Vertices Local Orientation (Vector field) ? ? scanline u v grid Neighborhood normal ? ? tangent bitangent Wei and Levoy mesh re-tiling [Turk’92] user-specified relaxation random flattening/resampling

Turk Solution Turk repulsion [Turk’91] User specified/ interpolated sweeping Surface marching image mesh Synthesis Order Pixels/Vertices Local Orientation (Vector field) ? ? scanline u v grid Neighborhood normal ? ? tangent bitangent

Both Papers Extend Fast Texture Synthesis using Tree-structured Vector Quantization Specifically by generalizing their definition of “search neighborhoods” to apply to meshes

Texture Synthesis by Neighborhood Search noise Input pyramid noise Output pyramid Search Copy

Surface Texture Synthesis by Neighborhood Search (Wei/Levoy) Input pyramid Output pyramid Search

Process 1. Build image/mesh pyramids 2. Assign texture orientation/Computation order 3. Generate texture

What aspects of image pyramids must we maintain in mesh pyramids? Uniform density Power-of-two complexity differences between levels

Image & Mesh Pyramids Mesh Retiling [Turk’92]

2 Turk-ish Methods for Mesh Retiling Turk ’92, used by Wei/Levoy –Uniformly distributes mesh vertices –Requires “shooting normals” to move between levels Turk ’91, used by Turk –Uniformly distributes mesh vertices –Maintains parent/child relationship between levels

Retiling Density vertices73728 vertices

Turk ‘91 Create a mesh hierarchy in which mesh M k = (V k, T k ) is defined by its Vertices and Triangles –For the lowest mesh in the hierarchy, place n points on the surface –Use repulsion to distribute points easily –Add 3n points to make the next level –Repeat Connect points by projecting nearby points onto a tangent plane Perform Delaunay triangulation to reduce triangles

Now, we need to determine orientation… In Turk’s method, we use this orientation to determine the computation order (Surface Sweeping) In Wei/Levoy’s method, orientation is needed to “flatten” the neighborhoods.

Texture Orientation Generate a coordinate frame Three orthogonal axes – s (texture right) – t (texture up) – n (surface normal)

Texture Orientation Methods for orienting textures –user-specified (Turk) –random (Wei/Levoy) –smooth or symmetric (Wei/Levoy)

Texture Orientation (User-specified)

4-way symmetric texture 4-way symmetric vector field Texture Orientation (Symmetry)

Texture Orientation random2-way symmetry4-way symmetry

Texture Orientation (Relaxation) Minimize an error function

Results (Wei/Levoy): Random Orientation

Results (Wei/Levoy): Other Orientations RandomUser-specified Relaxation 2-way symmetry 4-way symmetry

Texture Synthesis on Surfaces: Wei/Levoy Style

Synthesis

Synthesis : 2 Lowest Levels

Synthesis : Lowest Level Random copy

Synthesis Pass1 : Extrapolation Search Copy Shooting normal

Mesh Neighborhood Resample Compare Resampled Grid 2D Patch 3D Patch Flatten (Maillot’93)

Neighborhood Flattening Project the triangles adjacent to p onto p’s local texture coordinate system Add triangles one-at-a-time until neighborhood template is covered

Neighborhood Comparison compare ?

Synthesis Pass 2 : Full Neighborhood Search Copy

Multiresolution Synthesis

Texture Synthesis on Surfaces: According to Turk

Turk’s Surface Sweeping Select an anchor vertex A Assign s(v) = sweep distance to vertex v along the vector field from A (for all v) Consensus orientation b/w 2 vertices v and w: O vw = (O(v) + O(w))/2 Calculate a new s(v) as a weighted average that its neighboring values dictate it should have. Visit vertices in order by sweep distance

Turk’s Texture Synthesis (Pseudocode) I = Input texture N(v) = Neighborhood around v M(a,b) = Neighborhood around (a,b) D(M,N) = Match Value (sum of squared differences)

Turk’s Mesh Neighborhoods r = average distance between mesh vertices O(v) = Surface tangent vector P(v) = O(v) rotated 90 o about surface normal Together, O and P make a coordinate frame ! u v normal P O Now, we can traverse the surface by point repelling

Point Repelling Use color interpolation to determine color at current surface point Move r in the direction of O or P When an edge is reached, fold the path over the next polygon

Results smooth4-way symmetryrandom

Results Surface displacement smooth2-way symmetry

Results

Summary of Differences Turk’s approachOur approach Vector field Traversal order Neighborhood Mesh hierarchy smoothrandom, symmetric sweepingrandom surface marchingflattening/resampling explicit parent/childshooting normal