Near-Regular Texture Analysis and Manipulation Written by: Yanxi Liu Yanxi Liu Wen-Chieh Lin Wen-Chieh Lin James Hays James Hays Presented by: Alex Hadas Alex Hadas

What we will see today? Regular, Near-Regular Texture Definition Regular, Near-Regular Texture Definition Previous Approaches Previous Approaches Near-Regular Texture Analysis Near-Regular Texture Analysis Regularity Measurements Regularity Measurements Near-Regular Texture Manipulation Near-Regular Texture Manipulation Near-Regular Texture Synthesis Algorithm Near-Regular Texture Synthesis Algorithm

Regular, Near-Regular Texture Definition Regular Texture – wallpaper-like, congruent 2D tiling whose structural regularity can be completely characterized by 17 wallpaper groups Regular Texture – wallpaper-like, congruent 2D tiling whose structural regularity can be completely characterized by 17 wallpaper groups Example A: Cloth, Cloth, Tahiti TahitiTahiti Example taken from Wikipedia Wikipedia

Regular, Near-Regular Texture Definition Underlying lattice structure can be represented and generated by a pair of linear independent translations Underlying lattice structure can be represented and generated by a pair of linear independent translations Example C: Painted porcelain, porcelain China Example taken from Wikipedia Wikipedia T1T1 T2T2

Regular, Near-Regular Texture Definition The smallest bounded region that produces (under translation subgroup) simultaneously a covering (no gaps) and a packing (no overlaps) of the texture pattern on 2D plane is called a tile. The smallest bounded region that produces (under translation subgroup) simultaneously a covering (no gaps) and a packing (no overlaps) of the texture pattern on 2D plane is called a tile. Example B: Ornamental painting, NinevehNineveh, Assyria Assyria NinevehAssyria Example taken From Wikipedia Wikipedia

Regular, Near-Regular Texture Definition To Algorithm To Algorithm

Regular, Near-Regular Texture Definition Near-Regular Texture is statistical distortion of a regular, wallpaper like congruent tiling, possibly with individual variations in tile shape, size, color and lighting Near-Regular Texture is statistical distortion of a regular, wallpaper like congruent tiling, possibly with individual variations in tile shape, size, color and lighting

Regular, Near-Regular Texture Definition A Near – Regular Texture p = d(p r ), where A Near – Regular Texture p = d(p r ), where  p r is regular texture,  d = d geo ×d light ×d color, where d geo – Geometric Transformation d geo – Geometric Transformation d light – Lighting Changes d light – Lighting Changes d color – Color Alterations d color – Color Alterations

Regular, Near-Regular Texture Definition Examples of Near-Regular Textures Examples of Near-Regular Textures Brick wall Snake Cloth

Regular, Near-Regular Texture Definition Categorization of Near – Regular Textures (NRT) Type G eometry C olor SymbolsExample 0 R egular GRCR I Irregular GRCI II R egular GICR III Irregular GICI

Regular Texture

Near - Regular Texture Type I (GRCI)

Near - Regular Texture Type II (GICR)

Near – Regular Texture Type III (GICI)

Previous Approaches Generative model approach Generative model approach –Cost of model-specific parameter tuning

Previous Approaches Sample based approach Sample based approach –Neighborhood-based statistical analysis –Non-parametric estimation Tiling based approach Tiling based approach –Only Type I (Lui [2004b] –Only local boundaries preserved, but global near-regularity not addressed (Cohen et al.[2003]

Previous Approaches –Producing regular patterns with translational symmetry by generating tiling boundaries from closed planar contour Escherization [2000] Input Synthesized results Type I Kwatra et al. 2003

Previous Approaches Texture transfer problem Texture transfer problem –Image Analogies [Hertzmann et al. 2001] –Texture Quilting [Efros and Freeman 2001] Input Synthesized results Type II Efros and Freeman 2001

Previous Approaches Texture replacement on plane Texture replacement on plane –Surface is planar, texture is of type I (Tsin et al. [2001] Separation illuminance and texture using a non-linear filtering technique (Oh et al[2001] Separation illuminance and texture using a non-linear filtering technique (Oh et al[2001]

Near-Regular Texture Analysis Geometric Deformation Field Geometric Deformation Field Lighting Deformation Field Lighting Deformation Field Color Deformation Field Color Deformation Field A Pair of Regularity Measurements A Pair of Regularity Measurements

Geometric Deformation Field computer builds 2D lattice computer builds 2D lattice User adjusts misplaced points User adjusts misplaced points Computer finds optimized lattice Computer finds optimized lattice Using MFFD for capturing 1 to 1 warping field Using MFFD for capturing 1 to 1 warping field Represent warping field in HSV space Represent warping field in HSV space

Geometric Deformation Field t1t1t1t1 t2t2t2t2 t 1 +t 2 t2t2t2t2 t1t1t1t1 t 1 -t 2 t2t2t2t2 t1t1t1t1

NRT Analysis: Geometric Deformation Field Represent warping field in HSV space Represent warping field in HSV space dx dy Color scheme used Displacement Map

Lighting Deformation Field Straighten the NRT lattice using d geo Straighten the NRT lattice using d geo Apply Tsin et al.[2001]’s algorithm for lighting map extraction in the plain Apply Tsin et al.[2001]’s algorithm for lighting map extraction in the plain Apply inverse geometric field Apply inverse geometric field

Lighting Deformation Field

Color Deformation Field PCA method: create set of basis and coefficients PCA method: create set of basis and coefficients

Regularity Measurements Geometric Regularity Geometric Regularity Appearance Regularity Appearance Regularity

Regularity Measurements

Near-Regular Texture Manipulation Geometry Deformation Field Manipulation Geometry Deformation Field Manipulation Texture Replacement Texture Replacement Deformation Field Analogy Deformation Field Analogy Texture Regularity Manipulation Texture Regularity Manipulation

Geometry Deformation Field Manipulation

Results Comparison

Texture Replacement

Deformation Field Analogy AA’ B B’ : : Geometric Deformation Field Lighting Deformation Field Extracted from Input Texture Synthesized from A Result of Deformation Field Analogy

Texture Replacement

NRT Synthesis Algorithm Type I NRT only Type I NRT only What is Tile? What is Tile? What is Tile? What is Tile?

NRT Synthesis Algorithm Minimum tiles set {t i } Minimum tiles set {t i } Maximum tiles set {T i } Maximum tiles set {T i } Centered on half way shifted lattice points Centered on half way shifted lattice points

NRT Synthesis Algorithm Stage 1(analysis) Stage 1(analysis) –Determine from a given sample pattern –Determine lattice anchor points {t i } (user controlled) –For each t i construct maximum tile sets T (centered on lattice points) and T h (centered half way)

NRT Synthesis Algorithm Stage 2 (synthesis) Stage 2 (synthesis) 1)Start from top left corner with random tile chosen from T 2)Add tile to the synthesized texture in a scan line along with step When we reach right boundary place tile in direction with step from left most tile in a row

NRT Synthesis Algorithm Stage 2 (synthesis) (cont.) Stage 2 (synthesis) (cont.) 3)At each lattice or half-way lattice point select T or T h tile set and pick one of the best tiles. Error function value is less that threshold

NRT Synthesis Algorithm Error Function Distance Function Red values of the pixel Blue values of the pixel Green values of the pixel

NRT Synthesis Algorithm Stage 2 (synthesis) (cont.) Stage 2 (synthesis) (cont.) 4)Register selected candidate tile using a correlation-based method 5)Use dynamic programming to “stitch” the overlapping tiles. Apply it separately to horizontal and vertical directions

NRT Synthesis Algorithm Stage 2 (synthesis) (cont.) Stage 2 (synthesis) (cont.) 6)When pasting a tile to existing image apply blending where dynamic programming may have conflicting decisions. 7)Repeat steps 2-6 until the whole image is synthesized

NRT Synthesis Algorithm selected tile depends on distance of pixel to the boundary synthesized tile

NRT Synthesis Algorithm

Limitations Self occlusions Self occlusions Shadows caused by surface geometry Shadows caused by surface geometry Tiles are geometrically aligned Tiles are geometrically aligned

Summary User friendly (lattice definition, lighting map extraction) User friendly (lattice definition, lighting map extraction) Fast (1-20 min lattice adjustment, <1 min DF synthesis) Fast (1-20 min lattice adjustment, <1 min DF synthesis) Simple (MFFD control points number ~ tiles number) Simple (MFFD control points number ~ tiles number)

References Deformable Texture: The Irregular – Regular – Irregular Cycle (Yanxi Lui and Wen-Cheh Lin) Deformable Texture: The Irregular – Regular – Irregular Cycle (Yanxi Lui and Wen-Cheh Lin) Near-Regular Texture Analysis and Manipulation (Yanxi Lui,Wen-Cheh Lin, James Hays) Near-Regular Texture Analysis and Manipulation (Yanxi Lui,Wen-Cheh Lin, James Hays) Promise and Perils of Near-regular Texture(Yanxi Lui and Wen-Cheh Lin,Yanghai Tsin) Promise and Perils of Near-regular Texture(Yanxi Lui and Wen-Cheh Lin,Yanghai Tsin)