1. Near-Regular Texture Analysis and Manipulation Written by: Yanxi Liu Yanxi Liu Wen-Chieh Lin Wen-Chieh Lin James Hays James Hays Presented by: Alex Hadas Alex Hadas

2. What we will see today? Regular, Near-Regular Texture Definition Regular, Near-Regular Texture Definition Previous Approaches Previous Approaches Near-Regular Texture Analysis Near-Regular Texture Analysis Regularity Measurements Regularity Measurements Near-Regular Texture Manipulation Near-Regular Texture Manipulation Near-Regular Texture Synthesis Algorithm Near-Regular Texture Synthesis Algorithm

3. Regular, Near-Regular Texture Definition Regular Texture – wallpaper-like, congruent 2D tiling whose structural regularity can be completely characterized by 17 wallpaper groups Regular Texture – wallpaper-like, congruent 2D tiling whose structural regularity can be completely characterized by 17 wallpaper groups Example A: Cloth, Cloth, Tahiti TahitiTahiti Example taken from Wikipedia Wikipedia

4. Regular, Near-Regular Texture Definition Underlying lattice structure can be represented and generated by a pair of linear independent translations Underlying lattice structure can be represented and generated by a pair of linear independent translations Example C: Painted porcelain, porcelain China Example taken from Wikipedia Wikipedia T1T1 T2T2

5. Regular, Near-Regular Texture Definition The smallest bounded region that produces (under translation subgroup) simultaneously a covering (no gaps) and a packing (no overlaps) of the texture pattern on 2D plane is called a tile. The smallest bounded region that produces (under translation subgroup) simultaneously a covering (no gaps) and a packing (no overlaps) of the texture pattern on 2D plane is called a tile. Example B: Ornamental painting, NinevehNineveh, Assyria Assyria NinevehAssyria Example taken From Wikipedia Wikipedia

6. Regular, Near-Regular Texture Definition To Algorithm To Algorithm

7. Regular, Near-Regular Texture Definition Near-Regular Texture is statistical distortion of a regular, wallpaper like congruent tiling, possibly with individual variations in tile shape, size, color and lighting Near-Regular Texture is statistical distortion of a regular, wallpaper like congruent tiling, possibly with individual variations in tile shape, size, color and lighting

8. Regular, Near-Regular Texture Definition A Near – Regular Texture p = d(p r ), where A Near – Regular Texture p = d(p r ), where  p r is regular texture,  d = d geo ×d light ×d color, where d geo – Geometric Transformation d geo – Geometric Transformation d light – Lighting Changes d light – Lighting Changes d color – Color Alterations d color – Color Alterations

9. Regular, Near-Regular Texture Definition Examples of Near-Regular Textures Examples of Near-Regular Textures Brick wall Snake Cloth

10. Regular, Near-Regular Texture Definition Categorization of Near – Regular Textures (NRT) Type G eometry C olor SymbolsExample 0 R egular GRCR I Irregular GRCI II R egular GICR III Irregular GICI

11. Regular Texture

12. Near - Regular Texture Type I (GRCI)

13. Near - Regular Texture Type II (GICR)

14. Near – Regular Texture Type III (GICI)

15. Previous Approaches Generative model approach Generative model approach –Cost of model-specific parameter tuning

16. Previous Approaches Sample based approach Sample based approach –Neighborhood-based statistical analysis –Non-parametric estimation Tiling based approach Tiling based approach –Only Type I (Lui [2004b] –Only local boundaries preserved, but global near-regularity not addressed (Cohen et al.[2003]

17. Previous Approaches –Producing regular patterns with translational symmetry by generating tiling boundaries from closed planar contour Escherization [2000] Input Synthesized results Type I Kwatra et al. 2003

18. Previous Approaches Texture transfer problem Texture transfer problem –Image Analogies [Hertzmann et al. 2001] –Texture Quilting [Efros and Freeman 2001] Input Synthesized results Type II Efros and Freeman 2001

19. Previous Approaches Texture replacement on plane Texture replacement on plane –Surface is planar, texture is of type I (Tsin et al. [2001] Separation illuminance and texture using a non-linear filtering technique (Oh et al[2001] Separation illuminance and texture using a non-linear filtering technique (Oh et al[2001]

20. Near-Regular Texture Analysis Geometric Deformation Field Geometric Deformation Field Lighting Deformation Field Lighting Deformation Field Color Deformation Field Color Deformation Field A Pair of Regularity Measurements A Pair of Regularity Measurements

21. Geometric Deformation Field computer builds 2D lattice computer builds 2D lattice User adjusts misplaced points User adjusts misplaced points Computer finds optimized lattice Computer finds optimized lattice Using MFFD for capturing 1 to 1 warping field Using MFFD for capturing 1 to 1 warping field Represent warping field in HSV space Represent warping field in HSV space

22. Geometric Deformation Field t1t1t1t1 t2t2t2t2 t 1 +t 2 t2t2t2t2 t1t1t1t1 t 1 -t 2 t2t2t2t2 t1t1t1t1

23. NRT Analysis: Geometric Deformation Field Represent warping field in HSV space Represent warping field in HSV space dx dy Color scheme used Displacement Map

24. Lighting Deformation Field Straighten the NRT lattice using d geo Straighten the NRT lattice using d geo Apply Tsin et al.[2001]’s algorithm for lighting map extraction in the plain Apply Tsin et al.[2001]’s algorithm for lighting map extraction in the plain Apply inverse geometric field Apply inverse geometric field

25. Lighting Deformation Field

26. Color Deformation Field PCA method: create set of basis and coefficients PCA method: create set of basis and coefficients

27. Regularity Measurements Geometric Regularity Geometric Regularity Appearance Regularity Appearance Regularity

28. Regularity Measurements

29. Near-Regular Texture Manipulation Geometry Deformation Field Manipulation Geometry Deformation Field Manipulation Texture Replacement Texture Replacement Deformation Field Analogy Deformation Field Analogy Texture Regularity Manipulation Texture Regularity Manipulation

30. Geometry Deformation Field Manipulation

31. Results Comparison

32. Texture Replacement

34. Deformation Field Analogy AA’ B B’ : : Geometric Deformation Field Lighting Deformation Field Extracted from Input Texture Synthesized from A Result of Deformation Field Analogy

35. Texture Replacement

36. NRT Synthesis Algorithm Type I NRT only Type I NRT only What is Tile? What is Tile? What is Tile? What is Tile?

37. NRT Synthesis Algorithm Minimum tiles set {t i } Minimum tiles set {t i } Maximum tiles set {T i } Maximum tiles set {T i } Centered on half way shifted lattice points Centered on half way shifted lattice points

38. NRT Synthesis Algorithm Stage 1(analysis) Stage 1(analysis) –Determine from a given sample pattern –Determine lattice anchor points {t i } (user controlled) –For each t i construct maximum tile sets T (centered on lattice points) and T h (centered half way)

39. NRT Synthesis Algorithm Stage 2 (synthesis) Stage 2 (synthesis) 1)Start from top left corner with random tile chosen from T 2)Add tile to the synthesized texture in a scan line along with step When we reach right boundary place tile in direction with step from left most tile in a row

40. NRT Synthesis Algorithm Stage 2 (synthesis) (cont.) Stage 2 (synthesis) (cont.) 3)At each lattice or half-way lattice point select T or T h tile set and pick one of the best tiles. Error function value is less that threshold

41. NRT Synthesis Algorithm Error Function Distance Function Red values of the pixel Blue values of the pixel Green values of the pixel

42. NRT Synthesis Algorithm Stage 2 (synthesis) (cont.) Stage 2 (synthesis) (cont.) 4)Register selected candidate tile using a correlation-based method 5)Use dynamic programming to “stitch” the overlapping tiles. Apply it separately to horizontal and vertical directions

43. NRT Synthesis Algorithm Stage 2 (synthesis) (cont.) Stage 2 (synthesis) (cont.) 6)When pasting a tile to existing image apply blending where dynamic programming may have conflicting decisions. 7)Repeat steps 2-6 until the whole image is synthesized

44. NRT Synthesis Algorithm selected tile depends on distance of pixel to the boundary synthesized tile

45. NRT Synthesis Algorithm

51. Limitations Self occlusions Self occlusions Shadows caused by surface geometry Shadows caused by surface geometry Tiles are geometrically aligned Tiles are geometrically aligned

52. Summary User friendly (lattice definition, lighting map extraction) User friendly (lattice definition, lighting map extraction) Fast (1-20 min lattice adjustment, <1 min DF synthesis) Fast (1-20 min lattice adjustment, <1 min DF synthesis) Simple (MFFD control points number ~ tiles number) Simple (MFFD control points number ~ tiles number)

53. References Deformable Texture: The Irregular – Regular – Irregular Cycle (Yanxi Lui and Wen-Cheh Lin) Deformable Texture: The Irregular – Regular – Irregular Cycle (Yanxi Lui and Wen-Cheh Lin) Near-Regular Texture Analysis and Manipulation (Yanxi Lui,Wen-Cheh Lin, James Hays) Near-Regular Texture Analysis and Manipulation (Yanxi Lui,Wen-Cheh Lin, James Hays) Promise and Perils of Near-regular Texture(Yanxi Lui and Wen-Cheh Lin,Yanghai Tsin) Promise and Perils of Near-regular Texture(Yanxi Lui and Wen-Cheh Lin,Yanghai Tsin)