Efficient Editing of Aged Object Textures By: Olivier Clément Jocelyn Benoit Eric Paquette Multimedia Lab

2 Introduction Realistic image synthesis  Virtual reality, video games, special effects, etc. Aging (or weathering)  Many effects  Many objects  Time consuming

Multimedia Lab 3 Introduction Redesign iterations

Multimedia Lab 4 Outline Objectives Previous Work Aging Editing Process  Segmentation Phase  Elimination Phase  Reproduction Phase Results and Limitations Conclusion

Multimedia Lab 5 Objectives To build a system  To edit aging effects on textures  To increase realism  To reduce the amount of work  Adapted for artists adequate control interactive no complex parameters

Multimedia Lab 6 Outline Objectives Previous Work Aging Editing Process  Segmentation Phase  Elimination Phase  Reproduction Phase Results and Limitations Conclusion

Multimedia Lab 7 Previous Work Physically based methods [Dorsey and Hanharan 2000; Merillou et al. 2001; O’Brien et al. 2002; etc.]  Highly realistic results but lengthy calculations  Non-intuitive physical parameters Empirical methods [Chain et al. 2005; Gobron and Chiba 2001; Paquette et al. 2002; etc.]  More intuitive parameters Both approaches  Do not provide the control required by artists  Target a single aging effect Aging methods

Multimedia Lab 8 Previous Work Image based [Gu et al. 2006; Wang et al. 2006; etc.]  Capture the time-varying aspects of the material  Similar to our approach Focus of our approach  Simple capture process  Adequate control Aging methods

Multimedia Lab 9 Outline Objectives Previous Work Aging Editing Process  Segmentation Phase  Elimination Phase  Reproduction Phase Results and Limitations Conclusion

Multimedia Lab 10 Aging Editing Process Source image  Image, photograph  Containing aging effects Target aging mask  Binary image  Desired pattern Reproduction image  New aging effects Process overview

Multimedia Lab 11 Aging Editing Process Segmentation phase  Semi-automatic  Aged regions Elimination phase  Automatic  Aging removed Reproduction phase  Automatic  New aging effects Phase description R e d e s i g n i t e r a t i o n s

Multimedia Lab 12 Aging Editing Process Images summary

Multimedia Lab 13 Outline Objectives Previous Work Aging Editing Process  Segmentation Phase  Elimination Phase  Reproduction Phase Results and Limitations Conclusion 14

Multimedia Lab 14 Segmentation Phase Identifies aged regions Could be done with  Segmentation tools  Image editing software Stroke-based technique Lischinski et al. [2006]  Worked efficiently for semi-automatic identification

Multimedia Lab 15 Segmentation Phase Stroke-base technique - Video

Multimedia Lab 16 Outline Objectives Previous Work Aging Editing Process  Segmentation Phase  Elimination Phase  Reproduction Phase Results and Limitations Conclusion 17

Multimedia Lab 17 Elimination Phase Constrained texture synthesis Match the non-aged neighbourhood Search using ANN library Arya et al. [1998] The algorithm best match … new best match Elimination imageSource image copy the pixel color

Multimedia Lab 18 Elimination Phase The boundary pixels  Non-aged pixels in their neighbourhood  Must be filled first The aged region is filled iteratively Hole-filling

Multimedia Lab 19 Outline 20 Objectives Previous Work Aging Editing Process  Segmentation Phase  Elimination Phase  Reproduction Phase Results and Limitations Conclusion

Multimedia Lab 20 Reproduction Phase Extension of the elimination algorithm Consider the aged / non-aged context The new term

Multimedia Lab 21 Reproduction Phase Does not synthesize the entire image Only specified regions Iterative construction from multiple source images Aging effects transfer and combination

Multimedia Lab 22 Outline Objectives Previous Work Aging Editing Process  Segmentation Phase  Elimination Phase  Reproduction Phase Results and Limitations Conclusion

Multimedia Lab 23 Results Source imageElimination imageReproduction imageSource aging maskTarget aging mask

Multimedia Lab 24 Results Source imageElimination imageReproduction image

Multimedia Lab 25 Results Source imageElimination imageReproduction image

Multimedia Lab 26 Results Source imageAging masksReproduction image More results in the paper and the video…

Multimedia Lab 27 Results User interaction is minimal Interactive computation time Efficient for redesign iterations Efficiency 2.5 minutes - once 25 seconds - once 2 minutes every iteration 3 seconds every iteration Obtained on a PC with 3.2 GHz CPU and 3GB of RAM

Multimedia Lab 28 Limitations Apply only on surfaces  No fractures or deformations Camera-based texture acquisition  Specular lighting  Surface distortion Current implementation  Interactive on textures up to 512 x 512

Multimedia Lab 29 Outline Objectives Previous Work Aging Editing Process  Segmentation Phase  Elimination Phase  Reproduction Phase Results and Limitations Conclusion

Multimedia Lab 30 Conclusion A framework  To edit aging effects on textures  To reduce the amount of work needed during the redesign iterations Benefits  Appropriate for artists adequate control and interactivity no complex parameters  Works well for several types of aging effects

Multimedia Lab 31 Conclusion Synthesize the target aging mask  For numerous regions  Ex: scratches Handle layers in effects combination  Multiple effects over the same regions  Ex: dirt on top of rust Faster synthesis  To handle higher resolution textures Future work

Multimedia Lab 32 ? We would like to thank : And all our reviewers… Questions

Multimedia Lab 33 Previous Work Texture synthesis [Efros 1999; Hertzmann 2001; Kwatra 2003; Lefebvre 2006; Liang 2001; etc.]  Synthesis based on neighbourhood matching Our system  Extends from these algorithms  Specializes for the aging context Texture synthesis

Multimedia Lab 34 Previous Work Image analogies, Hertzmaan et al. [2001]  The output image is completely synthesized  Our approach uses a similar algorithm that synthesize only regions of the output  Our approach should be considered as an extension Texture synthesis

Multimedia Lab 35 Elimination Phase The replacement pixel is :  Selected from the non-aged pixels of the source image  One of the best neighbourhood matches The system seeks a replacement pixel that minimizes the following L 2 norm : The replacement pixel

Multimedia Lab 36 Elimination Phase An exhaustive search would require processing time far from interactive Thus, an approximation of the best match is found with the ANN library (Arya et al. [1998])  Approximate nearest neighbour searching algorithm based on a kd-tree structure  Our feature vector is composed of the RGB components of the non-aged pixels around the pixel to replace Interactivity