1. Image Quilting for Texture Synthesis and Transfer Alexei A. Efros (UC Berkeley) William T. Freeman (MERL) Siggraph01 ’

2. About author? Alexei A. Efros Assistant Professor Computer Science Department & The Robotics Institute School of Computer Science Carnegie Mellon University From St. Petersburg, Russia Got PhD from UC Berkeley in 2003 Then a year as a Visiting Research Fellow in Visual Geometry Group of Oxford Joined in CSD and RI in autumn of 2004 Computer graphics & computer vision

3. About author? William T. Freeman Professor of Electrical Engineering & Computer Science at the Artificial Intelligence Laboratory at MIT(September, 2001) Received a BS in physics and MS in electrical engineering from Stanford (1979), and an MS in applied physics from Cornell(1981) Got his PhD in 1992 from the MIT Worked at Mitsubishi Electric Research Labs (1992 – 2001, Cambridge) Computer vision

4. Quilting? Transfer? + =

5. Image vs. Texture

6. Example-based Texture Synthesis Input Example

7. The Goal of Texture Synthesis True (infinite) texture SYNTHESIS generated image input image Same in perceptual sense

8. The Challenge Texture analysis: how to capture the essence of texture? Need to model the whole spectrum: from repeated to stochastic texture RepeatedStochasticBoth?

9. Related Work Local region-growing method -Pixel-based -Patch-based-Patch-based Global optimization-based method Heeger and Bergen sig95,Pyramid-based texture synthesis Paget and Longstaff IEEE Tran… 98,Texture synthesis via a noncausal nonparametric multiscale markov random field ….. Physical Simulation et al

10. Input texture B1B2 Random placement of blocks block B1 B2 Neighboring blocks constrained by overlap B1B2 Minimal error boundary cut

11. min. error boundary Minimum Error Boundary Cut overlapping blocksvertical boundary _ = 2 overlap error

12. Minimum Error Boundary Cut

13. The Image Quilting Algorithm –Pick size of block and size of overlap –Synthesize blocks in raster order –Search input texture for block that satisfies overlap constraints (above and left) Easy to optimize using NN search [Liang et.al., ’ 01] –Paste new block into resulting texture Compute minimal error boundary cut

14. Synthesis Results

20. input image Portilla & Simoncelli Wei & LevoyImage Quilting Xu, Guo & Shum

21. Portilla & Simoncelli Wei & LevoyImage Quilting Xu, Guo & Shum input image

22. Portilla & Simoncelli Wei & LevoyImage Quilting input image Xu, Guo & Shum

23. Failures

24. Image Quilting vs. Graph Cut Input Image Quilting Graph Cut (siggraph 03’)

25. Texture Transfer + Luminance Constraint

27. += += parmesan rice

28. Conclusion –No multi-scale, no one-pixel-at-a-time! –fast and very simple –Improved stability –Results are not bad

30. Pixel-based Methods Compare local causal neighbourhoods Efros and Leung (ICCV ’ 99) Wei and Levoy (Siggraph 2000) Ashikhmin (I3D 2001) InputOutput

31. Patch-based Methods Copy patches of pixels rather than single pixels Chaos Mosaic, Xu et al, 1997 Patch-Based Sampling, Liang et al(ACM 2001) Image Quilting, Efros and Williams(Siggraph 2001)