1. Image Inpainting Marcelo Bertalmío, Minnesota
Guillermo Sapiro, Minnesota
Vicent Caselles, Barcelona
Coloma Ballester, Barcelona
SIGGRAPH ‘00

2. Overview What is image inpainting Related work Digital inpainting
Examples
Concluding remarks

3. What is inpainting? Modifying an image in a non-detectable form
Detail of "Cornelia, Mother of the Gracchi" by J. Suvee (Louvre).
Taken from Emile-Male “The Restorer’s Handbook of easel painting”.

4. Photo restoration
Restoration courtesy of D. Henry, Precious Photos Inc.

5. Object removal
From D. King, “The Commissar vanishes”.

6. Related work: Films n-1 n n+1 e.g. Kokaram et al.
Doesn’t work for stills or static objects
n-1 n
n+1

7. Related work: Texture synthesis
Hirani, Efros, Heeger, DeBonet, Simoncelli, etc.
Not practical for rich regions
Not designed for structured regions
“Copy” information instead of “learn and interpolate”

8. Related work: Disocclusion
Masnou-Morel, Nitzberg-Mumford, etc.
Limitations: Topology, angles
See also Chan-Shen ‘00

9. Our Contribution User only selects region to inpaint
Rich background and topology not an issue
Less than 5 minutes on a PC + =

10. How conservators inpaint
Minneapolis Institute of Art

11. Automatic digital inpainting
Propagate information
Evolutionary form

12. Digital inpainting (cont’d)
L = smoothness estimator (Laplacian)
N = isophote direction (time variant)

13. The equation Plus numerical schemes (Osher) Boundary conditions
Gray values (in a band)
Directions (in a band)

14. Example

15. Example: Text removal

16. Example: Photo restoration

17. Example: Special effects

18. Example: Scratch removal

19. Example: The evolution

20. Example: Structure but not texture is reproduced.

21. Extension: A variational formulation
Solved via E-L: Coupled 2nd order PDE’s
Full theory given
Similar practical results than 3rd order equation
See also Chan-Shen ‘00

22. Concluding remarks Technique imitates professionals Key concepts
Information propagation
Both gray values and directions are needed
Use a band surrounding the region
Sharp results
Low complexity
Texture is not reproduced

23. Concluding remarks (cont.)
Connected to thin film and fluid dynamics (A. Betozzi)
Opens then door to high order PDE’s
Extended to a variational formulation

24. Acknowledgments Institute Henri Poincare in Paris, France.
Tom Robbins, Elizabeth Buschor, Santiago Betelú, Stan Osher, Eero Simoncelli, Andrea Bertozzi.
Supported by ONR-Math, ONR Young Investigator Award, Presidential Early Career Awards for Scientists and Engineers, NSF CAREER Award, NSF-LIS, and IIE-Uruguay.

25. The end
Thank you