1. Eurographics 2012, Cagliari, Italy 3D Material Style Transfer Chuong H. Nguyen 1, Tobias Ritschel 2, Karol Myszkowski 1, Elmar Eisemann 2, Hans-Peter Seidel 1 1 Max-Planck-Institut Informatik 2 Telecom ParisTech / CNRS

2. Eurographics 2012, Cagliari, Italy Background – Material Perception [Ce Liu et al. 2010]

3. Eurographics 2012, Cagliari, Italy Background - Mood

4. Eurographics 2012, Cagliari, ItalyBackground

5. Motivation Scene without Materials Guide Image Scene with Materials

6. Eurographics 2012, Cagliari, Italy Previous Works - Colorization Color Transfer Reinhard et al: Color Transfer between Images, IEEE CG&A 2001 Colorization Welsh et al: Transfering Color to Greyscale Images, Siggraph 2001 Irony et al: Colorization by Examples, EGSR 2005

7. Eurographics 2012, Cagliari, Italy Previous Works – Detail Synthesis Improving the Realism Johnson et al: CG2Real: Improving the Realism of Computer Generated Images using a Large Collection of Photographs, IEEE TVCG 2010 Analogy Hertzmann et al: Image Analogies, Siggraph 2001 Mertens et al: Texture Transfer Using Geometry Correlation, EGSR 2007

8. Eurographics 2012, Cagliari, Italy Previous Work – Materials Image-Based Material Editing Khan et al: Image-Based Material Editing, Siggraph 2006 Material Perception Vangorp et al: The Influence of Shape on the Perception of Material Reflectance, Siggraph 2007

9. Eurographics 2012, Cagliari, Italy Problem statement Guide Source3D Scene ObjectsMaterials

10. Eurographics 2012, Cagliari, Italy Material Extraction Guide Source3D Scene ObjectsMaterials

11. Eurographics 2012, Cagliari, Italy Material Extraction Original Diffuse Specular 1 st K-Mean Merging2 nd K-Mean Extracted MaterialsTexture Diffuse and Specular Extraction Yang et al: Real-time Specular Highlight Removal Using Bilateral Filtering, ECCV, 2010

12. Eurographics 2012, Cagliari, Italy Material Extraction Results 12

13. Eurographics 2012, Cagliari, Italy Material Assignment Guide Source3D Scene ObjectsMaterials

14. Eurographics 2012, Cagliari, Italy Material Assignment Goal Guide Source ObjectsMaterials 1. 2. (Image Cost) (Geometry Cost)

15. Eurographics 2012, Cagliari, Italy Material Assignment Goal – Image Cost Guide Source ObjectsMaterials 1. 2. (Image Cost) (Geometry Cost)

16. Eurographics 2012, Cagliari, Italy Material Assignment - Image Cost Guide Source3D Scene ∑

17. Eurographics 2012, Cagliari, Italy Material Assignment Goal – Geometry Cost Guide Source ObjectsMaterials 1. 2. (Image Cost) (Geometry Cost)

18. Eurographics 2012, Cagliari, Italy 3D Scene Material Assignment - Geometry Cost Guide Source

19. Eurographics 2012, Cagliari, Italy Geometry Cost – Shape Descriptor 2D Angular Radial Transform (ART) Bober et al: MPEG-7 visual shape descriptors, IEEE Circuits and Sys. Vid. Tech. 11, 6 (2001) 35 coefs (B) 35 coefs (A) 35 coefs (C) 35 coefs (G) 35 coefs (H) 35 coefs(E) 35 coefs (D) 35 coefs (F) 35 coefs (O),…

20. Eurographics 2012, Cagliari, Italy Cost Function = Image Cost + Geometry Cost + ……

21. Eurographics 2012, Cagliari, Italy Material Assignment Guide Source3D Scene ObjectsMaterials

22. Eurographics 2012, Cagliari, Italy Optimization – Simulated Annealing + …… + …… otherwise assignment i th assignment i+1 th

23. Eurographics 2012, Cagliari, Italy Results: Pebbles

24. Eurographics 2012, Cagliari, Italy Results: Temple

25. Eurographics 2012, Cagliari, Italy Results: Kitchen

26. Eurographics 2012, Cagliari, Italy Results: Bedroom

27. Eurographics 2012, Cagliari, Italy Perceptual Study – Step 1

28. Eurographics 2012, Cagliari, Italy Perceptual Study – Step 2 60% best

29. Eurographics 2012, Cagliari, ItalyLimitations Plausible material extraction Image segmentation errors Fix lighting

30. Eurographics 2012, Cagliari, Italy Conclusion and Future Work Conclusion –Plausible material extraction from images –An automatic system for 3D material style transfer Future work –Material database for different objects –Better object recognition techniques

31. Eurographics 2012, Cagliari, Italy Thank you!