1. Accurate Image Based Relighting through Optimization Pieter Peers Philip Dutré Department of Computer Science K.U.Leuven, Belgium

2. What is Image Based Relighting? Basis Images Processing & Relighting Relit Image

3. Previous Work The Light Stage (Debevec et al. ’00) Images: ACM SIGGRAPH ‘00

4. Previous Work Environment Matting (Zonker et al. ’99, Chuang et al. ’00) Images: ACM SIGGRAPH ’99

5. Previous work Diffuse Materials Specular Materials The Light Stage +- Environment Matting -+ Sparse sampling of light source directions Sufficient angular sampling Large span of incoming light directions Bundled incoming light directions Matusik et al. EGWR’02

6. Three-step algorithm Basis Image Recording Relit image = Captured reflectance + Light-map 1 3 2 Reflectance field calculation

7. Basis Image Recording

9. ... Basis Image Recording HDR photographs

10. Practical Setup HDR photographs Monitor calibration 4x4 grid, 9 patterns per lightpatch ±600 basis images

11. Three-step algorithm Basis Image Recording 1 3 2 Relit image = Captured reflectance + Light-map Reflectance field calculation

12. Support and reflection coefficient

13. Support and reflectance … Patterns Pixel × R pix,patch Reflection Coefficient = = = = Resulting Value … … → Error 1 → Error 2 → Error n-1 → Error n Find and R pix,patch that minimize:  Error k

14. cli ck Support and reflectance: examples top left back right

15. Three-step algorithm Basis Image Recording 1 3 2 Relit image = Captured reflectance + Light-map Reflectance field calculation

16. Re-lighting For each pixel: Pixel value = ∑ Filter i (LightMap) × R pix,i support i

17. Results A C B D

18. Results A C B D

19. Results A C B D

20. Results A C B D

21. Comparison Stochastic ray tracing Relit from rendered basis images

22. Comparison Stochastic ray tracing Relit from rendered basis images

23. Comparison Stochastic ray tracing Relit from rendered basis images

24. Conclusion Accurate relighting: –Diffuse materials –Specular materials Combine strengths from: –The Light Stage –Environment matting

25. Future Work Alternative emitting devices Investigate other patterns & optimization algorithms

26. Acknowledgements Frank Suykens and Vincent Masselus Some environment maps from: http://www.debevec.org/Probes

27. Thank you… Questions?