1. Interactive Time-Dependent Tone Mapping Using Programmable Graphics Hardware Nolan GoodnightGreg HumphreysCliff WoolleyRui Wang University of Virginia Eurographics Symposium on Rendering 2003 25-27 th June - Leuven, Belgium

2. HDR and Tone Mapping Clamped to [0,1] Compressed

3. Advances in graphics hardware n Physically-based rendering on the GPU (Purcell et al, 2003) n High dynamic range texture mapping (Debevec et al, 2001)

4. System Overview n Interactive tone mapping system for an OpenGL application tone mapping systemapplication HDR image LDR image Frame buffer Display callback

5. Interface to the application tmInitialize(); // Initialize the system tmEnable(); // Retarget GL calls n Draw geometry tmCompress(); // Compress output tmDisable(); // Restore app context tone mapping system application

6. Choosing a tone mapping operator n Photographic Tone Reproduction for High Contrast Images (Reinhard et al, 2002) n Global operator is a simple transfer function scaled luminance 0 1

7. Choosing a tone mapping operator n Local operator n Digital analog to ‘burning’ and ‘dodging’ local area luminance Center-surround

8. n Global operator is simple and fast to compute n Only one global computation n We can dynamically choose the number of zones Why use this tone mapping operator?

9. Variable number of zones: 3 3 Zones

10. Variable number of zones: 4 3 Zones

11. Variable number of zones: 5 3 Zones

12. Variable number of zones: 6 3 Zones

13. Variable number of zones: 7 3 Zones

14. Variable number of zones: 8 3 Zones

15. System block diagram

16. Implementation n Target architecture n ATI Radeon 9800 (R350) n Data storage n Floating-point off-screen buffers (pbuffers) n Multiple rendering surfaces (GL_AUXi) n Algorithms n ARB fragment and vertex assembly n Generate fragments with image-sized quads n Data representation n Vector vs. scalar organization

17. Global operator block diagram

18. Implementation: global operator n Simple luminance transform n Store luminance and log luminance in separate channels HDR image Luminance Log luminance Mipmap reduction LDR image Single buffer luminance log luminance

19. Implementation: global operator HDR image Luminance Log luminance Mipmap reduction LDR image Single buffer Single rendering surface log luminance channel log average luminance

20. Implementation: global operator HDR image Luminance Log luminance Mipmap reduction LDR image Single buffer operator shader texture 0 texture 1 texture 2

21. Local operator block diagram

22. Implementation: GPU-based convolutions n Transform n-vector product into multiple 4-vector products filter luminance + + …………

23. Vectorizing the luminance n Output 4 pixels at the same time n Useful for expensive algorithms n Requires a conversion back to scalar form. Stacked domain

24. n A simple method for luminance vectorization: Vectorizing the luminance R G B A luminance

25. n A simple method for luminance vectorization: Vectorizing the luminance R G B A luminance

26. n A simple method for luminance vectorization: Vectorizing the luminance R G B A luminance

27. n A simple method for luminance vectorization: Vectorizing the luminance R G B A luminance

28. n A simple method for luminance vectorization: n Preserves spatial locality Vectorizing the luminance R G B A luminance

29. filter image Example:1 x n inner product stacked image GPU-based convolutions

30. filter image stacked image GPU-based convolutions Pass 1

31. filter image stacked image GPU-based convolutions Pass 1Pass 2 +

32. filter image stacked image GPU-based convolutions Pass 1Pass 2Pass 3 ++

33. GPU-based convolutions n Compute multiple 4-vector products per pass n Less shader and texture switching stacked image ++ Single render pass

34. GPU-based convolutions n Compute multiple 4-vector products per pass n Less shader and texture switching stacked image ++ Single render pass

35. GPU-based convolutions n Compute multiple 4-vector products per pass n Less shader and texture switching stacked image ++ Single render pass

36. GPU-based convolutions n Compute multiple 4-vector products per pass n Less shader and texture switching stacked image ++ Single render pass

37. GPU-based convolutions n Compute multiple 4-vector products per pass n Less shader and texture switching stacked image ++ Single render pass

38. GPU-based convolutions n Advantages : n Handles large kernels n Efficient memory access n No transform back to scalar values 21 x 21 kernel ~ 10 ms 41 x 41 kernel ~ 16 ms 11 x 11 kernel ~ 6 ms 512 X 512 image

39. System block diagram

40. Calculating adaptation zones on the GPU luminance 0 Buffer 0Buffer 1 FRONT BACK 1 filtered

41. Calculating adaptation zones on the GPU luminance 2 Buffer 0Buffer 1 FRONT BACK 1 filtered

42. Calculating adaptation zones on the GPU luminance 2 Buffer 0 FRONT BACK 3 Buffer 1 filtered

43. Calculating adaptation zones on the GPU luminance 4 FRONT BACK 3 Buffer 0Buffer 1 filtered

44. Image size Frames per second 16 bit floats 32 bit floats Performance: global operator

45. Performance: local operator Number of zones 16 bit floats 32 bit floats Frames per second

46. Performance comparison: CPU vs. GPU

47. Results: Accuracy n Comparison with CPU: 512 x 512 image ImageRMS % error Scaled luminance0.022 % Convolution (5 x 5)0.026 % Convolution (49 x 49)0.032 % Final image1.051 %

48. False-color zone images CPUGPU

49. Compressed: 2 zonesClamped [0,1] Images generated at ~30Hz

50. Compressed: 2 zonesClamped [0,1]

51. Compressed: 2 zonesClamped [0,1] Images generated at ~30Hz

52. Compressed: 2 zonesClamped [0,1] Images generated at ~30Hz

53. Compressed: 2 zonesClamped [0,1] Images generated at ~30Hz

54. Compressed: 2 zonesClamped [0,1] Images generated at ~30Hz

55. Conclusion and Future Work n Summary n System for interactively compressing HDR output from an OpenGL application n Complex tone mapping operator on the GPU n Future Work n Other tone mapping operators n Further optimizations n Non-invasive implementation