2. A Perceptual Heuristic for Shadow Computation in Photo-Realistic Images Wednesday, 2 August 2006 Peter VangorpOlivier DumontToon LenaertsPhilip Dutré Katholieke Universiteit Leuven {peter.vangorp,olivier.dumont,toon.lenaerts,philip.dutre}@cs.kuleuven.be

3. Introduction 3 types of realistic rendering

4. Introduction – radiometric accuracy “render everything a photometer can detect”

5. Introduction 3 types of realistic rendering – radiometric accuracy “render everything a photometer can detect” – physiological perception “render only what the eye can see”

6. Introduction 3 types of realistic rendering – radiometric accuracy “render everything a photometer can detect” – physiological perception “render only what the eye can see” – psychological perception “render only what the brain can see”

7. Radiometric accuracy measure shapes, light sources, materials,... photometer Cornell box [Meyer et al. 1986]

8. Physiological perception use low-level limitations of human visual system threshold vs intensity, contrast sensitivity,... referencethreshold mapvisibly indistinguishable adaptive rendering [Ramasubramanian et al. 1999]

9. Physiological perception use low-level limitations of human visual system threshold vs intensity, contrast sensitivity,... referencethreshold mapvisibly indistinguishable adaptive rendering [Ramasubramanian et al. 1999]

10. Psychological perception use higher-level heuristics “Is a shadow necessary for the realism of a scene?” [Thompson et al. 1998]

11. Psychological perception use higher-level heuristics “Is a shadow necessary for the realism of a scene?” “Do we need highlights to convey material properties?” [Thompson et al. 1998] [Fleming et al. 2004]

12. Psychological perception use higher-level heuristics “Is a shadow necessary for the realism of a scene?” “Do we need highlights to convey material properties?” “How detailed should the geometry be?” [Thompson et al. 1998] [Luebke 2001] [Fleming et al. 2004]

13. Motivation & Goals Motivation – shadows are important for perception of realism Goal – detect perceptually important shadows in the scene – render important shadows accurately – approximate unimportant shadows [Kersten et al. 1997]

14. Methodology 1. Psycho-physical experiments 2. Derive a heuristic predicting shadow importance 3. Design a perceptually driven algorithm 4. Experimental validation

15. Methodology 1. Psycho-physical experiments 2. Derive a heuristic predicting shadow importance 3. Design a perceptually driven algorithm 4. Experimental validation

16. Methodology 1. Psycho-physical experiments 2. Derive a heuristic predicting shadow importance 3. Design a perceptually driven algorithm 4. Experimental validation

17. Methodology 1. Psycho-physical experiments 2. Derive a heuristic predicting shadow importance 3. Design a perceptually driven algorithm 4. Experimental validation

18. 1. Perceptual experiments Test setup: 162 images, varying sphere over 9 radii and 9 heights correct shadow: no shadow: (avg. illumination)

19. 1. Perceptual experiments “Does the lighting in this image look realistic?” – single stimulus – 5000+ decisions – avg. 2 sec / decision

20. 2. Heuristic large difference in realism, e.g. small difference in realism, e.g.  sphere radius  sphere height difference in realism

21. 3. A perceptually driven algorithm Ray tracing

22. 3. A perceptually driven algorithm Ray tracing – shoot viewing ray

23. 3. A perceptually driven algorithm Ray tracing – shoot viewing ray – evaluate heuristic in hit point to be shaded in function of distance and solid angle solid angledistance

24. 3. A perceptually driven algorithm Preprocessing step – shadow photon map [Jensen and Christensen 1995]

25. 3. A perceptually driven algorithm Preprocessing step – shadow photon map – shadow photons augmented with heuristic [Jensen and Christensen 1995] shadow photon map

26. 3. A perceptually driven algorithm For each viewing ray Gather nearest shadow photons Calculate average perceptual value avg < threshold approximate: photon map render accurately: shadow rays yesno

27. 3. A perceptually driven algorithm For each viewing ray Gather nearest shadow photons Calculate average perceptual value avg < threshold approximate: photon map render accurately: shadow rays yesno

28. 3. A perceptually driven algorithm For each viewing ray Gather nearest shadow photons Calculate average perceptual value avg < threshold approximate: photon map render accurately: shadow rays yesno

29. For each viewing ray Gather nearest shadow photons Calculate average perceptual value avg < threshold approximate: photon map render accurately: shadow rays yesno user-defined threshold 3. A perceptually driven algorithm

30. For each viewing ray Gather nearest shadow photons Calculate average perceptual value avg < threshold approximate: photon map render accurately: shadow rays yesno user-defined threshold

31. 3. A perceptually driven algorithm For each viewing ray Gather nearest shadow photons Calculate average perceptual value avg < threshold approximate: photon map render accurately: shadow rays yesno user-defined threshold

32. 3. A perceptually driven algorithm

33. 4. Validation Similar perceptual experiment – “Do the lighting and the shadows look realistic?”

34. 4. Validation Similar perceptual experiment – “Do the lighting and the shadows look realistic?” Stimuli: 6 scenes – threshold 25%, 50%, 75% – reference rendering (threshold 0%)

35. 4. Validation Similar perceptual experiment – “Do the lighting and the shadows look realistic?” Stimuli: 6 scenes – threshold 25%, 50%, 75% – reference rendering (threshold 0%) 15 subjects, almost 6000 decisions, avg. 5 seconds

36. 4. Validation Up to 50% of the pixels can be approximated, without loss of perceptual realism Threshold 25% Approx px 13% Threshold 50% Approx px 24% Threshold 75% Approx px 48%

37. Results referencethreshold 90%

38. Results reference threshold 90%

39. Results referencethreshold 90%

40. Results referencethreshold 90%

41. reference Results threshold 80%

42. Results referencethreshold 80%

43. Results referencethreshold 80%

44. Results referencethreshold 80%

45. reference Results threshold 90%

46. Results referencethreshold 90%

47. Results referencethreshold 90%

48. Conclusions & Future Work Intuitions confirmed by statistical data Rendering algorithm driven by perceptual information

49. Conclusions & Future Work Intuitions confirmed by statistical data Rendering algorithm driven by perceptual information Extend methodology to other phenomena Different questions than “Does this look realistic?” Better ways to incorporate perceptual information – currently no significant acceleration yet

50. Questions?