1. High-Quality Parallel Depth-of- Field Using Line Samples Stanley Tzeng, Anjul Patney, Andrew Davidson, Mohamed S. Ebeida, Scott A. Mitchell, John D. Owens University of California, Davis Sandia National Laboratories 1

2. Depth of Field is Beautiful Pixar’s Toy Story 3

3. Depth of Field is Beautiful and in games

4. The Problem 4 Good noise-reducing Depth-of-Field effects takes a lot of point samples!

5. The Problem 5 Good noise-reducing Depth-of-Field effects takes a lot of point samples!

6. The Problem 6 Good noise-reducing Depth-of-Field effects takes a lot of point samples!

7. The Problem 7 Good noise-reducing Depth-of-Field effects takes a lot of point samples!

8. This Talk Instead of point samples, use line samples! – Heavier compute per sample, but need fewer samples for good results A tiled line sampling renderer for graphics hardware – How to make line sampling work within tight memory constraints?

9. Line Samples A dimensional extension of point samples Gribel et al. : High-Quality Spatio- Temporal Rendering using Semi-Analytical Visibility Jones and Perry: Antialiasing with Line Samples 9

10. Point Samples in a pixel 10 Scene primitive Pixel

11. Point Samples in a pixel 11 Pixel Shoot samples

12. Point Samples in a pixel 12 Pixel Intersect & Generate colored samples

13. Point Samples in a pixel 13 Pixel Composite

14. Line Samples in a pixel 14 Pixel

15. Line Samples in a pixel 15 Pixel Shoot samples

16. Line Samples in a pixel 16 Pixel Intersect & Generate colored line segments

17. Line Samples in a pixel 17 Pixel Composite

18. Side Note 18 Pixel Samples can be in any arbitrary orientation.

19. Line Samples in a pixel 19

20. Depth of Field Sampling Point Sampling: Generate 4D point samples in (x,y,u,v) space. – Point on screen (x,y) + – Point on lens (u,v) Line Sampling: Fix a point in screen space (x,y), sample along the lens in (u,v) space.

21. How to Sample Along the Lens? Good area coverage Uneven line lengths Grid Pinwheel Even Line Lengths Bias towards center

22. How to Sample Along the Lens? Pinwheel Even Line Lengths Bias towards center Reweigh the samples obtained

23. Line Samples for DoF u x 23

24. Line Samples for DoF u v 24 Step 1: Project scene geometry from screen space to lens space. y x

25. Line Samples for DoF u v 25 Step 2: Sample the lens using a wagonwheel pattern y x

26. Line Samples for DoF u v 26 Step 3: Record intersected segments y x

27. Line Samples for DoF u v 27 Step 4: Composite segments to pixel color y x

28. Line Samples using DoF 28

29. Line Samples using DoF 29

30. Line Samples using DoF 30

31. Line Samples using DoF 31

32. Comparison Point SamplesLine Samples 32

33. Comparison Point SamplesLine Samples 33 Metric of Comparison!

34. Line Sampling on the GPU What’s really important to us: Easy parallel processing Bounded memory requirements – Shared memory is limited – Games require fixed storage

35. Parallel Implementation: 2 Kernels 35 Intersect line samples with scene geometry, & generate line segments Global Line Segment Buffer Composite and filter segments into final pixels

36. Parallel Implementation: 2 Kernels 36 Intersect line samples with scene geometry, & generate line segments Global Line Segment Buffer Composite and filter segments into final pixels Problem Occurs Here!

37. Problem 800 x 600 x 16 x 16 x 24 widthheight# line samples # line segments Bytes per segment 37

38. Binning Split screen into tiles – use shared memory instead of global memory Combine the two kernels into one 38 Global Memory Shared Memory

39. New Problem New problem: Shared memory is small. ? Line Segment Buffer Line Segments 39

40. Line Segments Solution: Keep only the important line segments! Line Segments 40

41. Heuristics Only keep segments that contribute the most to the color of the sample Green Segment Too Small Green Segment Occluded 41

42. Heuristics Only keep segments that contribute the most to the color of the sample Green Segment Too Small Green Segment Occluded 42 Blue Segment Merged

43. Heuristics Rules: Discard if segment is too short Discard if segment is occluded Merge if segments are similar A small amount of shared memory = small tiles Another Problem! 43

44. Level of Detail 4x4 bins, 256 max triangles 4x4 bins, 1024 max triangles

45. Bins Overflow 4x4 bins, 256 max triangles 45

46. Bins Overflow 4x4 bins, 512 max triangles 46

47. Bins Overflow 4x4 bins, 1024 max triangles 47

48. Bins Overflow 48 Overflow because of depth Overflow because of large CoC.

49. Bins Overflow 49 Overflow because of depth Overflow because of large CoC.

50. Bins Overflow 50 Overflow because of depth Overflow because of large CoC.

51. Bins Overflow 51 Overflow because of depth Overflow because of large CoC.

52. Level of Detail 52 Close and blurred Low detail Close and sharp High detail Far and blurred Low detail

53. Level of Detail c / a says, in additional to a traditional LoD scheme: If object is close but very blurred – choose lower LoD If object is far but still sharp – choose higher LoD No LoD LoD Ground Truth 1/3 as many triangles, and 1.6 x improvement in speed

54. Final System Structure 54

55. Final System Structure 55 Stage 1: Vertex Transform and CoC computation.

56. Final System Structure 56 Stage 2: Triangle Binning into Tiles

57. Final System Structure 57 Stage 3: Tile Sample, Composite, and Filter

58. Results 256 Point Samples 0.49 fps 16 Line Samples 2.04 fps 58

59. Results 59 256 Point Samples 0.64 fps 16 Line Samples 2.86 fps

60. Drawbacks Several Problems to the current model Heavily dependent on shared memory. Not completely accurate – sometimes significant segments may be discarded. Shading is done once per line segment. – Can cause oversmoothing. 60

61. Artifacts Point Samples Too few point samples causes noise Line Samples Too few line samples causes ghosting

62. Convergence Point Samples Takes many samples to converge Line Samples Converges much faster than point samples

63. Contributions Depth-of-field using line sampling techniques Implementing tiled line sampler on GPUs Blur dependent level of detail 63

64. Future Work Line sampling in more aggressive stochastic rendering systems – Area lighting / Soft shadows – Global Illumination Better memory management – Better heuristics? 64

65. Thank you! 65 Special thanks to Intel’s ISTC-VC and NSF for funding! Alduin from TES V: Skyrim