1. Fast and Accurate Soft Shadows using a Real-Time Beam Tracer Ravi Ramamoorthi Columbia Vision and Graphics Center Columbia University http://www.cs.columbia.edu/~ravir Intel, Mar 29, 2007 Joint work with Ryan Overbeck (Columbia), Bill Mark (UT Austin)

2. Beam Tracing: Reference Quality (Our Method) Ray Tracing: Comparable Time 4 Shadow Rays Ray Tracing: Comparable Quality 256 Shadow Rays 3 Seconds 99 Seconds  Note that secondary effects (soft shadows) are often hard to accelerate in modern fast ray tracers Result: Beam-Traced Soft Shadows

3. Outline  Motivation  Beam Tracing Algorithm  Beam-Triangle Intersection  Beam-KD-tree Traversal  Analysis – Primary Visibility  Soft Shadows  Future Work

4. Background Renaissance of interest in ray tracing [Whitted 80]  Real-time ray tracing has become a reality by exploiting geometric coherence of rays in the scene  Ray Packets [Wald et al. 2001]  Frustum Proxies [Reshetov et al. 2005 (MLRTA)]  SIMD instructions (Intel SSE2) to trace multiple rays  Ray tracing can produce significantly higher quality images than standard rasterization based algorithms  Potential for a revolution in interactive graphics pipeline

5. Challenges Problems with current Real-Time Ray Tracing algorithms  How to use packets/frusta well for secondary visibility?  More geometric coherence than current ray tracers use  Per pixel operations (even simple shading) bottleneck  Can’t yet afford to subsample pixels leading to aliasing Our solution: Real-Time Beam Tracing  Trace beams instead of individual rays  Old technique [Heckbert Hanrahan 84] but previously considered too slow, complicated

6. Beam Tracing Old technique [Heckbert and Hanrahan 84]  Trace beams instead of individual rays  Automatically adapts to geometric coherence  Provides area samples (can be antialiased)  Well suited secondary effects (shadows, reflections)

7. Beam Tracing  Start with beam and scene geometry  For primary visibility, the beam is the view frustum

8. Beam Tracing  The beam is intersected against the first geometry primitive

9. Beam Tracing  The beam is split against the primitive’s boundaries (triangle edges)

10. Beam Tracing  The beam is split against the primitive’s boundaries (triangle edges)

11. Beam Tracing  Each sub-beam is intersected against the other primitives and is recursively split

12. Beam Tracing  Each sub-beam is intersected against the other primitives and is recursively split

13. Beam Tracing  Each sub-beam is intersected against the other primitives and is recursively split

14. Beam Tracing  Each sub-beam is intersected against the other primitives and is recursively split

15. Beam Tracing  Each sub-beam is intersected against the other primitives and is recursively split

16. Beam Tracing  Each sub-beam is intersected against the other primitives and is recursively split

17. Beam Tracing  The result is the visible surface of the scene

18. Beam Tracing  The final hit beams often contain many samples/pixels, for large performance gains

19. Beam Tracing addresses Challenges  How to efficiently address secondary visibility?  Large coherent area samples for hard shadows  Only need area coverage on light for soft shadows  More geometric coherence than current ray tracers  Automatically adapts to (all) available coherence  Only split beam when necessary, at triangle boundaries  Per pixel (even simple shading) bottleneck  Returns area samples instead of point samples  Independent of resolution, combines with GPU, software  Aliasing: Can’t yet afford to subsample pixels  Area elements: can inherently be antialiased on GPU

20. Beam Tracing: Cons and Contrib. Little work since HH84 (considered slow technique)  Beam-Triangle intersection is slow vs ray-triangle  No effective acceleration structures, fast traversals  Often used in acoustics [Funkhouser 98,99] Contributions  Fast Beam-Triangle Intersections  Fast Beam-Kd-tree traversal  Includes modern ray tracing accelerations, SSE  Application to primary, secondary visibility

21. Beam Tracing: In Action

22. Outline  Motivation  Beam Tracing Algorithm  Beam-Triangle Intersection  Beam-KD-tree Traversal  Analysis – Primary Visibility  Soft Shadows  Future Work

23. Beam-Triangle Intersection  Beam intersects triangle and splits  That part that hits, and that part that misses  High-Level Algorithmic Decisions  Mirror ray-triangle intersection, diverging only if needed  Parallelize through use of SIMD SSE Instructions  Operate on (3 or 4) corners of beam as if single ray  Algorithm Overview  Triangle projection (into 2D image plane)  Handle trivial cases (no splitting)  Beam splitting

24. Beam—Triangle Intersection  Key to efficient intersection is separating trivial from non-trivial interactions. Beam Triangle

25. Beam—Triangle Intersection  Beam—Triangle Interaction  Trivial  Triangle Edge Separator  Non-Trivial

26. Beam—Triangle Intersection  Beam—Triangle Interaction  Trivial  Triangle Edge Separator  Beam Edge Separator  Non-Trivial

27. Beam—Triangle Intersection  Beam—Triangle Interaction  Trivial  Triangle Edge Separator  Beam Edge Separator  Beam Inside Triangle  Non-Trivial

28. Beam—Triangle Intersection  Beam—Triangle Interaction  Trivial  Triangle Edge Separator  Beam Edge Separator  Beam Inside Triangle  Non-Trivial  Everything else  SPLIT the beam

29. Beam—Triangle Intersection  Beam—Triangle Interaction  Trivial  Triangle Edge Separator  Beam Edge Separator  Beam Inside Triangle  Non-Trivial  Everything else  SPLIT the beam

30. Beam—Triangle: Split Procedure  For each edge  Split beam into 2 sub-beams  Result is 2 sets of sub-beams  Possible Hit beams  Miss beams Miss Beam Possible Hit Beam

31. Beam—Triangle: Split Procedure  For each edge  Split each potential hit beam into 2 sets of sub-beams  Result is 2 sets of sub-beams  Potential Hit beams  Miss beams

32. Beam—Triangle: Split Procedure  For each edge  Split each potential hit beam into 2 sets of sub-beams  Result is 2 sets of sub-beams  Potential Hit beams  Miss beams

33. Beam—Triangle: Split Procedure  Sometimes an extra split is required to restrict beams to having only 3 or 4 corner rays.

34. Beam—Triangle: Split Procedure  Final result is 2 lists of beams  Hit Beams (yellow)  Miss Beams (red)

35. Beam-Triangle – Other splits  Other split cases handled in same way  In the paper we describe a fast new data parallel algorithm (using SIMD SSE instructions) for performing this beam—triangle clipping.

36. Beam-Triangle Intersect. – Performance 15-40 FPS5-10 FPS ~800 Triangles~2000 Triangles

37. Outline  Motivation  Beam Tracing Algorithm  Beam-Triangle Intersection  Beam-KD-tree Traversal  Analysis – Primary Visibility  Soft Shadows  Future Work

38. Rays vs. kd-tree -> Rays vs. Slabs  Ray – kd-tree traversal based on slabs

39. Rays vs. Slabs  The current bounding box is the intersection of slabs X slab Y slab Current Bounding Box

40. Rays vs. Slabs  This ray intersects the current bounding box

41. Rays vs. Slabs  We calculate the ray’s entry (tmin) and exit (tmax) distance for each slab

42. Rays vs. Slabs  We calculate the ray’s entry (tmin) and exit (tmax) distance for each slab

43. Rays vs. Slabs  If the ray enters each slab before it exits any other slab, then it hits the bounding box

44. Rays vs. Slabs  How do we know that this ray misses the bounding box?

45. Rays vs. Slabs  Because it exits the x slab before it enters the y slab

46. Rays vs. kd-tree  For a ray in a kd-tree, it is enough for a ray to keep track of where the ray enters the current bounding box (tmin) and where it exits (tmax)

47. Beams vs. kd-tree  How do we trace a beam through a kd-tree?

48. Beams vs. kd-tree  How do we trace a beam through a kd-tree? Kd-tree Current Bounding Box

49. Beams vs. kd-tree  How do we trace a beam through a kd-tree? Beam

50. Beams vs. kd-tree  If we just use the tmins and tmaxes from the corner rays, the tmin/tmax points define a near and far plane for the active part of the beam.  This works as long as all four corners take the same path through the tree. tmins tmaxes

51. Beams vs. kd-tree  But if the next split plane causes the beam’s rays to take different paths… Next split plane

52. Beams vs. kd-tree  …the new tmax value lies on a different kd plane from the other tmax values… New tmax

53. Beams vs. kd-tree  …and the new tmaxes define a new far plane…

54. Beams vs. kd-tree  …which doesn’t fully represent the active portion of the beam. Under-represented area

55. Beams vs. kd-tree  This causes problems in 3D, when the corner of a split plane passes through a face of the beam.  The beam on the right will only traverse the far side of the split plane when both sides should be visited

56. Beams vs. kd-tree  Another option…

57. Beams vs. kd-tree  Split the beam where the split plane passes through the far plane of the active beam. Beam split

58. Beams vs. kd-tree  This assures that the tmins/tmaxes of all corner rays lie on the same kd-plane.  This works, but excessive splitting leads to slower render times.

59. Beams vs. kd-tree  Instead we keep multiple tmin/tmax values, one for each dimension. tmax x y

60. Beams vs. kd-tree  The active part of the beam is the intersection of the beams defined by these planes. Active part of beam

61. Beams vs. kd-tree  To traverse the tree, we much check the tmins/tmaxes on all dimensions Example: If all of the beam’s tmax values for the x-slab are less than the distance to the current split plane, only the near side of the plane needs to be traversed.

62. Beams vs. kd-tree  This beam will hit the large red triangle before it sees the smaller green triangle.  Hit beams must continue KD tree traversal until its hit distance is fully contained by one of the beam’s tmin planes and all of its tmax planes.

63. Outline  Motivation  Beam Tracing Algorithm  Beam-Triangle Intersection  Beam-KD-tree Traversal  Analysis – Primary Visibility  Soft Shadows  Future Work

64. Beam Tracing – Primary Visibility  Trace primary beams, obtain hit beams (areas)  Hit beams as quads to GPU for rasterization  Area elements exactly represent visible surface of scene (often much more compact than all triangles)  Compact area elements enable GPU per-pixel shaders, 6x antialiasing (ray tracing slows down)  Resolution independent (same time for 1024x1024 or higher as for 512x512)

65. Comparison Setup  Average numbers over multiple views (all methods)  Beams vs Rays (our optimized) vs MLRT (Intel)  Our ray tracer optimized, but one at a time (useful for comparing statistics)  MLRT uses one thread, with shading, display no quad optimization.  Diffuse shading, single source at viewer  kd-trees from MLRT (tuned to their performance)  1024x1024 resolution  Beams antialiased (for free). MLRT, rays not.  3.0 GHZ Pentium 4, 1.5GB RAM, ATI 9800 card

66. Beam Tracing – Primary Visibility

67. Simple Scene Erw6 Beam Tracer: 169.49 FPS MLRT: 12.99 FPS 816 Triangles (150 visible tris ; few hit beams) BeamsRays Kd steps per pixel 0.008513.86 Intersect. per pixel 0.0033 6.81 Hits per Pixel 0.0010 0.95 Frame Rate 169.49 fps 0.99 fps

68. Complex Scene: Soda Hall Beam Tracer: 21.28 FPS MLRT: 5.56 FPS Z-buffer (GPU): 13-15 FPS 2M Triangles (1500 visible tris) BeamsRays Kd steps per pixel 0.1342.36 Intersect. per pixel 0.064 4.29 Hits per Pixel 0.0097 0.98 Frame Rate 21.28 fps 0.52 fps

69. Conference (High Tesselation) Beam Tracer: 5.56 FPS MLRT: 7.12 FPS 282K Triangles (4600 visible). Few kd-steps, int. BeamsRays Kd steps per pixel 0.3025.67 Intersect. per pixel 0.4816.34 Hits per Pixel 0.027 0.91 Frame Rate 5.56 fps 0.52 fps

70. Armadillo (subpixel tris; worst case) Beam Tracer: 1.72 FPS MLRT: 5.99 FPS 345K Triangles (26000 visible tris) BeamsRays Kd steps per pixel 1.0746.46 Intersect. per pixel 0.60 5.47 Hits per Pixel 0.14 0.99 Frame Rate 1.72 fps 0.44 fps

71. Number of Hit Beams: Beam Splitting  # Hit Beams ~= 6 x # Visible Triangles

72. Performance vs # Visible Triangles  Performance linear w.r.t. # Visible Triangles

73. Performance Comparison: Summary  Usually 1-2+ orders of magnitude fewer kd-steps, intersection tests per pixel than ray tracing  Faster on all scenes than our raytracer, on many than MLRT (and potential for further optimization)  Soda Hall among most impressive  Absolute size of model slows MLRT down in the kd- tree, while beams keep to leaves  High quality hard shadows (secondary effects)  Simply connect point light to (large) primary hit beams  Can obtain antialiasing essentially for free

74. Beam Tracing: Hard Shadows  Qualitative benefits over ray tracing:  Large coherent regions for secondary visibility

75. Beam Analysis – Primary Rays  Qualitative benefits over ray tracing:  Large coherent regions for secondary beams  High quality anti-aliased primary visibility AND secondary visibility Beam Tracer anti-aliased MLRT (ray tracer) without anti-aliasing

76. Video 1

77. Outline  Motivation  Beam Tracing Algorithm  Beam-Triangle Intersection  Beam-KD-tree Traversal  Analysis – Primary Visibility  Soft Shadows  Future Work

78. Soft Shadows  Connect pixel with beam to area light vertices  Only need total area coverage on light source

79. Suitability for Soft Shadows  Range where beam tracing is most useful  Typically less than 10 visible tris per pixel  Observe that hit beams 2x number visible triangles (not 6x as in primary visibility)  10x – 40x faster than ray tracing  Can be much faster than soft shadow volumes [Laine et al. 2005, Lehtinen et al. 2006]

80. Comparison Setup  Use our optimized ray tracer for comparison  Not clear how to adapt frustum methods/ray packets  512x512 resolution (linear scaling with res)  Our kd-tree builder (basic surface area heuristic)  Lambertian materials, single square light  3.0 GHZ Pentium 4, 1.5GB RAM, ATI 9800 card

81. Soft Shadows – Plant (512x512)  Only 5245 triangles, but high shadow complexity  Almost every edge a silhouette edge Ray Tracing: Comparable Time Beam Tracing: Exact Ray Tracing: Comparable Quality (256 shadow rays) 5 seconds / 9 shadow rays 5 seconds 108 seconds

82. Soft Shadows – Soda (512x512) Ray Tracing: Comparable Time Beam Tracing: Exact Ray Tracing: Comparable Quality (256 shadow rays) 1.84 seconds / 4 shadow rays 1.78 seconds 79 seconds

83. Video 2

84. Summary  New algorithms for real-time beam tracing  New method for fast soft shadows  Beam tracing has many benefits over ray tracing  Can be significantly faster when required sample density is higher than the required geometric fidelity  Area Samples vs. Point Samples  Antialiasing, shading  Perfect soft shadows and other secondary effects  Beam tracing old technique, assumed slow  No longer true, viable method for real-time, shadows

85. Future Work  Area to area beams for faster soft shadows  Fall back on point samples when geometric fidelity is too high (tris smaller than pixel)  Multiresolution representations (aka Razor)  Explore other visual effects  Specular to diffuse interactions (Caustics)  Glossy reflections

86. The Future  What is the future rendering algorithm?  Hybrid Beam Tracing/GPU ?  Hybrid Beam Tracing/Ray Tracing?  How does it depend on visual effects needed?  On future architectures?

87. Research Projects  High quality real-time rendering  Real-time ray tracing  All-frequency interactive relighting  Volumetric scattering in mist, fog, rain  Data-driven appearance acquisition, rendering  Complex lighting, materials in computer vision  Mathematical foundations of appearance

88. The End