1. 1 Interactive Point Based Ray Tracing June 21, 2005 Interactive Ray Tracing of Point-based Models Ingo Wald SCI Institute, University of Utah wald@sci.utah.edu

2. 2 Interactive Point Based Ray Tracing June 21, 2005 Motivation Why point-based rendering ? – Many advantages, but no details here … But: Why point-based ray tracing ? – Actually, two different questions: Q1: Why ray tracing for point-based models ? Q2: Why use point-based techniques in ray tracing ?

3. 3 Interactive Point Based Ray Tracing June 21, 2005 Q1: Why Ray Tracing ? Reason 1: Logarithmic scalability to large models – Very handy for today‘s huge models – But: For PBR not that important (multiresolution)… Reason 2 : Image quality / shading quality – Ray traced image quality not yet standard in PBG… But would be nice to have

4. 4 Interactive Point Based Ray Tracing June 21, 2005 Q2: Why Point-based techniques in Ray Tracing? Native point-based data (acquisition) – Direct ray tracing avoids tesselation Much simpler And for native triangular models ? – Triangular model  Point-based model  PBRT Does this make sense ? – Highly controversial issue amongst ray tracing researchers RT is log. in #tris, anyway. Don‘t need multiresolution… Many new problems with points as primitives – Consistency, efficiency, loss of information…

5. 5 Interactive Point Based Ray Tracing June 21, 2005 Q2: Why Point-based techniques in Ray Tracing? Interactive ray tracing: Three recent examples

6. 6 Interactive Point Based Ray Tracing June 21, 2005 Q2: Why Point-based techniques in Ray Tracing? Interactive ray tracing: Three recent examples – Example 1: Boeing 777 (350 million triangles, 40 GB data)

7. 7 Interactive Point Based Ray Tracing June 21, 2005 Q2: Why Point-based techniques in Ray Tracing? Interactive ray tracing: Three recent examples – Example 2: Lawrence Livermore isosurface (8 billion voxels)

8. 8 Interactive Point Based Ray Tracing June 21, 2005 Q2: Why Point-based techniques in Ray Tracing? Interactive ray tracing: Three recent examples – Example 3: Forest landscape (1.5 billion triangles)

9. 9 Interactive Point Based Ray Tracing June 21, 2005 Q2: Why Point-based techniques in Ray Tracing? Interactive ray tracing: Three recent examples – Yes, can render all three already today without PB techniques – But: Doesn‘t make sense (>1G prim. for <1M pixels ?) Too much data: Tiny camera move  totally different data – Even though image doesn‘t change at all… Sample only every 1000 th pixel  Severe aliasing – Aliasing: THE most important problem of RTRT today  Need multiresolution approach also for RT – Probably best via PB approach Current results only intermediate step… … towards long-term goal of multiresolution (PB)RT

10. 10 Interactive Point Based Ray Tracing June 21, 2005 How to ray trace PB models ? Problem: Zero ray-point hit probability – Rays infinitely thin – Points infitnitely thin, too Two possible solutions – Use „thick“ rays that have volume (cones, beams, …) Reconstruct from all points overlapping ray‘s volume – „Grow“ points to have a surface can then be found by „thin“ ray – Long history of debate on which is better…

11. 11 Interactive Point Based Ray Tracing June 21, 2005 How to ray trace PB models ? Alternative 1: Growing rays – Experience: Tracing beams very costly (much more than rays)

12. 12 Interactive Point Based Ray Tracing June 21, 2005 How to ray trace PB models ? Alternative 1: Growing rays – Many un-answered questions Where exactly is the hit point ? – E.g., where to start the shadow ray from ? Hitpoint position ? Position A points gathered by beam

13. 13 Interactive Point Based Ray Tracing June 21, 2005 How to ray trace PB models ? Alternative 1: Growing rays – Many un-answered questions Where exactly is the hit point ? How stable is the decision ? Hitpoint position ? Position A points gathered by beam

14. 14 Interactive Point Based Ray Tracing June 21, 2005 How to ray trace PB models ? Alternative 1: Growing rays – Many un-answered questions Where exactly is the hit point ? How stable is the decision ? Is it consistent ? – E.g., when viewed from different angle (shadow ray) Hitpoint position ? Same surface, different viewpoint: Same hitpoint ? Is it in shadow ? surface seen from A Position A Position B (Light ?)

15. 15 Interactive Point Based Ray Tracing June 21, 2005 How to ray trace PB models ? Alternative 1: Growing rays – Many un-answered questions Where exactly is the hit point ? How stable is the decision? Is it consistent ?  Too many open questions

16. 16 Interactive Point Based Ray Tracing June 21, 2005 How to ray trace PB models ? Alternative 2: Growing points – Define unique surface through points Always consistent – Most trivial solution: Use disks/ellipses/… Trivial

17. 17 Interactive Point Based Ray Tracing June 21, 2005 How to ray trace PB models ? Alternative 2: Growing points – Define unique surface through points Always consistent – Most trivial solution: Use disks/ellipses/… Trivial, but shading discontinuities (worse w/ shadows)

18. 18 Interactive Point Based Ray Tracing June 21, 2005 How to ray trace PB models ? Better: Define smooth surface – Use Adamson/Alexa‘s model Each point x i has normal n i and radius r i of influence Influence w i falls off with distance to x i – With w i (x i )=1 and w i (x)=0 for all |x-x i |>r i Define weighted average position X(x) and N(x) Define „distance“ function F(x) := ((x-X(x)).N(x)  Implicit fct F(x)=0 is smooth surface – Need: Useful r i ‘s (not given) Fast ray/surface intersection framework

19. 19 Interactive Point Based Ray Tracing June 21, 2005 Choosing Splat Radii Step 1: Choosing the splat radii – Too small  Holes in the model – Too large  Many regions of influence will overlap (slow) Use Wu et al.‘s optimal subsampling technique – Wu [EG04]: Determine r i such as to Optimally cover surface Achieve minimal overlap  Optimal solution: Use exactly that technique Note: Can also use ‚other‘ radii – e.g., user-specified… – Might just be slower or contain holes

20. 20 Interactive Point Based Ray Tracing June 21, 2005 Intersecting the Surface Step 2: Determining the splats to intersect with – Each point x always influenced by only few x i A B C D

21. 21 Interactive Point Based Ray Tracing June 21, 2005 Intersecting the Surface Step 2: Determining the splats to intersect with Enclose each splat‘s ROI with tight fitting box A B C D

22. 22 Interactive Point Based Ray Tracing June 21, 2005 Intersecting the Surface Step 2: Determining the splats to intersect with Enclose each splat‘s ROI with tight fitting box Build kd-tree over those AA boxes A B C D

23. 23 Interactive Point Based Ray Tracing June 21, 2005 Intersecting the Surface Step 2: Determining the splats to intersect with Enclose each splat‘s ROI with tight fitting box Build kd-tree over those AA boxes A B C D

24. 24 Interactive Point Based Ray Tracing June 21, 2005 Intersecting the Surface Step 2: Determining the splats to intersect with Enclose each splat‘s ROI with tight fitting box Build kd-tree over those AA boxes A B C D

25. 25 Interactive Point Based Ray Tracing June 21, 2005 Intersecting the Surface Step 2: Determining the splats to intersect with Enclose each splat‘s ROI with tight fitting box Build kd-tree over those AA boxes A B C D

26. 26 Interactive Point Based Ray Tracing June 21, 2005 Intersecting the Surface Step 2: Determining the splats to intersect with Enclose each splat‘s ROI with tight fitting box Build kd-tree over those AA boxes Leaves: Store list of splats whose ROI overlaps cell A B C D A,B A AA,B,C B C B,C -- - - (right side incomplete)

27. 27 Interactive Point Based Ray Tracing June 21, 2005 Intersecting the Surface Step 2: Determining the splats to intersect with Enclose each splat‘s ROI with tight fitting box Build kd-tree over those AA boxes Leaves: Store list of splats whose ROI overlaps cell – KD-Tree: Inherit advantages of fast triangular ray tracing Fast traversal, occlusion culling, good kd-trees (SAH)… A B C D A,B A AA,B,C B C B,C -- - -

28. 28 Interactive Point Based Ray Tracing June 21, 2005 Intersecting the Surface Step 3: Ray/surface intersection in each voxel – Find distance t : F(t)=0 – Equidistant sampling along t Have intersection if sgn(F(t i+1 )) != sign(F(t i )) But: Too coarse sampling can miss the surface – Optimizations: Fast SSE implementation (no details) Quite fast, but: Still main cost factor  Further optimize kd-tree to minimize F(x) calls – Need to take closer look at kd-tree

29. 29 Interactive Point Based Ray Tracing June 21, 2005 KD-Tree Post-optimizations Observation 1: Many full cells don‘t contain surf. – Reason: KD-tree was built over ROI‘s ROI‘s bound surface, but not closely

30. 30 Interactive Point Based Ray Tracing June 21, 2005 KD-Tree Post-optimizations Observation 1: Many full cells don‘t contain surf. – Example from before A B C A,B A AA,B,C B C B,C -- - A,B,C A,B,C

31. 31 Interactive Point Based Ray Tracing June 21, 2005 KD-Tree Post-optimizations Observation 1: Many full cells don‘t contain surf. – Actual surface only touches few voxels A B C A,B A AA,B,C B C B,C -- -A,B,C A,B,C A,B,C

32. 32 Interactive Point Based Ray Tracing June 21, 2005 KD-Tree Post-optimizations Observation 1: Many full cells don‘t contain surf. – Many non-empty cells that are not necessary A B C A,B A AA,B,C B C B,C -- -A,B,C A,B,C A,B,C

33. 33 Interactive Point Based Ray Tracing June 21, 2005 KD-Tree Post-optimizations Observation 1: Many full cells don‘t contain surf. – Many non-empty cells that are not necessary Can make quite a difference A B C A,B A AA,B,C B C B,C -- -A,B,C A,B,C A,B,C

34. 34 Interactive Point Based Ray Tracing June 21, 2005 KD-Tree Post-optimizations Observation 1: Many full cells don‘t contain surf. Fix: Determine min(F()) and max(F()) in each cell  Cull cells with min > 0 or max < 0 – Currently: Sample F with random x  Can produce false culling (holes) – Better: Interval arithmetic (Not implemented yet)

35. 35 Interactive Point Based Ray Tracing June 21, 2005 KD-Tree Post-optimizations Observation 2: Most cells too large – Reason: Only split planes at ROI borders – Example from earlier-on: A B C A,B A AA,B,C B C B,C -- -

36. 36 Interactive Point Based Ray Tracing June 21, 2005 KD-Tree Post-optimizations Observation 2: Most cells too large – Reason: Only split planes at ROI borders – Example from earlier-on: Green regions would suffice… A B C A,B A AA,B,C B C B,C -- - A

37. 37 Interactive Point Based Ray Tracing June 21, 2005 KD-Tree Post-optimizations Observation 2: Most cells too large – Reason: Only split planes at ROI borders – Example from earlier-on: Green regions would suffice… Fix: Post-shrinking of cells – Cut voxel into slices, cull slices Same problem as before: False culling… Important: Not only fewer cell intersections… – Also: Cells much thinner: Finer sampling, fewer artifacts

38. 38 Interactive Point Based Ray Tracing June 21, 2005 KD-Tree Post-optimizations Observation 3: After shrink/cull, many cells empty – Example from earlier-on: Most cells empty after cull/shrink A,B A AA,B,C B C B,C -- -

39. 39 Interactive Point Based Ray Tracing June 21, 2005 KD-Tree Post-optimizations Observation 3: After shrink/cull, many cells empty – Example from earlier-on: Most cells empty after cull/shrink C A,B B,C A

40. 40 Interactive Point Based Ray Tracing June 21, 2005 KD-Tree Post-optimizations Observation 3: After shrink/cull, many cells empty – Example from earlier-on: Most cells empty after cull/shrink – Optimization: Post-collapse Undo splits that don‘t make sense any more C A,B B,C A

41. 41 Interactive Point Based Ray Tracing June 21, 2005 Results Overall Framerate:1Dual-Opteron 2.4GHz, 512x512 ModelSizeDiffusePhong w/Shadows Iphigenia Head1M6.8Not measured Iphigenia Full1M15.9Not measured Octopus465K8.97.5 Dragon1.3M7.47.1 David1.5M11.27.9

42. 42 Interactive Point Based Ray Tracing June 21, 2005 Results Overall Framerate: ~5-15 fps @ 1PC Nice scalability to very complex models – „ManyIffis“: 24M points w/ shadows  ~2fps@1PC

43. 43 Interactive Point Based Ray Tracing June 21, 2005 Results Overall Framerate: ~5-15 fps @ 1PC Nice scalability to very complex models High-quality shading – Global Illumination

44. 44 Interactive Point Based Ray Tracing June 21, 2005 Results Overall Framerate: ~5-15 fps @ 1PC Nice scalability to very complex models High-quality shading Problems and limitations: – Sometimes small holes (false culling, undersampling, …) – All is static: Can‘t move points, can‘t change radii – Assumes continuous surface: Not applicable to random point cloud (forest leaves…)

45. 45 Interactive Point Based Ray Tracing June 21, 2005 Summary & Future Work Summary – Motivated/Argued for Point-based Ray Tracing – Discussed „thin rays“ vs „thick rays“ issue – Outlined framework for Interactive PBRT Future work – Performance & Stability Bounding the surface, interval arithmetic, … – Multiresolution PBRT In particular, for massively complex models

46. 46 Interactive Point Based Ray Tracing June 21, 2005 Questions ?

47. 47 Interactive Point Based Ray Tracing June 21, 2005 Questions ?

48. 48 Interactive Point Based Ray Tracing June 21, 2005 Q1: Why Ray Tracing ? Impact of Shadows and Reflections: Example

49. 49 Interactive Point Based Ray Tracing June 21, 2005 Q2: Why Point-based techniques in Ray Tracing? Need multiresolution approach also for RT – Trivial for „nice“ (manifold) meshes (Progressive meshes,…) – But: „problematic“ for real-world models (Forest, Boeing, …) Should be much easier for point-based representation  Eventual goal of (our) PBRT activities – Not ray tracing (nice) point-based models – Rather: Multiresolution ray tracing using a PB approach Reduce (in-core) data Fight aliasing – Current activities only first step in that direction…

50. 50 Interactive Point Based Ray Tracing June 21, 2005 Adamson&Alexa‘s model Observations – Need position and normal for each point  assume as given – Need radius of influence for each point – For each x, most w i (x) will be zero Fast computation: Need to know which... – Evaluating F(x) will be costly Need to minimize F(x) calls Need to quickly determine where surface can NOT be Need to minimize number of points contributing to x  Need to minimize regions of influence (minimize r i !)

51. 51 Interactive Point Based Ray Tracing June 21, 2005 So: Why Interactive Point- based Ray Tracing ? Advantages of PBG in particular for ray tracers: Native point-based data (acquisition) – Avoid tesselation … sure, makes sense. And for native triangular models ? – Triangular model  Point-based model  PBRT Does this make sense ?

52. 52 Interactive Point Based Ray Tracing June 21, 2005 So: Why Interactive Point- based Ray Tracing ? Advantages of PBG in particular for ray tracers:  Highly controversial issue amongst ray tracers – Ray tracing is logarithmic in scene size, anyway Don‘t need point-based multiresolution capabilities – Triangles very well suited for fast ray tracing How to (efficiently) intersect points ? – Triangles: well-defined surface How to consistently intersect points ? In particular for secondary rays – shadow ray has to see exactly the same surface Sampling: Loss of information ?  Many new problems, PBG advantages over rast. don‘t apply…  Where are the advantages ?

53. 53 Interactive Point Based Ray Tracing June 21, 2005 Q1: Why Ray Tracing Impact of Shadows and Reflections: Example

54. 54 Interactive Point Based Ray Tracing June 21, 2005 Motivation: Why Point-based Rendering ? Point-based models become commonplace – Acquisition: Almost all models are point clouds Direct PBR simplifies 3D processing pipeline – Today: Acquisition  triangulation  rendering – Direct PBR: Get rid of (non-trivial) triangulation – Plus: Increasing use in rendering/modelling Multiresolution, no connectivity/topology, much simpler,…

55. 55 Interactive Point Based Ray Tracing June 21, 2005 Motivation: Why Point-based Rendering ? Point-based models become commonplace – Acquisition: Almost all models are point clouds Direct PBR simplifies 3D processing pipeline – Today: Acquisition  triangulation  rendering – Direct PBR: Get rid of (non-trivial) triangulation – Plus: Increasing use in rendering/modelling Multiresolution, no connectivity/topology, much simpler,… But – Why point-based ray tracing ? – Actually, two different questions: Q1: Why ray tracing for point-based models ? Q2: Why point-based approach to ray tracing ?

56. 56 Interactive Point Based Ray Tracing June 21, 2005 Outline Motivation – Why point based ray tracing Our approach – Thin rays vs thick rays – Fast traversal and intersection Results Summary and Future Work

57. 57 Interactive Point Based Ray Tracing June 21, 2005 How to ray trace PB models ? Alternative 1: Growing rays – Many un-answered questions Where exactly is the hit point ? How sensitive is the procedure to exact form of beam ? – E.g., what if the beam were traced a little bit further ? Hitpoint position ? Position A points gathered by beam Same surface ?