1 Interactive Point Based Ray Tracing June 21, 2005 Interactive Ray Tracing of Point-based Models Ingo Wald SCI Institute, University of Utah

2 Interactive Point Based Ray Tracing June 21, 2005 Motivation Why point-based rendering ? – Many advantages, but no details here … But: Why point-based ray tracing ? – Actually, two different questions: Q1: Why ray tracing for point-based models ? Q2: Why use point-based techniques in ray tracing ?

3 Interactive Point Based Ray Tracing June 21, 2005 Q1: Why Ray Tracing ? Reason 1: Logarithmic scalability to large models – Very handy for today‘s huge models – But: For PBR not that important (multiresolution)… Reason 2 : Image quality / shading quality – Ray traced image quality not yet standard in PBG… But would be nice to have

4 Interactive Point Based Ray Tracing June 21, 2005 Q2: Why Point-based techniques in Ray Tracing? Native point-based data (acquisition) – Direct ray tracing avoids tesselation Much simpler And for native triangular models ? – Triangular model  Point-based model  PBRT Does this make sense ? – Highly controversial issue amongst ray tracing researchers RT is log. in #tris, anyway. Don‘t need multiresolution… Many new problems with points as primitives – Consistency, efficiency, loss of information…

5 Interactive Point Based Ray Tracing June 21, 2005 Q2: Why Point-based techniques in Ray Tracing? Interactive ray tracing: Three recent examples

6 Interactive Point Based Ray Tracing June 21, 2005 Q2: Why Point-based techniques in Ray Tracing? Interactive ray tracing: Three recent examples – Example 1: Boeing 777 (350 million triangles, 40 GB data)

7 Interactive Point Based Ray Tracing June 21, 2005 Q2: Why Point-based techniques in Ray Tracing? Interactive ray tracing: Three recent examples – Example 2: Lawrence Livermore isosurface (8 billion voxels)

8 Interactive Point Based Ray Tracing June 21, 2005 Q2: Why Point-based techniques in Ray Tracing? Interactive ray tracing: Three recent examples – Example 3: Forest landscape (1.5 billion triangles)

9 Interactive Point Based Ray Tracing June 21, 2005 Q2: Why Point-based techniques in Ray Tracing? Interactive ray tracing: Three recent examples – Yes, can render all three already today without PB techniques – But: Doesn‘t make sense (>1G prim. for <1M pixels ?) Too much data: Tiny camera move  totally different data – Even though image doesn‘t change at all… Sample only every 1000 th pixel  Severe aliasing – Aliasing: THE most important problem of RTRT today  Need multiresolution approach also for RT – Probably best via PB approach Current results only intermediate step… … towards long-term goal of multiresolution (PB)RT

10 Interactive Point Based Ray Tracing June 21, 2005 How to ray trace PB models ? Problem: Zero ray-point hit probability – Rays infinitely thin – Points infitnitely thin, too Two possible solutions – Use „thick“ rays that have volume (cones, beams, …) Reconstruct from all points overlapping ray‘s volume – „Grow“ points to have a surface can then be found by „thin“ ray – Long history of debate on which is better…

11 Interactive Point Based Ray Tracing June 21, 2005 How to ray trace PB models ? Alternative 1: Growing rays – Experience: Tracing beams very costly (much more than rays)

12 Interactive Point Based Ray Tracing June 21, 2005 How to ray trace PB models ? Alternative 1: Growing rays – Many un-answered questions Where exactly is the hit point ? – E.g., where to start the shadow ray from ? Hitpoint position ? Position A points gathered by beam

13 Interactive Point Based Ray Tracing June 21, 2005 How to ray trace PB models ? Alternative 1: Growing rays – Many un-answered questions Where exactly is the hit point ? How stable is the decision ? Hitpoint position ? Position A points gathered by beam

14 Interactive Point Based Ray Tracing June 21, 2005 How to ray trace PB models ? Alternative 1: Growing rays – Many un-answered questions Where exactly is the hit point ? How stable is the decision ? Is it consistent ? – E.g., when viewed from different angle (shadow ray) Hitpoint position ? Same surface, different viewpoint: Same hitpoint ? Is it in shadow ? surface seen from A Position A Position B (Light ?)

15 Interactive Point Based Ray Tracing June 21, 2005 How to ray trace PB models ? Alternative 1: Growing rays – Many un-answered questions Where exactly is the hit point ? How stable is the decision? Is it consistent ?  Too many open questions

16 Interactive Point Based Ray Tracing June 21, 2005 How to ray trace PB models ? Alternative 2: Growing points – Define unique surface through points Always consistent – Most trivial solution: Use disks/ellipses/… Trivial

17 Interactive Point Based Ray Tracing June 21, 2005 How to ray trace PB models ? Alternative 2: Growing points – Define unique surface through points Always consistent – Most trivial solution: Use disks/ellipses/… Trivial, but shading discontinuities (worse w/ shadows)

18 Interactive Point Based Ray Tracing June 21, 2005 How to ray trace PB models ? Better: Define smooth surface – Use Adamson/Alexa‘s model Each point x i has normal n i and radius r i of influence Influence w i falls off with distance to x i – With w i (x i )=1 and w i (x)=0 for all |x-x i |>r i Define weighted average position X(x) and N(x) Define „distance“ function F(x) := ((x-X(x)).N(x)  Implicit fct F(x)=0 is smooth surface – Need: Useful r i ‘s (not given) Fast ray/surface intersection framework

19 Interactive Point Based Ray Tracing June 21, 2005 Choosing Splat Radii Step 1: Choosing the splat radii – Too small  Holes in the model – Too large  Many regions of influence will overlap (slow) Use Wu et al.‘s optimal subsampling technique – Wu [EG04]: Determine r i such as to Optimally cover surface Achieve minimal overlap  Optimal solution: Use exactly that technique Note: Can also use ‚other‘ radii – e.g., user-specified… – Might just be slower or contain holes

20 Interactive Point Based Ray Tracing June 21, 2005 Intersecting the Surface Step 2: Determining the splats to intersect with – Each point x always influenced by only few x i A B C D

21 Interactive Point Based Ray Tracing June 21, 2005 Intersecting the Surface Step 2: Determining the splats to intersect with Enclose each splat‘s ROI with tight fitting box A B C D

22 Interactive Point Based Ray Tracing June 21, 2005 Intersecting the Surface Step 2: Determining the splats to intersect with Enclose each splat‘s ROI with tight fitting box Build kd-tree over those AA boxes A B C D

23 Interactive Point Based Ray Tracing June 21, 2005 Intersecting the Surface Step 2: Determining the splats to intersect with Enclose each splat‘s ROI with tight fitting box Build kd-tree over those AA boxes A B C D

24 Interactive Point Based Ray Tracing June 21, 2005 Intersecting the Surface Step 2: Determining the splats to intersect with Enclose each splat‘s ROI with tight fitting box Build kd-tree over those AA boxes A B C D

25 Interactive Point Based Ray Tracing June 21, 2005 Intersecting the Surface Step 2: Determining the splats to intersect with Enclose each splat‘s ROI with tight fitting box Build kd-tree over those AA boxes A B C D

26 Interactive Point Based Ray Tracing June 21, 2005 Intersecting the Surface Step 2: Determining the splats to intersect with Enclose each splat‘s ROI with tight fitting box Build kd-tree over those AA boxes Leaves: Store list of splats whose ROI overlaps cell A B C D A,B A AA,B,C B C B,C (right side incomplete)

27 Interactive Point Based Ray Tracing June 21, 2005 Intersecting the Surface Step 2: Determining the splats to intersect with Enclose each splat‘s ROI with tight fitting box Build kd-tree over those AA boxes Leaves: Store list of splats whose ROI overlaps cell – KD-Tree: Inherit advantages of fast triangular ray tracing Fast traversal, occlusion culling, good kd-trees (SAH)… A B C D A,B A AA,B,C B C B,C

28 Interactive Point Based Ray Tracing June 21, 2005 Intersecting the Surface Step 3: Ray/surface intersection in each voxel – Find distance t : F(t)=0 – Equidistant sampling along t Have intersection if sgn(F(t i+1 )) != sign(F(t i )) But: Too coarse sampling can miss the surface – Optimizations: Fast SSE implementation (no details) Quite fast, but: Still main cost factor  Further optimize kd-tree to minimize F(x) calls – Need to take closer look at kd-tree

29 Interactive Point Based Ray Tracing June 21, 2005 KD-Tree Post-optimizations Observation 1: Many full cells don‘t contain surf. – Reason: KD-tree was built over ROI‘s ROI‘s bound surface, but not closely

30 Interactive Point Based Ray Tracing June 21, 2005 KD-Tree Post-optimizations Observation 1: Many full cells don‘t contain surf. – Example from before A B C A,B A AA,B,C B C B,C -- - A,B,C A,B,C

31 Interactive Point Based Ray Tracing June 21, 2005 KD-Tree Post-optimizations Observation 1: Many full cells don‘t contain surf. – Actual surface only touches few voxels A B C A,B A AA,B,C B C B,C -- -A,B,C A,B,C A,B,C

32 Interactive Point Based Ray Tracing June 21, 2005 KD-Tree Post-optimizations Observation 1: Many full cells don‘t contain surf. – Many non-empty cells that are not necessary A B C A,B A AA,B,C B C B,C -- -A,B,C A,B,C A,B,C

33 Interactive Point Based Ray Tracing June 21, 2005 KD-Tree Post-optimizations Observation 1: Many full cells don‘t contain surf. – Many non-empty cells that are not necessary Can make quite a difference A B C A,B A AA,B,C B C B,C -- -A,B,C A,B,C A,B,C

34 Interactive Point Based Ray Tracing June 21, 2005 KD-Tree Post-optimizations Observation 1: Many full cells don‘t contain surf. Fix: Determine min(F()) and max(F()) in each cell  Cull cells with min > 0 or max < 0 – Currently: Sample F with random x  Can produce false culling (holes) – Better: Interval arithmetic (Not implemented yet)

35 Interactive Point Based Ray Tracing June 21, 2005 KD-Tree Post-optimizations Observation 2: Most cells too large – Reason: Only split planes at ROI borders – Example from earlier-on: A B C A,B A AA,B,C B C B,C -- -

36 Interactive Point Based Ray Tracing June 21, 2005 KD-Tree Post-optimizations Observation 2: Most cells too large – Reason: Only split planes at ROI borders – Example from earlier-on: Green regions would suffice… A B C A,B A AA,B,C B C B,C -- - A

37 Interactive Point Based Ray Tracing June 21, 2005 KD-Tree Post-optimizations Observation 2: Most cells too large – Reason: Only split planes at ROI borders – Example from earlier-on: Green regions would suffice… Fix: Post-shrinking of cells – Cut voxel into slices, cull slices Same problem as before: False culling… Important: Not only fewer cell intersections… – Also: Cells much thinner: Finer sampling, fewer artifacts

38 Interactive Point Based Ray Tracing June 21, 2005 KD-Tree Post-optimizations Observation 3: After shrink/cull, many cells empty – Example from earlier-on: Most cells empty after cull/shrink A,B A AA,B,C B C B,C -- -

39 Interactive Point Based Ray Tracing June 21, 2005 KD-Tree Post-optimizations Observation 3: After shrink/cull, many cells empty – Example from earlier-on: Most cells empty after cull/shrink C A,B B,C A

40 Interactive Point Based Ray Tracing June 21, 2005 KD-Tree Post-optimizations Observation 3: After shrink/cull, many cells empty – Example from earlier-on: Most cells empty after cull/shrink – Optimization: Post-collapse Undo splits that don‘t make sense any more C A,B B,C A

41 Interactive Point Based Ray Tracing June 21, 2005 Results Overall Framerate:1Dual-Opteron 2.4GHz, 512x512 ModelSizeDiffusePhong w/Shadows Iphigenia Head1M6.8Not measured Iphigenia Full1M15.9Not measured Octopus465K Dragon1.3M David1.5M

42 Interactive Point Based Ray Tracing June 21, 2005 Results Overall Framerate: ~5-15 1PC Nice scalability to very complex models – „ManyIffis“: 24M points w/ shadows 

43 Interactive Point Based Ray Tracing June 21, 2005 Results Overall Framerate: ~5-15 1PC Nice scalability to very complex models High-quality shading – Global Illumination

44 Interactive Point Based Ray Tracing June 21, 2005 Results Overall Framerate: ~5-15 1PC Nice scalability to very complex models High-quality shading Problems and limitations: – Sometimes small holes (false culling, undersampling, …) – All is static: Can‘t move points, can‘t change radii – Assumes continuous surface: Not applicable to random point cloud (forest leaves…)

45 Interactive Point Based Ray Tracing June 21, 2005 Summary & Future Work Summary – Motivated/Argued for Point-based Ray Tracing – Discussed „thin rays“ vs „thick rays“ issue – Outlined framework for Interactive PBRT Future work – Performance & Stability Bounding the surface, interval arithmetic, … – Multiresolution PBRT In particular, for massively complex models

46 Interactive Point Based Ray Tracing June 21, 2005 Questions ?

47 Interactive Point Based Ray Tracing June 21, 2005 Questions ?

48 Interactive Point Based Ray Tracing June 21, 2005 Q1: Why Ray Tracing ? Impact of Shadows and Reflections: Example

49 Interactive Point Based Ray Tracing June 21, 2005 Q2: Why Point-based techniques in Ray Tracing? Need multiresolution approach also for RT – Trivial for „nice“ (manifold) meshes (Progressive meshes,…) – But: „problematic“ for real-world models (Forest, Boeing, …) Should be much easier for point-based representation  Eventual goal of (our) PBRT activities – Not ray tracing (nice) point-based models – Rather: Multiresolution ray tracing using a PB approach Reduce (in-core) data Fight aliasing – Current activities only first step in that direction…

50 Interactive Point Based Ray Tracing June 21, 2005 Adamson&Alexa‘s model Observations – Need position and normal for each point  assume as given – Need radius of influence for each point – For each x, most w i (x) will be zero Fast computation: Need to know which... – Evaluating F(x) will be costly Need to minimize F(x) calls Need to quickly determine where surface can NOT be Need to minimize number of points contributing to x  Need to minimize regions of influence (minimize r i !)

51 Interactive Point Based Ray Tracing June 21, 2005 So: Why Interactive Point- based Ray Tracing ? Advantages of PBG in particular for ray tracers: Native point-based data (acquisition) – Avoid tesselation … sure, makes sense. And for native triangular models ? – Triangular model  Point-based model  PBRT Does this make sense ?

52 Interactive Point Based Ray Tracing June 21, 2005 So: Why Interactive Point- based Ray Tracing ? Advantages of PBG in particular for ray tracers:  Highly controversial issue amongst ray tracers – Ray tracing is logarithmic in scene size, anyway Don‘t need point-based multiresolution capabilities – Triangles very well suited for fast ray tracing How to (efficiently) intersect points ? – Triangles: well-defined surface How to consistently intersect points ? In particular for secondary rays – shadow ray has to see exactly the same surface Sampling: Loss of information ?  Many new problems, PBG advantages over rast. don‘t apply…  Where are the advantages ?

53 Interactive Point Based Ray Tracing June 21, 2005 Q1: Why Ray Tracing Impact of Shadows and Reflections: Example

54 Interactive Point Based Ray Tracing June 21, 2005 Motivation: Why Point-based Rendering ? Point-based models become commonplace – Acquisition: Almost all models are point clouds Direct PBR simplifies 3D processing pipeline – Today: Acquisition  triangulation  rendering – Direct PBR: Get rid of (non-trivial) triangulation – Plus: Increasing use in rendering/modelling Multiresolution, no connectivity/topology, much simpler,…

55 Interactive Point Based Ray Tracing June 21, 2005 Motivation: Why Point-based Rendering ? Point-based models become commonplace – Acquisition: Almost all models are point clouds Direct PBR simplifies 3D processing pipeline – Today: Acquisition  triangulation  rendering – Direct PBR: Get rid of (non-trivial) triangulation – Plus: Increasing use in rendering/modelling Multiresolution, no connectivity/topology, much simpler,… But – Why point-based ray tracing ? – Actually, two different questions: Q1: Why ray tracing for point-based models ? Q2: Why point-based approach to ray tracing ?

56 Interactive Point Based Ray Tracing June 21, 2005 Outline Motivation – Why point based ray tracing Our approach – Thin rays vs thick rays – Fast traversal and intersection Results Summary and Future Work

57 Interactive Point Based Ray Tracing June 21, 2005 How to ray trace PB models ? Alternative 1: Growing rays – Many un-answered questions Where exactly is the hit point ? How sensitive is the procedure to exact form of beam ? – E.g., what if the beam were traced a little bit further ? Hitpoint position ? Position A points gathered by beam Same surface ?