1. Splatting Jian Huang, CS 594, Spring 2002 This set of slides reference slides made by Ohio State University alumuni over the past several years.

2. Volumetric Ray Integration color opacity object (color, opacity) 1.0

3. Splatting Lee Westover - Vis 1989; SIGGRAPH 1990 Object order method Front-To-Back or Back-To-Front Original method - fast, poor quality Many many improvements since then! –Crawfis’93: textured splats –Swan’96, Mueller’97: anti-aliasing –Mueller’98: image-aligned sheet-based splatting –Mueller’99: post-classified splatting –Huang’00: new splat primitive: FastSplats

4. Splatting Volume = field of 3D interpolation kernel –One kernel at each grid voxel Each kernel leaves a 2D footprint on screen –Voxel contribution = footprint ·(C, opacity) Weighted footprints accumulate into image voxel kernels screen footprints = splats screen

5. Splatting Volume = field of 3D interpolation kernel –One kernel at each grid voxel Each kernel leaves a 2D footprint on screen –Voxel contribution = footprint ·(C, opacity) Weighted footprints accumulate into image voxel kernels screen footprints = splats screen

6. Splatting Volume = field of 3D interpolation kernel –One kernel at each grid voxel Each kernel leaves a 2D footprint on screen –Voxel contribution = footprint ·(C, opacity) Weighted footprints accumulate into image voxel kernels screen footprints = splats screen

7. Splatting Volume = field of 3D interpolation kernel –One kernel at each grid voxel Each kernel leaves a 2D footprint on screen –Voxel contribution = footprint ·(C, opacity) Weighted footprints accumulate into image voxel kernels screen footprints = splats screen

8. Ray-casting - revisited color c = c s  s (1 -  ) + c opacity  =  s (1 -  ) +  1.0 object (color, opacity) volumetric compositing Interpolation kernel

9. Ray-casting - revisited (ideally) we would reconstruct the continuous volume (cloud) using the interpolation kernel h: the we would compute the analytic integral along a ray r: this can only be approximated by discretization (hey! Which optical model is this equation??)

10. Splatting - principal idea This last equation can be rewritten in the following way: Splatting Kernel or “Splat” analytically 4 Which can be computed analytically: known as footprint

11. Footprint Extent Approximate the 3D kernel (h(x,y,z))extent by a sphere

12. Footprint Table A popular kernel is a three-dimensional Gaussian (radially symmetric) As 1D integration of 3D Gaussian is still a 2D Gaussian – we can just skip the Z integration and evaluate the Gaussian function on 2D image space after voxel projection Generic footprint table preprocessing

13. View-dependent footprint It is possible to transform a sphere kernel into A ellipsoid The projection of an ellipsoid is an ellipse We need to transform the generic footprint table to the ellipse

14. View-dependent footprint (2)

15. Example Footprint at Different Resolutions

16. Footprint - principal idea Draw each voxel as a cloud of points (footprint) that spreads the voxel contribution across multiple pixels. Larger footprint -> larger spatial kernel extent -> lower frequency components -> more blurring –Large pixel/voxel ratio

17. Rendering a Splat Use texture mapping hardware to resample footprint table (either single density channel or separate classified r,g,b,a channels) Or, use FastSplats to render each splat as a graphics primitive of itself

18. Splatting - efficiency “footprint” - splatted (integrated) kernel if interpolation kernel is isotropic (spherical) then its footprint is independent of the view point (for orthographic viewing) for perspective - footprint can be approximated with an ellipse Hence, for common cases, we can pre-integrate it (efficient!) for perspective projection, to approximate, we have to compute the orientation of the ellipse

19. Splatting - Highlights Footprints can be pre-integrated –fast voxel projection Advantages over ray-casting: –Fast: voxel interpolation is in 2D on screen –More accurate integration (analytic for X-ray) –More accurate reconstruction (afford better kernels) –Only relevant voxels must be projected

20. Early Implementation – Axis Aligned Splatting Voxel kernels are added within axis-aligned sheets Sheets are composited front-to-back Sheets = volume slices most perpendicular to the image plane image plane at 70°image plane at 30° volume slices x y z

21. Early Implementation – Axis Aligned Splatting sheet buffer compositing buffer volume slices image plane Volume

22. Early Implementation – Axis Aligned Splatting sheet buffer compositing buffer volume slices image plane Add voxel kernels within first sheet

23. Early Implementation – Axis Aligned Splatting sheet buffer compositing buffer volume slices image plane Transfer to compositing buffer

24. Early Implementation – Axis Aligned Splatting sheet buffer compositing buffer volume slices image plane Add voxel kernels within second sheet

25. Early Implementation – Axis Aligned Splatting sheet buffer compositing buffer volume slices image plane Composite sheet with compositing buffer

26. Early Implementation – Axis Aligned Splatting sheet buffer compositing buffer volume slices image plane Add voxel kernels within third sheet

27. Early Implementation – Axis Aligned Splatting sheet buffer compositing buffer volume slices image plane Composite sheet with compositing buffer

28. What Doesn’t Work? Mathematically, the early splatting methods only work for X- ray type of rendering, where voxel ordering is not important –Bad approximation for other types of optical models Object ordering is important in volume rendering, front objects hide back objects –need to composite splats in proper order, else we get bleeding of background objects into the image (color bleeding!) Axis- aligned approach add all splats that fall within a volume slice most parallel to the image plane, composite these sheets in front- to- back order –Incorrect accumulating on axis-aligned face cause popping A better approximation with Riemann sum is to use the image- aligned sheet-based approach

29. Problems Early Implementation – Axis Aligned Splatting In-accurate compositing, result in color bleeding and popping artifacts (Demo)! Part of this voxel gets composited before part of this voxel

30. Image-Aligned Sheet-Buffer sheet buffer compositing buffer Slicing slab cuts kernels into sections Kernel sections are added into sheet-buffer Sheet-buffers are composited image plane

31. Image-Aligned Sheet-Buffer sheet buffer compositing buffer Slicing slab cuts kernels into sections Kernel sections are added into sheet-buffer Sheet-buffers are composited image plane

32. Image-Aligned Sheet-Buffer sheet buffer compositing buffer Slicing slab cuts kernels into sections Kernel sections are added into sheet-buffer Sheet-buffers are composited image plane

33. Image-Aligned Sheet-Buffer sheet buffer compositing buffer Slicing slab cuts kernels into sections Kernel sections are added into sheet-buffer Sheet-buffers are composited image plane

34. Image-Aligned Sheet-Buffer sheet buffer compositing buffer Slicing slab cuts kernels into sections Kernel sections are added into sheet-buffer Sheet-buffers are composited image plane

35. Image-Aligned Sheet-Buffer sheet buffer compositing buffer Slicing slab cuts kernels into sections Kernel sections are added into sheet-buffer Sheet-buffers are composited image plane

36. Image-Aligned Sheet-Buffer sheet buffer compositing buffer Slicing slab cuts kernels into sections Kernel sections are added into sheet-buffer Sheet-buffers are composited image plane

37. Image-Aligned Splatting Note: We need an array of footprint tables now. A separate footprint table for each slice of the 3D reconstruction kernel.

38. Volume Rendering Pipeline: Two Variations

40. IASB Splatting No popping or color bleeding Sharp, noise-free images

41. Occlusion Culling A voxel is only visible if the volume material in front is not opaque occluded voxel: does not pass visibility test wall of occluding voxels occlusion map = opacity image screen

42. Visibility Test Based on SAT of Occlusion Buffer opacity  threshold occlusion map Do not project Project opacity < threshold opacity = 0 Compute occlusion map after each sheet Cull both individual voxel and voxel sets with a summed area table of occlusion map

43. Occlusion Culling Build a summed area table (SAT) from the opacity buffer To test whether a rectangular region is opaque or not, check the four corners Can cull voxel sets directly

44. Anti-aliasing Needed to preserve small features Needed for the diverging rays in perspective In splatting, resize the footprint according to depth Aliasedanti-aliased

45. Motion Blur Stretch the reconstruction kernel in the direction of movement. Stretch the splat footprint in the direction of the projected movement (2D).

46. Camera Depth-of-Field Two possible approaches: –Low-pass filter the splats –Low-pass filter the sheets Plainwith Depth-of-Field

47. Procedural Textures Easily done with pre-coloring Per-pixel

48. Bump-Mapping If calculating the normal per-pixel, we can modulate it to achieve bump mapping.

49. Per-pixel Classification Per-pixel classification can be based on gray scale, position, gradient, or... 7.25 sec 9.41 sec (procedural) 7.99 sec

50. Scan-Converting A Splat Scan-convert an arbitrary-size radially symmetric 2D function centered at arbitrary position –circular or elliptical Texture mapping hardware is not the solution We want a hardware accelerated splat or point primitive

51. Fast Splats FastSplat We desire – fast scan conversion – minimum or controllable errors – compact storage – simple integer operation

52. FastSplats  1D Linear  1D Squared  2D  1D with Radius Look Up Table (RLUT)

53. 1D Linear, 1D Squared FastSplats On the radial line 1D Linear FastSplat, indexed by r x,y 1D Squared FastSplat, indexed by r 2 x,y (x,y) (x +1,y) r x,y (x o,y o )

54. 1D Squared FastSplat (Elliptical) For elliptical kernels, if we define a canonical radius: The incremental scan-conversion still works at the same low cost

55. FastSplats  1D Linear  1D Squared  2D 1D with Radius Look Up Table (RLUT)

56. 2D FastSplat (BitBlt,VoxBlt) No run-time computation Pre-rasterized footprint with center at (x o,y o ), radius r Pre-rasterized footprints for all possible center positions, radius r Snap splat center to a k by k sub- pixel grid

57. 2D FastSplats No run-time computation Pre-rasterized footprints for all possible center positions, for all possible radii Snap splat center to a k by k sub- pixel grid Allow for a set of radius values

58. 2D FastSplats (2) The storage need: When storage is limited, the quality is limited too [Mora’00], similar

59. FastSplats  1D Linear  1D Squared  2D 1D with Radius Look Up Table (RLUT)

60. 1D RLUT FastSplat For hardware, we need finer parallelism than scan-line

61. 1D FastSplat with RLUT RLUT

62. 1D FastSplat with RLUT At a k by k subpixel precision

63. 1D FastSplat with RLUT At a k by k subpixel precision

64. 1D FastSplat with RLUT At a k by k subpixel precision

65. 1D FastSplat with RLUT At a k by k subpixel precision

66. 1D FastSplat with RLUT At a k by k subpixel precision

67. 1D FastSplat with RLUT At a k by k subpixel precision

68. 1D FastSplat with RLUT At a k by k subpixel precision

69. 1D FastSplat with RLUT At a k by k subpixel precision

70. 1D FastSplat with RLUT At a k by k subpixel precision

71. 1D FastSplat with RLUT At a k by k subpixel precision

72. 1D FastSplat with RLUT At a k by k subpixel precision

73. 1D FastSplat with RLUT At a k by k subpixel precision

74. 1D FastSplat with RLUT At a k by k subpixel precision

75. 1D FastSplat with RLUT At a k by k subpixel precision

76. 1D FastSplat with RLUT At a k by k subpixel precision

77. 1D FastSplat with RLUT At a k by k subpixel precision

78. 1D FastSplat with RLUT k by k subpixel precision

79. 1D FastSplat with RLUT Due to symmetry, the RLUT set for x component is the same as the RLUT set for the y component one RLUT set –k 1D tables –each of splat_extent length x or y offset k splat_extent k k

80. Comparisons Among the FastSplats 1D Linear: very accurate, compact, slow 1D Squared: accurate, compact, fast 1D RLUT: accurate, compact, intended for hardware 2D FastSplat: can be very fast, accurate and compact under constrained conditions