1. High-Quality Pre-Integrated Volume Rendering Using Hardware Accelerated Pixel Shading Klaus Engel, Martin Kraus, Thomas Ertl Visualization and Interactive Systems Group University of Stuttgart, Germany Siggraph/Eurographics Workshop on Graphics Hardware 2001

2. Visualization and Interactive Systems Group,University of Stuttgart Motivation Problems of texture-based volume graphics on PCs: –scientific visualization limited quality 3D textures not commonly available pre-classification shading –computer games high rasterization requirements => low performance high texture memory requirements

3. Visualization and Interactive Systems Group,University of Stuttgart Texture-based Volume Rendering 2D textures (axis-aligned slices) 3D textures (view-aligned Slices) texturing (trilinear interpolation) texturing (trilinear interpolation) compositing (blending) compositing (blending) texturing (bilinear interpolation) texturing (bilinear interpolation) compositing (blending) compositing (blending)

4. Visualization and Interactive Systems Group,University of Stuttgart Volume Rendering Quality Improvement Higher sampling rates: additional tri-linearly interpolated slices –3D textures: render additional slice polygons –2D textures: render intermediate slices using multi-texture approach (Rezk-Salama,Engel et al., GH2000) But: –decreases rendering speed due to additional rasterization –many additional slices required even for low-resolution volumes –non-linear transfer functions: not sufficient to sample volume with the Nyquist frequency of scalar data

5. Visualization and Interactive Systems Group,University of Stuttgart Volume Classification voxels Post- classification interpolation Pre- classification transfer functions classification

6. Visualization and Interactive Systems Group,University of Stuttgart Pre-Integrated Volume Rendering Our approach: –texture-based (2D/3D) –ray-segments computed in a pre-processing step –pre-computed ray-segment lookup (dependent texture) –small number of slices => fast Especially suited for: –low resolution volume data –non-linear transfer functions Capable of: –high-quality rendering –direct volume rendering, volume shading, isosurfaces

7. Visualization and Interactive Systems Group,University of Stuttgart Pre-integrated Volume Rendering - Idea slice-by-slice slab-by-slab sbsb sfsf sfsf sbsb fetch integral from dependent texture sbsb sfsf pre-integrate all possible combinations hardware-accelerated implementation on NVidia GeForce3 chip sfsf sbsb front slice back slice project slice texture polygon

8. Visualization and Interactive Systems Group,University of Stuttgart Texture Shaders Texture Shaders (when enabled) replace the standard OpenGL texture fetch mechanism With texture shaders, results from previous stages can affect the lookup of a subsequent stage ARB_multitexture with NV_texture_shader ARB_multitexture only (s,t,r,q) 0 (s,t,r,q) 1 (s,t,r,q) 2 (s,t,r,q) 0 (s,t,r,q) 1 (s,t,r,q) 2 stage 0 math & fetch stage 1 math & fetch stage 2 math & fetch unit 0 fetch unit 1 fetch unit 2 fetch

9. Visualization and Interactive Systems Group,University of Stuttgart Pre-integrated Volume Rendering - Texel Fetch RGB 1 (s 2,t 2,r 2 )  (1,0,0) stage 2 DOT_PRODUCT_NV (1,0,0) RGB 1 =s b stage 2 DOT_PRODUCT_NV (1,0,0) RGB 1 =s b RGB 0 (s 3,t 3,r 3 )  (1,0,0) stage 3 DOT_PRODUCT_ TEXTURE2D_NV (1,0,0) RGB 0 =s f stage 3 DOT_PRODUCT_ TEXTURE2D_NV (1,0,0) RGB 0 =s f RGBA result on to register combiners front slice front slice back slice back slice sbsb sbsb stage 1 TEXTURE_2D stage 1 TEXTURE_2D (s 1,t 1 ) RGBA result sfsf sfsf stage 0 TEXTURE_2D stage 0 TEXTURE_2D (s 0,t 0 ) RGBA result sbsb sfsf

10. Visualization and Interactive Systems Group,University of Stuttgart Isosurfaces Isosurfaces: particular dependent texture 3296 sfsf sbsb 32 96 front slice back slice 1. 1 front slice back slice 4. 4 front slice back slice 2. 2 front slice back slice 3. 3 3a

11. Visualization and Interactive Systems Group,University of Stuttgart Isosurfaces – Gradient Interpolation g = IP g b + (1-IP) g f interpolation weight: IP = (s iso – s f )/(s b – s f ) store gradient with scalar data in RGBA texture store IP in dependent texture implement interpolation in reg. combiners sbsb sbsb front slice front slice back slice back slice gfgf gbgb g

12. Visualization and Interactive Systems Group,University of Stuttgart Pre-integrated Volume Rendering – Register Combiners Combiner 0+1: rebuild gradients (front + back) g y g z g x RBGARBGA Combiner 2: interpolate gradients g x g y g z I = I a + I d ( n. l ) + I s ( n. h ) p Combiner 3-7: shading (isosurface or volume) p max = 256 IP = (s iso – s f )/(s b – s f ) g = IP g b + (1-IP) g f s s

13. Visualization and Interactive Systems Group,University of Stuttgart Interactive Transfer Function Update 8 bit data: 256 x 256 = 65536 ray segments => ~20 seconds on Athlon 650 (for constant ray segment length) 3 optimizations: –assume constant ray segment length –local modification of TF requires local update of DT –neglect self-attenuation within ray-segments and use integral functions (as in Max et al., VolVis’90)

14. Visualization and Interactive Systems Group,University of Stuttgart Results – Direct Volume Rendering 128 slices pre- classification 128 slices pre-integrated 284 slices post- classification 128 slices post- classification

15. Visualization and Interactive Systems Group,University of Stuttgart Results – Direct Volume Rendering, Random TF

16. Visualization and Interactive Systems Group,University of Stuttgart Results - Isosurfaces

17. Visualization and Interactive Systems Group,University of Stuttgart Results Performance PC Nvidia GeForce3 64 MB DDR SDRAM (3.8 ns), 200 MHz internal clock, 460 MHz memory clock, 512 x 512 viewport 25 % slower than approach that used 1 or 2 texture fetches => loopback in texture_shaders for 3 rd and 4 th fetch (2 clock cycles) Interactive transfer function update when neglecting self-attenuation

18. Visualization and Interactive Systems Group,University of Stuttgart Problems artifacts in isosurfaces unequal ray segment lengths only linear interpolation for each ray segment artifacts on volume boundaries clipping planes front slice front slice back slice back slice

19. Visualization and Interactive Systems Group,University of Stuttgart Conclusions Pre-Integrated Volume Rendering –High quality even for small volumes with small number of slices with non-linear transfer functions reproduces high frequencies in the transfer function –Low rasterization requirements –Semi-transparent Volume Rendering –Volume Shading –Isosurfaces arbitrary number independently colored per-pixel lighting

20. Visualization and Interactive Systems Group,University of Stuttgart Demo Demo and Data Download: http://wwwvis.informatik.uni-stuttgart.de/~engel/pre-integrated/

21. Visualization and Interactive Systems Group,University of Stuttgart Appendix: Volume Shading and Isosurfaces Per-pixel shading requires per-voxel gradient or dot product –dynamic lighting –static lighting (fixed light sources) voxel gradient x y z voxel intensity RBGA dot productvoxel intensity LA

22. Visualization and Interactive Systems Group,University of Stuttgart Appendix: Gradient Rebuild dot product affects only the RGB components  voxel intensity stored in R,  gradient in GBA voxel gradient y z x voxel intensity RBGA Reroute x-component of gradient to R channel in the register combiners for lighting

23. Visualization and Interactive Systems Group,University of Stuttgart Appendix: Register Combiners

24. Visualization and Interactive Systems Group,University of Stuttgart Appendix: Register Combiners input registers RGB A const. color1 G h l spare 1 IdId IaIa prim. color sec. color IsIs const. color0 const. 0 lighting A B C D A BA B dot product A B C D AB A B C D GC3 GC5 GC6 I = I a + I d ( n. l ) + I s ( n. h ) p GC7 A B C D AB+CD sum I d ( n. l ) + I s ( n. h ) p I d ( n. l ) + I s ( n. h ) p C DC D A B C D AB GC4 FC sum A 1 0 D AB+ (1-A)C +D

25. Visualization and Interactive Systems Group,University of Stuttgart Appendix: Dependent Texture Shaders DEPENDENT_AR_-TEXTURE_2D_NV: The previous texture shader’s alpha and red colors are used as the (s,t) for a 2D texture lookup not suitable for us, because we need (s,t) from two different textures (s 0,t 0 ) RGBA result on to texture environment or register combiners Texture coordinates unused … RGBA stage 0 TEXTURE_2D GL_RGBA8 internal format stage 0 TEXTURE_2D GL_RGBA8 internal format stage 1 DEPENDENT_AR_- TEXTURE_2D_NV nonprojective_lookup_2D( A, R ) stage 1 DEPENDENT_AR_- TEXTURE_2D_NV nonprojective_lookup_2D( A, R )

26. Visualization and Interactive Systems Group,University of Stuttgart Appendix: Ray-segment Integration

27. Visualization and Interactive Systems Group,University of Stuttgart Appendix: Results - Quality Comparison: –pre-classification –post-classification –pre-integrated classification pre-integrated dependent texture post-classification dependent texture for tri-linear ip. intermediate slices sfsf sbsb asbasb (1- a )s f

28. Visualization and Interactive Systems Group,University of Stuttgart Appendix: Performance Results volume size16 2 32 2 64 2 128 2 256 2 direct905025134 isosurface704021111 TF update (seconds)globallocal self-attenuation20< 10 w/o self-attenuation0.3< 0.15