High-Quality Volume Graphics on Consumer PC Hardware Klaus Engel Joe Kniss Markus Hadwiger Christof Rezk-Salama

a f RGB Map data value f to color and opacity Shading, Compositing… Human Tooth CT a(f) RGB(f) f Shading, Compositing…

Optical Properties Color (RGB) Emissive term ~ =

Optical Properties ~ Alpha (a.k.a. opacity or extinction) = = Attenuates light based on density ~ = =

Optical Properties ~ Traditional Volume Rendering Equation Emission & absorption ~

Optical Properties Scale alpha values based on sample rate (sr) 1.3 .7

Transfer Function Transform scalar data value into optical properties Use optical properties to solve integral Very easy to implement Data is a texture Transfer function is a lookup Compute Riemann sum using “over operator”

Transfer Function Pre-classification Post-classification Transfer function evaluated before interpolation, i.e. interpolation of colors not data Transfer function evaluated after interpolation, i.e. data is interpolated first

Implementation Two choices: Texture Lookup Table glColorTable* Varies depending on hardware, may require a special texture data format Fast Dependent Texture Read Very hardware dependent Becoming more general, see OpenGl 2.0 Slower but more flexible…

Transfer Function Problem: f(x,y,z) No shape or depth queues No shading. color alpha f(x,y,z)

Transfer Function Solution: Faux Shading f(x,y,z) Ramp color to black with alpha Silhouette edges color alpha f(x,y,z)

Transfer Function Better: Surface Shading Slower, requires normal More flexible

Transfer Function Better: Surface Shading Slower, requires normal More flexible Faux shading enhances

Transfer Function Problem: Can’t surface shade homogeneous regions Need good gradients Sensitive to noise

Transfer Function Solution: Surface scalar (s) Only shade high gradient magnitudes: Or, add s to TF Interpolate…

Transfer Function Problem: How did the light get there? No attenuation through volume Not realistic!

Transfer Function Solution: Shadows Better depth queues Dramatic effects

Shadows Image plane r1 r0 Eye Light

Shadows Sample ri (s) Image plane r1 r0 IL l0 Eye ~

Shadows Implementation: 2 passes Attenuate from the light source Render from the eye Store light attenuation in second volume Multiply color by attenuation from shadow volume

Shadows Disadvantages: Difficult to build shadow volume on the card Slow to build off the card Additional volume required Attenuation leakage Blurry shadow boundaries Low resolution shadows!

Shadows Alternative: Incremental shadows Generate shadows one slice at a time Only use a 2D buffer Image space shadow computation All on card Half angle slicing

Slicing from light’s point of view Shadows Eye Slicing from light’s point of view

Slicing from eye’s point of view Shadows Eye Slicing from eye’s point of view

Half angle slicing: good from either point of view Incremental Shadows Eye Half angle slicing: good from either point of view

Similar aspect ratio from both points of view Incremental Shadows Similar aspect ratio from both points of view

Incremental Shadows * Slice pass 1

Incremental Shadows Slice pass 2

Incremental Shadows Advantages: Disadvantages: Screen space shadows No leakage Use render to texture to optimize Shades perturbed volumes Simple implementation Disadvantages: Aliasing at sharp opacity changes Fix with slightly larger light buffer

Shadows Problem: Shadows still too dark Direct attenuation is inadequate Need to handle higher order light transport effects Shadows

Shadows Solution: Translucency One consequence of light scattering Smoke, clouds, skin, wax…. Shadows Translucent

Translucency Wax: Real Shadows

Translucency Wax: Real Shadows Translucent

Translucency Add indirect attenuation Direct attenuation, same as shadows Blurred indirect attenuation, includes an indirect alpha

Direct (Id) and indirect (Ii) attenuation Translucency Direct (Id) and indirect (Ii) attenuation

Translucency How? Same as shadows (two light buffers) Sample previous light buffer multiple times for blur Ping-pong blending Store indirect in a color component Sum direct and indirect in fragment shader for the eye pass Only use direct attn. for eye pass

Translucency Problem: Still doesn’t look right Real Shadows Translucent

Translucency Solution: add spectral attenuation Real Translucent w/spectral attn.

Translucency Spectral attenuation: Attenuate some colors more than others Spectral indirect attn. is simplest Need separate alpha for RGB Store in RGB components of light buffer

Translucency Indirect alpha vs. transport color Alpha: Transport: Transport color is easier to specify

Optical Properties Recap: Reflective color/Emission (RGB) Direct attenuation/alpha (A) Surface scalar (s) Indirect attenuation (Ar,Ag,Ab) Others? Scattering, absorption, phase function, density, emission, index of refraction…

Transfer function Specification Simple, easy Expressive Guided

Transfer function Typical: 1D linear ramps f(x,y,z) Independent R,G,B,A control Difficult Trial and error alpha f(x,y,z)

Transfer function Problem: f(x,y,z) RGB = bad color space for humans No concept of features alpha f(x,y,z)

Transfer function Better: set color at control points f(x,y,z) Use HSV or HLS color spaces Simplified interface Still no guidance alpha f(x,y,z)

Transfer function Guided techniques: Design Galleries Thumbnails Semi-Automatic Dual-domain interaction

Design Galleries Treat TF and rendering as a high dimensional parameter space Stochastically sample space Cluster images based on fitness Select best looking image

Design Galleries

Design Galleries Computationally expensive Difficult to implement Not guided by dataset specifics Only handles 1D transfer functions

Thumbnails Visual history of changes Show important regions of TF Show effects of potential changes

Thumbnails spreadsheets

Thumbnails Parameterization

Semi-automatic Volumetric edge detection Use first and second derivatives Build histogram volumes Use simplified interface Generate 1D or 2D TFs

Semi-automatic Position of boundary center

Semi-automatic

Semi-automatic Identifies data ranges of boundaries Still requires hand-editing Not really an interactive process Demonstrates the value of histograms

Transfer function What is a 2D transfer function Multiple values per sample point 2D lookup table Each value is an axis of TF Better specificity More complicated to use

Transfer function RGB( ) Generalize… ( )

Transfer function ( , ) RGB( , ) Modify…

Transfer function ( , ) RGB( , ) Modify…

- + Transfer function RGB ( , , ) Second directional derivative measured with Hessian -

Transfer function + RGB ( , , ) Done -

Transfer function Multi-dimensional TFs Data value, gradient magnitude, second derivative Multivariate, i.e. multiple values Implement as dependent texture read, 1D, 2D, 3D texture for TF

Transfer function Multiple scans Color data Multiple variables MRI-T1,T2,PD Cryosection Simulation

Transfer function Problem: How do we interact with a higher dimensional TF? Larger parameter space Unintuitive feature identification Greater demands on user interface

Transfer function Solution: It is a process!! Data set guidance Improved classification Intuitive feature identification

1 “Default” transfer function Initial

1 Probing Initial 2 Explore

1 Initial 2 Explore 3 Specify Transfer function widget Manual Dual-domain interaction Initial 2 Explore 3 Specify

1 Initial 2 Explore 4 Refine 3 Specify Use widgets to tune the transfer function Initial 2 Explore 4 Refine 3 Specify

Interactive 1 Initial 2 Explore 4 Refine 3 Specify Understanding from immediate feedback Initial 2 Explore 4 Refine 3 Specify Interactive

Classification Discrete features Basic functions Direct manipulation widgets V3 slider(s) V2 V1

Guidance Histograms Blue Red Green

Guidance Probing Identify features in transfer function Chapel Hill CT

Guidance Dual-domain interaction Classify features by pointing at them

Refinement Manipulate well defined control points

Transfer function Advantages: Better feature discrimination

Technique Animated transfer function