Data Visualization Lecture 8 3D Scalar Visualization Volume Rendering : Further Ray Casting plus Other Approaches

Classical Approach - Volume Rendering Integral C(s)=light reflected at point s Cast rays through image plane into volume, and measure light received Kajiya and von Hertzen (1984) Max (1995) volume m(s) = density at point s image plane s L I = L0C(s)m(s) exp[ -s0 m(t)dt ] ds light density attenuation

Simplifying the Integral I = L0C(s)m(s) exp[ -s0 m(t)dt ] ds Approximate using Riemann sums (n = number of steps) Approximate exponential by Taylor series and introduce opacity, a, and unit spacing Calculate recursively front-to-back as... I = Sni=0 C(iDs)m(iDs)Ds Pi-1j=0 exp [-m (jDs)Ds] I = Sni=0 C(i)a(i) Pi-1j=0 (1 - a(j)) Cout = Cin + (1-ain)aiCi aout = ain + ai(1 - ain) {stop when a = 1} Compositing associative but not commutative ie can group but cannot re-order

Interpolation Sample points occur within cells, not at grid points, so we need to interpolate Do we: classify at grid points, then interpolate colour / opacity interpolate data values, then classify ? Classify - interpolate classification done as pre-processing smoothing effect can obscure detail Interpolate - classify classification now within the inner loop of the ray cast (sample points are view dependent) in return, fine detail can be picked out

Classify - Interpolate There is a danger in interpolation after classification Naïve colour interpolation would assign 3 parts yellow, 1 part blue to centre point… … but if opacity of bottom left is zero? Correct approach is to weight according to opacity, so colour at centre is yellow!

A Note on Shading The appearance of volume rendered images depends critically on the reflectance calculation used to shade the samples...

A Note on Shading Recall from isosurface rendering - that surface normal is equal to the gradient vector of f At (xi,yj,zk), df/dx = 0.5*(f i+1,j,k - f i-1,j,k) df/dy = 0.5*(f i,j+1,k - f i,j-1,k) df/dz = 0.5*(f i,j,k+1 - f i,j,k-1) This gives us normals at all grid points

A Note on Shading If the classification is done at the vertices, there is a choice: Gouraud-type: shade vertices and then interpolate to get reflected colour at sample point Phong-type: interpolate normals at vertices to get sample normal and then calculate shading Phong-type gives better quality - at the cost of more computational effort

Ray Casting Advantages Disadvantages Non binary classification Shows structure between surfaces Readily parallelisable Disadvantages Computationally expensive - cost proportional to number of voxels (compositing is expensive) Does not take advantage of triangle rendering hardware

Ray Casting Examples The following sequence of slides were produced using the ray casting technique available in the vtk software The slides show for the bonsai tree data set how different aspects can be highlighted by control of the opacity transfer The slides also show, by animation, a comparison of different approaches to interpolation and shading Thanks to Chris Goodyer for creating the images

Legend NN = classify vertex, shade, nearest neighbour interpolate Trilin Vertex = classify vertex, trilinear interpolate, shade Trilin Sample = trilinear interpolate, classify, shade

Bonsai Tree NN – TrilinVertex – TrilinSample

Bonsai Tree Branches NN – TrilinVertex - TrilinSample

Bonsai Tree Leaves NN – TrilinVertex - TrilinSample

Skull NN – TrilinVertex - TrilinSample

Texture-based Volume Rendering Volume rendering by ray casting is time-consuming one ray per pixel each ray involves tracking through volume calculating samples, and then compositing different for each viewpoint Alternative approach - using texture maps - can exploit graphics hardware

Texture Mapping Modern graphics hardware includes facility to draw a textured polygon The texture is an image with red, green, blue and alpha components… … this is used in computer graphics to avoid constructing complex geometric models … and we can exploit this in volume rendering

Texture-based Volume Rendering Draw from back-to-front a set of rectangles first rectangle drawn as an area of coloured pixels, with associated opacity, as determined by transfer function and interpolation - and merged with background in a compositing operation (supported by hardware) successive rectangles drawn on top

3D Texture-based Volume Rendering For a given viewing direction, we would need to select slices perpendicular to this direction This requires interpolation to get the values on the slices Until recently this has only been possible with expensive graphics boards volume image plane 3D texture mapping

2D Texture-based Volume Rendering Simpler solution - 2D texture mapping: view volume as set of slices parallel to co-ordinate planes -Precompute the textured planes for each of the 3 directions -Choose the orientation best suited to viewing direction

Texture-based Volume Rendering

Comparison of Ray Casting and Texture Approaches Texture-based Texture-based Ray casting http://www.cora.nwra.com/Ogle/ http://www.amiravis.com http://vg.swan.ac.uk/vlib

Close Up Ogle: texture-based Vlib: ray casting

Shear Warp Rendering To get fast traversal, shear volume by translating each slice… then can resample as shown Project front-to-back to get intermediate image Then warp image

Example of Shear Warp Rendering

New Hardware Advances Holy grail: real-time volume rendering Main searcher has been Kaufman through Cube architectures VolumePro System first commercially available from Mitsubishi’s RealTime Visualization, now from TeraRecon Uses shear-warp rendering http://www.rtviz.com/home.html

Splatting Another commonly used method is splatting Fuzzy balls around each voxel projected on to image plane Composited in the image plane VolumeToGeom in IRIS Explorer

Summary of Volume Rendering Techniques Ray casting Splatting Both high quality, computationally expensive Texture-based Shear-warp Both lower quality, but faster

Reading Overviews of Volume Visualization Ray Casting - Classic Paper Todd Elvins, `A Survey of Algorithms for Volume Visualization’, Computer Graphics, 1992 Ken Brodlie and Jason Wood, Computer Graphics Forum, 2001 Michael Meissner et al, ‘A Practical Evaluation of Popular Volume Rendering Algorithms’, http://www.gris.uni-tuebingen.de/~bartz/Publications/paper/volvis2000.pdf Ray Casting - Classic Paper Marc Levoy, `Display of Surfaces from Volume Data’, IEEE CG&A, 1988 Book Introduction to Volume Rendering, Lichtenbelt, Crane, Naqvi, Hewlett-Packard Professional Books