Spectral Analysis of Function Composition and Its Implications for Sampling in Direct Volume Visualization Steven Bergner GrUVi-Lab/SFU Torsten Möller Daniel Weiskopf David J Muraki Dept. of Mathematics/SFU

Overview Frequency domain intuition Function Composition in Frequency Domain Application to Adaptive Sampling Future Directions

Frequency domain standard analysis tool Intuition Analysis Application Motivation Frequency domain standard analysis tool Assumption of band-limitedness we know how to sample in the spatial domain Given by Nyquist frequency f

Sampling in Frequency domain Intuition Analysis Application Sampling in Frequency domain  F f(x) f  F Wie komme ich mit den wenigsten samplen aus??  F x f

Convolution Theorem Spatial Domain: Frequency Domain:  F Convolution: Intuition Analysis Application Convolution Theorem Spatial Domain: Frequency Domain:  F Convolution: Multiplication:

Combining 2 different signals Intuition Analysis Application Combining 2 different signals Convolution / Multiplication: E.g. filtering in the spatial domain => multiplication in the frequency domain Compositing: What about

Opacity f Transfer Function g Map data value f to optical properties, such as opacity and colour Then shading+compositing g(f(x)) Opacity f g

Estimates for band-limit of h(x) Intuition Analysis Application Estimates for band-limit of h(x) Considering M. Kraus et al. Can be a gross over-estimation Our solution

Example of g(f(x)) Original function f(x) g(f(x)) sampled by Intuition Analysis Application Example of g(f(x)) Original function f(x) g(f(x)) sampled by Transfer function g(y) g(f(x)) sampled by The actual sampling rate is actual four times as much - that’s good for doing linear interpolation …

Analysis of Composition in Frequency Domain

Composition in Frequency Domain Intuition Analysis Application Composition in Frequency Domain y y

Composition as Integral Kernel Intuition Analysis Application Composition as Integral Kernel

Intuition Analysis Application Visualizing P(k,l)

Slopes of lines in P(k,l) are related to 1/f‘(x) Intuition Analysis Application Visualizing P(k,l) Slopes of lines in P(k,l) are related to 1/f‘(x) Extremal slopes bounding the wedge are 1/max(f’)

Contribution insignificant for rapidly changing Intuition Analysis Application Analysis of P(k,l) For general Contribution insignificant for rapidly changing Contributions large when These points are called points of stationary phase: The largest such k is of interest:

Second order Taylor expansion Exponential drop-off Intuition Analysis Application Exponential decay Second order Taylor expansion Exponential drop-off

Adaptive Sampling for Raycasting Application Adaptive Sampling for Raycasting

Compute the gradient-magnitude volume Intuition Analysis Application Adaptive Raycasting Compute the gradient-magnitude volume For each point along a ray - determine max|f’| in a local neighborhood Use this to determine stepping distance

Adaptive sampling - 25% less samples Intuition Analysis Application Adaptive Raycasting Adaptive sampling - 25% less samples Uniform sampling Fig. 5. Examples of hipiph data set sampled at a fixed rate (0.5) (a) and sampled with adaptive stepping (b). The adaptive method in (b) uses about 25% fewer samples than (a) only measuring in areas of non-zero opacity to not account for effects of empty-space skipping. The similarity of both images indicates that visual quality is preserved in the adaptive, reduced sampling.

Adaptive Raycasting Same number of samples Intuition Analysis Application Adaptive Raycasting Same number of samples

Adaptive Raycasting SNR Intuition Analysis Application Adaptive Raycasting SNR Ground-truth: computed at a fixed sampling distance of 0.06125 unit grid point spacing of 1. The result of the evaluation is shown in Figure 6. The error plot is based on the signal to noise ratio (SNR), computed as Fig. 6. Quality vs. performance, where quality is measured using signal to noise ratio (SNR) and performance is indicated by the number of samples taken along all rays cast into the volume. Adaptive sampling clearly outperforms the uniform (fixed) sampling. Only samples in areas of non-zero opacity are taken into account, i.e., both sampling schemes equally make use of empty space skipping.

Pre-integration approach Intuition Analysis Application Pre-integration approach Standard fix for high-quality rendering Assumes linearity of f between sample points Fails for High-dynamic range data Multi-dimensional transfer function Shading approximation between samples A return to direct computation of integrals is possible

Future directions Exploit statistical measures of the data contained in P(k,l) Combined space-frequency analysis Other interpretations of g(f(x)) change in parametrization of g activation function in artificial neural networks Fourier Volume Rendering

Proper sampling of combined signal g(f(x)): Intuition Analysis Applications Summary Proper sampling of combined signal g(f(x)): Solved a fundamental problem of rendering Applicable to other areas Use the ideas for better algorithms

Acknowledgements NSERC Canada BC Advanced Systems Institute Canadian Foundation of Innovation

Thanks… … for your attention! Any Questions?

Transfer Functions (TFs) Intuition Analysis Application Transfer Functions (TFs) a(g) RGB(g) a Simple (usual) case: Map data value g to color and opacity g Shading, Compositing… Human Tooth CT For the most part, TFs map from the data values to color and opacity:RGBa. So in this TF, I have three ranges of values which receive opacity, according to these tents, And within those ranges I’m assigning constant colors: red, cyan, yellow. Here’s a sample dataset. When we map the values through the transfer function, we get Colors and opacities, since, the data values are the DOMAIN of the transfer function, and the Colors and opacities are the range. But the important thing is that color and opacity are something tangible, as far as graphics operations, So then we can apply shading and compositing algorithms to produce a final rendering. This is a volume rendering of this dataset, using this transfer function.

Motivation - Volume Rendering Intuition Analysis Application Motivation - Volume Rendering Convolution used all the time: interpolation ray-casting multi-resolution pyramids gradient estimation Compositing used all the time: transfer functions Given Needed

Assume a linear function f(x) = ax Intuition Analysis Application Analysis of P(k,l) Assume a linear function f(x) = ax If phase is zero - integral infinite Non-zero - integral is zero

Intuition Analysis Application Analysis of P(k,l)

Proper sampling of g(f(x)) Intuition Analysis Application Proper sampling of g(f(x)) Our solution: