1. Hardware-Based Nonlinear Filtering and Segmentation using High-Level Shading Languages I. Viola, A. Kanitsar, M. E. Gröller Institute of Computer Graphics and Algorithms Vienna University of Technology Vienna, Austria

2. Ivan Viola Vienna University of Technology2 / 23 Volume Visualization Pipeline CPU GPU CPU DATA ACQUISITION DATA ENHANCEMENT VISUALIZATION MAPPING RENDERING VISUALIZATION MAPPING DATA ENHANCEMENT

3. Ivan Viola Vienna University of Technology3 / 23 GPU-based Algorithms high performance high performance high flexibility high flexibility easy implementation: HLSL easy implementation: HLSL necessary features: necessary features:  floating point precision  long shader programs  latest commodity graphics hardware DATA ENHANCEMENT liver dataset segmented vessels

4. Ivan Viola Vienna University of Technology4 / 23 Talk Outline processing pipeline processing pipeline GPU-based filtering GPU-based filtering  per-vertex stage  per-fragment stage median filter median filter bilateral filter bilateral filter rotated mask filter rotated mask filter GPU-based segmentation GPU-based segmentation

5. Ivan Viola Vienna University of Technology5 / 23 GPU CPU Liver Vessel Tree Visualization pre-filtering pre-filtering  improving thresholding segmentation  edge-preserving filters interactive threshold adjustment interactive threshold adjustment mask generation mask generation volumetric clipping volumetric clipping volume rendering volume rendering

6. Ivan Viola Vienna University of Technology6 / 23 Processing Pipeline

7. Ivan Viola Vienna University of Technology7 / 23 Talk Outline processing pipeline processing pipeline GPU-based filtering GPU-based filtering  per-vertex stage  per-fragment stage median filter median filter bilateral filter bilateral filter rotated mask filter rotated mask filter GPU-based segmentation GPU-based segmentation

8. Ivan Viola Vienna University of Technology8 / 23 Filtering in Graphics Hardware Issues data representation: textures data representation: textures  3D texture  stack of 2D textures access to value: texture fetch access to value: texture fetch neighborhood addressing: texture offset  we use 5×5×5 neighborhood neighborhood addressing: texture offset  we use 5×5×5 neighborhood filter implementation: per-fragment stage filter implementation: per-fragment stage results: rendered into off-screen buffer results: rendered into off-screen buffer

9. Ivan Viola Vienna University of Technology9 / 23 TEXTURE STACKOFF-SCREEN BUFFER STACK Data Representation TEXTURE STACKOFF-SCREEN BUFFER STACK

10. Ivan Viola Vienna University of Technology10 / 23 Neighborhood Addressing Two alternatives: directly in fragment program  requires additional computation directly in fragment program  requires additional computation pre-compute in per-vertex stage pre-compute in per-vertex stage store in vertex attributes store in vertex attributes interpolation “for-free” interpolation “for-free” swizzle operator swizzle operator

11. Ivan Viola Vienna University of Technology11 / 23 + float4(-2, 2, -1,1)IN.TEXCOORD0.x yxy OUT.TEXCOORD0.xy zw= Address Pre-computation IN.TEXCOORD0.x y OUT.TEXCOORD0.xy = + float4(-2, 2 FILTER KERNEL XY X-2 Y+2 X-1 Y+1 PER-VERTEX STAGE TEXCOORD0.xy TEXCOORD0.zw TEXCOORD0.zy TEXCOORD0.xw XYXW ZWZY

12. Ivan Viola Vienna University of Technology12 / 23 Per-fragment Stage medical data - 12 bit precision medical data - 12 bit precision  fixed point 12-bit arithmetics  fixed point 12-bit arithmetics use cache coherence use cache coherence exploit 4D instructions exploit 4D instructions reduce conditionals reduce conditionals reduce number of registers reduce number of registers push computation to per-vertex stage push computation to per-vertex stage

13. Ivan Viola Vienna University of Technology13 / 23 Median Filter central value of ordered set central value of ordered set implementation implementation  CPU-based  sorting  GPU-based  similar to quickselect()

14. Ivan Viola Vienna University of Technology14 / 23 GPU-based Median Filter input data 12 bit [0..4095] input data 12 bit [0..4095] multi-pass approach multi-pass approach not efficient on CPU not efficient on CPU exploiting GPU 4D arithmetics exploiting GPU 4D arithmetics 01234567

15. Ivan Viola Vienna University of Technology15 / 23 edge preservation: anisotropic filter kernel edge preservation: anisotropic filter kernel product of two weights: product of two weights:  geometric:  photometric: Bilateral Filter x f(x ) high geometric weightlow geometric weight high geometric weightlow photometric weight

16. Ivan Viola Vienna University of Technology16 / 23 GPU-based Bilateral Filter weights are precomputed weights are precomputed geometric weight stored in unused vertex attributes ( COLOR0 ) geometric weight stored in unused vertex attributes ( COLOR0 ) photometric weight stored in 1D mirror LUT photometric weight stored in 1D mirror LUT weight product weight product sum-up contributions & weights sum-up contributions & weights normalize normalize

17. Ivan Viola Vienna University of Technology17 / 23 Rotated Mask Filter anisotropic noise removal with edge preservation anisotropic noise removal with edge preservation splits filter mask into sub-regions splits filter mask into sub-regions mean and variance value for each sub- region mean and variance value for each sub- region result – mean value of sub-region with minimal variance result – mean value of sub-region with minimal variance GPU implementation GPU implementation  single pass - slow  multiple passes - reduce temp. registers

18. Ivan Viola Vienna University of Technology18 / 23 Talk Outline processing pipeline processing pipeline GPU-based filtering GPU-based filtering  per-vertex stage  per-fragment stage median filter median filter bilateral filter bilateral filter rotated mask filter rotated mask filter GPU-based segmentation GPU-based segmentation

19. Ivan Viola Vienna University of Technology19 / 23 Segmentation input: pre-filtered data after noise removal input: pre-filtered data after noise removal thresholding segmentation thresholding segmentation  0 outside interval  1 within interval interactive threshold adjustment interactive threshold adjustment output: compressed form  32 slices in one 32 bit slice output: compressed form  32 slices in one 32 bit slice

20. Ivan Viola Vienna University of Technology20 / 23 Results OperationGPU [ms]CPU [ms]Speedup Median filter24678486391.97 Bilateral filter9668147061.52 Rotated mask f.7989580037.26 Thresholding403498.73 Thresholding & compression64–– GPU: NVIDIA GeForceFX 5900 Ultra GPU: NVIDIA GeForceFX 5900 Ultra CPU: AMD AthlonXP 2.4 GHz, 1GB DDR RAM CPU: AMD AthlonXP 2.4 GHz, 1GB DDR RAM liver dataset: 512×512×72 liver dataset: 512×512×72

21. Ivan Viola Vienna University of Technology21 / 23 Results

22. Ivan Viola Vienna University of Technology22 / 23 Conclusions data enhancement step on GPU! data enhancement step on GPU! simple tasks  better speedup simple tasks  better speedup optimization HW specific optimization HW specific high-level programming high-level programming  friendly  many implementation possibilities  compiler efficiency

23. Ivan Viola Vienna University of Technology23 / 23 Thank you for your attention!

24. Ivan Viola Vienna University of Technology24 / 23 edge preservation: anisotropic filter kernel edge preservation: anisotropic filter kernel product of two weights: product of two weights:  geometric: h g (spatial difference)  photometric: h p (density difference) Bilateral Filter central sample neighbor: high geometric low photometric weight neighbor: low geometric high photometric weight neighbor: low geometric low photometric weight low contribution