1. Ultrasound Visualization Pipeline A Survey
Å. Birkeland, V. Šoltészová, D. Hönigmann, O. H. Gilja, S. Brekke, T. Ropinski and I. Viola
1University of Bergen, Norway
2n22 Research & Technology Transfer, Wiener Neustadt, Austria
3National Centre for Ultrasound in Gastroenterology, Haukeland University Hospital, Norway
4University of Münster, Germany

2. Definition of Visualization
the use of (usually) computer graphics to reveal insight into data to a user
to form a mental vision, image, or picture of (something not visible or present to the sight, or of an abstraction); to make visible to the mind or imagination [Oxford Engl. Dict., 1989]
computer graphics, but not photo-realistic rendering
The purpose of computing is insight, not numbers [R. Hamming, 1962]
The purpose of visualization is insight, not pictures [B. Schneidermann, 1999]
Åsmund Birkeland, University of Bergen

3. Ultrasound Imaging Modality
Non-invasive
Relatively inexpensive
High resolution
Spatially
Temporally
Noise
Random
Speckle
Difficult to interpret
Åsmund Birkeland, University of Bergen

4. Usage of Medical Ultrasound
Diagnostics
Intraoperative imaging
Intervention
Diagnostically
Fetal examinations
Functonal heart examination
Intra-operatively
biopsies
Intervention
drug delivery,
Åsmund Birkeland, University of Bergen

5. Ultrasound Image Modality
2D ultrasound
3D ultrasound
3D freehand ultrasound
Dedicated 3D ultrasound probe
4D ultrasound
Functional imaging
Doppler
B-flow
Strain
Contrast enhanced US
Åsmund Birkeland, University of Bergen

6. Ultrasound Visualization – The Pipeline
Pre-processing: Processing ultrasound data prior to segmentation, registration or rendering
Segmentation: Extracting features from ultrasound data
Registration: Combining ultrasound data with other types of medical imaging modalities
Rendering: Presenting ultrasound data
Augmented Reality: Combining ultrasound rendering with the real world
Pre-processing
Registration
Rendering
Augmented Reality
The essential parts of the visualization pipeline
Registration
Åsmund Birkeland, University of Bergen

7. Ultrasound Visualization - The Pipeline
60 papers
Different classifications
Åsmund Birkeland, University of Bergen

8. Åsmund Birkeland, University of Bergen
Pre-processing
Scan-conversion
Reconstruction of freehand ultrasound
Data Enhancement
Reconstruction – interpolation between images
Filtering – Noise removal prior to segmentation
Data enhancement prior to registration
Gee et al. Processing and Visualizing Three-Dimensional Ultrasound Data
Åsmund Birkeland, University of Bergen

9. Åsmund Birkeland, University of Bergen
Segmentation
Automated assessment of ovarian follicles using a novel three-dimensional ultrasound software
MagiCut – Interactive clipping for 3D ultrasound
One autmated techniques in US workstations
Clipping tools are common – planar – MagiCut
Åsmund Birkeland, University of Bergen

10. Vessel Extraction from Ultrasound
Ultrasound Painting of Vascular Tree
Quickly extract 3D models live during examination
Minimize interaction
Using basic 2D B-mode ultrasound + tracking
Åsmund Birkeland, University of Bergen

11. Vessel Extraction from Ultrasound

12. Åsmund Birkeland, University of Bergen
Registration
Combining two or more image modalities in the same reference frame
Rigid Registration
Non-Rigid Registration
Difficult due to nature of imaging modalities
Feature based
Olesch et al. - Matching CT and Ultrasound data of the Liver by Landmark constrained Image Registration 2009
Image based
W. Wein et al. - Automatic CT-ultrasound registration for diagnostic imaging and image-guided intervention
Olesch et al. Matching CT and Ultrasound data of the Liver by Landmark constrained Image Registration
Wiein et al Automatic CT-ultrasound registration for diagnostic imaging and image-guided intervention
Åsmund Birkeland, University of Bergen

13. Åsmund Birkeland, University of Bergen
Rendering
Surface Rendering
3D freehand rendering
4D ultrasound
Difficulties with ultrasound
Paolo -
Fattal and Lischinski Variational Classification for Visualization of 3D Ultrasound Data
Åsmund Birkeland, University of Bergen

14. Åsmund Birkeland, University of Bergen
Volume: Set of Slices
Åsmund Birkeland, University of Bergen

15. Rendering – Volume Visualization
3D→2D, e.g., a density volume from CT etc.
direct volume rendering (semi-transparent volume)
Åsmund Birkeland, University of Bergen

16. Rendering – Transfer Functions
Mapping data to color and opacity
Non-regular data intensities for the same tissue
Linear OTF
Manually designed piecewice linear OTF
Adaptively designed OTF
Tube-cores – selection of voxels along the viewing direction
Hönigmann et al. Adaptive Design of a Global Opacity Transfer Function for Direct Volume Rendering of Ultrasound Data
Åsmund Birkeland, University of Bergen

17. Rendering – Multi-Modal
Doppler
MRI / CT
Increase in image clutter
Petersch et al. – Blood flow in its context: Combining 3D B-Mode and Color doppler Ultrasonic Data 2007
Viola et al. - Illustrated Ultrasound for Multimodal Data Interpretation of Liver Examinations
Viola et al. Illustrated Ultrasound for Multimodal Data Interpretation of Liver Examinations
Petersch et al. Blood flow in its context: Combining 3D B-Mode and Color doppler Ultrasonic Data
Åsmund Birkeland, University of Bergen

18. Åsmund Birkeland, University of Bergen
Rendering - Shading
Ropinski et al. Interactive Volumetric Lighting Simulating Scattering and Shadowing
Šoltészová et al. Multi-Directional Occlusion shading
Chromatic Depth coloring
Solteszova – Multi-Directional occlusion shading 2010
Ropinksi - Interactive Volumetric Lighting Simulating Scattering and Shadowing
Chromatic Depth Coloring
Åsmund Birkeland, University of Bergen

19. Ultrasound and Augmented Reality
Top: early work in augmented ultrasound: Merging Virtual Objects with the Real World: Seeing Ultrasound Imagery within the Patient
Shelton et al. - Ultrasound Visualization with the Sonic Flashlight
Sato: Image Guidance of Breast Cancer Surgery Using 3-D Ultrasound Images and Augmented Reality Visualization
Åsmund Birkeland, University of Bergen
Åsmund Birkeland, University of Bergen 19

20. Åsmund Birkeland, University of Bergen
Conclusion
Story
Åsmund Birkeland, University of Bergen

21. Åsmund Birkeland, University of Bergen
Acknowledgements
Illustrasound Project, VERDIKT, The Norwegian Research Council
Helwig Hauser – VisGroup, University of Bergen, Norway
Wolfgang Wein - White Lion Technologies, Munich, Germany
Åsmund Birkeland, University of Bergen

22. Åsmund Birkeland, University of Bergen
Questions?
Thank you!
Åsmund Birkeland, University of Bergen