Ultrasound Visualization Pipeline A Survey Å. Birkeland, V. Šoltészová, D. Hönigmann, O. H. Gilja, S. Brekke, T. Ropinski and I. Viola 1. University of Bergen, Norway 3. Haukeland University Hospital, Norway 1University of Bergen, Norway 2n22 Research & Technology Transfer, Wiener Neustadt, Austria 3National Centre for Ultrasound in Gastroenterology, Haukeland University Hospital, Norway 4University of Münster, Germany 2 . n22 Research & Technology Transfer, Austria 4. University of Münster, Germany

Motivation for Ultrasound Visualization Fetal examination Cardiac Gastro Fetal examinations Functonal heart examination Åsmund Birkeland, University of Bergen

Ultrasound Imaging Modality Non-invasive Cheap High resolution Spatially Temporally Noise Random Speckle Åsmund Birkeland, University of Bergen

Ultrasound Image Modality 2D ultrasound 3D ultrasound Dedicated 3D ultrasound probe 3D freehand ultrasound 4D ultrasound Blood flow Doppler B-flow Åsmund Birkeland, University of Bergen

Ultrasound Visualization – The Pipeline Pre-processing: Processing ultrasound data prior to segmentation, registration or rendering. Segmentation: Extracting features from ultrasound data. Registration: Combining ultrasound data with other types of medical imaging modalities. Rendering: Presenting ultrasound data. Augmented Reality: Combining ultrasound rendering The essential parts of the visualization pipeline Åsmund Birkeland, University of Bergen

Ultrasound Visualization - The Pipeline 60 papers Different classifications Åsmund Birkeland, University of Bergen

Åsmund Birkeland, University of Bergen Pre-processing Reconstruction of freehand ultrasound Large volumes Data Enhancement Filltering Reconstruction – interpolation between images Filtering – Noise removal prior to segmentation Data enhancement prior to registration Gee et al. Processing and Visualizing Three-Dimensional Ultrasound Data Åsmund Birkeland, University of Bergen

Åsmund Birkeland, University of Bergen Segmentation Automated assessment of ovarian follicles using a novel three-dimensional ultrasound software MagiCut – Interactive clipping for 3D ultrasound One autmated techniques in US workstations Clipping tools are common – planar – MagiCut Åsmund Birkeland, University of Bergen

Åsmund Birkeland, University of Bergen Registration Combining two or more image modalities in the same reference frame Rigid Registration Non-Rigid Registration Olesch et al. Matching CT and Ultrasound data of the Liver by Landmark constrained Image Registration Olesch et al. - Matching CT and Ultrasound data of the Liver by Landmark constrained Image Registration 2009 Åsmund Birkeland, University of Bergen

Åsmund Birkeland, University of Bergen Rendering Surface Rendering 3D freehand rendering 4D ultrasound Difficulties with ultrasound Fattal and Lischinski Variational Classification for Visualization of 3D Ultrasound Data Åsmund Birkeland, University of Bergen

Rendering – Transfer Functions Mapping data to color and opacity Non-uniform data intensities for the same tissue Hönigmann et al. Adaptive Design of a Global Opacity Transfer Function for Direct Volume Rendering of Ultrasound Data Åsmund Birkeland, University of Bergen

Rendering – Multi-Modal Doppler MRI / CT Increase in image clutter Petersch et al. – Blood flow in its context: Combining 3D B-Mode and Color doppler Ultrasonic Data 2007 Viola et al. - Illustrated Ultrasound for Multimodal Data Interpretation of Liver Examinations Viola et al. Illustrated Ultrasound for Multimodal Data Interpretation of Liver Examinations Petersch et al. Blood flow in its context: Combining 3D B-Mode and Color doppler Ultrasonic Data Åsmund Birkeland, University of Bergen

Åsmund Birkeland, University of Bergen Rendering - Shading Ropinski et al. Interactive Volumetric Lighting Simulating Scattering and Shadowing Šoltészová et al. Multi-Directional Occlusion shading Chromatic Depth coloring Solteszova – Multi-Directional occlusion shading 2010 Ropinksi - Interactive Volumetric Lighting Simulating Scattering and Shadowing Chromatic Depth Coloring Åsmund Birkeland, University of Bergen

Ultrasound and Augmented Reality Top: early work in augmented ultrasound: Merging Virtual Objects with the Real World: Seeing Ultrasound Imagery within the Patient Shelton et al. - Ultrasound Visualization with the Sonic Flashlight Sato: Image Guidance of Breast Cancer Surgery Using 3-D Ultrasound Images and Augmented Reality Visualization Åsmund Birkeland, University of Bergen Åsmund Birkeland, University of Bergen 14

Åsmund Birkeland, University of Bergen Acknowledgements Illustrasound Project, VERDIKT, The Norwegian Research Council Helwig Hauser – VisGroup, University of Bergen, Norway Åsmund Birkeland, University of Bergen

Åsmund Birkeland, University of Bergen Questions? Thank you! Åsmund Birkeland, University of Bergen