1. Michael Burns Martin Haidacher Eduard Gröller Ivan ViolaWolfgang Wein

2. CT scan with embedded Ultrasound data Michael Burns - Contextual Medical Visualization

3.  Poking needles Liver biopsy Radio frequency ablation  Procedure: Patient has CT scan Needle path is planned Uses ultrasound probe to help guide needle Doctor views CT scan at time of procedure Michael Burns - Contextual Medical Visualization

4. CT Scan DataUltrasound Data Michael Burns - Contextual Medical Visualization

5. Ultrasound embedded in dense volume Michael Burns - Contextual Medical Visualization

6. Ultrasound embedded in sparse volume Michael Burns - Contextual Medical Visualization

7. Ultrasound with contextual cutaway Michael Burns - Contextual Medical Visualization

8.  Volumetric data Tissue types differentiated and ranked ○ Important materials most visible ○ Unimportant materials provide context  Ultrasound image Captured with 3D position and orientation of probe Registration between coordinate frames [Wein05] Michael Burns - Contextual Medical Visualization

9. Volumetric Data (e.g. CT Scan) Importance in the Transfer Function Viewpoint Information Object of Interest (e.g. Ultrasound Plane) Contextual Cutaway Views Importance-Driven Shading Integrating Occlusion with Importance Michael Burns - Contextual Medical Visualization

10. Volumetric Data (e.g. CT Scan) Importance in the Transfer Function Viewpoint Information Object of Interest (e.g. Ultrasound Plane) Contextual Cutaway Views Importance-Driven Shading Integrating Occlusion with Importance Michael Burns - Contextual Medical Visualization

11. Rank materials by relevance 1. Definition in volumetric space Uses auxiliary volume Requires preprocessing per dataset 2. Definition in transfer function space Extra value in transfer function Shared among datasets Michael Burns - Contextual Medical Visualization

12. Material boundaries F(x, y, z) |  F(x, y, z)| Michael Burns - Contextual Medical Visualization

13. Volumetric Data (e.g. CT Scan) Importance in the Transfer Function Viewpoint Information Object of Interest (e.g. Ultrasound Plane) Contextual Cutaway Views Importance-Driven Shading Integrating Occlusion with Importance Michael Burns - Contextual Medical Visualization

14.  Visual distinction between materials Emphasis of important materials  Material properties Color Opacity  Lighting properties Shading conveys detail Michael Burns - Contextual Medical Visualization

15. Full ShadingNo Shading Michael Burns - Contextual Medical Visualization

16. Full ShadingImportance Shading Michael Burns - Contextual Medical Visualization

17. Importance Shading  Emphasis: E  Shaded color: C shaded  Subdued color: C subdued = E * C unshaded + (1 – E ) * C shaded  Final color: C final = I * C shaded + (1 – I ) * C subdued Michael Burns - Contextual Medical Visualization

19. Volumetric Data (e.g. CT Scan) Importance in the Transfer Function Viewpoint Information Object of Interest (e.g. Ultrasound Plane) Contextual Cutaway Views Importance-Driven Shading Integrating Occlusion with Importance Michael Burns - Contextual Medical Visualization

20.  Object of interest obscured by volume High importance ○ Should be visible ○ May obscure object Low importance ○ Not necessarily visible ○ May not obscure object  View-dependent cutaway structure Michael Burns - Contextual Medical Visualization

21. v  Base Clear Object of Interest Michael Burns - Contextual Medical Visualization

22. Small  Large  Michael Burns - Contextual Medical Visualization

23. v 11 22 Base Transition Clear Overlay d Michael Burns - Contextual Medical Visualization

24. BaseTransition Michael Burns - Contextual Medical Visualization

25. Base, TransitionOverlay Michael Burns - Contextual Medical Visualization

26. Base, Transition, OverlayBase Michael Burns - Contextual Medical Visualization

27. Volumetric Data (e.g. CT Scan) Importance in the Transfer Function Viewpoint Information Object of Interest (e.g. Ultrasound Plane) Contextual Cutaway Views Importance-Driven Shading Integrating Occlusion with Importance Michael Burns - Contextual Medical Visualization

28. v  = 1  = 0 1 0.75 0.5 0.25 0 Michael Burns - Contextual Medical Visualization

29.  Fade material based on occlusion value between two occlusion thresholds Thresholds based on importance  Modify opacity: τ u = I τ l = max(2 * I – 1, 0) α’ = α * (1 – ramp(τ u, τ l, Ω)) Michael Burns - Contextual Medical Visualization

30. BaseTransitionClearOverlay Liver (0.5) Vessels (0.99) Occlusion value (Ω) α scale factor 010.5 Flesh (0.01) 0 1 τ u = I τ l = max(2 * I – 1, 0) Michael Burns - Contextual Medical Visualization

31. No cutaway (over draw) Michael Burns - Contextual Medical Visualization

32. Simple cutaway Michael Burns - Contextual Medical Visualization

33. Simple cutaway with interior Michael Burns - Contextual Medical Visualization

34. Interior becomes overlay region Michael Burns - Contextual Medical Visualization

35. Transition area added Michael Burns - Contextual Medical Visualization

37.  GPU Raycaster in GLSL  Cutaway represented as height field Created by rendering extruded geometry Requires only 1 lookup per ray  Performance Interactive frame rates 10-15 fps on high-end hardware Dependent on sample rate, volume size, empty space skipping, etc. Michael Burns - Contextual Medical Visualization

38.  Visualization Material importance defined within transfer function Important materials emphasized through shading View-dependent cutaway structure determines occlusion of object-of-interest Materials removed in occluding areas, according to their importance  Application Visualize ultrasound data within CT scan for needle driven operations Initial feedback has been positive Currently being evaluated for clinical use Michael Burns - Contextual Medical Visualization

39.  Gianluca Paladini Siemens Corporate Research  Adam Finkelstein Princeton University  Paper reviewers  EuroVis 2007 hosts Michael Burns - Contextual Medical Visualization