1. -1- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing 3D Visualization Sonia Pujol, Ph.D. Surgical Planning Laboratory Harvard Medical School Leonardo da Vinci (1452-1519), Virgin and Child Alte Pinakothek, München

2. -2- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Slicer3 An end-user application for image analysis An open-source environment for software development A software platform that is both easy to use for clinical researchers and easy to extend for programmers

3. -3- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Material This course requires the installation of the Slicer3 software and training dataset Slicer 3 Software: www.slicer.org/pages/Downloads Training Dataset: SlicerSampleVisualization.tar.gz Disclaimer It is the responsibility of the user of 3DSlicer to comply with both the terms of the license and with the applicable laws, regulations and rules.

4. -4- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Data This course is built upon three datasets of a single healthy subject brain: MR DICOM GRASS MR Nrrd SPGR Pre-computed Label Map tar –zxvf SlicerSampleVisualization.tar.gz

5. -5- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Learning objective Following this tutorial, you’ll be able to load and visualize volumes within Slicer3, and to interact in 3D with structural images and models.

6. -6- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Overview Loading and visualizing multiple volumes simultaneously Loading and visualizing segmented structures overlaid on grayscale images Loading and visualizing 3D models Loading and saving a scene

7. -7- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing If you have downloaded a pre-compiled version of the software, launch Slicer3 from the directory Slicer3 3.2.2008-08-08/: Mac/Linux Run ‘./Slicer3’ in Slicer3-3.2.2008-08-08/ Windows Run ‘./Slicer3’ in Program Files/Slicer3 3.2.2008-08-08/ or launch Start  All Programs  Slicer3 3.2.2008-08-08/Slicer Launch Slicer3

8. -8- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing If you have built a version of Slicer3 on your machine, launch Slicer3 from the directory Slicer3-build: Mac/Linux/Windows Run ‘./Slicer3’ in Slicer3-build/ Launch Slicer3 Slicer3 Slicer3-build Slicer3-lib

9. -9- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Slicer3 GUI The Graphical User Interface (GUI) of Slicer3 integrates five components: the Menu Toolbar the Module GUI Panel the 3D Viewer the Slice Viewer the Slice and 3D View Controller Slice Viewer 3DViewer Module GUI Panel Slice and 3D View Controller Menu Toolbar

10. -10- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Part 1: Loading and visualizing multiple volumes simultaneously

11. -11- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Loading Volumes The Data module is the module displayed by default in the GUI.

12. -12- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Loading Volumes Left-click on the module selection menu to load the module Volumes

13. -13- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Loading Volumes The panel of the module Volumes appears in the interface.

14. -14- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Loading Volumes Click on Select Volume File

15. -15- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Loading Volumes Browse to find the first image 001.dcm of the dataset located in the directory SlicerSampleVisualization/dicom and click on Open

16. -16- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Loading Volumes Select Image Origin: Centered Click on Apply to load the DICOM dataset

17. -17- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Loading Volumes The DICOM images appear in the Background display of the 2DViewer.

18. -18- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Loading Volumes Click on the tab Info to display the characteristics of the DICOM volume

19. -19- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Viewing Volume Information The information contained in the DICOM header appears in the interface: Volume dimensions: 256 x 256 x 124 Volume Spacing: 0.9375x0.9375x1.3 mm Scan Order: LR Scalar Type: Short Image Origin: (-79.95, 119.53,119.53)

20. -20- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Click on the choose view icon Exploring the data

21. -21- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Select the red slice only layout from the menu Exploring the data

22. -22- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Exploring the data The axial slices of the dicom volume appear in the 3D viewer. Slicer automatically adjusts the dimensions of the image to the size of the window.

23. -23- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Exploring the data Browse the images using the slider to display the ventricles (~slice 38)

24. -24- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Exploring the data Click on the conventional layout icon to come back to the standard view

25. -25- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Loading Volumes Click on the ‘adjust slices fit to window’ icon

26. -26- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Loading Volumes Click on Select Volume File

27. -27- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Loading Volumes Browse to find the header file of the volume spgr.nhdr located in the directory SlicerSampleVisualization/nrrd and click on Open.

28. -28- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Loading Volumes Select Image Origin: Centered and Click on Apply

29. -29- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Loading Volumes The spgr images appear in the Background display of the 2DViewer.

30. -30- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Loading Volumes Select the module Data from the Modules Menu.

31. -31- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Loading Volumes The two datasets 001.dcm ad spgr.nhdr are loaded in Slicer.

32. -32- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Overview Loading and visualizing multiple volumes simultaneously Loading and visualizing segmented structures overlaid on grayscale images Loading and visualizing 3D models Loading and saving a scene

33. -33- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Part 2: Loading and visualizing segmented structures overlaid on grayscale images

34. -34- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Loading a label map Come back to the module Volumes and click on Select Volume File

35. -35- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Loading a label map Browse to find the header file all.nhdr of the label map dataset located in the directory SlicerSampleVisualization/nrrd and click on Open

36. -36- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Visualizing a label map Set Image Origin to Centered, Check the Label Map box and click on Apply

37. -37- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Visualizing a label map Slicer displays the label map all in the Label layer

38. -38- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Visualizing Multiple Volumes Foreground Viewer Left click the drop- down menu to the right of the F icon and select the labelmap all Label Viewer Left click the drop- down menu to the right of the L icon and select None

39. -39- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Visualizing Multiple Volumes Background Viewer Left click on the drop- down menu to the right of the B icon and select the volume spgr

40. -40- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Visualizing Multiple Volumes Select Manipulate Slice Views Use the slider to fade between the labelmap all (Foreground) and the spgr volume (Background).

41. -41- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Visualizing Multiple Volumes Select the Red Slice View mode Select the red slice only layout from the menu

42. -42- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Visualizing Multiple Volumes Browse to display the segmented ventricles overlaid on the spgr images (~slice 41)

43. -43- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Overview Loading and visualizing multiple volumes simultaneously Loading and visualizing segmented structures overlaid on grayscale images Loading and visualizing 3D models Loading and saving a scene

44. -44- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Part 3: Loading and visualizing 3D models

45. -45- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Loading a 3D model Select the module Models from the main menu

46. -46- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Loading a 3D model Click on Load Model Directory and select the directory models: SlicerSampleVisualization/models/

47. -47- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Loading a 3D model Slicer loads the 3D models of the skin, the ventricles and the vessels.

48. -48- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Loading a 3D model Click on the Display panel to access the module’s display components

49. -49- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Loading a 3D model Select the model Skin.vtk and scroll down in the Display window to display the icon Set Color.

50. -50- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Visualizing a 3D model Click on the icon Set Color and choose a new color for the 3D model of the head.

51. -51- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Visualizing a 3D model Use the slider to lower the opacity of the 3D model.

52. -52- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Visualizing a 3D model The 3D models of the vessels and the ventricles appear through the skin.

53. -53- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Visualizing a 3D model Select the model Vessels.vtk and change the color of the vasculature

54. -54- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Manipulating a 3D model Click on the R letter (Right) in the control window

55. -55- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Manipulating a 3D model The 3D Viewer shows the model from the right side of the patient

56. -56- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Manipulating a 3D model Position the mouse in the 3D Viewer, hold down the left mouse button and drag to rotate the model.

57. -57- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Manipulating a 3D model Hold down the right mouse button, and move the mouse up and down to zoom out and in.

58. -58- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Manipulating a 3D model Click on the A letter to display the standard Anterior View. Click on the Spin icon

59. -59- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Manipulating a 3D model The model starts spinning inside the Viewer Panel.

60. -60- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Manipulating a 3D model Click on the icon Rock

61. -61- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Manipulating a 3D model The model starts rocking inside the Viewer Panel.

62. -62- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Manipulating a 3D model Click on the Slice Visibility icon to display the slices in the 3D Viewer

63. -63- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Manipulating the images The anatomical slices appear in the 3D Viewer.

64. -64- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Manipulating the images Use the sliders to slice through the volume in the anterior-posterior direction.

65. -65- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Manipulating the images Select the model Skin and set the opacity to 1.0

66. -66- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Manipulating the images Select the option Clipping

67. -67- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Manipulating the images Scroll down and expand the component Clipping.

68. -68- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Manipulating the images Set the Red Slice Clipping, Yellow Slice Clipping and Green Slice Clipping to Negative Space

69. -69- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Manipulating the images Slicer displays the result of the clipping.

70. -70- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Overview Loading and visualizing multiple volumes simultaneously Loading and visualizing segmented structures overlaid on grayscale images Loading and visualizing 3D models Loading and saving a scene

71. -71- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Part 4: Loading and saving a Scene

72. -72- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Saving Data Click on File and Select Save

73. -73- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Saving Data The list of elements currently loaded into Slicer3 appears.

74. -74- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Saving Data Click on Scene File to browse to the location where the scene will be saved

75. -75- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Saving Data Enter the name MyScene.mrml, and click on Save

76. -76- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Saving Data Check the box Save Scene and click on Save

77. -77- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Creating Scene Snapshots Choose a 3D view of the scene and select the module Data.

78. -78- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Creating Scene Snapshots Click on the capture snapshot icon

79. -79- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Creating Scene Snapshots Enter the Snapshot Name MySceneSnapshot1 and click on OK

80. -80- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Creating Scene Snapshots Select a Superior view of the dataset, and click on the capture snapshot icon

81. -81- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Creating Scene Snapshots Enter the name MySceneSnapshot2 and click on OK

82. -82- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Creating Scene Snapshots Select File  Save and click on Save to include the two scene snapshots in the saved scene

83. -83- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Saving Data Select File  Close Scene in the main menu and click on OK

84. -84- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Saving Data The elements of the previous scene disappear from the Viewer.

85. -85- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Saving Data Select File  Load Scene from the main menu

86. -86- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Saving Data Select the file myScene.mrml and click on Open

87. -87- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Loading a Scene Slicer loads the elements from the scene myScene.mrml

88. -88- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Loading a Scene Left-click on the restore snapshot icon. Select MySceneSnapshot2 and click on restore

89. -89- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Loading a Scene Slicer restores the scene snapshot MySceneSnapshot2

90. -90- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Overview Loading and visualizing multiple volumes simultaneously Loading and visualizing segmented structures overlaid on grayscale images Loading and visualizing 3D models Loading and saving a scene

91. -91- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Conclusion Interactive interface to load and manipulate greyscale volumes, labelmaps and 3D models. 3D interaction with anatomical views Open-source platforms for Linux, Mac and Windows

92. -92- 3D Visualization. Sonia Pujol, Ph.D., Harvard Medical School National Alliance for Medical Image Computing Acknowledgments National Alliance for Medical Image Computing NIH U54EB005149 Neuroimage Analysis Center NIH P41RR013218