1. NA-MIC National Alliance for Medical Image Computing http://na-mic.org Slicer 2.x Architecture Steve Pieper, PhD

2. National Alliance for Medical Image Computing http://na-mic.org Overall Goals Data Model –Mrml Tree –Mrml Nodes –Mrml Data Module Architecture –Flow of Control, Startup Sequence –Standard Developer Widgets –GUI Modules –Editor Modules –I/O Modules Writing Custom Modules

3. National Alliance for Medical Image Computing http://na-mic.org Mrml Data Model Tree-structured scene description –Accessible as Mrml(dataTree) Serialized to XML file Compiled to VTK class instances in memory for display and manipulation

4. National Alliance for Medical Image Computing http://na-mic.org Mrml http://www.slicer.org/mrml

5. National Alliance for Medical Image Computing http://na-mic.org Mrml XML <Volume name='all' filePattern='%s.%03d' filePrefix='labels/all' rasToIjkMatrix='0 -1.06667 0 128 0 0 -1.06667 128 0.666667 0 0 62 0 0 0 1' rasToVtkMatrix='0 -1.06667 0 128 0 0 1.06667 128 0.666667 0 0 62 0 0 0 1' positionMatrix='0 0 1 -93 -1 0 0 120 0 1 0 -120 0 0 0 1' description='LR' colorLUT='-1' labelMap='yes' interpolate='no' window='4' level='1' lowerThreshold='0' upperThreshold='0' imageRange='1 124'>

6. National Alliance for Medical Image Computing http://na-mic.org Mrml XML Describes the data as it exists on disk Composes the scene by positioning data elements Retains visualization and other state parameters

7. National Alliance for Medical Image Computing http://na-mic.org Data Model – the Mrml Tree Mrml Tree is an in-memory data description corresponding to the mrml xml file –vtkMrmlNode subclasses contain the metadata –Mapping of metadata to run-time state is managed mainly in Tcl –vtkMrmlData subclasses are helpers for classes that need C++ help to efficiently manage runtime state

8. National Alliance for Medical Image Computing http://na-mic.org vtkMrmlNode

9. National Alliance for Medical Image Computing http://na-mic.org Mrml Nodes vtkMrmlNode abstract base class –int ID; –char *Description; –char *Options; –char *Name; –char *Title; –virtual void Write(ofstream& of, int indent); IsA vtkObject Can be subclassed in run time loadable Modules

10. National Alliance for Medical Image Computing http://na-mic.org vtkMrmlModelNode vtkMrmlModelNode Example Instance Variables // Strings char *ModelID; char *FileName; char *FullFileName; char *Color; // Numbers float Opacity; // Booleans int Visibility; int Clipping; int BackfaceCulling; int ScalarVisibility;

11. National Alliance for Medical Image Computing http://na-mic.org vtkMrmlModelNode - methods vtkMrmlModelNode -- Example Methods SetModelID with 1 arg GetModelID SetFileName with 1 arg GetFileName SetFullFileName with 1 arg GetFullFileName SetColor with 1 arg GetColor SetOpacity with 1 arg GetOpacity VisibilityOn VisibilityOff GetVisibility SetVisibility with 1 arg ClippingOn ClippingOff GetClipping SetClipping with 1 arg BackfaceCullingOn BackfaceCullingOff GetBackfaceCulling SetBackfaceCulling with 1 arg ScalarVisibilityOn ScalarVisibilityOff GetScalarVisibility SetScalarVisibility with 1 arg

12. National Alliance for Medical Image Computing http://na-mic.org vtkMrmlModelNode DTD XML Representation Example: DTD entry from Base/tcl/mrml20.dtd: <!ATTLIST Model name NMTOKEN "" fileName CDATA #REQUIRED color NMTOKEN "" description CDATA "" opacity NMTOKEN "1.0" visibility (yes | no) "yes" clipping (yes | no) "no" backfaceCulling (yes | no) "yes" scalarVisibility (yes | no) "no" ignore (yes | no) "no" scalarRange NMTOKENS "0 100">

13. National Alliance for Medical Image Computing http://na-mic.org Example Slicer Control Flow Event Manager User Module MainUpdateMRML VTK Visualization 1 2 3 4 5 Module Modules 6

14. National Alliance for Medical Image Computing http://na-mic.org How Mrml Files are Read MainMrmlReadVersion2.x –In Base/tcl/tcl-main/Parse.tcl –Reads xml file and returns tcl list of nodes and values MainMrmlBuildTreesVersion2.0 –In Base/tcl/tcl-main/MainMrml.tcl –Converts list into vtkMrmlTree and vtkMrmlNode instances –Modules can define Module($m,procMRMLLoad) to extend Mrml

15. National Alliance for Medical Image Computing http://na-mic.org Editing the MrmlTree Examples MainMrmlAddNode –Creates a new vtkMrmlNode instance and adds it to Mrml(dataTree) and returns nodeId –Use vtkMrmlNode methods to set data Mrml(dataTree) RemoveItem $node –Remove a node Mrml(dataTree) InsertAfterItem $nodeBefore $node –Add your new node to specific spot in the hierarchy DevCreateNewCopiedVolume {Description ""} { VolName ""} –Create a copy node of existing volume with optional new name and description

16. National Alliance for Medical Image Computing http://na-mic.org Compiling the Mrml Tree MainUpdateMRML –In Base/tcl/tcl-main/Main.tcl –Is called after read or after changes to the Mrml Tree –Invokes Main*UpdateMRML Models, Volumes, etc Instances VTK classes, reads data, changes render properties, etc to make scene match the MRML Tree Keeps Dev* GUI elements in sync with Tree and Selections –Invokes Modules($m,procMRML) for each Module

17. National Alliance for Medical Image Computing http://na-mic.org Rendering To make the newly updated data visible to the user: –Render3D Redraw just the 3D view –RenderSlice Redraw slice view 0, 1, or 2 –RenderSlices Redraw all slice views –RenderAll Redraw 3D and Slices

18. National Alliance for Medical Image Computing http://na-mic.org Iterating Through the Mrml Tree set nitems [Mrml(dataTree) GetNumberOfItems] for {set widx 0} {$widx < $nitems} {incr widx} { set item [Mrml(dataTree) GetNthItem $widx] puts “$item is a [$item GetClassName]” }

19. National Alliance for Medical Image Computing http://na-mic.org Overall Module Goals Allow Independent Development of Run-Time Loadable Modules –Enforce code independence –Allow modules delivered under different licenses –Support development outside of main cvs prior to code sharing The SLICER_MODULES environment variable –Space-separated list of directories containing slicer module directory trees –Used by cmaker, launch, and tarup –Allows you to keep your own code in a distinct cvs repository/working directory, but integrate with build and distribution scripts

20. National Alliance for Medical Image Computing http://na-mic.org Execution Environment Slicer runs inside a VTK Tcl/Tk shell –Uses the vtk executable that gets built when Tcl wrapping is enabled for the VTK build (or the wish84.exe shell on Windows) –vtkSlicerBase is a tcl package that gets pulled in at run time –Each Module is a tcl package –Note that vtkSlicerBase and the Modules can be wrapped for other languages if desired (python, java)

21. National Alliance for Medical Image Computing http://na-mic.org What is a Module? C++ code (optional) –VTK Subclasses –ITK Subclasses –Extra Helper code (GSL, OpenGL, etc) Tcl code –Application Logic –User Interface Code

22. National Alliance for Medical Image Computing http://na-mic.org Tcl Packages / Modules Includes Tcl code to be interpreted Includes compiled C++ code as a shared library (optional) –Can reference and subclass C++ classes from other packages Package path that can be used to find module- specific logos, data, etc 3D Slicer Application Slicer BaseModule 1Module N… VTKTcl OpenGL Window System Computer Hardware Tcl packages

23. National Alliance for Medical Image Computing http://na-mic.org Directory Organization Modules/vtk –cxx CMakeListsLocal.txt lists the source files plus custom build directives –tcl.tcl has standard slicer entry points –builds Platform specific binary directories –Wrapping Standard tcl/java/python wrapping support –data Additional files your module needs (will be included in distribution by tarup.tcl)

24. National Alliance for Medical Image Computing http://na-mic.org Flow of Control Outline Base/tcl/Go.tcl is main entry point –Parse command line args –Starts tkcon console –Load vtkSlicerBase and all module packages Uses tcl ‘package require’ command –Run MainBoot

25. National Alliance for Medical Image Computing http://na-mic.org Anatomy of A Module See Modules/vtkCustomModule/tcl/@ModuleName@. tcl.in Modules/vtkCustomModule/tcl/@ModuleName@. tcl.in –For example, Custom CustomInit –Analogous to a Constructor CustomBuildGUI CustomEnter / CustomExit –When user module pane is visible/hidden Custom* –Any other “Methods” for the Module –These procs Edit the MrmlTree and call MainUpdateMRML and Render

26. National Alliance for Medical Image Computing http://na-mic.org The Shared Dev* Procs Base/tcl/tcl-shared/Developer.tcl –Interact with MrmlTree –Use standard look and feel conventions DevAddSelectButton –Automatically updating popup menu for Volumes, Models, etc –Adds entries when objects are created DevUpdateNodeSelectButton –Updates to currently selected DevAddButton DevAddLabel DevAddFileBrowse DevInfoWindow, DevErrorWindow, DevWarningWindow… Etc…

27. National Alliance for Medical Image Computing http://na-mic.org Event Management The Enter and Exit procs can be used to set up custom mouse/keyboard bindings Base/tcl/tcl-shared/Events.tcl –pushEventManager / popEventManager –Arguments are list of: Widget, event, callback

28. National Alliance for Medical Image Computing http://na-mic.org Event Example In EMSegmentInit: foreach s $Slice(idList) { set widget $Gui(fSl${s}Win) append EMSegment(eventManager) " \ {$widget {EMSegmentBindingCallback Sample %x %y}}” } In EMSegmentEnter: pushEventManager $EMSegment(eventManager) In EMSegmentExit: pushEventManager $EMSegment(eventManager)

29. National Alliance for Medical Image Computing http://na-mic.org Event Example (continued) EMSegmentBindingCallback calls EMSegmentReadGreyValue $x $y 1 In EMSegmentReadGreyValue: set s $Interactor(s) ;# get the slice window where user clicked scan [MainInteractorXY $s $x $y] "%d %d %d %d" xs ys x y ;# convert to pixels Slicer SetReformatPoint $s $x $y scan [Slicer GetIjkPoint] "%g %g %g" xIjk yIjk zIjk ;# convert to array coords set xIjk [expr int($xIjk)] set yIjk [expr int($yIjk)] set zIjk [expr int($zIjk)] # get the pixel data from the volume set ImageData [Volume($v,vol) GetOutput] set pixel [$ImageData GetScalarComponentAsDouble $xIjk $yIjk $zIjk 0]

30. National Alliance for Medical Image Computing http://na-mic.org Progress Bar Progress Bar on Lower part of slicer control window For a vtkProcessObject instance X x AddObserver StartEvent MainStartProgress x AddObserver ProgressEvent "MainShowProgress x" x AddObserver EndEvent MainEndProgress

31. National Alliance for Medical Image Computing http://na-mic.org Fiducials Example proc intensities {vol} { # Note: does not account for any transforms applied to Volume catch "m Delete" vtkMatrix4x4 m eval m DeepCopy [Volume($vol,node) GetRasToVtkMatrix] set ids [FiducialsGetPointIdListFromName default] foreach id $ids { set pt [eval m MultiplyPoint [FiducialsGetPointCoordinates $id] 1] set i [expr int([lindex $pt 0])] set j [expr int([lindex $pt 1])] set k [expr int([lindex $pt 2])] set imageData [Volume($vol,vol) GetOutput] set pixel [$imageData GetScalarComponentAsDouble $i $j $k 0] puts "fiducial $id value $pixel at $i $j $k" }

32. National Alliance for Medical Image Computing http://na-mic.org Reader Modules Motivation vtkMrmlVolumeNodes can be created by –User Interface –Reading in XML File –Programmatically Once the Node exists, the reader proc provides the vtkImageData when called by MainUpdateMRML

33. National Alliance for Medical Image Computing http://na-mic.org Reader Modules Managed by Base/tcl/tcl-modules/Volumes.tcl ReaderModule needs an entry point proc named Vol Init –For example: VolGenericInit –This will be invoked by VolumesInit ReaderModule sets –Volume(readerModules,,procGUI) to fill in type-specific controls as sub-pane of Volumes Modules – Module(Volumes,readerProc, ) so that MainUpdateMRML has a hook to call the reader when if found in the FileType attribute of a vtkMrmlVolumeNode

34. National Alliance for Medical Image Computing http://na-mic.org Editor Modules Motivation The Editor manages manually guided segmentation of a –source volume into a working labelmap Editor Modules add to the palette of tools Ed(editor) is an instance of vtkImageEditorEffects –Allows you to apply a filtering pipeline to modify the labelmap

35. National Alliance for Medical Image Computing http://na-mic.org Editor Modules Similar to Readers, these are managed by Base/tcl/tcl- modules/Editor.tcl Editor Module needs an entry point proc named Ed Init –For example: EdWatershedInit –This will be invoked by EditorInit EditorModule sets –Ed(,procGUI) to fill in type-specific controls on the this editor type’s own subframe of the editor Segmentation is done In the UI control callbacks –Create a vtk (or vtkITK-based) processing pipeline of vtkImageToImageFilter subclasses –Set parameters from UI elements or Slicer data such as Fiducial locations –call Ed(editor) Apply –Ed(editor) manages progress bar, updating label map, undo (single)

36. National Alliance for Medical Image Computing http://na-mic.org Conclusions Slicer provides a usable and extensible application framework for medical image computing research A significant base code provides a reference for many common coding tasks and user interface paradigms

37. National Alliance for Medical Image Computing http://na-mic.org Resources www.slicer.org www.na-mic.org/Wiki www.na-mic.org/Bug www.na-mic.org/Testing slicer-devel@bwh.harvard.edu slicer-users@bwh.harvard.edu

38. NA-MIC National Alliance for Medical Image Computing http://na-mic.org Slicer Architecture Supplement Adding A New View Option Nicole Aucoin

39. National Alliance for Medical Image Computing http://na-mic.org Adding a new option example To add a new View option, X, to the Scene Options node so can save and restore it via Mrml file: cxx/vtkMrmlSceneOptionsNode.h –add vtkGet/SetStringMacros(ViewX) –declare char *ViewX cxx/vtkMrmlSceneOptionsNode.cxx –Add ViewX in constructor, Write, Copy, PrintSelf

40. National Alliance for Medical Image Computing http://na-mic.org Adding a new option, con’t tcl/tcl-main/MainView.tcl –MainViewInit: add to Module(View,presets) list, set View(X) to same default value. –Add a new proc MainViewSetX, with default arg set to default value –MainViewStorePresets: add setting X's Preset from current value –MainViewRecallPresets: add setting X from preset value

41. National Alliance for Medical Image Computing http://na-mic.org Adding a new option, con’t tcl/tcl-main/MainOptions.tcl –MainOptionsRetrievePresetValues: set Preset(View,n,X) from the node –MainOptionsUnparsePresets: set the node's X from the Preset

42. National Alliance for Medical Image Computing http://na-mic.org Adding a new option, con’t tcl/tcl-main/MainMrml.tcl: – MainMrmlBuildTreesVersion2.0: add to parsing of SceneOptions node, refer to vtkMrmlSceneOptionsNode C++ code