1. 3D Slicer Architecture and Implementation
You’ve just seen the why and the what, now we’ll discuss the how…
Steve Pieper, PhD

2. Goals
NA-MIC Kit: Software and Methodologies for Medical Image Computing
Facilitate Research
Promote Interoperability
Stable, Cross-Platform Run Time Environment
Full Set of Core Features
Avoid Duplicated Effort
Flexible Module Architecture
Plug-ins should be As Simple As Possible

3. Overview MRML, Logic, GUI Core Libraries and Dependencies
Some Details of MRML
Modules

4. Slicer3 “Observer MVC” Pattern
MRML (Model)
For Scene Description and Application State
MRML Nodes are Persistent and Undoable
Scene and Nodes are Observable
Logic Encapsulate VTK and ITK Pipelines (Controller)
Observe MRML to Configure Pipelines
Help Create/Manage Nodes
No UI Components (no Widgets, Actors, Mappers, Renderers or RenderWindows)
GUI (View)
Observe and Edit MRML
Interact with User and Display Hardware
Modules Should Follow Same Conventions
GUI
Renderers
Widgets
Edit
Observe
Logic
Observe
Edit
Observe
Edit
MRML Nodes
Motivation for this architecture: modularize! Make the parts resuable.
MVC is model/view/controller – often used as an idealized design pattern, but almost always gets customized for any give application
“Observe” means generic event mechanisms are used to pass information.
“Edit” means code can directly call methods.
Example: GUI can call methods in Logic classes, but Logic cannot call GUI methods.
MRML cannot call Logic or GUI methods.
There can be many observers for any event.

5. Example: EM in Slicer3 MRML Logic GUI Manage MRML nodes
API for access to parameters
Manage hierarchy
Segmentation algorithm
Window to parameter set
Wizard
Interaction with images
Global Parameters
Hierarchical parameters
Image data
Segmentation output

6. Example: EM Batch Tool MRML Logic CLI Manage MRML nodes
API for access to parameters
Manage hierarchy
Segmentation algorithm
Command Line Parsing
Load Parameter Description MRML Scenes
Global Parameters
Hierarchical parameters
Image data
Segmentation output

7. Library Dependencies ITK VTK Tcl/Tk Teem KWWidgets vtkTeem vtkITK MRML
SlicerBaseLogic
SlicerBaseGUI
Slicer3

8. vtkITK, vtkTeem
Slicer3
Teem: Multidimensional Raster Image Library (Gordon Kindlmann)
Includes NRRD File Format
ExtensiveTensor Manipulation Utilities
vtkTeem, vtkITK: Create VTK Compatible Filters using ITK and Teem Code for Implementation
Includes vtkITKArchetype* Readers and Writers Implemented with ITK I/O Factories
SlicerBaseGUI
SlicerBaseLogic
MRML
KWWidgets
vtkTeem
vtkITK
Tcl/Tk
VTK
Teem
ITK

9. MRML MRML: Medical Reality Markup Language
Slicer3
MRML: Medical Reality Markup Language
Library Provides Central Data Representation for Slicer3
Application State is Explicitly Described in MRML
GUI and Modules:
Observe MRML to Learn of Changes to State
Manipulate State to Reflect User Interaction and Calculation Results
SlicerBaseGUI
SlicerBaseLogic
MRML
KWWidgets
vtkTeem
vtkITK
Tcl/Tk
VTK
Teem
ITK

10. SlicerBaseLogic
Slicer3
Utility Code for Implementing Base Application Functionality
Input/Output
Volume Reslicing
Image Filters
No Rendering, and No GUI Code
SlicerBaseGUI
SlicerBaseLogic
MRML
KWWidgets
vtkTeem
vtkITK
Tcl/Tk
VTK
Teem
ITK

11. SlicerBaseGUI KWWidgets Subclasses vtkSlicerApplication
Custom KWWidgets
Rendering Utility Code
VTK 3D Widget Subclasses
SlicerBaseGUI
SlicerBaseLogic
MRML
KWWidgets
vtkTeem
vtkITK
Tcl/Tk
VTK
Teem
ITK

12. Scene Description
Provided by D. Gering

13. MRML Concepts Node Unit of Organization for MRML
Group of Variables Representing an Object or Concept State
Scene
Slicer Application has Single Scene Instance that has Collection of Nodes and API for Access/Create/Delete
Undo/Redo & Scene Snapshots
Scene Swaps Nodes in to / out of Current State
Logic/GUI Auto-Update through Observers
Serialization / Deserialization
Nodes Responsible for Read/Write of XML Version of State
Scene Read/Write Analogous to Snapshots

14. MRML Concepts (cont) Observers / Events
Scene and Nodes Invoke Custom Observable Events for Specific Actions (e.g. NodeAddedEvent) or ModifiedEvents for General Updates
Node API Allows Disabling ModifiedEvent Invocation to Allow Groups of Operations with Single ModifiedEvent
Node API Includes MRMLObserverManager Class to Simplify Addition/Removal of Multiple Event Observers
Scene Provides EventBroker to Support Observer Introspection, Asynchronous Invocation, Logging, and Event Compression

15. MRML Concepts (cont) IDs & References
Each Node is Given a Unique ID for Retrieval in Scene
Nodes Can Refer to Other Nodes by ID
E.g. TransformableNodes can Refer to a TransformNode by ID
E.g. ModelNode can Refer to ModelDisplayNode by ID
Scene Maintains ReferencedIDs and ReferencingNodes Lists to Maintain One-to-One Relationship During Scene Import

16. Node Types Displayable/Storable/Transformable
Display/Storage/Transform
Hierarchy
Parameter
Selection
Slice
SliceComposite
Color
Fiducial
Snapshot
Command Line Module
Other Module-Specific Nodes

17. “Observer MVC” Example
Event: User Picks Add Volume in Volumes GUI (KWWidget)
Volumes GUI calls AddArchetypeVolume in VolumesLogic
Volumes Logic Creates VolumeNode and VolumeStorageNode, Reads Data, and Adds to Scene
Scene Invokes NodeAddedEvent
NodeSelector Widgets Update Menus
Volumes GUI Sets Active Volume on SelectionNode
Slice Viewers Updated to New Volume
GUI
Renderers
Widgets
Edit
Observe
Logic
Observe
Edit
Observe
Edit
MRML Nodes
Motivation for this architecture: modularize! Make the parts resuable.
MVC is model/view/controller – often used as an idealized design pattern, but almost always gets customized for any give application
Notes: Event invocation happens synchronously in Slicer 3.2, meaning volume is read and updated before AddArchetypeVolume method returns. Asynchronous event processing may become the default in the future.
GUI Currently propagates selection to each slice composite node in the scene. In the future the slice GUIs should be observing the scene and optionally display newly loaded volumes.

18. Coordinate Systems IJK (Index Coordinates of Volume)
vtkMRMLVolumeNode uses vtkImageData to store voxels, with Origin 0,0,0 and Spacing 1,1,1
RAS (Right-Anterior-Superior)
vtkMRMLVolumeNode::GetIJKToRASMatrix provides vtkMatrix4x4 that includes origin, spacing, and direction vectors to map to patient space
Origin is at the center of the voxel
Vectors Point in Specified Direction (i.e. R goes from Left to Right)
LPS (Left-Posterior-Superior)
Used by ITK code and DICOM
Slicer Automatically Converts RAS->LPS

19. Transforms TransformToParent
vtkMRMLTransformableNode Instances Contain TransformNodeID Reference to a vtkMRMLTransformNode
Allows Many Nodes Inside Each Transform and Transform Hierarchies
TranfrormNode Defines how to Go from Child to Parent Space (e.g. a ModelNode points to a TransfromNode that Scales it to Global RAS Coordinates)
Transforms Represented by vtkGeneralTransforms (linear/nonlinear)
(TransformFromParent – not yet implemented)
Nonlinear Registration Results Often Better Represented as Resampling Transforms, Opposite of Current Modeling Transforms

20. Modularity Goals Keep the base package “lean and mean”
Modules have individual identity
Per-module web site, svn, downloads, mailing lists, wiki…
Allow users to assemble their own set of tools
Customized ‘Bundles’ by task or application
Easy to download compatible extensions
Analogous to Firefox extensions
Integrate extension builds into developer/nightly/release processs
NITRC Supplement to NA-MIC helping to pay for needed infrastructure (Neuroimaging Informatics Tools and Resources Clearinghouse)
NITRC can host neuroimaging projects (gforge implementation)

21. Base Features vs. Modules
Visualization
MRML: Models, Volumes, Volume Rendering, Fiducials, Continuum Meshes, Labeled Data
Can create custom MRML Nodes and behavior
Filtering
None
Implemented as Modules using ITK or other Libraries
Registration
Transform Display and Edit, Save/Restore
Calculate Transforms, Resample Data
Segmentation
Label Maps, Parcellated Surfaces
Segmentation Algorithms in ITK or other Libraries
Quantification
Label, Image, Volume Statistics; Numpy access to MRML
Applications in Python or MATLAB
Real-time Integration
VTK Rendering, KWWidgets framework, Tracker Support (as Transforms)
Direct Manipulation of the MRML Scene; 2D/3D Widgets; Device Interfaces, OpenIGTLink Module
Diffusion Imaging
DWI, DTI, Fiber Bundles
Tractography, Clustering, Atlases
Applications
“Bundles” of Modules in Distribution: Registration, Editor, some Filters,
Customized Extensions, Domain-specific code, Optimized Interfaces…

22. Module Terminology Built-In Modules
Few Core Features Linked to Slicer3.cxx
Non-Core Modules becoming Loadable
Loadable Modules
Dynamically Discovered and Loaded
Can Access Application’s MRML Scene Instance and Call SlicerBaseGUI and Related APIs (vtkRendering, KWWidgets…)
Scripted Modules
Like Loadable, but Written in Tcl or Python
Command Line Modules
Can Run as Stand Alone Executables
Provide Standard Command Line Parsing, which Allows Slicer to Generate GUI Automatically
Can be Compiled as Shared Library for Dynamic Link to Slicer

23. Loadable Modules Create vtkSlicerModuleGUI Subclass
Create vtkSlicerModuleGUI Subclass
Override Virtual Methods for CreateGUI, ProcessMRMLEvents, etc.
Provide Custom vtkMRMLNode Subclass(es) to Represent, Save and Restore state
Create Logic Classes to Implement Module Functionality
Create CMake Configuration Files to Build and Install Shared Libraries for Runtime Discovery and Packaging

24. Scripted Modules
A Generic ScriptedModule Module Provides Mapping of C++ Method Entry Points to a Corresponding Naming Convention for Scripts
Scripts are Dynamically Discovered at Runtime
A Generic ScriptedModuleNode Allows Save/Restore of Keyword / Value Pairs to Represent Module State
Full API of Slicer, MRML, KWWidgets, VTK, available (except where not wrappable; these APIs have been designed for scriptability)
Editor is extensive Tcl-based Module
Several Python examples are included in current svn trunk
Python modules can also access Numpy routines for Numerical processing (Volumes and other DataArrays are directly mapped to Numpy arrays for easy manipulation)

25. Command Line Modules
Use the GenerateCLP command (provided with Slicer) to convert XML description of program arguments into a C++ .h file
Main program of module uses PARSE_ARGS macro provided by .h file to instance variables corresponding to parameters defined in XML file
PARSE_ARGS also defines a --xml argument that returns original XML string on stdout
Slicer parses the XML to create GUI for each module
Are Covered Extensively in the HelloWorld Tutorial
Note: Command Line Modules can be written in any language. C++ utilities are provided, but can be emulated / recreated as needed.