1. NA-MIC National Alliance for Medical Image Computing http://na-mic.org Slicer 3 Update for Core 1 NA-MIC Algorithms Meeting UNC May 23, 2006

2. National Alliance for Medical Image Computing http://na-mic.org Why Slicer3? Address Longstanding “Legacy” Issues –Cores 1, 3: Simplify coordinate systems used in processing, interaction, and display –Core 3: Increase UI Flexibility, Improve Look & Feel –Core 2: Modularize the Base for Reusability Simplify Module Writing –Core 1: Define interface to data model –Core 1: Allow variety in programming styles –Core 1, 3: Allow Modules to be usable with scripts, grid, pipeline

3. National Alliance for Medical Image Computing http://na-mic.org Slicer3 Opportunities Core 1 –Provide Platform for Next Generation Algorithms Core 2 –Leverage NA-MIC Kit and Engineering Methodology Core 3 –Work with DBP Users on Functionality and Usability

4. National Alliance for Medical Image Computing http://na-mic.org Slicer3 Constraints Limited Resources for Implementation Preserve Investment by Users and Developers in Existing Slicer Modules Adhere to NA-MIC Licensing Goals

5. National Alliance for Medical Image Computing http://na-mic.org Slicer3 Collaborating Resources NA-MIC Core 2 NAC (P41 at BWH) –Neuroimaging Analysis Center BIRN –Biomedical Informatics Research Network (function and morphometry) NCIGT (U41 at BWH) –National Center for Image Guided Therapy Several other collaborating projects contributing to Base and Modules

6. National Alliance for Medical Image Computing http://na-mic.org Slicer3 Plan Re-implement Base Layer –Slice/Volume/Model Viewing –Interaction, Events, Usability –Module Interface –Data Model Port Slicer2 Modules –EMSegmenter, fMRIEngine/Ibrowser, DTMRI, ITK Registration Integrate New Modules

7. National Alliance for Medical Image Computing http://na-mic.org Slicer3 Timeline July 2005 –Kickoff, Concept/Planning at Programmer Week January 2006 –Beginning of New Repository March 2006 –“Alpha” Prototype of New Architecture July 2006 –Alpha2/Beta1: testing with Core 3 November 2006 –Beta2 January 2007 –3.0 Release to NA-MIC at AHM

8. National Alliance for Medical Image Computing http://na-mic.org Slicer3 Software Environment CMake / CTest / CPack (Version 2.4) Dart (Version 2) VTK (5) ITK (2.6) KWWidgets (CVS Head) Teem (1.9)

9. National Alliance for Medical Image Computing http://na-mic.org Slicer3 Engineering Tools Dart (version 2) CMake (version 2.4) –CTest –CPack subversion phpBugTracker MediaWiki Your Favorite Editor and Compiler

10. National Alliance for Medical Image Computing http://na-mic.org User Desktop Slicer 3.0 Architecure AlgorithmsITKVTK Slicer Modules VTK Apps Using ITK Scripts of Slicer Mods Batch Programs Non-NAMIC Cmd tools LONI Pipeline Birn Grid Data/Compute Slicer 3.0

11. National Alliance for Medical Image Computing http://na-mic.org Slicer3 Base Programming MRML, Logic, and KWWidgets all C++ to Maximize Reusability (callable from C++) –Base classes are all wrapped and accessible from Tcl at runtime MRML is a Library that can be used Independent of Slicer3 –Uses VTK Object Model (Reference Counting, Observer Pattern, Wrapping…) –Relies on ITK ImageIO and VTK PolyData IO GUI KWWidgets are VTK-Like C++ wrapping of Tcl/Tk –KWWidgets are wrapped and callable from C++, Tcl, Python –KWWidgets provide Application level support (Main Window, Menus, Registry…) –VTK-specific Widgets (RenderWidget, Transfer Function Editor, Material Properties Editor…) –GUI separated from MRML and Logic to support possible alternate GUIs

12. National Alliance for Medical Image Computing http://na-mic.org Slicer3 Module Programming Many Options to Implement Modules –Pure C++ Libraries vtkITK Library –Pure Tcl code –C++/Tcl packages (Slicer2 Modules) –External Executables (more later on this topic) –Loosely Coupled Modules Built on Other Platforms (teem, FreeSurfer, MATLAB, etc)

13. National Alliance for Medical Image Computing http://na-mic.org Slicer3 “Observer MVC” Pattern MRML (Model) –For Scene Description and Application State –MRML Nodes are Persistent and Undoable –Scene and Nodes are Observable Logic Encapsulate VTK and ITK Pipelines (Controller) –Observe MRML to Configure Pipelines –No UI Components (no Widgets, Actors, Mappers, Renderers or RenderWindows) GUI (View) –Observes and Edits MRML –Interacts with User and Displays Logic Outputs Logic MRML Nodes GUI WidgetsRenderers Edit Observe

14. MRML Base Logic Links to: ITK MRML VTK (except Rendering) GUI Links to: VTK Rendering KWWidgets Observe Modified Events Edit Mrml Scene/Nodes Mediators Interactive Module Logic Links to: Base Logic ITK MRML VTK (except Rendering) GUI Links to: VTK Rendering KWWidgets Mediators Observe Modified Events on Application State Edit Mrml Scene/Nodes Registers GUI Event Observers Observe Modified Events Provides Frames for Widgets, Routing of User Events Execution Model Manager Logic Links to: Base Logic GUI Links to: KWWidgets Mediators External Processes (Grid/Pipeline) Speak MRML (link to libmrml or have data staged by slicer) Uses the XML interface (Managed by Base and communicats like other Modules) Describe Input options Formulates Command Line and possibly stages data MRML Aware Processes Can Edit Mrml Scene/Nodes and Observe Modified Events Slicer 3 Architecture Diagram (2006-01-26) Auto-Generated GUIs from XML Descriptions Notes: All classes in the Logic directory should be able to run ‘headless’ without OpenGL or window system for scripting and testing Base/Logic contains transient application state (cursor location, focus, mrml scene connection…) Interactive Modules are ones which interact with the VTK scene and/or User events Interactive Modules interact with 3D scene by creating objects in MRML scene (not by direct manipulation of the Renderer) Logic classes encapsulate and manage internal vtk/itk pipelines GUI classes are implemented as KWWidget subclasses Each Logic class defines a set of Get/Set methods for internal state and Modified Events that GUI classes can Observe

15. data layerdata mediatorlogic layergui mediatorgui layer Main program; Bootstrap application logic, Create application gui MRML data model VTK pipeline Logic code adds modified event observers to MRML scene and MRML nodes. Logic methods apply changes to the MRML scene. (VTK, ITK, no KWWidgets, no VTKRendering) vtkObject vtkSlicerLogic - Undo manager, - tracing vtkSlicerApplicationLogic - MRML slice # - “transient state” - open views, slices, modules (and active) - cursor location - version - focus, event routing vtkSlicerViewLogic vtkSlicerLinkedSliceLogic vtkSliceLogic - vtkImageReslice - vtkImageComposite vtkSlicerSliceLayerLogic - MRML node - Opacity - Compositing - Window/level vtkSlicerViewLogic vtkSlicerModuleLogic - Undo/tracing helpers vtkSlicerEditorLogic vtkSlicerMRMLLogic vtkSlicerCustomModuleLogic GUI code adds observers to modified events generated by/to logic Apply methods from GUI classes call set method in logic to change state vtkKWWidget vtkSlicerGUI - look & feel? - set command helpers? vtkSlicerApplicationGUI - connect to window system - vtkKWApplication - vtkKWWindow vtkSlicerViewGUI - Render windows - mappers, actors vtkSlicerSliceGUI - vtkImageMapper - vtkKWRenderWindow vtkSlicerSliceLogicGUI vtkSlicerModuleGUI vtkSlicerCustomModuleGUI

16. National Alliance for Medical Image Computing http://na-mic.org Slicer3 New Features ITK IO Framework (Archetype Based) Multiple Undo/Redo Modules as Executables –For Grid and Pipeline Use Object Oriented Module Framework New UI Framework Extensive Testing Framework Much More…

17. National Alliance for Medical Image Computing http://na-mic.org Core 3 Feature Requests Image Editor!!! –Efficient Manual ROI Creation Unified Save –Summary view of the state of the scene –Checkbox the items you want to save “Photoshop Layers” Style Volume Viewing “Module Dock” of Frequently Used Modules Core 1 ideas?

18. National Alliance for Medical Image Computing http://na-mic.org Slicer3 Projects Architecture/Features (Steve Pieper, Group: Bill Lorensen, Ron Kikinis, Mike Halle, Noby Hata)Architecture/Features Data Model (Steve Pieper, Alex Yarmarkovich, Xiaodong Tao, Mathieu Malaterre)Data Model Execution Model (Dan Blezek, Bill Lorensen, Mike Halle, Jim Miller)Execution Model Visualization and UI Infrastructure (Mathieu Malaterre with Sebastien Barre, Wendy Plesniak, Mike Halle)Visualization and UI Infrastructure Interface Design and Usability (Wendy Plesniak, Mike Halle, Sebastien Barre)Interface Design and Usability Transition of Selected Slicer2.x Modules to Slicer3 (Nicole Aucoin, Katie Hayes, Wendy Plesniak, Mathieu Malaterre)Transition of Selected Slicer2.x Modules to Slicer3 Build/Test/Deploy System (Andy Cedilnik)Build/Test/Deploy System Pipeline Integration (Mike Pan, Jags Rajendiran)Pipeline Integration Grid Integration (Jeff Grethe, Brendan Faherty)Grid Integration

19. National Alliance for Medical Image Computing http://na-mic.org Transition of Slicer 2.x Modules: Goals Slicer 2: –Port C++ classes to compile against VTK 5.x Maintain backward compatibility with VTK 4.x –Edit TCL interface to properly call new class versions –Write test cases to define gold standard Slicer 3: –Port threads of control from TCL to C++ –Implement dynamic discovery and loading of modules

20. National Alliance for Medical Image Computing http://na-mic.org Transition of Slicer 2.x Modules: Progress Slicer 2: –Nightly builds against VTK5 –TCL revisions underway –Many tests written and run nightly

21. National Alliance for Medical Image Computing http://na-mic.org Slicer 3 Execution Model: Goals Provide interface to batch programs –Simplify and unify command line processing Auto generate C++ command line processing –Simplify and unify GUI Auto generate GUI

22. National Alliance for Medical Image Computing http://na-mic.org Execution Model: Progress XML Interface Description –Describes command line options –Describes suggestions for GUI GenerateCLP –Generates C++ command line processing code from XML –Reports XML description

23. National Alliance for Medical Image Computing http://na-mic.org Program Flow User Desktop prog.xmlGenerateCLPprogCLP.h Slicer Modules prog.cxx Non-NAMIC Cmd tools LONI Pipeline Birn Grid Data/Compute Slicer 3.0

24. National Alliance for Medical Image Computing http://na-mic.org Resample --xml filtering Resample Volume Resamples a volume 1.0 Bill Lorensen Resampling Parameters Parameters used for resampling outputPixelSpacing -s --spacing Spacing along each dimension (0 means use input spacing) Spacing 0,0,0 IO Input/output parameters InputVolume input Volume input 0 Input volume to be resampled OutputVolume output 1 Resampled Volume

25. National Alliance for Medical Image Computing http://na-mic.org Resample.cxx. #include “ResampleCLP.h” Main (int argc, char *argv[]) { PARSE_ARGS; algorithm code }

26. National Alliance for Medical Image Computing http://na-mic.org Resample --help USAGE: ResampleVolume [--xml] [--echo] [-s >] [--] [--version] [-h] Where: --xml Produce xml description of command line arguments (default: 0) --echo Echo the command line arguments (default: 0) -s >, --spacing > Spacing along each dimension (0 means use input spacing) (default: 0,0,0) --, --ignore_rest Ignores the rest of the labeled arguments following this flag. --version Displays version information and exits. -h, --help Displays usage information and exits. (required) Input volume to be resampled (default: None) (required) Resampled Volume (default: None) Resamples a volume

27. National Alliance for Medical Image Computing http://na-mic.org Slicer Daemon Slicerd listens on local socket slicerget –gets volume and writes to stdout in nrrd slicerput [name] –reads nrrd on stdin and puts in slicer % slicerget 0 | unu 1op exp - | slicerput

28. National Alliance for Medical Image Computing http://na-mic.org ITK IO and Slicer Daemon AntiAliasBinaryImageFilter.exe bunny.z.nrrd "| sh slicerput.tcl"

29. National Alliance for Medical Image Computing http://na-mic.org Undo Architecture for Slicer3 Versioning “Commit Aside” Strategy Encapsulated within MRML Store “Initial Scene” (from file) and “Delta Scenes” (scenes containing undoable changes) Delta Scenes ‘are’ MRML Scenes Some Nodes are Reference Nodes Setting the MRML scene in the Application Logic causes the cascade of observer callbacks => All Undoable operations must store their data as MRML nodes

30. National Alliance for Medical Image Computing http://na-mic.org Undo 1 st Pass Initial Scene Application Logic N1 N2 N3 Delta1 Scene R N4 R Delta2 Scene R R N5 Saved Scene N1 N4 N5 Undo Stack

31. National Alliance for Medical Image Computing http://na-mic.org Undo Current Initial Scene MRML Scene N1 N2 N3 Delta1 Scene N4 Delta2 Scene N5 Saved Scene N1 N4 N5 Undo Stack

32. National Alliance for Medical Image Computing http://na-mic.org Slicer3 Slice Coordinates XY Image Space Coordinates In RenderWindow Slice Coordinates RAS to RAS Coordinates IJK Volume Coordinates XYToSlice Matrix SliceToRAS Matrix RASToRAS Transforms RASToIJK Matrix IJK = RASToIJK * RASToRAS * SliceToRAS * XYToSlice * XY SliceNode XYToIJK Matrix Calculated by SliceLayerLogic Coordinates are Cell Centered (no VTK flip) From GUI IJK = XYToIJK * XY TransformNodeVolumeNode

33. National Alliance for Medical Image Computing http://na-mic.org Slice Management Foreground Slice Layer Logic Slice Node Slice Composite Node Background Slice Layer Logic Slice Logic Volume Display Node Volume Node Volume Display Node Volume Node Slice GUI MRML Nodes: Persistent, Undoable State Logic: Encapsulate VTK Pipelines, Observe MRML Nodes GUI: Manage Widgets and Rendering; Manipulated Nodes and Logic; Observe Nodes and Logic

34. National Alliance for Medical Image Computing http://na-mic.org Slice Management Foreground Slice Layer Logic Slice Node Slice Composite Node Background Slice Layer Logic Slice Logic Volume Display Node Volume Node Volume Display Node Volume Node Slice GUI MRML Nodes: Persistent, Undoable State Logic: Encapsulate VTK Pipelines, Observe MRML Nodes GUI: Manage Widgets and Rendering; Manipulated Nodes and Logic; Observe Nodes and Logic Stores: Window/Level Lookup Table Transparency Threshold

35. National Alliance for Medical Image Computing http://na-mic.org Slice Management Foreground Slice Layer Logic Slice Node Slice Composite Node Background Slice Layer Logic Slice Logic Volume Display Node Volume Node Volume Display Node Volume Node Slice GUI MRML Nodes: Persistent, Undoable State Logic: Encapsulate VTK Pipelines, Observe MRML Nodes GUI: Manage Widgets and Rendering; Manipulated Nodes and Logic; Observe Nodes and Logic Stores: RAS Location of Slice (Origin and Orientation) Field of View Pixel Dimensions

36. National Alliance for Medical Image Computing http://na-mic.org Slice Management Foreground Slice Layer Logic Slice Node Slice Composite Node Background Slice Layer Logic Slice Logic Volume Display Node Volume Node Volume Display Node Volume Node Slice GUI MRML Nodes: Persistent, Undoable State Logic: Encapsulate VTK Pipelines, Observe MRML Nodes GUI: Manage Widgets and Rendering; Manipulated Nodes and Logic; Observe Nodes and Logic Stores: Volume IDs per Layer Opacity of Overlay

37. National Alliance for Medical Image Computing http://na-mic.org Slice Management Foreground Slice Layer Logic Slice Node Slice Composite Node Background Slice Layer Logic Slice Logic Volume Display Node Volume Node Volume Display Node Volume Node Slice GUI MRML Nodes: Persistent, Undoable State Logic: Encapsulate VTK Pipelines, Observe MRML Nodes GUI: Manage Widgets and Rendering; Manipulated Nodes and Logic; Observe Nodes and Logic Stores: Image Data IJKToRAS Matrix Name…

38. National Alliance for Medical Image Computing http://na-mic.org Slice Management Foreground Slice Layer Logic Slice Node Slice Composite Node Background Slice Layer Logic Slice Logic Volume Display Node Volume Node Volume Display Node Volume Node Slice GUI MRML Nodes: Persistent, Undoable State Logic: Encapsulate VTK Pipelines, Observe MRML Nodes GUI: Manage Widgets and Rendering; Manipulated Nodes and Logic; Observe Nodes and Logic Pipeline with: vtkImageReslice vtkImageMapToWindowLevelColors vtkImageMapToRGBA

39. National Alliance for Medical Image Computing http://na-mic.org Slice Management Foreground Slice Layer Logic Slice Node Slice Composite Node Background Slice Layer Logic Slice Logic Volume Display Node Volume Node Volume Display Node Volume Node Slice GUI MRML Nodes: Persistent, Undoable State Logic: Encapsulate VTK Pipelines, Observe MRML Nodes GUI: Manage Widgets and Rendering; Manipulated Nodes and Logic; Observe Nodes and Logic Pipeline with: vtkImageBlend (Annotations, Crosshairs…)

40. National Alliance for Medical Image Computing http://na-mic.org Slice Management Foreground Slice Layer Logic Slice Node Slice Composite Node Background Slice Layer Logic Slice Logic Volume Display Node Volume Node Volume Display Node Volume Node Slice GUI MRML Nodes: Persistent, Undoable State Logic: Encapsulate VTK Pipelines, Observe MRML Nodes GUI: Manage Widgets and Rendering; Manipulated Nodes and Logic; Observe Nodes and Logic GUI: SliceControllerWidget (Menus, Scale, Buttons…) SliceViewer (RenderWidget, ImageMaper, ImageActor…)

41. National Alliance for Medical Image Computing http://na-mic.org Core 3 Feature Requests Image Editor!!! –Efficient Manual ROI Creation Unified Save –Summary view of the state of the scene –Checkbox the items you want to save “Photoshop Layers” Style Volume Viewing “Module Dock” of Frequently Used Modules Core 1 ideas?

42. National Alliance for Medical Image Computing http://na-mic.org Unified Save Template Would show currently loaded data Show save state of each element User option to save each element Save Page from NA-MIC / Kitware Wiki To Web Project

43. NA-MIC National Alliance for Medical Image Computing http://na-mic.org Slicer3 user interface and usability http://www.na-mic.org/Wiki/index.php/Slicer3:Interface_Design Task: Design & engineer Slicer3 GUI. Employ user-centered approach where possible, and establish persistent usability guidelines for developers. 3 goals focused on users and developers: Enable a user to understand and effectively use the content and tools being presented; Enable a user to accomplish a principle task by following an appropriate and satisfying workflow or a curiosity-driven exploration, at an appropriate pace. Enhance and support the developer’s experience with GUI infrastructure and guidelines that simplify and clarify their work.

44. NA-MIC National Alliance for Medical Image Computing http://na-mic.org Engineering and design requirements: Separate GUI from application layer; Preserve application testability; Use consistent event handling methods and API to application logic and MRML; Provide mechanism for undo; Centrally specify application look & feel ( permitting overrides); Rethink GUI layout: keep good elements, brand elements, make improvements; Develop Slicer3 brand: logo, visual icons, splash screen, style sheets for www; Develop dialog with key user communities; Test designs of key modules with members of the user community and refine; Monitor mechanisms for user feedback, bug reporting and feature requests; Publish design guidelines for all new development. engineering design usability

45. NA-MIC National Alliance for Medical Image Computing http://na-mic.org Engineering and design requirements: Separate GUI from application layer; Preserve application testability; Use consistent event handling methods and API to application logic and MRML; Provide mechanism for undo; Centrally specify application look & feel ( permitting overrides); Rethink GUI layout: keep good elements, brand elements, make improvements; Develop Slicer3 brand: logo, visual icons, splash screen, style sheets for www; Develop dialog with key user communities; Test designs of key modules with members of the user community and refine; Monitor mechanisms for user feedback, bug reporting and feature requests; Publish design guidelines for all new development. engineering

46. NA-MIC National Alliance for Medical Image Computing http://na-mic.org Tk options database manages look & feel GUI layer architecture and interface to application layer: Base classes defined and implemented. Main application GUI development underway. Module GUI development underway.

47. NA-MIC National Alliance for Medical Image Computing http://na-mic.org Engineering and design requirements: Separate GUI from application layer; Preserve application testability; Use consistent event handling methods and API to application logic and MRML; Provide mechanism for undo; Centrally specify application look & feel ( permitting overrides); Rethink GUI layout: keep good elements, brand elements, make improvements; Develop Slicer3 brand: logo, visual icons, splash screen, style sheets for www; Develop dialog with key user communities; Test designs of key modules with members of the user community and refine; Monitor mechanisms for user feedback, bug reporting and feature requests; Publish design guidelines for all new development. design

48. NA-MIC National Alliance for Medical Image Computing http://na-mic.org GUI design sketches: Keep default view; Keep “slicer-blue” 3D view; Simplify and clarify module navigation; Group slice window controls, Group 3D view controls.

49. NA-MIC National Alliance for Medical Image Computing http://na-mic.org module navigation application menu module GUI panel with collapsing frames and scrollbar slice control panel 3D view control panel application status bar

50. NA-MIC National Alliance for Medical Image Computing http://na-mic.org spec mockup: identify and arrange common-use top-level functionality and anticipate “Tk-look”; prescribe widget to display “3D nav” by default, and “slice-zoom” during slice-window mouseover.

51. NA-MIC National Alliance for Medical Image Computing http://na-mic.org current status: added toolbar and re- designing module navigation populating GUI panel with KWWidgets and custom widgets; adjusting layout to clarify functional groupings of widgets; adding scrollbar to module GUI panel. designing “3D navigation / slice zoom” widget in view control panel.

52. NA-MIC National Alliance for Medical Image Computing http://na-mic.org Toolbar icon design, (ongoing) design studies for slicer3 logo

53. NA-MIC National Alliance for Medical Image Computing http://na-mic.org Engineering and design requirements: Separate GUI from application layer; Preserve application testability; Use consistent event handling methods and API to application logic and MRML; Provide mechanism for undo; Centrally specify application look & feel ( permitting overrides); Rethink GUI layout: keep good elements, brand elements, make improvements; Develop Slicer3 brand: logo, visual icons, splash screen, style sheets for www; Develop dialog with key user communities; Test designs of key modules with members of the user community and refine; Monitor mechanisms for user feedback, bug reporting and feature requests; Publish design guidelines for all new development. usability

54. NA-MIC National Alliance for Medical Image Computing http://na-mic.org ongoing dialog with local user communities; for now, keeping wiki page repository of feature and resource requests (which also incorporates appropriate slicer2 feature requests): http://www.na-mic.org/Wiki/index.php/Slicer3:FeatureRequests will schedule early user testing sessions after beta