1. SimITK and SimVTK: ITK and VTK in Simulink DG Gobbi, P Mousavi, KM Li, J Xiang, A Campigotto, A LaPointe, G Fichtinger, P Abolmaesumi Medical Image Analysis Laboratory (MedIA) http://www.media.queensu.ca School of Computing Queen’s University

2. Motivation Visual programming for ITK and VTK Visual programming for ITK and VTK Drag & drop approach to application development Drag & drop approach to application development Increase understandability & maintainability Increase understandability & maintainability Instructional value Instructional value MATLAB is our friend MATLAB is our friend Use MATLAB’s Simulink visual environment Use MATLAB’s Simulink visual environment Hence, avoid reinventing the wheel Hence, avoid reinventing the wheel Allow mix of MATLAB code and ITK / VTK Allow mix of MATLAB code and ITK / VTK

3. Related Work Visual Programming of VTK and ITK Visual Programming of VTK and ITK ITKBoard (CSIRO) ITKBoard (CSIRO) SciRun (Utah) SciRun (Utah) MeVisLab (MeVis Research GmbH) MeVisLab (MeVis Research GmbH) caBIG XIP (caBIG, Siemens, Washington U. in St Louis) caBIG XIP (caBIG, Siemens, Washington U. in St Louis) ITK and VTK in MATLAB (without Simulink) ITK and VTK in MATLAB (without Simulink) MATITK (Chu and Hamarneh) MATITK (Chu and Hamarneh) VTK plus MATLAB – various VTK plus MATLAB – various Us: Visual ITK/VTK Programming in MATLAB Us: Visual ITK/VTK Programming in MATLAB

4. MATLAB’s Simulink Environment LibraryModel

5. Our Own Simulink Libraries

6. 3D Processing with ITK

7. Combine ITK Processing and VTK Rendering Input: cthead1.png Output: VTK Window

8. Method: Wrapping Process Simulink wrappers for ITK, VTK Simulink wrappers for ITK, VTK Each Simulink block is an “S-Function” with callbacks Each Simulink block is an “S-Function” with callbacks S-Function callbacks are C++ code, calls ITK or VTK S-Function callbacks are C++ code, calls ITK or VTK CMake build process CMake build process Start with an XML description of each filter (ITK) Start with an XML description of each filter (ITK) or Start with class header files (VTK) or Start with class header files (VTK) CMake calls perl script to generate C++ code CMake calls perl script to generate C++ code CMake then calls perl to generate.mdl library file CMake then calls perl to generate.mdl library file Finally, build libraries with VC++ or gcc Finally, build libraries with VC++ or gcc

9. Method: Data Flow for SimITK SimITK, SimVTK use different architecture SimITK, SimVTK use different architecture ITK is primarily image based ITK is primarily image based VTK handles many data set types VTK handles many data set types SimITK Data Flow SimITK Data Flow ITK images -> MATLAB arrays (“info” and “data”) ITK images -> MATLAB arrays (“info” and “data”) Use shared memory for efficiency Use shared memory for efficiency Conversion of ITK parameter data types Conversion of ITK parameter data types ITK uses several specialized types e.g. versor ITK uses several specialized types e.g. versor

10. SimITK Helper Class InputPort Conversion itkImage Run ITK Image Filter itkImage Conversion OutputPort Parameter Variables Conversion Spacing & Origin Data Spacing & Origin Data Input Output Parameters

11. Method: Data Flow for SimVTK SimVTK Data Flow SimVTK Data Flow Use MATLAB arrays to pass vtkObject pointers Use MATLAB arrays to pass vtkObject pointers Define “VTKOBJECT” MATLAB type for safety Define “VTKOBJECT” MATLAB type for safety When run starts, Simulink builds VTK pipeline on-the-fly When run starts, Simulink builds VTK pipeline on-the-fly Then, Simulink calls Update or Render at each time step Then, Simulink calls Update or Render at each time step SimVTK filter parameters SimVTK filter parameters VTK parameter types need no conversion VTK parameter types need no conversion VTK and MATLAB both use basic C data types VTK and MATLAB both use basic C data types

12. Registration Examples Registration in ITK Registration in ITK Immediate, interactive feedback Immediate, interactive feedback Great way to tune parameters Great way to tune parameters Easy to swap out components Easy to swap out components

14. Intervention Examples Computer Assisted Intervention in VTK Computer Assisted Intervention in VTK Incorporate real-time data input Incorporate real-time data input Use “Subsystems” to manage complexity Use “Subsystems” to manage complexity

17. Discussion “Diagrams” that actually run as code “Diagrams” that actually run as code Easy to verify that code matches design Easy to verify that code matches design Great teaching tool Great teaching tool Understandability improves maintainability Understandability improves maintainability Utility Utility Simulink models can run as stand-alone exec Simulink models can run as stand-alone exec Nearly all VTK, ITK classes can be wrapped Nearly all VTK, ITK classes can be wrapped

18. Future Work WrapITK WrapITK To automate ITK build (VTK is already done!) To automate ITK build (VTK is already done!) GUI integration GUI integration Slicer module interface in Simulink Slicer module interface in Simulink KWWidgets (later Qt, maybe FLTK) KWWidgets (later Qt, maybe FLTK) Community Building Community Building Mailing list, website Mailing list, website We want to hear from you! We want to hear from you!