Segmentation support in Slicer Csaba Pinter Laboratory for Percutaneous Surgery, Queen’s University, Canada

- 2 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2015 Current state in Slicer Labelmaps: scalar volume node with attribute Models Separate conversion modules

- 3 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2015 Issues Correspondence – Overlap is not allowed within a labelmap Per-structure volumes are complex and allow overlaps only temporarily – Which nodes represent the same segmentation? (consistent naming is the only option) – Multi-label labelmap and models: 1 to N

- 4 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2015 Issues cont’d Conversion – Needed because structures cannot be shown in 3D without a manual conversion to models – User needs to manually convert each node or write a script Support for DICOM Segmentation Object – Overlapping segments grouped together Smooth contours are not supported Limited grouping (labelmap: no grouping; models: one simple tree)

- 5 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2015 Objective Create Segmentation node and infrastructure that supports Both volume and model representations (and potentially more) Multiple segments (DICOM Segmentation Object) that can overlap On-demand automatic conversion between representations

- 6 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2015 Representations Several default representations and converters: Binary labelmap, Closed surface, Planar contours, Fractional labelmap (smooth brush, probabilities, etc.) Extensible: representation types and conversion algorithms can be registered Conversion graph – Nodes are representations – Edges are converters (weighted) – Automatic path search and conversion

- 7 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2015 Design #1 – nodes vtkMRMLLabelmapVolumeNode replaces attributed scalar volume node for labelmaps vtkMRMLSegmentationNode subclasses labelmap volume node – Provides a “merged labelmap” for volume display and implicit CLI support – Contains multiple segments

- 8 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2015 Design #2 vtkOrientedImageData class – Used in segments for labelmap representations – Created to keep segments MRML-independent – This should eventually replace vtkImageData throughout Slicer and eventually added to VTK. We already have a couple of modified VTK filters that support vtkOrientedImageData

- 9 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2015 Design – classes vtkMRMLSegmentationNode : vtkMRMLLabelmapVolumeNode -Provide merged labelmap -Access segmentations -Handle transforms vtkMRMLSegmentationNode : vtkMRMLLabelmapVolumeNode -Provide merged labelmap -Access segmentations -Handle transforms vtkSegmentation -Non-MRML VTK class -Handle individual segments -Common properties: -Master representation -Volume geometry vtkSegmentation -Non-MRML VTK class -Handle individual segments -Common properties: -Master representation -Volume geometry vtkSegment -Handle representations -Properties: Name, Color, Opacity, Tags (string vector) vtkSegment -Handle representations -Properties: Name, Color, Opacity, Tags (string vector) vtkSegmentationConverter -Conversion path search -Handle conversion parameter -Initiate conversion vtkSegmentationConverter -Conversion path search -Handle conversion parameter -Initiate conversion vtkSegmentationConverterRule -Conversion cost computation -Conversion -Expose conversion parameters vtkSegmentationConverterRule -Conversion cost computation -Conversion -Expose conversion parameters vtkSegmentationConverter Factory -Register/unregister converter rule vtkSegmentationConverter Factory -Register/unregister converter rule Segmentations/MRML SegmentationLib

- 10 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2015 Progress vtkMRMLLabelmapVolumeNode implemented – Only possible issues in extensions where they use comboboxes for labelmaps with show subclasses option on, but otherwise solid, integration possible. SegmentationLib and Segmentation module created in SlicerRT – Critical feature for the toolkit, we need it as soon as possible – Integration work can start afterwards

- 11 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2015 To do Phase #1 – Segment operations: add, move, remove – Display: merged labelmap, displayable manager – Conversion: graph search, conversion params – Transform: apply linear and non-linear, bounds – Subject hierarchy: individual segment handling, segmentation operations (convert, extract) – Editor: per-segment box, extract then write back Detailed plans at