NA-MIC National Alliance for Medical Image Computing Slicer3 EMSegment Tutorial January 2008 NAMIC All-Hands Meeting Brad Davis, Yuman Yuan, Sebastien Barre, Will Schroeder, Polina Goland, Ron Kikinis, Kilian Pohl

National Alliance for Medical Image Computing Program EM Segmenter basics Algorithm views in Slicer3 Slicer3 EM segment under the hood Tutorial –Data –Simple Segmentation Interface –Template Builder Interface Questions (all week…)

National Alliance for Medical Image Computing EM Segment Basics Figure: Pohl et al.

National Alliance for Medical Image Computing EM Segment Basics Figures: Pohl et al.

National Alliance for Medical Image Computing Atlas Images Target Images Parameters Segmentation Result EM Segment Basics GM WM CSF Air Skin/Skull

National Alliance for Medical Image Computing Atlas Images Parameters Segmentation Result EM Segment Basics Collection of images (input channels) that will be segmented T1T2 Target Images

National Alliance for Medical Image Computing Atlas Images Target Images Parameters Segmentation Result EM Segment Basics Hierarchy of anatomical structures –Intensity distributions, prior probabilities Default preprocessing –Intensity normalization, registration EM segmentation algorithm –Smoothing, weights, # iterations Stored as XML MRML file Root ICC Background AirSkin/Skull GMWMCSF

National Alliance for Medical Image Computing Target Images Parameters Segmentation Result EM Segment Basics Atlas images (spatial priors) –Aligned in common coordinate frame GM AirSkin/Skull WM CSF Atlas Images

National Alliance for Medical Image Computing Atlas Images Target Images Parameters Segmentation Result EM Segment Basics Labelmap volume –Voxel values specified in parameters Root ICC Background AirSkin/Skull GMWMCSF

National Alliance for Medical Image Computing Algorithm Development 12-year history of algorithm development –Wells et al (TMI): EM framework for simultaneous estimation of bias field and label map –Kapur et al (PhD Thesis): Model noise via Markov Random Field –Van Leemput et al (TMI): Non-spatial tissue priors –Pohl et al (MICCAI): Deformable registration to align atlas 2004 (ISBI): Hierarchical framework to model anatomical dependencies 2007 (TMI): Up-to-date, detailed description

National Alliance for Medical Image Computing Slicer3 Segmentation Code Preprocessing Wizard GUI Recent Development Effort Goals: –Easy to use –Adapts to a variety of scenarios –Is a research tool –Simplified interfaces –Minimize preprocessing requirements –Validation and testing ITK/VTK

National Alliance for Medical Image Computing EMSegmenter Logic / Preprocessing / Algorithm Slicer3 GUI EM Template Builder Module EM Simple Module Command Line EM Command Line Executable Slicer3 EM segment views Advanced interface for creating/editing segmentation parameters Simple interface for applying parameters to new images Command-line interface for script/batch processing

National Alliance for Medical Image Computing Template Builder View Runs inside Slicer3 Purpose: Build/Adapt parameter sets Run segmentation Wizard guides user through interface panels

National Alliance for Medical Image Computing `One-Click View Runs inside Slicer3 Purpose: Simple interface for segmenting new data Requires predefined parameter set

National Alliance for Medical Image Computing Command-line View EMSegmentCommandLine executable bundled with Slicer3 Purpose: Scripted/batch processing Requires predefined parameter set Specify new target/atlas images on command line Atlas Images Target Images Parameters Segmentation Result

National Alliance for Medical Image Computing Specify Inputs Parameters Target Images Atlas Images Slicer3 EM Segment Workflow

National Alliance for Medical Image Computing Specify Inputs Default Pre- Processing Parameters Target Images Atlas Images Target Image Normalization Target-to-target Registration Atlas-to-target Registration Slicer3 EM Segment Workflow

National Alliance for Medical Image Computing Specify Inputs Default Pre- Processing Parameters Target Images Atlas Images Target Image Normalization Target-to-target Registration Atlas-to-target Registration Segmentation EM Segment Algorithm: Pohl et al. Slicer3 EM Segment Workflow

National Alliance for Medical Image Computing Specify Inputs Default Pre- Processing Parameters Target Images Atlas Images Target Image Normalization Target-to-target Registration Atlas-to-target Registration Segmentation EM Segment Algorithm: Pohl et al. Review Results Slicer3 Slice Views Slicer3 Model Maker External Program Slicer3 EM Segment Workflow

National Alliance for Medical Image Computing Default Pre- Processing Target Image Normalization Target-to-target Registration Atlas-to-target Registration New Development: Default Preprocessing Lower bar for entry Starting point for building effective pipeline Not intended for all scenarios No interface for complex parameter tweaking Easily bypassed by advanced users

National Alliance for Medical Image Computing Default Pre- Processing Target Image Normalization Target-to-target Registration Atlas-to-target Registration New Development: Default Preprocessing VTK filter by Kilian Pohl Scale intensities to achieve predefined mean –Heuristics determine histogram cutoffs Default settings for T1, T2 images

National Alliance for Medical Image Computing Default Pre- Processing Target Image Normalization Target-to-target Registration Atlas-to-target Registration New Development: Default Preprocessing Target image 1 is fixed image Perform registration for each additional target image –Align image centers –Rigid registration, mutual information, gradient descent optimization (ITK) –Coarse-to-fine

National Alliance for Medical Image Computing Default Pre- Processing Target Image Normalization Target-to-target Registration Atlas-to-target Registration New Development: Default Preprocessing Target image 1 is fixed image Perform registration with selected atlas image –Align image centers, Rigid registration, B-Spline registration –Mutual information, (normalized cross-correlation, sse, not yet tested) –Rigid: gradient descent, B-Spline: ITK LBFGSB optimizer, 5x5x5 knot points –Coarse-to-fine –Fast/Standard/Slow: sample points, max iterations –Resample all atlas images via transform; guess background value

National Alliance for Medical Image Computing Default Registration Results

National Alliance for Medical Image Computing Tutorial Data Walkthrough 1/3 README: extra release notes Predefined parameter file –MR T1 & T2 brain segmentation: White matter, gray matter, CSF Target Images –ImageData_Input sub directory –Target : T1 & T2 images used in parameter file –Target_NewPatient1, Target_NewPatient2 : New data to try with the parameter file

National Alliance for Medical Image Computing Tutorial Data Walkthrough 2/3 Atlas Images –ImageData_Input sub directory –Atlas : atlas images used in parameter file –Atlas_NewAtlas : Extra atlas data for demonstration purposes (currently a copy of first atlas)

National Alliance for Medical Image Computing Tutorial Data Walkthrough 3/3 Low resolution data for testing –Full sized data: 30 minutes –Low resolution data: 1 minute Data, parameters have _small appended to filename

National Alliance for Medical Image Computing `One-Click Module Tutorial 1/3 EMSegment tutorial data is required Within Slicer3, open EMSegment Simple Module

National Alliance for Medical Image Computing `One-Click Module Tutorial 2/3 MRML Scene : Select the tutorial parameter set Result Labelmap : Select a new image filename Target Volumes : Select input target volumes from one of the target directories (e.g., Target_NewPatient1 ) Note: target volume order is important Click `Apply, wait (~30 minutes for full size images)…

National Alliance for Medical Image Computing `One-Click Module Tutorial 3/3 Review the results in Slicer3 –Load target images –Load result labelmap –Use slice views to review segmentation results Apply Model Maker to generate surfaces

National Alliance for Medical Image Computing EMSegment Command-line Examples Script examples available in tutorial directory –New images –New atlas –New image and atlas –See README

National Alliance for Medical Image Computing Template Builder Tutorial EMSegment tutorial data is required Within Slicer3, open EMSegment Module

National Alliance for Medical Image Computing Template Builder Tutorial Load tutorial parameters –File->Load Scene –Select mrml file included with tutorial Explore volumes included with scene –Target volumes, atlas volumes –An old segmentation result is also included

National Alliance for Medical Image Computing Template Builder Tutorial Step 1: Choose EMSegmenter parameter set For now, use tutorial set Later use this interface to create a new parameter set

National Alliance for Medical Image Computing Template Builder Tutorial Define anatomical structures Right click to add or delete nodes Label corresponds to eventual voxel values in segmentation result

National Alliance for Medical Image Computing Template Builder Tutorial Choose an atlas volume for each leaf node To change or add new atlas volumes –Load new volumes into Slicer3 –Select new volumes at this step

National Alliance for Medical Image Computing Template Builder Tutorial Choose target images To change or add target images –Load new volumes into Slicer3 –Select new volumes at this step You can reorder target images; order is important

National Alliance for Medical Image Computing Template Builder Tutorial You can choose to align target images First target is fixed image Rigid, mutual information registration

National Alliance for Medical Image Computing Template Builder Tutorial You can choose to normalize target images Simple, default strategy Default parameter sets available from pulldown

National Alliance for Medical Image Computing Template Builder Tutorial Intensity distributions define appearance of each leaf structure Gaussian Dimensionality equal to number of target images Two methods –Sample voxels from images –Specify mean and covariance manually Tip: sample first, then fine tune manually

National Alliance for Medical Image Computing Template Builder Tutorial Sample voxels from images –Load first target image into Slicer3 slicer view –Choose anatomical structure –Choose `manual sampling –Ctrl-left-click on image to add voxels

National Alliance for Medical Image Computing Template Builder Tutorial Sample voxels from images (continued) –To remove an unwanted sample right-click on it and choose remove –Change back to manual mode to tweak distribution

National Alliance for Medical Image Computing Template Builder Tutorial Segmentation parameters for every tree node Influence –Prior weight relative to other structures –Atlas –Input channels –Smoothing (parent nodes only) Stopping Conditions

National Alliance for Medical Image Computing Template Builder Tutorial Atlas-to-target registration parameters Moving image registered to first target image –You can choose any image loaded into Slicer3 Same transformation applied to all atlas images before segmentation begins

National Alliance for Medical Image Computing Template Builder Tutorial Select output and run registration You can troubleshoot preprocessing by saving intermediate results Choose an output labelmap ROI governs segmentation processing (one-based, not zero-based)

National Alliance for Medical Image Computing Acknowledgements Wendy Plesniak, SPL BWH Steve Pieper, Isomics Luis Ibáñez, Kitware Sylvain Bouix, PNL BWH William Wells, BWH Funding provided by NAMIC

NA-MIC National Alliance for Medical Image Computing Questions…

National Alliance for Medical Image Computing Specify Inputs Default Pre- Processing Parameters Target Images Atlas Images Target Image Normalization Target-to-target Registration Atlas-to-target Registration Segmentation EM Segment Algorithm: Pohl et al. Review Results Slicer3 Slice Views Slicer3 Model Maker External Program Slicer3 EM Segment Workflow

National Alliance for Medical Image Computing Atlas Images Target Images Parameters Segmentation Result Specify Inputs Default Pre- Processing Parameters Target Images Atlas Images Target Image Normalization Target-to-target Registration Atlas-to-target Registration Segmentation EM Segment Algorithm: Pohl et al. Review Results Slicer3 Slice Views Slicer3 Model Maker External Program