1. NA-MIC National Alliance for Medical Image Computing http://na-mic.org Slicer3 EMSegment Tutorial January 2008 NAMIC All-Hands Meeting Brad Davis, Yuman Yuan, Sebastien Barre, Will Schroeder, Polina Goland, Ron Kikinis, Kilian Pohl

2. National Alliance for Medical Image Computing http://na-mic.org Program EM Segmenter basics Algorithm views in Slicer3 Slicer3 EM segment under the hood Tutorial –Data –Simple Segmentation Interface –Template Builder Interface Questions (all week…)

3. National Alliance for Medical Image Computing http://na-mic.org EM Segment Basics Figure: Pohl et al.

4. National Alliance for Medical Image Computing http://na-mic.org EM Segment Basics Figures: Pohl et al.

5. National Alliance for Medical Image Computing http://na-mic.org Atlas Images Target Images Parameters Segmentation Result EM Segment Basics GM WM CSF Air Skin/Skull

6. National Alliance for Medical Image Computing http://na-mic.org Atlas Images Parameters Segmentation Result EM Segment Basics Collection of images (input channels) that will be segmented T1T2 Target Images

7. National Alliance for Medical Image Computing http://na-mic.org Atlas Images Target Images Parameters Segmentation Result EM Segment Basics Hierarchy of anatomical structures –Intensity distributions, prior probabilities Default preprocessing –Intensity normalization, registration EM segmentation algorithm –Smoothing, weights, # iterations Stored as XML MRML file Root ICC Background AirSkin/Skull GMWMCSF

8. National Alliance for Medical Image Computing http://na-mic.org Target Images Parameters Segmentation Result EM Segment Basics Atlas images (spatial priors) –Aligned in common coordinate frame GM AirSkin/Skull WM CSF Atlas Images

9. National Alliance for Medical Image Computing http://na-mic.org Atlas Images Target Images Parameters Segmentation Result EM Segment Basics Labelmap volume –Voxel values specified in parameters Root ICC Background AirSkin/Skull GMWMCSF

10. National Alliance for Medical Image Computing http://na-mic.org Algorithm Development 12-year history of algorithm development –Wells et al. 1996 (TMI): EM framework for simultaneous estimation of bias field and label map –Kapur et al. 1999 (PhD Thesis): Model noise via Markov Random Field –Van Leemput et al. 1999 (TMI): Non-spatial tissue priors –Pohl et al. 2002 (MICCAI): Deformable registration to align atlas 2004 (ISBI): Hierarchical framework to model anatomical dependencies 2007 (TMI): Up-to-date, detailed description

11. National Alliance for Medical Image Computing http://na-mic.org Slicer3 Segmentation Code Preprocessing Wizard GUI Recent Development Effort Goals: –Easy to use –Adapts to a variety of scenarios –Is a research tool –Simplified interfaces –Minimize preprocessing requirements –Validation and testing ITK/VTK

12. National Alliance for Medical Image Computing http://na-mic.org EMSegmenter Logic / Preprocessing / Algorithm Slicer3 GUI EM Template Builder Module EM Simple Module Command Line EM Command Line Executable Slicer3 EM segment views Advanced interface for creating/editing segmentation parameters Simple interface for applying parameters to new images Command-line interface for script/batch processing

13. National Alliance for Medical Image Computing http://na-mic.org Template Builder View Runs inside Slicer3 Purpose: Build/Adapt parameter sets Run segmentation Wizard guides user through interface panels

14. National Alliance for Medical Image Computing http://na-mic.org `One-Click View Runs inside Slicer3 Purpose: Simple interface for segmenting new data Requires predefined parameter set

15. National Alliance for Medical Image Computing http://na-mic.org Command-line View EMSegmentCommandLine executable bundled with Slicer3 Purpose: Scripted/batch processing Requires predefined parameter set Specify new target/atlas images on command line Atlas Images Target Images Parameters Segmentation Result

16. National Alliance for Medical Image Computing http://na-mic.org Specify Inputs Parameters Target Images Atlas Images Slicer3 EM Segment Workflow

17. National Alliance for Medical Image Computing http://na-mic.org Specify Inputs Default Pre- Processing Parameters Target Images Atlas Images Target Image Normalization Target-to-target Registration Atlas-to-target Registration Slicer3 EM Segment Workflow

18. National Alliance for Medical Image Computing http://na-mic.org Specify Inputs Default Pre- Processing Parameters Target Images Atlas Images Target Image Normalization Target-to-target Registration Atlas-to-target Registration Segmentation EM Segment Algorithm: Pohl et al. Slicer3 EM Segment Workflow

19. National Alliance for Medical Image Computing http://na-mic.org Specify Inputs Default Pre- Processing Parameters Target Images Atlas Images Target Image Normalization Target-to-target Registration Atlas-to-target Registration Segmentation EM Segment Algorithm: Pohl et al. Review Results Slicer3 Slice Views Slicer3 Model Maker External Program Slicer3 EM Segment Workflow

20. National Alliance for Medical Image Computing http://na-mic.org Default Pre- Processing Target Image Normalization Target-to-target Registration Atlas-to-target Registration New Development: Default Preprocessing Lower bar for entry Starting point for building effective pipeline Not intended for all scenarios No interface for complex parameter tweaking Easily bypassed by advanced users

21. National Alliance for Medical Image Computing http://na-mic.org Default Pre- Processing Target Image Normalization Target-to-target Registration Atlas-to-target Registration New Development: Default Preprocessing VTK filter by Kilian Pohl Scale intensities to achieve predefined mean –Heuristics determine histogram cutoffs Default settings for T1, T2 images

22. National Alliance for Medical Image Computing http://na-mic.org Default Pre- Processing Target Image Normalization Target-to-target Registration Atlas-to-target Registration New Development: Default Preprocessing Target image 1 is fixed image Perform registration for each additional target image –Align image centers –Rigid registration, mutual information, gradient descent optimization (ITK) –Coarse-to-fine

23. National Alliance for Medical Image Computing http://na-mic.org Default Pre- Processing Target Image Normalization Target-to-target Registration Atlas-to-target Registration New Development: Default Preprocessing Target image 1 is fixed image Perform registration with selected atlas image –Align image centers, Rigid registration, B-Spline registration –Mutual information, (normalized cross-correlation, sse, not yet tested) –Rigid: gradient descent, B-Spline: ITK LBFGSB optimizer, 5x5x5 knot points –Coarse-to-fine –Fast/Standard/Slow: sample points, max iterations –Resample all atlas images via transform; guess background value

24. National Alliance for Medical Image Computing http://na-mic.org Default Registration Results

25. National Alliance for Medical Image Computing http://na-mic.org Tutorial Data Walkthrough 1/3 README: extra release notes Predefined parameter file –MR T1 & T2 brain segmentation: White matter, gray matter, CSF Target Images –ImageData_Input sub directory –Target : T1 & T2 images used in parameter file –Target_NewPatient1, Target_NewPatient2 : New data to try with the parameter file

26. National Alliance for Medical Image Computing http://na-mic.org Tutorial Data Walkthrough 2/3 Atlas Images –ImageData_Input sub directory –Atlas : atlas images used in parameter file –Atlas_NewAtlas : Extra atlas data for demonstration purposes (currently a copy of first atlas)

27. National Alliance for Medical Image Computing http://na-mic.org Tutorial Data Walkthrough 3/3 Low resolution data for testing –Full sized data: 30 minutes –Low resolution data: 1 minute Data, parameters have _small appended to filename

28. National Alliance for Medical Image Computing http://na-mic.org `One-Click Module Tutorial 1/3 EMSegment tutorial data is required Within Slicer3, open EMSegment Simple Module

29. National Alliance for Medical Image Computing http://na-mic.org `One-Click Module Tutorial 2/3 MRML Scene : Select the tutorial parameter set Result Labelmap : Select a new image filename Target Volumes : Select input target volumes from one of the target directories (e.g., Target_NewPatient1 ) Note: target volume order is important Click `Apply, wait (~30 minutes for full size images)…

30. National Alliance for Medical Image Computing http://na-mic.org `One-Click Module Tutorial 3/3 Review the results in Slicer3 –Load target images –Load result labelmap –Use slice views to review segmentation results Apply Model Maker to generate surfaces

31. National Alliance for Medical Image Computing http://na-mic.org EMSegment Command-line Examples Script examples available in tutorial directory –New images –New atlas –New image and atlas –See README

32. National Alliance for Medical Image Computing http://na-mic.org Template Builder Tutorial EMSegment tutorial data is required Within Slicer3, open EMSegment Module

33. National Alliance for Medical Image Computing http://na-mic.org Template Builder Tutorial Load tutorial parameters –File->Load Scene –Select mrml file included with tutorial Explore volumes included with scene –Target volumes, atlas volumes –An old segmentation result is also included

34. National Alliance for Medical Image Computing http://na-mic.org Template Builder Tutorial Step 1: Choose EMSegmenter parameter set For now, use tutorial set Later use this interface to create a new parameter set

35. National Alliance for Medical Image Computing http://na-mic.org Template Builder Tutorial Define anatomical structures Right click to add or delete nodes Label corresponds to eventual voxel values in segmentation result

36. National Alliance for Medical Image Computing http://na-mic.org Template Builder Tutorial Choose an atlas volume for each leaf node To change or add new atlas volumes –Load new volumes into Slicer3 –Select new volumes at this step

37. National Alliance for Medical Image Computing http://na-mic.org Template Builder Tutorial Choose target images To change or add target images –Load new volumes into Slicer3 –Select new volumes at this step You can reorder target images; order is important

38. National Alliance for Medical Image Computing http://na-mic.org Template Builder Tutorial You can choose to align target images First target is fixed image Rigid, mutual information registration

39. National Alliance for Medical Image Computing http://na-mic.org Template Builder Tutorial You can choose to normalize target images Simple, default strategy Default parameter sets available from pulldown

40. National Alliance for Medical Image Computing http://na-mic.org Template Builder Tutorial Intensity distributions define appearance of each leaf structure Gaussian Dimensionality equal to number of target images Two methods –Sample voxels from images –Specify mean and covariance manually Tip: sample first, then fine tune manually

41. National Alliance for Medical Image Computing http://na-mic.org Template Builder Tutorial Sample voxels from images –Load first target image into Slicer3 slicer view –Choose anatomical structure –Choose `manual sampling –Ctrl-left-click on image to add voxels

42. National Alliance for Medical Image Computing http://na-mic.org Template Builder Tutorial Sample voxels from images (continued) –To remove an unwanted sample right-click on it and choose remove –Change back to manual mode to tweak distribution

43. National Alliance for Medical Image Computing http://na-mic.org Template Builder Tutorial Segmentation parameters for every tree node Influence –Prior weight relative to other structures –Atlas –Input channels –Smoothing (parent nodes only) Stopping Conditions

44. National Alliance for Medical Image Computing http://na-mic.org Template Builder Tutorial Atlas-to-target registration parameters Moving image registered to first target image –You can choose any image loaded into Slicer3 Same transformation applied to all atlas images before segmentation begins

45. National Alliance for Medical Image Computing http://na-mic.org Template Builder Tutorial Select output and run registration You can troubleshoot preprocessing by saving intermediate results Choose an output labelmap ROI governs segmentation processing (one-based, not zero-based)

46. National Alliance for Medical Image Computing http://na-mic.org Acknowledgements Wendy Plesniak, SPL BWH Steve Pieper, Isomics Luis Ibáñez, Kitware Sylvain Bouix, PNL BWH William Wells, BWH Funding provided by NAMIC

47. NA-MIC National Alliance for Medical Image Computing http://na-mic.org Questions…

48. National Alliance for Medical Image Computing http://na-mic.org Specify Inputs Default Pre- Processing Parameters Target Images Atlas Images Target Image Normalization Target-to-target Registration Atlas-to-target Registration Segmentation EM Segment Algorithm: Pohl et al. Review Results Slicer3 Slice Views Slicer3 Model Maker External Program Slicer3 EM Segment Workflow

49. National Alliance for Medical Image Computing http://na-mic.org Atlas Images Target Images Parameters Segmentation Result Specify Inputs Default Pre- Processing Parameters Target Images Atlas Images Target Image Normalization Target-to-target Registration Atlas-to-target Registration Segmentation EM Segment Algorithm: Pohl et al. Review Results Slicer3 Slice Views Slicer3 Model Maker External Program