1. Model-Based Organ Segmentation in CT Scans
Jiun-Hung Chen

2. Input
Output

3. Organ Segmentation in 3D CT scans
First step for further analysis
Computer-aided detection, computer-aided diagnosis, radiation treatment, or organ shape analysis.
Manual organ segmentation: accurate but very time-consuming
An efficient and effective automatic or semi-automatic organ segmentation method

4. Problem Statement
Learn how to segment unseen testing CT images from a set of training CT images with ground truths
Goal: Minimize the training errors while can generalize to unseen testing CT images

5. Why Difficult? (Shape Variations)

6. Why Difficult? (Similar Appearances)

7. Why Difficult? (Appearance Changes)

8. Why Difficult? (Position Changes)

9. Outline Introduction Previous Work Initial Results Proposal
Conclusions

10. Active Shape Model Based Framework
Input
Training CT Volumes
3D point correspondence
Learning Statistical Shape Models
Known point correspondences
Learning Organ Detection
Learning Boundary Intensity Model
Boundary Refinement
Organ Detection
A Testing CT Volume
Shape Model Initialization
Learned Statistical Shape Model
Learned Boundary Intensity Model
Organ Detector
Detected Bounding Box
Initial Shape
Final Shape …
Training phase
Testing phase … …

11. P1 P2 P3
Organ 1
Organ 2
Organ N

12. Input Output Correspondence 1 Correspondence 2 P1 P2 P3 P1 P2 P3 P1 P2

13. P1 P2 P3 x y z
Input
Output

14. Learned Boundary Intensity Model
Input
Output
Intensity profiles P1 P2 P3
Learned Boundary Intensity Model
Prob.
Intensity profiles

15. x,y,z rx ry rz sx sy sz …
Input
Output

16. Learned Statistical
Shape Model
Organ Detector
Test Organ
x,y,z rx ry rz sx sy sz
Test Organ
Test Organ

17. P
Learned Boundary Intensity Model
SSM
SSM

18. Outline Introduction Previous Work Initial Results Proposal
Conclusions

19. 3D Point Correspondence (MDL)
Goal: Find 3D Point Correspondence
Idea: Minimize MDL-based objective function
Evaluate the quality of the correspondence
How: Gradient descent
Manipulate correspondences by parameterization and re-parameterization.
Davies et al. [IEEE TMI’02]

20. Statistical Shape Models
Principal Component Analysis (PCA)
Kernel PCA
Boundary Intensity Models
Gaussian distribution
AdaBoosted histogram classifiers
Heuristics
Cootes et al. [IEEE PAMI 01], Twining et al. [BMVC’01]
Cootes et al. [IEEE PAMI 01], Li [ICCV’05], Kainmuller et al. [MICCAI’07]

21. Organ Detection (MSL) Goal: Find the bounding box Idea
The parameter space is 9D.
3D positions, 3D scales and 3D orientations.
Idea
Uniform and exhaustive search is unnecessary
How: decompose the problem into three steps
position estimation, position-scale estimation and finally position-scale-orientation estimation.
Zheng et al. [ICCV’07]

22. Two ASM-based Systems Ling et al. [CVPR’08] Statistical shape models
Kainmuller et al. [MICCAI’07]
Ling et al. [CVPR’08]
Statistical shape models
PCA
43 CT volumes
Boundary intensity model
Heuristics
Liver detection
Lungs detection and DICOM info
Performance
Ranked first in a recent liver segmentation competition.
10 testing volumes
1.1mm (the average symmetric surface distance)
15 minutes.
Statistical Shape Models
Statistical Shape models
PCA, hierarchical shape pyramids
75 volumes
Boundary intensity model
A boundary classifier
Liver detection
MSL
Performance
5 fold cross validation
1.59 mm (the average symmetric surface distance)
1.38 mm (the median)
12 seconds.
Boundary Intensity Model
Liver Detection
Performance

23. Other approaches Active contour Level Sets Graph cuts
Adaptive thresholding
Kass et al. [IJCV 88], Osher et al. [J. Comput. Phys. 88], Boykov et. al. [IEEE PAMI 01], Shi et. al. [IEEE PAMI 00], Kobashi et. al. [PR 95]

24. Outline Introduction Previous Work Initial Results Proposal
Conclusions

25. ASM Framework with Our Methods
Training CT Volumes
MDL with 2DPCA for 3D Point Correspondence
Tensor-based Statistical Shape Models
Known point correspondences
Learning Organ Detection
Learning Boundary Intensity Model
Graph Cut based Boundary Refinement
Organ Detection
A Testing CT Volume
Shape Model Initialization
Learned Statistical Shape Model
Organ Detector
Detected Bounding Box
Initial Shape
Final Shape
Learned Boundary Intensity Model
Training phase
Testing phase

26. Experimental Setting Datasets: Leave-one-out cross validation
4 types of organs (livers, left kidneys, right kidneys, spleens)
15-20 subjects
Leave-one-out cross validation
Measure the reconstruction error
Metrics: Euclidean and Hausdorff distance

27. MDL-2DPCA MDL-based objective function
Idea: Generalize the objective function to 2DPCA space
Replace eigenvalues from PCA with from 2DPCA
How: Gradient descent
Comparisons: original MDL vs. MDL-2DPCA
* Chen and Shapiro [to appear in EMBC’09]

28. Results (3D Point Correspondences)
Livers
Left Kidneys
Reconstruction error
Original MDL
MDL-2DPCA
# eigenvectors
Spleens
Right Kidneys

29. Tensor-based SSM Idea: Tensor-based dimension reduction methods
2DPCA
Parafac model
Tucker decomposition
Comparisons: PCA vs. Tensor-based dimension reduction
* Chen and Shapiro [to appear in EMBC’09]

30. Results (Statistical Shape Models)
Livers
Left Kidneys
Parafac
Reconstruction error
PCA
Tucker
2DPCA
# eigenvectors
Right Kidneys
Spleens

31. Organ Detection (Boosting Approach)
Idea: Classify whether an image block contains an organ of interest
How:
Partition slices into non-overlapping 32x32 blocks
Global features: gray-tone histogram of the image slice and its slice index
Local features: the position of a block, the mean and variance of its intensity values, and its intensity histogram.
20,000 SVM linear classifiers + Adaboosting
Comparisons: Manual vs. Adaboosting

32. Results (Organ Detection)
Positive (predicted)
Negative (predicted)
Positive (actual)
96.23%
3.77%
Negative (actual)
4.57%
95.43%
Livers (Training)
Positive (predicted)
Negative (predicted)
Positive (actual)
91.23%
8.77%
Negative (actual)
6.57%
93.43%
Livers (Testing)

33. True Positive : Green, False Positive: Blue
False Negative: Red, True Negative: Cyan
Sub. 1
Sub. 2

34. Graph Cuts Based Boundary Refinement
Idea: Adding hard constraints to min s-t cuts
Min s-t cuts with side constraints
NP-hard in general cases
Approximation algorithm: standard rounding algorithm
Comparisons: with constraints vs. without constraints
Chen and Shapiro [ICPR’08]

35. Results (Boundary Refinement)
Initial Contour
Slice 1
Slice 2
without
with

36. Outline Introduction Previous Work Initial Results Proposal
Conclusions

37. Framework with Proposal
Training CT Volumes
Nonlinear and Robust MDL for 3D Point Correspondence
Tensor-based Statistical Shape Models
Known point correspondences
Boosting/Bagging MSL
Learning Boundary Intensity Model
Robust Graph Cut based Boundary Refinement with TU constraints
Organ Detection
A Testing CT Volume
Shape Model Initialization
Learned Statistical Shape Model
Organ Detector
Detected Bounding Box
Initial Shape
Final Shape
Learned Boundary Intensity Model
Training phase
Testing phase

38. Nonlinearity and Robustness
Idea: generalize MDL objective function to KPCA, tensor-based decompositions and robust PCA
How: replace eigenvalues from PCA with
Eigenvalues from KPCA
High-order singular values from Tensor decomposition
Eigenvalues from Robust PCA
Gradient descent for optimization

39. Boosting/Bagging MSL Goal: Improve MSL for organ detection
Idea: Boosting/Bagging MSL
How:
Treat MSL as a base classifier
Use AdaBoosting/Bagging to combine many MSL classifiers

40. Robust Graph Cuts Goal: Achieve robustness by Minimax
minimize the worst case loss in objective values
Idea: Some edge weights are unreliable
Minimize maximum cut values caused by unreliable edge weights
How: Solve a series of modified min s-t cut problems
Polynomial time algorithm

41. Totally Unimodular (TU) Hard Constraints
Goal: Balance the performance and efficiency
Hard constraints: helpful but NP-hard
Idea: Special cases where global optimal solutions can be found in polynomial time
How:
Find interesting semantic constraints that can be represented as TU constraints
Specialized algorithms for TU problems

42. Outline Introduction Previous Work Initial Results Proposal
Conclusions

43. Training CT Volumes
MDL with 2DPCA for 3D Point Correspondence
Tensor-based Statistical Shape Models
Known point correspondences
Learning Organ Detection
Learning Boundary Intensity Model
Graph Cut based Boundary Refinement
Organ Detection
A Testing CT Volume
Shape Model Initialization
Learned Statistical Shape Model
Organ Detector
Detected Bounding Box
Initial Shape
Final Shape
Learned Boundary Intensity Model
Training CT Volumes
3D point correspondence
Learning Statistical Shape Models
Known point correspondences
Learning Organ Detection
Learning Boundary Intensity Model
Boundary Refinement
Organ Detection
A Testing CT Volume
Shape Model Initialization
Learned Statistical Shape Model
Learned Boundary Intensity Model
Organ Detector
Detected Bounding Box
Initial Shape
Final Shape
Training CT Volumes
Nonlinear and Robust MDL for 3D Point Correspondence
Tensor-based Statistical Shape Models
Known point correspondences
Boosting/Bagging MSL
Learning Boundary Intensity Model
Robust Graph Cut based Boundary Refinement with TU constraints
Organ Detection
A Testing CT Volume
Shape Model Initialization
Learned Statistical Shape Model
Organ Detector
Detected Bounding Box
Initial Shape
Final Shape
Learned Boundary Intensity Model

44. Proposal General Approach and Many Applications
2D point correspondence
Image segmentation
Stereo
Image restoration
Interactive image segmentation
Face alignment
Clustering

45. Acknowledgement Committee Members Linda’s group members
Professor Linda Shapiro (CSE) as Chair
Professor James F. Brinkley (BioStr)
Professor Jenq-Neng Hwang (EE)
Professor Rajesh Rao (CSE)
Professor Mark Ganter (ME) as GSR
Linda’s group members

46. Thank You.

47. Gantt Chart Summer 09 Autumn 10 Winter 10 Spring 10 A working system
Summer 09
Autumn 10
Winter 10
Spring 10
A working
system
Better organ
detector
Better boundary
refinement
Better 3D point
correspondence
System Integration
/Tuning/Testing
Dissertation Writing
Defense