1. 3D Human Airway Segmentation for Virtual Bronchoscopy
Atilla P. Kiraly,1 William E. Higgins,1,2 Eric A. Hoffman,2
Geoffrey McLennan,2 and Joseph M. Reinhardt2
1Penn State University, University Park, PA 16802
2University of Iowa, Iowa City, IA 52246
SPIE Medical Imaging 2002, San Diego, CA, 24 February 2002

2. 4. Virtual Bronchoscopy Applications
Outline
1. Introduction
2. Method
3. Segmentation Results
4. Virtual Bronchoscopy Applications
Here is the outline of the talk, in the introduction we’ll review the images we are analyzing
Along with the data contained in the images

3. Introduction New 3D CT Images can be large: 512 X 512 X 400
Partial volume effects
Reconstruction artifacts
Patient breathing artifacts
Airway segmentation necessary for Virtual Bronchoscopy
Path planning, rendering, quantitative analysis
* Manual segmentation not an option
…analysis can be very dependent on segmentation results

4. Previous Research 1. Knowledge-based
W. Park et al., IEEE Trans. Med. Imaging, Aug. 1998
2. Central-axis analysis
R. Swift et al., Comp. Med. Imag. Graph., Feb. 2002
3. 3D Region growing (RG)  not robust
R. M. Summers et al., Radiology, Sept. 1996
K. Mori et al., 13th ICPR, 1996
4. Mathematical morphology  too slow
F. Preteux et al., J. Elect. Imaging, Jan. 1999
D. Bilgen et al., IEEE Trans. Med. Imaging, submitted 2001
VB systems need pre-processing of the data segmentation and path planning most important
Sonka – knowledge based – structural relationships
Park improved on specificity via fuzzy logic
---not tested on human data, more robust(less knowledge dept. methods exist)
Law genetic algorithm
--global threshold , partial volume effects
Morph 2 step process, candidate determination followed by reconstruction
Fetita – fractal analysis
--time consuming

5. Proposed Hybrid Approach
Combines 3D RG and Morphology based methods
Use filtering to improve robustness of both methods
Use results of 3D RG to reduce application area of the larger operators in the Morphology method
Order of magnitude improvement in execution time
Our proposed approach combines…
In addition We add filtering

6. 3D Airway Segmentation Overview
3D image I
Modified 3D Region Growing
Optional Filter
Lung Region Definition
Optional Filter W
Morphology
This shows an overview of the segmentation method
Wi i{1,…,Zsize}
2D Candidate Labeling
3D Reconstruction
Airway Segmentation IS

7. 3D Airway Segmentation Overview
3D image I
Modified 3D Region Growing
Optional Filter
Lung Region Definition
Optional Filter W
Morphology
This shows an overview of the segmentation method
Wi i{1,…,Zsize}
2D Candidate Labeling
3D Reconstruction
Airway Segmentation IS

8. Optional Pre-Filtering of the Data
PURPOSE:
3D RG can successfully complete without parenchymal leakage
Can help reduce false candidates in morphology method
COST:
Lose some peripheral branches
METHODS:
4-connected or 3 X 3 Median filter applied to each slice on 2D basis

9. 3D Airway Segmentation Overview
3D image I
Modified 3D Region Growing
Optional Filter
Lung Region Definition
Optional Filter W
Morphology
This shows an overview of the segmentation method
Wi i{1,…,Zsize}
2D Candidate Labeling
3D Reconstruction
Airway Segmentation IS

10. Modified Adaptive 3D Region Growing
Seed = s
Threshold = T
T = T + 1
Volume < Explosion
Yes No
T = T - 1
3D Region Growing
Post Processing

11. Post Processing PURPOSE:
1. RG result contains cavities due to noisy data
2. Edges of segmentation can be very rough
METHOD:
Cavity deletion and binary closing of RG segmentation

12. 3D Airway Segmentation Overview
3D image I
Modified 3D Region Growing
Optional Filter
Lung Region Definition
Optional Filter W
Morphology
PURPOSE: Mask area of the lungs to constrain operator area
METHOD:
Partial version of S. Hu et al., IEEE Trans. Med. Imaging, June 2001
Perform 3D RG at root-site with optimally determined threshold
Wi i{1,…,Zsize}
2D Candidate Labeling
3D Reconstruction
Airway Segmentation IS

13. 3D Airway Segmentation Overview
3D image I
Modified 3D Region Growing
Optional Filter
Lung Region Definition
Optional Filter W
Morphology
This shows an overview of the segmentation method
Wi i{1,…,Zsize}
2D Candidate Labeling
3D Reconstruction
Airway Segmentation IS

14. Morphology-Based Segmentation
Two-Step Process
1. 2D Candidate Labeling
Identify potential airways on a 2D basis
Uses gray-scale reconstruction with different operators
2. 3D Reconstruction
HYBRID:
Use results of 3D RG and Lung Region Definition to limit application area of step 1

15. 2D Candidate Labeling
Basis Operator
bth order homothetic operators

16. 1 Sample and threshold slice z from Image I
2D Candidate Labeling
1 Sample and threshold slice z from Image I
J inf. The local min smaller then operator 20% diff bet max and min values

17. 2D Candidate Labeling
Perform gray-scale closing with operator of size b
Erode image and take maximum with original
Repeat above step until max no longer involves S
J inf. The local min smaller then operator 20% diff bet max and min values

18. 2D Candidate Labeling Threshold result into binary image C
Union of results for all b determines candidate locations

19. 3D Airway Segmentation Overview
3D image I
Modified 3D Region Growing
Optional Filter
Lung Region Definition
Optional Filter W
Morphology
This shows an overview of the segmentation method
Wi i{1,…,Zsize}
2D Candidate Labeling
3D Reconstruction
Airway Segmentation IS

20. 3D Reconstruction
PURPOSE: Determine valid candidates to form final result
METHOD:
Closed space dilation with unit kernel radius
3D 6-connected region growing IS

21. Case h006: 512X512X574 287MB (0. 72mm X 0.72mm X 0.60mm)
Results: case h006
Maximum Intensity Projections (MIP) of resultant segmentations
3D RG
Hybrid
Morphology
Morphology method failed
Different branches segmented
No filtering used
Case h006: 512X512X MB (0. 72mm X 0.72mm X 0.60mm)
Case h006_512_85, root site=(273,248,0), seger=(RegGrow,no filter,explode at T=50000)

22. Case h007: 512X512X488 244MB (0.65mm X 0. 65mm X 0.60mm)
Results: case h007
MIP of resultant segmentations
3D RG
Hybrid
Morphology
4-connected median filter
3D RG and Morphology methods show leakage
Case h007: 512X512X MB (0.65mm X 0. 65mm X 0.60mm)
Case h007_512_85, root site=(266,221,0), seger=(RegGrow,star median,explode at T=50000)

23. Case h007: 512X512X488 244MB (0.65mm X 0. 65mm X 0.60mm)
Results: case h007
Tree Renderings
3D RG
Hybrid
Morphology
4-connected median filter
3D RG and Morphology methods show leakage
Case h007: 512X512X MB (0.65mm X 0. 65mm X 0.60mm)
Case h007_512_85, root site=(266,221,0), seger=(RegGrow,star median,explode at T=50000)

24. Case h008: 512X512X389 194MB (0.59mm X 0.59mm X 0.06mm)
Results: case h008
MIP of resultant segmentations
3D RG
Hybrid
Morphology
Only hybrid method succeeded
No filtering used
Case h008: 512X512X MB (0.59mm X 0.59mm X 0.06mm)
Case h008_512_85, root site=(242,211,0), seger=(RegGrow,no filter,explode at T=50000)

25. Segmentation Time Results
Method
Labeling
seconds
Reconstruction
Total
3D RG
N.A. 64
Hybrid
1700
1580
3280
Morphology
15380
3200
18580
Over order of magnitude increase in labeling speed – concerning the labeling portion
Improved results over standard method in some cases
Hybrid demonstrates 10X improvement in labeling time

26. Edge Localization Segmented by both RG and Hybrid methods
Segmented by Hybrid method only
Hybrid method demonstrates better edge localization

27. H012 case: papilloma
(-1000,-800) WINDOWING
Hybrid and Morphology method fail in capturing papilloma

28. Virtual Bronchoscopy Applications
Airway Analysis
Peripheral Nodule Biopsy
Mediastinal Lymph-Node Biopsy
Use the Virtual Navigator.
Sherbondy et al., SPIE Medical Imaging 2000, vol. 3978
Helferty et al., SPIE Medical Imaging 2001, vol. 4321
Helferty et al., ICIP 2002
We now apply our segmentation methodology to three application in Virtual Bronchoscopy: Airway analysis, peripheral nodule biopsy, and mediastinal lymph-node biopsy. For this work, we use our previously devised Virtual Bronchoscopy system, the Virtual Navigator, introduced in these publications.

29. Virtual Navigator: architecture
Data
Sources
CT Scan
Bronchoscope
Stage 1: 3D CT Assessment
Identify Target ROI Sites
Segment Airway Tree
Calculate Centerline Paths
Virtual Endoluminal Movies
Cross-Section Area Calculations
Volume Slices, Slabs, Projections
Stage 2: Live Bronchoscopy
Capture Endoscope Video
Correct Barrel Distortion
Interactive Virtual Views
Register Virtual CT to Video
Draw Target Regions on Video
Image
Processing
Analysis
The physician applies the Virtual Navigator in two Stages. In Stage 1, the physician uses a 3D CT scan to identify target biopsy sites, construct the main airway tree, and define centerline paths through the major airways. This information is stored in a Case Study. Later, during Stage 2 Live Bronchoscopy, the Virtual Navigator is interfaced directly to the bronchoscope. The case study information is used during live bronchoscopy to help guide biopsy.
HTML Multimedia Case Report
ROI
List
Segmented Airway Tree
Centerline Paths
Screen Snapshots
Recorded Movies
Physician Notes

30. Virtual Navigator: Hardware
Video
Stream
AVI File
Endoscope
Scope Monitor
Scope Processor
Light Source
RGB,Sync,Video
Matrox
Cable
Matrox PCI card
Main Thread
Video Tracking
OpenGL Rendering
Worker Thread
Mutual Information
Dual CPU System
Video AGP card
Capture
Rendered
Image
Polygons, Viewpoint
PC Enclosure
The Virtual Navigator runs on a standard Windows-based PC. It uses a Matrox frame grabber card to interface to a bronchoscope. All software is written in Visual C++ and draws upon vtk and OpenGL.
The system resides on a standard Windows-based PC. A Matrox video card serves as the interface between the PC and the videobronchoscope. The main software system,written in Visual C++, can run on an inexpensive laptop computer.

31. Airway Analysis (work in progress)
Discuss. Point out that it’s a work in progress.
Case h16_512_85, root site=(263,233,45), seger=(RegGrow,star median,explode at T=50000)

32. Peripheral Nodule Biopsy (work in progress)
For Peripheral Nodule Biopsy, we: (1) segment the main airway tree, (2) compute centerline paths, and (3) define the nodule of interest. We use the tools of the Virtual Navigator to interrogate the case VIRTUALLY, before the procedure. DESCRIBE BRIEFLY. Point out the nodule on the views. This could then conceivably be used to guide live bronchoscopic biopsy of the nodule. This is a work in progress.
Case h001_512_85, root site=(273,292,0), seger=(RegGrow,star median,explode at T=-948),slab=(focus=20,vision=30,maxwin=400)

33. VB-Guided Mediastinal Lymph-Node Biopsy
Human Study underway
29 cases to date (2/2002)
VB-Guided approach being compared to standard approach which uses CT film.

34. Mediastinal Lymph-Node Biopsy (study underway progress)
Case h005_512_85. Root site = (253,217,0), seger = (RegGrow, no filter), ROI #2 considered (Blue)

35. Integrated segmentation tool-kit used for VB
Conclusion
Hybrid method
Clinically feasible
Similar results to Morphology
No method superior
No method consistently recovered more airways
Hybrid and Morphology methods localize edges better
Only Region Growing succeeded in papilloma case
Integrated segmentation tool-kit used for VB