Path Planning in Virtual Bronchoscopy Mohamadreza Negahdar Supervisor : Dr. Ahmadian Co-supervisor : Prof. Navab Tehran University of Medical Sciences January 2006 Progress Report

Clinical background (Motivation) Lung cancer is the most common cause of cancer related death* Lung cancer is the most common cause of cancer related death* 164,000 new cases and 156,000 deaths estimated in 2003 in the US The average 5 yr. survival rate is only 12% 164,000 new cases and 156,000 deaths estimated in 2003 in the US The average 5 yr. survival rate is only 12% Diagnosis of disease at early stage with subsequent treatment may dramatically increase cure and survival rate Diagnosis of disease at early stage with subsequent treatment may dramatically increase cure and survival rate Since its introduction in 1990 spiral CT has helped physicians visualize pulmonary nodules with a better diagnostic confidence compared to chest X-ray Since its introduction in 1990 spiral CT has helped physicians visualize pulmonary nodules with a better diagnostic confidence compared to chest X-ray *American Cancer Assc. Update 2003

Introduction High-resolution 3D CT pulmonary images permit evaluation of thin tubular structures (e.g., airways) and provide 3D position/shape information (e.g., for cancers) High-resolution 3D CT pulmonary images permit evaluation of thin tubular structures (e.g., airways) and provide 3D position/shape information (e.g., for cancers) However, 3D images are hard to assess manually. Virtual Bronchoscopic (VB) system enable 3D image probing and treatment planning Virtual Bronchoscopic (VB) system enable 3D image probing and treatment planning Both for ease of use and for quantitative assessment, Virtual Bronchoscopic systems need airway paths for effective use Both for ease of use and for quantitative assessment, Virtual Bronchoscopic systems need airway paths for effective use

Virtual Bronchoscopy VB is a computer-based approach for navigating virtually through airways captured in a 3-D MDCT image VB is a computer-based approach for navigating virtually through airways captured in a 3-D MDCT image VB 3-D image analysis: VB 3-D image analysis: Guidance of bronchoscopy Guidance of bronchoscopy Human lung-cancer assessment Human lung-cancer assessment Planning and guiding bronchoscopic biopsies Planning and guiding bronchoscopic biopsies Quantitative airway analysis –noninvasively- Quantitative airway analysis –noninvasively- Smooth virtual navigation Smooth virtual navigation A suitable method must: A suitable method must: Provide a detailed, smooth structure of the airway tree’s central axes Provide a detailed, smooth structure of the airway tree’s central axes Require little human interaction Require little human interaction Function over a wide range of conditions as observed in typical lung- cancer patients Function over a wide range of conditions as observed in typical lung- cancer patients

Virtual Bronchoscopy A major component of path planning system for VB is a method for computing the central airway paths (centerlines) from a 3-D MDCT chest image A major component of path planning system for VB is a method for computing the central airway paths (centerlines) from a 3-D MDCT chest image Two general approaches for path definition: Two general approaches for path definition: Manual-Path definition: time-consuming, error-prone, cannot readily get many paths. Manual-Path definition: time-consuming, error-prone, cannot readily get many paths. Recent automated techniques: don’t use gray-scale information, Recent automated techniques: don’t use gray-scale information,

Virtual Bronchoscopy Quicksee-Basic operation: Quicksee-Basic operation: 1. Load Data 3D radiologic image 3D radiologic image 2. Do Automatic Analysis Compute Compute Paths (axes) through airways Paths (axes) through airways Extract regions (airways) Extract regions (airways) Save results for interactive navigation Save results for interactive navigation 3. Perform Interactive navigation/assessment View, Edit, create paths through 3D image View, Edit, create paths through 3D image View structure; get quantitative data View structure; get quantitative data Many visual aids and viewers available Many visual aids and viewers available

OUR WORK Goals: Goals: The aim of our work is to build trajectories for virtual endoscopy inside 3D medical images, using the most automatic way. The aim of our work is to build trajectories for virtual endoscopy inside 3D medical images, using the most automatic way. Virtual endoscopy results are shown in various anatomical regions (bronchi, colon, brain vessels, arteries) with different 3D imaging protocols (CT, MR). Virtual endoscopy results are shown in various anatomical regions (bronchi, colon, brain vessels, arteries) with different 3D imaging protocols (CT, MR). In my thesis, an automatic centerline determination algorithm for three dimensional virtual bronchoscopy CT image will be presented. In my thesis, an automatic centerline determination algorithm for three dimensional virtual bronchoscopy CT image will be presented. We try that our method: We try that our method: Be faster Be faster Needs less interaction Needs less interaction Be more robust and reproducible Be more robust and reproducible

Path Path through a tubular structure defines a trajectory along tube’s central axis Path through a tubular structure defines a trajectory along tube’s central axis A Path denoted as: A Path denoted as: Medial (central) axes of branches Medial (central) axes of branches Preserve homotopy of structure Preserve homotopy of structure Continuous for smooth visualization Continuous for smooth visualization Path is spine of cylinder

Previous Path-Finding Methods  Automated approaches:  Segmentation followed by 3D skeletonization  Active contour models  Morphological operations  Estimation of principal eigenvectors  Vector fields  Shortcomings: Some lead to imprecise/missing paths and require long processing time

Our Method 2D 2D Morphological Operations (Algorithm I) Morphological Operations (Algorithm I) Distance Transformation (Algorithm II) Distance Transformation (Algorithm II) 3D 3D A Combination of Methods with Novelty A Combination of Methods with Novelty Phantom Phantom Human airways Human airways

2-D (basic shape-Algorithm I) 2-D (basic shape-Algorithm I) Load an Object Load an Object

2-D (basic shape-Algorithm I) 2-D (basic shape-Algorithm I) Distance from Boundary Distance from Boundary

2-D (basic shape-Algorithm I) 2-D (basic shape-Algorithm I) Gradient of DT from Boundary Gradient of DT from Boundary Gradient < 0

2-D (basic shape-Algorithm I) 2-D (basic shape-Algorithm I) Thinning Thinning

2-D (branching shape-Algorithm I) 2-D (branching shape-Algorithm I) Skeletonization Skeletonization False Branches

2-D (branching shape-Algorithm I) 2-D (branching shape-Algorithm I) Length-based Elimination Length-based Elimination End Points Branch Points

2 2-D (basic shape-Algorithm II) Distance Transformation (Chamfer Distance) Start Point

Distance Transformation City block City block dist 4 (p,q) = | p x – q x | + | p y – q y | dist 4 (p,q) = | p x – q x | + | p y – q y | Chess board Chess board dist 8 (p,q) = max { | p x – q x |,| p y – q y | } dist 8 (p,q) = max { | p x – q x |,| p y – q y | } Chamfer Chamfer dist cha (p,q) = A. max { | p x – q x |,| p y – q y | } + dist cha (p,q) = A. max { | p x – q x |,| p y – q y | } + (B-A). min { | p x – q x |,| p y – q y | } (B-A). min { | p x – q x |,| p y – q y | } Euclidean Euclidean dist e (p,q) = √( ( p x – q x ) 2 + ( p y – q y ) 2 ) dist e (p,q) = √( ( p x – q x ) 2 + ( p y – q y ) 2 ) Squared Euclidean Squared Euclidean dist E (p,q) = ( p x – q x ) 2 + ( p y – q y ) 2 dist E (p,q) = ( p x – q x ) 2 + ( p y – q y ) 2

2-D (basic shape-Algorithm II) 2-D (basic shape-Algorithm II) End Point Detection & Shortest Path End Point Detection & Shortest Path Steepest Descent Local Maxima Start Point End Point End Points Shortest Path

3D 3D  Our Procedure  Prepare the Data  Start Point Detection  Boundary Extraction  End Points Detection  Path Initialization  Centering  Refinements

Prepare the Input Prepare the Input Segmentation & Create the 3D Image Segmentation & Create the 3D Image Slicing the Segmented Image Slicing the Segmented Image Feed the Slice Images Feed the Slice Images Refine slices & Create 3D Image Matrices Refine slices & Create 3D Image Matrices Binarize the Object Binarize the Object Optimize the dataset Optimize the dataset

Load Data Load Data

Start Point Detection Start Point Detection

Boundary Extraction Boundary Extraction Morphological Operations Morphological Operations Boundary = Dilated Image – Original Image Boundary = Dilated Image – Original Image Boundary = Original Image – Eroded Image Boundary = Original Image – Eroded Image Distance Transformation from boundary to middle Distance Transformation from boundary to middle Boundary = ( DT == 1 ) Boundary = ( DT == 1 )

End Point Detection End Point Detection Distance Transformation Distance Transformation Assigns larger number to voxels with region growing in comparison to exact Euclidean metric More accurate approximation of true Euclidean distance metric Allocate integer values to voxels which speeds up the next computations EDT

Chamfer Distance Transformation  dist cha (p,q) = A. max { | p x – q x |,| p y – q y |,|p z – q z | } + (B-A). max{ min{ | p x – q x |,| p y – q y | }, (B-A). max{ min{ | p x – q x |,| p y – q y | }, min{ | p x – q x |,| p z – q z | }, min{ | p x – q x |,| p z – q z | }, min{ | p y – q y |,| p z – q z | } } + min{ | p y – q y |,| p z – q z | } } + (C-B-A). min{ | p x – q x |,| p y – q y |,|p z – q z | } (C-B-A). min{ | p x – q x |,| p y – q y |,|p z – q z | }  dist cha (p,Origin) = A. p x + (B-A). p y + (E-B-A). p z if p x >= p y +p z A. p x + (B-A). p y + (E-B-A). p z if p x >= p y +p z (E-C). p x + (C+B-E). p y + (C-B). p z if p x <= p y +p z (E-C). p x + (C+B-E). p y + (C-B). p z if p x <= p y +p z (E >= A+B) & (E >= B/2+C) (E >= A+B) & (E >= B/2+C)

End Point Detection End Point Detection Neighboring Window Neighboring Window

End Point Detection End Point Detection

Path Initialization Path Initialization Neighboring Neighboring

Path Initialization Path Initialization Start Point End Points Farest End Point

Centering Centering What is a snake? What is a snake? An energy minimizing spline guided by external constraint forces and pulled by image forces toward features: Edge detection Subjective contours Motion tracking Stereo matching … Basically, snakes are trying to match a deformable model to an image by means of energy minimization. D G

Centering Centering Energy & Gradient of Image Energy & Gradient of Image D = EDT from Boundary to middle G (i,j,k) = Gradient ( D(i,j,k) ) G (i,j,k) = Gradient ( D(i,j,k) ) Gx = 0.5 ( D(i+1,j,k) – D(i-1,j,k) ) Gy = 0.5 ( D(i,j+1,k) – D(i,j-1,k) ) Gz = 0.5 ( D(i,j,k+1) – D(i,j,k-1) ) D G Middle axis has minimum of Gradient

Centering Centering Snake Snake Path is considered as a parameterized curve (snake) v(s) = ( x(s),y(s),z(s) ) v(s) = ( x(s),y(s),z(s) ) T s [0,1] The Snake evolves in order to minimize an energy defined as: Smoothing terms Image term Decreasing function of the image gradient

Centering Centering Image force Image force v(i) is the discrete representation of the curve v v(i) is the discrete representation of the curve v In our experiments, the snake converges in a few iterations (< 20) and stabilizes itself very robustly In our experiments, the snake converges in a few iterations (< 20) and stabilizes itself very robustly

Centering Centering Start Point End Points Farest End Point

Refinements Refinements Length-based Elimination Length-based Elimination In Path Initialization Stage: In Path Initialization Stage: Remove branches which has length less than 10 voxel After Centering Stage: After Centering Stage: Remove branches which has length less than 5 voxel

Refinements Refinements Continuous Path Continuous Path Lose of continuity after centering Lose of continuity after centering Detect of discontinuity and make continue the path Detect of discontinuity and make continue the path

& now … & now … Virtual navigation and virtual endoscopy Virtual navigation and virtual endoscopy Segmentation & Registration Segmentation & Registration Virtual-guided bronchoscopy & Biopsy Virtual-guided bronchoscopy & Biopsy Quantification of anatomical structures Quantification of anatomical structures Surgical planning Surgical planning Radiation treatment Radiation treatment Curved planner reformation Curved planner reformation Stenosis detection Stenosis detection Aneurism and wall bronchia classification detection Aneurism and wall bronchia classification detection Deforming volumes Deforming volumes … …

Virtual Bronchoscopy

Discussion No single method is good for everything … No single method is good for everything … then we use combination of distance field & potential field Fully automated Fully automated without any interaction by physician No miss branch, No false branch No miss branch, No false branch 42 branch out of 42 Robust Robust less sensitivity to noise Too fast Too fast less than 1 minute for (512 x 512 x 416) – ( mm)

Future work Evaluate our method with more dataset Evaluate our method with more dataset Test the final path in a virtual environment Test the final path in a virtual environment More refinements of the path planning method More refinements of the path planning method Comparing of our method with others Comparing of our method with others

My thanks to … Dr. Alireza Ahmadian Prof. Nassir Navab Dr. Joerg Traub & My Family For nothing is hidden, except to be revealed; Nor has been secret, but that is should come to light.

Questions …. Suggestions …. Comments …. Ideas …. ? Questions …. Suggestions …. Comments …. Ideas …. ?