1. 1 Head and Neck Lymph Node Region Delineation with Auto-segmentation and Image Registration Chia-Chi Teng Department of Electrical Engineering University of Washington

2. 2 Outline Introduction Introduction Related Work Related Work Lymph Node Region Contouring with Image Registration Lymph Node Region Contouring with Image Registration Automatic Segmentation of Landmark Structures Automatic Segmentation of Landmark Structures Geometrical Feature Based Similarity Geometrical Feature Based Similarity Results Results Conclusion Conclusion

3. 3 Context 3D Conformal Radiotherapy (beams are shaped to match the tumor) 3D Conformal Radiotherapy (beams are shaped to match the tumor) Intensity Modulated Radiation Therapy (controls intensity in small volumes) Intensity Modulated Radiation Therapy (controls intensity in small volumes)

4. 4 Target Volumes GTV / CTV / PTV GTV / CTV / PTV

5. 5 Motivation Improve the process of target volume delineation for radiation therapy planning. Improve the process of target volume delineation for radiation therapy planning. Objective: Objective: –Auto-contour lymph node regions. –Initial focus on head and neck.

6. 6 Problem Where are the lymph nodes? Where are the lymph nodes? Where are the lymph node regions? Where are the lymph node regions? none of the structures are lymph nodes

7. 7 Solution Create reference (canonical) models. Create reference (canonical) models. Map reference nodal regions to patients. Map reference nodal regions to patients.

8. 8 System Overview

9. 9 Image Registration Align the transformed reference image f R ° g to the target image f T. Align the transformed reference image f R ° g to the target image f T. Find the optimal set of transformation parameters  that maximize an image similarity function S : Find the optimal set of transformation parameters  that maximize an image similarity function S :  optimal = argmax  S(  )

10. 10 Mattes’ Method Similarity Function Similarity Function S(  )  f R ° g, f T ) S(  ) =  mutual_information( f R ° g, f T ) Transformation Funciton Transformation Funciton g(x|  ) = R(x - x C ) – T(x - x C ) + D(x|  ) x = [x, y, z] T in the reference image coordinates.

11. 11 Deformable Transformation Control points (15*15*11). Control points (15*15*11). Each control point is associated with a 3- element deformation vector , describing x-, y-, z-components of the deformation. Each control point is associated with a 3- element deformation vector , describing x-, y-, z-components of the deformation.

12. 12 Project Target Lymph Regions Image registration aligns reference and target CT sets. Image registration aligns reference and target CT sets. Apply result transformation g to reference lymph node regions. Apply result transformation g to reference lymph node regions. Incorporate anatomical landmark correspondences. Incorporate anatomical landmark correspondences. Use surface mesh of outer body contour, mandible, hyoid … Use surface mesh of outer body contour, mandible, hyoid …

13. 13 Surface Warping Shelton’s method used to find Shelton’s method used to find correspondences between surfaces. correspondences between surfaces. Energy based surface mesh warping. Energy based surface mesh warping. E(C) = E sim (C) +  E str (C) +  E pri (C) S R S T C is the function which maps points from reference surface S R to target surface S T.

14. 14 Landmark Correspondence The deformation  at landmark points The deformation  at landmark points  k =  k   k  k : points from reference surface mesh S R.  k : corresponding locations on transformed reference surface S R ° C matching the target surface mesh S T.

15. 15 S R Surface S R S T Surface S T S R ° C  k =  k   k

16. 16 Using Landmark Correspondence Deformation vectors D( j ) are modified according to landmark correspondences  k in the proximity of the control points j. Deformation vectors D( j ) are modified according to landmark correspondences  k in the proximity of the control points j. Landmark structures align better. Landmark structures align better. Faster convergence. Faster convergence.

17. 17 Reference Mattes w/ Landmark Target Compare Image Registration Results

18. 18 Reference Mattes w/ Landmark Target

19. 19 Automatic Segmentation of Landmark Structures Given: Cancer radiation treatment patient’s head and neck CT image. Find: – –Skull base & thoracic inlet. – –Anatomical structures: cervical spine (white) respiratory tract (dark green) mandible (turquoise) hyoid (yellow) thyroid cartilage internal jugular veins (pink) carotid arteries (dark yellow) sternocleidomastoid muscles (light green, orange)

20. 20 Method 2D knowledge-based segmentation – –Based on Kobashi’s work – –Dynamic thresholding – –Progressive landmarking Combined with 3D active contouring –Do not require successful 2D segmentation on every axial slice – –Initialize with 2D segmentation result

21. 21 A B C D E 2D Segmentation Results

22. 22 2D/3D Iteration 135135 246246 Identify objects that are easy to find, use them to find harder ones.

23. 23 2D/3D Iteration – cont. 7911 81012

24. 24 Geometrical Feature- Based Similarity Given: A stored database DB of CT scans from prototypical reference head and neck cancer patients and a single query CT scan Q from a target patient. Find: Similarity between Q and each database image d in DB in order to find the most similar database images { d s }.

25. 25 Structures Outer body contour Outer body contour Mandible Mandible Hyoid Hyoid Internal jugular veins Internal jugular veins

26. 26 Feature Types Simple numeric 3D regional properties: volume and extents. Simple numeric 3D regional properties: volume and extents. Vector properties: relative location between structures. Vector properties: relative location between structures. Shape properties: surface meshes of structures. Shape properties: surface meshes of structures.

27. 27 Features for Similarity Measure Volume and extents of the overall region Volume and extents of the overall region Normalized centroid of hyoid and mandible Normalized centroid of hyoid and mandible 3D centroid difference vector between mandible and hyoid 3D centroid difference vector between mandible and hyoid 2D centroid difference vectors between hyoid and jugular veins 2D centroid difference vectors between hyoid and jugular veins Surface meshes of mandible and outer body contour Surface meshes of mandible and outer body contour

28. 28 Mesh Feature Distance Register reference mesh and target mesh with Iterative Closest Point (ICP), result. Register reference mesh S R and target mesh S T with Iterative Closest Point (ICP), result T. Hausdorff distance between Hausdorff distance between two aligned surface meshes, TS R and S T ),(max),( T Sp TRh STpdSTSd R   The Hausdorff distance is the maximum distance from any point in the transformed reference image to the test image.

29. 29 Feature Vector Distance Given feature vectors F d and F Q for model d and query Q in the feature vector space R N. Given feature vectors F d and F Q for model d and query Q in the feature vector space R N. 2 1 2 1 ),(),(          i Q N i i diiQdF FFdwFFD

30. 30 Evaluation Surface mesh distance after full image registration D H – slow. Surface mesh distance after full image registration D H – slow. Feature vector distance D F – fast. Feature vector distance D F – fast. DHDH DFDF corr_coef( D H, D F ) = 0.72 = 0.72 Images with small feature vector distance should produce the best results after registration.

31. 31 Experiment Results 50 head and neck patient CT sets. 50 head and neck patient CT sets. 34 subjects are segmented. 34 subjects are segmented. 20 subjects with lymph node regions drawn by experts. 20 subjects with lymph node regions drawn by experts. Image registration Image registration 20 * (20 – 1) = 380 total cases.

32. 32 Auto-segmentation Results Correct Segmentations Correct Segmentations

33. 33 Auto-segmentation cont. Incorrect Segmentations Incorrect Segmentations Carotid artery misidentified as Hyoid partly missing due to jugular vein due to surgery. too low inter-slice resolution.

34. 34 SuccesssFailureIncorrect% of success Cervical Spine3400100.00% Respiratory Tract3400100.00% Mandible3400100.00% Hyoid3400100.00% ThyroidCartilage330197.06% Left Internal Jugular Vein273479.41% Right Internal Jugular Vein311291.18% Left Carotid Artery259073.53% Right Carotid Artery304088.24% Left SCM2410070.59% Right SCM259073.53% Auto-segmentation cont.

35. 35 Image Registration Results Total casesSuccessfulSuccess rate (%) Mattes method38036796.57% New method using landmark correspondence380 100.00% AverageStandard deviation Mattes method32 minutes6 minutes New method using landmark correspondence26 minutes5 minutes Time of Convergence Success/Failure

36. 36 Quantitative Evaluation - Surface Mesh Distance ° g Projected Region S R ° g Color is distance to truth. Ground Truth: Expert Drawn Target Region S T ° g D H (S R ° g, S T, n) : Hausdorff distance n : lymph node region

37. 37 AverageStandard deviation Mattes method2.851.44 New method using landmark correspondence2.120.64 ° g D H (S R ° g, S T, 1B) for all S R, S T. Measurement in centimeter. Mattes distance larger than landmark distance.

38. 38 AverageStandard deviation Mattes method1.020.51 New method using landmark correspondence0.590.21 ° g Mean_distance(S R ° g, S T, 1B) for all S R, S T. Measurement in centimeter.

39. 39 Similarity Evaluation R H (i, Q) : the i th ranked reference subject for target Q based on the image registration results, D H. R F (i, Q) : the i th ranked reference subject based on geometrical features, D F. P(R F (1, Q) =R H (1, Q)) = 80% P(R F (1, Q) =R H (2, Q)) = 10% P(R F (1, Q) =R H (3, Q)) = 4%

40. 40 Similarity Evaluation Examples DHDH DHDH DFDF DFDF corr_coef( D H, D F ) = 0.74 = 0.74 corr_coef( D H, D F ) = 0.68 = 0.68

41. 41 Similarity Evaluation – Surface Mesh Distance Average Standard deviation D H for the closest reference subject to each target based on feature distance 1.280.31 D H for all reference and target subjects 2.590.90 Measurement in centimeter. So its better to find the closest subject.

42. 42 Qualitative Evaluation – 1.1 Mattes Expert w/ Landmark Drawn Clinically acceptable target projection. Clinically acceptable target projection.

43. 43 Mattes Expert w/ Landmark Drawn Qualitative Evaluation – 1.2 Clinically acceptable target projection. Clinically acceptable target projection.

44. 44 Mattes Expert w/ Landmark Drawn Qualitative Evaluation – 1.3 Clinically acceptable target projection. Clinically acceptable target projection.

45. 45 Mattes Expert w/ Landmark Drawn Qualitative Evaluation – 2 Clinically unacceptable target projection. Clinically unacceptable target projection.

46. 46 Conclusion Inter-subject image registration technique shows promise for lymph node region auto-contouring. Inter-subject image registration technique shows promise for lymph node region auto-contouring. Knowledge-based auto-segmentation is useful for head and neck CT. Knowledge-based auto-segmentation is useful for head and neck CT. Fast similar subject search is possible and critical as reference database grows. Fast similar subject search is possible and critical as reference database grows.

47. 47 Future Work Integrate and evaluated in a clinical environment. Integrate and evaluated in a clinical environment. Generalize to other types of cancer. Generalize to other types of cancer. Regional lymphatic involvement prediction. Regional lymphatic involvement prediction. Improve image registration results. Improve image registration results. Improve auto-segmentation results. Improve auto-segmentation results. –Validation logic –Knowledge-based 3D active contour constraints

48. 48 Acknowledgement Linda Shapiro Linda Shapiro Ira Kalet Ira Kalet Jim Brinkley Jim Brinkley David Haynor David Haynor David Mattes David Mattes Mark Whipple Mark Whipple Jerry Barker Jerry Barker Carolyn Rutter Carolyn Rutter Rizwan Nurani Rizwan Nurani

49. 49 Contributions The first auto target contouring tool for radiation therapy. (AMIA 2002) The first auto target contouring tool for radiation therapy. (AMIA 2002) An auto-segmentation method combining 2D dynamic thresholding and 3D active contouring. (IEEE CBMS 2006) An auto-segmentation method combining 2D dynamic thresholding and 3D active contouring. (IEEE CBMS 2006) An image registration method using landmark correspondences in conjunction with mutual information optimization. (IEEE ISBI 2006) An image registration method using landmark correspondences in conjunction with mutual information optimization. (IEEE ISBI 2006) A patient similarity measurement using 3D geometrical features of anatomical structures. (IEEE ISBI 2007) A patient similarity measurement using 3D geometrical features of anatomical structures. (IEEE ISBI 2007)