1. ANALYSIS OF A LOCALLY VARYING INTENSITY TEMPLATE FOR SEGMENTATION OF KIDNEYS IN CT IMAGES MANJARI I RAO UNIVERSITY OF NORTH CAROLINA AT CHAPEL HILL

2. OVERVIEW  Introduction  Background  Materials and Methods  Results  Conclusions and Discussion

3. External Beam Radiation Therapy (Image Types : CT, MRI, Ultrasound, PET etc.)  Simulation  Treatment Planning

4. Segmentation  Segmentation is a process of extraction of information from an image in such a way that the output image contains much less information than the original one, but the little information that it contains is much more relevant to the purpose of the task.  Medical Image Segmentation – Extraction of anatomical structures such as the kidney.

5. Segmentation in Treatment Planning  Manual Segmentation  Computer-based models  Combination techniques

6. Virtual Simulation  Combination of simulation and treatment planning  Detect tumor sites from different viewpoints  Design irradiation beam and orientation  Calculate dose distribution  Presence of patient not required

7. Manual Segmentation - Limitations  Dependent on the experience of the operator  Inter-operator and Intra-operator variability  Difficult to identify 3D structures on a 2D slice  Complexity is increased due to presence of infiltrating tumor and disease

8. OVERVIEW  Introduction  Background  Materials and Methods  Results  Conclusions and Discussion

9. Shape Representation Using Medial Models  A medial axis is the locus of centers of spheres that are bitangent to the surface of the object being represented  The center point of each such sphere is a point on the medial surface  M-reps are a class of medial models

10. Object Representation via M-reps  An m-rep consists of a grid of medial atoms, each of which is made of a hub and a pair of spokes  The atoms in an m-rep define the following: A medial position x in the mesh Two vectors of length r, which is the radius of the bitangent sphere inscribed inside the object. The vectors are the spokes that point to these points of tangency. The angle Θ formed between each of the spokes and the angular bisector b. A frame F = (n,b,b ┴ ), where n is normal to both b and b ┴, which defines the tangent plane to the medial sheet at x. The curvature η of the object about a point.

11. Object Representation via M-reps  M-reps define an object-based intrinsic coordinate system represented by (u, v, t,  ), where u and v represent the row and column corresponding to the position of the atom in the medial mesh t indicates which side of the medial locus the point lies on; t= -1 or +1 for internal medial points and varies around the crest from -1 through 0 at the boundary to +1  measures the distance along the spokes from the boundary, with  > 0 outside and  < 0 inside the boundary and  = -1 at the medial locus u v t  

12. Object Representation via M-reps  The medial locus is a curve for objects in 2D and a sheet for objects in 3D  It is sparsely sampled to produce an approximate surface of the m-rep or the “implied boundary”.

13. Segmentation using M-reps  Transformation of the whole kidney model based on both the similarity transform and Principal Geodesic Analysis or PGA -translation, rotation, scaling of the entire model and shape variation according to the principal modes of variation of a mean model.

14. Segmentation using M-reps  Deformation of each medial atom in the model- translation, rotation and scaling of each individual atom of the medial mesh.

15. Segmentation using M-reps  Atom deformation in this step is based on Geometric Typicality - Calculated by relating an atom’s coordinates predicted by the stage immediately previous to the current deformation stage, and by its neighbors at the current stage. Geometry to Image Match – Calculated based on a template defined in a ‘collar’ region about the boundary.

16. Segmentation using M-reps  Fine-scale surface refinement, i.e., each surface tile of the implied surface is shifted along its normal to optimize the objective function. This scale was not used in the present study.

17. Kidney Segmentation  Surrounded by crowded soft tissue environment – most soft tissue does not contrast well against its background.  High risk due to radiation exposure during treatment of abdominal and pelvic tumors. Hence, important in segmentation during treatment planning.  Simple organ, can be represented by a single-figure m-rep

18. Kidney as an object of interest for this study  Variation of intensities along the boundary of the kidney  Examples

19. Axial ViewCoronal View

21. OVERVIEW  Introduction  Background  Materials and Methods  Results  Conclusions and Discussion

22. Training and Target Images  Training Images To generate a mean kidney m-rep model To generate intensity profiles for the locally varying template

23. Training Images  CT Images – Siemens Somatom Plus 4 scanner  Raster resolution (number of pixels per slice) – 512 x 512  Pixel size – ranged from 0.098mm 2 to 0.156mm 2

24. Criteria for selection of Training Images  Presence of whole kidney(s)  Position of patient during scan

25. Criteria for selection of Training Images  Absence of contrast agent  Absence of tumor, disease or kidney stones

26. Criteria for selection of Training Images  No more than moderate motion artifacts

27. Criteria for selection of Training Images  Margin of at least 2cm on the superior and inferior edges of the kidney.  Slice thickness less than or equal to 5mm per slice.

28. OVERVIEW  Introduction  Background  Materials and Methods  Results  Conclusions and Discussion

29. Image Match based on a ‘Template’  A target pattern at different locations in an image - maximum response when the intensity values of the pixels in the image correlate to the values at the same locations specified by the template  Determined by training the model on a population of user-approved segmentations or the “answers”

30. Image match based on a ‘Template’  For an m-rep, the template is defined in the collar region about the boundary  The width of this region ranged from –0.3r to 0.3r with 0 at the boundary in this study  The template thus corresponds to a Gaussian centered about the boundary with a standard deviation of 0.15

31. Locally varying intensity template  Function of figural positions along the boundary of an m-rep  Generated from a population of training images  Several ‘profile types’ as compared to the Gaussian which had only one

32. Training  Manual segmentation of kidneys from the training data set  Conversion of resulting contours into blurred binary images  Deformation of an m-rep model into the blurred binary images to generate a mean model with principal geometric modes of variation via PGA  Segmentation of the training kidneys by deforming the mean model into the gray-scale training images

33. Training  Generation of intensity profiles at many points on the surface of the segmented kidneys  Classification of the each of the intensity profiles into one of three categories, based on the best match among three analytic filter types Final Step: Segmentation of kidneys in the target images using the locally varying intensity template to compute image match

34. Manual segmentation  Software used for radiation treatment planning – similar to drawing tools  Contours were traced on each cross-sectional slice  Intensity windowing to enhance the quality of the displayed image

35. Manual Slice-by-slice contouring using “Anastruct-Editor”

36. Generation of Training Binary Images  Output of hand-segmentation – series of contour ‘stacks’  Converted to binary images (have two intensity values, 0 for black and 1 for white)  Gaussian smoothing operator was used to blur the images. The  of the operator was approximately 1mm

37. Generation of the mean kidney m-rep  Single figure mesh of 15 atoms (5 rows and 3 columns) was fit into each of the training blurred binary images  The mean model and the corresponding principal modes of variation were obtained from this population of m-reps  Initial m-rep was replaced by the mean and the process was iterated until convergence

38. Generation of the locally varying Kidney Template  Initial Analytic Filters Light-to-dark - higher intensities in the kidney as compared to surrounding structures Dark-to-light - lower intensities in the kidney as compared to surrounding structures Notch - similar intensities in the kidney as well as surrounding structures with a narrow dark region in between

39. Initial Analytic Filters Light-to-dark Dark-to-light Notch Points along the normal Intensity

40.  Image corresponding to the light-to-dark filter

41.  Image corresponding to the dark-to-light filter

42.  Image corresponding to the notch filter

43. Generation of Profiles for the Template  End points of the spokes of the outer atoms in the medial lattice of an m-rep are linked together to form a quadrangular mesh surface  Surface was subdivided to provided a natural framework for defining the boundary at 2562 points on the surface of the m-rep

44. Generation of a Profiles for the Template  At each of the 2562 boundary points, normals were drawn (length from -0.3r to +0.3r with 0 at the boundary as defined by the collar)  Each normal was sampled at 11 points for each of the 2562 boundary points

45. Generation of Profiles for the Template  Segmentation of the training blurred binary images with the mean kidney m- rep  Generation of ‘profiles’ for each m- rep in this population with intensity information from corresponding gray- scale images

46. Generation of the locally varying Kidney Template  Responses of the profiles to each of the three analytic filters, given by the dot product, X.Y=  x i y i, i= 1,2,…, n, X -vector representing the profile Y -vector representing the filter  The highest response among the three filters for each point determined the filter type classification for that point

47. Classification of the profiles according to responses 40.66 16.82 -34.74 Light-to-dark Dark-to-light Notch

48. Converged Mean Filters Points along the normal Intensity Light-to-dark Dark-to-light Notch

49. Filter distribution on the surface of the left and right kidneys in the templates Left KidneyRight Kidney

50. Segmentation of Target Images

51. Axial ViewCoronal View

52. Segmentation of Target Images Axial ViewCoronal View

53. Accuracy Evaluation  VALMET –Software package involving voxel-based comparision Volume overlap - intersection of the two volumes divided by their union Maximum Surface Distance or Hausdorff Distance - the largest distance between two surfaces (Not symmetric) Mean absolute surface distance - how much on average the two surfaces differ (Also not symmetric) Curve 1 Curve 2 a b

54. OVERVIEW  Introduction  Background  Materials and Methods  Results  Conclusions and Discussion

55. Comparison using VALMET– Gaussian Vs Locally Varying Template  24 Target Cases (12 left and 12 right kidneys)  Segmentations were compared to human segmentations performed by two experts  Average increase in volume overlap over all the cases - 1.3%  Mean surface separation between human segmentations and M-rep segmentation - 0.33cm for the Gaussian template and 0.32cm for the locally varying template

56. Comparison using VALMET– Gaussian Vs Locally Varying Template Mean Surface Separation Rater A Vs M-rep Rater B Vs M-rep

57. Results (Contd.) Axial ViewCoronal View

58. Results (Contd.) Axial ViewCoronal View

59. Results (Contd.) – ‘Error’ in Human Segmentation Sagittal View Coronal View

60. Results (Contd.) Coronal ViewSagittal View

61. OVERVIEW  Introduction  Background  Materials and Methods  Results  Conclusions and Discussion

62. Analysis of the Results  Locally varying template based segmentation showed improvement for 65% of the cases  Achieved a greater degree of automation in the entire segmentation process

63. Analysis of the Results Coronal ViewAxial View

64. Analysis of the Results Axial ViewCoronal View

65. Instances of Human Intervention Axial ViewCoronal View

66. Instances of Human Intervention Axial ViewCoronal View

67. Future Directions  More than three filter types to define the template  Statistical evaluation of variation of profiles along the boundary  Templates based on ‘weighted’ profiles, responses which are neighbor dependent  Higher density m-rep model