1. Edward J. Delp  Texture Analysis  February 2000  Slide 1 Texture Analysis and Its Applications in Medical Imaging Edward J. Delp Purdue University School of Electrical and Computer Engineering Video and Image Processing Laboratory (VIPER) West Lafayette, Indiana, USA email: ace@ecn.purdue.edu http://www.ece.purdue.edu/~ace

2. Edward J. Delp  Texture Analysis  February 2000  Slide 2 Breast Cancer Second major cause of cancer death among women in the United States (after lung cancer) Leading cause of nonpreventable cancer death 1 in 8 women will develop breast cancer in her lifetime 1 in 30 women will die from breast cancer

3. Edward J. Delp  Texture Analysis  February 2000  Slide 3 Mammography Mammograms are X-ray images of the breast Screening mammography is currently the best technique for reliable detection of early, non-palpable, potentially curable breast cancer Studies show that mammogram can reduce the overall mortality from breast cancer by up to 30%

4. Edward J. Delp  Texture Analysis  February 2000  Slide 4 Screening Mammography

5. Edward J. Delp  Texture Analysis  February 2000  Slide 5 A Digital Mammogram (normal)

6. Edward J. Delp  Texture Analysis  February 2000  Slide 6 Analysis of Mammograms Density 1Density 2Density 3Density 4

7. Edward J. Delp  Texture Analysis  February 2000  Slide 7 Digital Mammography Resolution - 50  pixel size –3000 x 4000 pixels (12,000,000 pixels) –8-16 bits/pixels 8 bits/pixel (12 MB) 16 bits/pixel (24 MB) Each study consists of 48-96 MB! 200 patients per day can results to 20GB/day Problems with storage and retrieval

8. Edward J. Delp  Texture Analysis  February 2000  Slide 8 Problems in Screening Mammography Radiologists vary in their interpretation of the same mammogram False negative rate is 4 – 20% in current clinical mammography Only 15 – 34% of women who are sent for a biopsy actually have cancer

9. Edward J. Delp  Texture Analysis  February 2000  Slide 9 Current Research in Computer Aided Diagnosis (CAD) The goal is to increase diagnostic accuracy as well as the reproducibility of mammographic interpretation Most work aims at detecting one of the three abnormal structures Some have explored classifying breast lesions as benign or malignant The implementation of CAD systems in everyday clinical applications will change the practice of radiology

10. Edward J. Delp  Texture Analysis  February 2000  Slide 10 Three Types of Breast Abnormalities Micro- calcification Circumscribed Lesion Spiculated Lesion

11. Edward J. Delp  Texture Analysis  February 2000  Slide 11 Malignant Microcalcifications Extremely vary in form, size, density, and number, usually clustered within one area of the breast, often within one lobe Granular: dot-like or elongated, tiny, innumerable Casting: fragments with irregular contour, differ in length

12. Edward J. Delp  Texture Analysis  February 2000  Slide 12 Benign Microcalcifications Homogenous, solid, sharply outlined, spherical, pearl-like, very fine and dense Crescent-shaped or elongate Ring surrounds dilated duct, oval or elongated, varying lucent center, very dense periphery Linear, often needle like, high and uniform density

13. Edward J. Delp  Texture Analysis  February 2000  Slide 13 Benign Microcalcifications (Cont.) Ring-shaped, oval, center radiolucent, occur within skin Egg-shell, center radiolucent or of parenchymal density Coarse, irregular, sharply outlined and very dense Similar to raspberry, high density but often contain small, oval-shaped lucent areas

14. Edward J. Delp  Texture Analysis  February 2000  Slide 14 Malignant Masses High density radiopaqueSolid tumor, may be smooth or lobulated, random orientation

15. Edward J. Delp  Texture Analysis  February 2000  Slide 15 Benign Masses Halo: a narrow radiolucent ring or a segment of a ring around the periphery of a lesion Capsule: a thin, curved, radiopaque line that surrounds lesions containing fat Cyst: spherical or ovoid with smooth borders, orient in the direction of the nipple following the trabecular structure of the breast

16. Edward J. Delp  Texture Analysis  February 2000  Slide 16 Benign Masses (Cont.) Radiolucent densityRadiolucent and radiopaque combined Low density radiopaque

17. Edward J. Delp  Texture Analysis  February 2000  Slide 17 Malignant Spiculated Lesions Scirrhous carcinoma: distinct central tumor mass, dense spicules radiate in all directions, spicule length increases with tumor size Early stage scirrhous carcinoma: tumor center small, may be imperceptible, only a lace-like, fine reticular radiating structure which causes parenchymal distortion and/or asymmetry

18. Edward J. Delp  Texture Analysis  February 2000  Slide 18 Benign Spiculated Lesions Sclerosing duct hyperplasia: translucent, oval or circular center, the longest spicules are very thin and long, spicules close to the lesion center become numerous and clumped together in thick aggregates Traumatic fat necrosis: translucent areas are within a loose, reticular structure, spicules are fine and of low density

19. Edward J. Delp  Texture Analysis  February 2000  Slide 19 Statistical Segmentation of Mammograms Mary L. Comer, Sheng Liu, and Edward J. Delp Abnormalities in mammograms are disruptions of the normal structures It is desirable to partition a mammogram into texture regions Study the use of a new statistical method for the detection of abnormalities in mammograms

20. Edward J. Delp  Texture Analysis  February 2000  Slide 20 Non-statistical Approaches Use a series of heuristics, such as filtering, thresholding, and texture analysis Suffer from a lack of robustness when the number of images to be classified is large

21. Edward J. Delp  Texture Analysis  February 2000  Slide 21 EM/MPM Algorithm Assign each pixel in the mammogram membership to one of 3 texture classes: tumor, normal tissue, and background, depending on statistical properties of the pixel and its neighborhood Both the original mammogram and its class labels are modeled as discrete parameter random fields Use a combination of the expectation-maximization and the maximization of the posterior marginals (EM/MPM) algorithms to segment mammograms

22. Edward J. Delp  Texture Analysis  February 2000  Slide 22 Image Models

23. Edward J. Delp  Texture Analysis  February 2000  Slide 23 Segmentation Algorithm

24. Edward J. Delp  Texture Analysis  February 2000  Slide 24 Advantages The values of all parameters of the MPM algorithm need not be known a priori Provide indication of the reliability of each classified pixel Detect various types of tumors within the same framework

25. Edward J. Delp  Texture Analysis  February 2000  Slide 25 Database Images used in this research were provided courtesy of the Center for Engineering and Medical Image Analysis at the University of South Florida Abnormal mammograms have an interpretation file that indicates the types and positions of abnormalities 220 micron resolution

26. Edward J. Delp  Texture Analysis  February 2000  Slide 26 Experiments The spatial interaction parameter  and cost parameters  were determined experimentally using a variety of mammography images a priori knowledge is used to initialize the model parameter vector  The reliability information is displayed as an image where pixel values are proportional to the estimated marginal conditional probability mass function of the label field: larger graylevel  higher reliability of classification

27. Edward J. Delp  Texture Analysis  February 2000  Slide 27 Experimental Results Original mammogram Segmented image Ground truthReliability image

28. Edward J. Delp  Texture Analysis  February 2000  Slide 28 Experimental Results (Cont.) Original mammogram Segmented image Ground truthReliability image

29. Edward J. Delp  Texture Analysis  February 2000  Slide 29 Multiresolution Detection of Spiculated Lesions in Digital Mammograms Sheng Liu and Edward J. Delp Spiculations or a more stellate appearance in mammograms indicates with near certainty the presence of breast cancer Detection of spiculated lesions is very important in the characterization of breast cancer

30. Edward J. Delp  Texture Analysis  February 2000  Slide 30 Difficulties Center masses of spiculated lesions are usually irregular with ill-defined borders In some cases, the center masses are too small to be perceptible Spiculated lesions vary from a few millimeters to several centimeters in size

31. Edward J. Delp  Texture Analysis  February 2000  Slide 31 Difficulties (Cont.) Computer aided diagnosis of digital mammograms generally consists of feature extraction followed by classification It is very difficult to determine the neighborhood size that should be used to extract features which are local If the neighborhood is too large, small lesions may be missed If the neighborhood is too small, one may not be able to capture features of larger lesions

32. Edward J. Delp  Texture Analysis  February 2000  Slide 32 Appearance of A Spiculated Lesion at Multiple Resolutions

33. Edward J. Delp  Texture Analysis  February 2000  Slide 33 Block Diagram of Multiresolution Detection of Spiculated Lesions

34. Edward J. Delp  Texture Analysis  February 2000  Slide 34 Multiresolution Decomposition Linear phase nonseparable 2D perfect reconstruction wavelet transform –does not introduce phase distortions in the decomposed images –no bias is introduced in the horizontal and vertical directions as a separable transform would The impulse response of the analysis low pass filter

35. Edward J. Delp  Texture Analysis  February 2000  Slide 35 Advantages of Multiresolution Approach Overcomes the difficulty of choosing a neighborhood size a priori (variable lesion size) Requires less computation by –starting with the least amount of data –propagating detection results to finer resolutions

36. Edward J. Delp  Texture Analysis  February 2000  Slide 36 A Spiculated Lesion Distorts the Normal Breast Duct Structure Normal duct structures of the breast radiate from the nipple to the chest wall Spiculated lesion radiates spicules in all directions

37. Edward J. Delp  Texture Analysis  February 2000  Slide 37 Gradient Orientation Histogram Has a peak at the ductal structure orientation near a normal pixel Flat near a lesion pixel

38. Edward J. Delp  Texture Analysis  February 2000  Slide 38 Example Histograms A normal region A spiculated region

39. Edward J. Delp  Texture Analysis  February 2000  Slide 39 Notation (i, j) — spatial location at row i and column j f(i, j) — pixel intensity at (i, j)  S ij — some neighborhood of (i, j) M — the number of pixels within  S ij D y (i, j) and D x (i, j) — estimate of the vertical and horizontal spatial derivatives of f at (i, j), respectively  (i, j) = tan -1 {D y (i, j)/D x (i, j)}  (-  /2,  /2] — estimate of the gradient orientation at (i, j)

40. Edward J. Delp  Texture Analysis  February 2000  Slide 40 Notation (Cont.) hist ij — histogram of  within  S ij using 256 bins hist ij (n) — # of pixels in  S ij that have gradient orientations, where n = 0, 1, …, 255 — average bin height of hist ij

41. Edward J. Delp  Texture Analysis  February 2000  Slide 41 Folded Gradient Orientation M  + (i, j) and M  - (i, j) — number of positive and negative gradient orientations within  S ij, respectively and — average positive and negative gradient orientations, respectively — folded gradient orientation

42. Edward J. Delp  Texture Analysis  February 2000  Slide 42 Features Differentiate Spiculated Lesions from Normal Tissue Mean pixel intensity in  S ij — Standard deviation of pixel intensities in  S ij —

43. Edward J. Delp  Texture Analysis  February 2000  Slide 43 Features Differentiate Spiculated Lesions from Normal Tissue (Cont.) Standard deviation of gradient orientation histogram in  S ij — Standard deviation of the folded gradient orientations in  S ij —

44. Edward J. Delp  Texture Analysis  February 2000  Slide 44 Why Folded Gradient Orientation? So that is not sensitive to the nominal value of , but to the actual gradient orientation variances The gradient orientation distance between  /2 and -  /4 is the same as that between  /2 and  /4, however –   ([  /2, -  /4]) = 2.8 –   ([  /2,  /4]) = 0.3 -  /4 folds to 3  /4, now   ’ ([  /2, -  /4]) =   ’ ([  /2,  /4]) = 0.3

45. Edward J. Delp  Texture Analysis  February 2000  Slide 45 Multiresolution Feature Analysis An M  M region at a coarser spatial resolution N/n  N/n corresponds to an nM  nM region in the original mammogram with spatial resolution N  N if a set of features extracted within an 8  8 window at the original resolution N  N can capture spiculated lesions of size 1mm, then the same set of features extracted at the coarser resolution N/4  N/4, using the same sized 8  8 window, should be able to detect spiculated lesions of size 4mm.

46. Edward J. Delp  Texture Analysis  February 2000  Slide 46 Multiresolution Feature Analysis (Cont.) Choose a neighborhood that is small enough to capture the smallest possible spiculated lesion in the finest resolution Fix this neighborhood size for feature extraction at all resolutions Larger lesions will be detected at a coarser resolution Smaller lesions can be detected at a finer resolution

47. Edward J. Delp  Texture Analysis  February 2000  Slide 47 Test Pattern at Multiple Resolutions An ideal spiculated lesion and normal duct structures embedded in uncorrelated Gaussian distributed noise

48. Edward J. Delp  Texture Analysis  February 2000  Slide 48 Multiresolution Feature Extraction Each feature at different resolutions is extracted within same sized circular neighborhoods Features are able to discriminate a spiculated lesion from complex background when extracted within an appropriate neighborhood whose size matches to that of the lesion Fail when the sizes mismatch

49. Edward J. Delp  Texture Analysis  February 2000  Slide 49 Feature   ’ at Multiple Resolutions

50. Edward J. Delp  Texture Analysis  February 2000  Slide 50 Feature  hist at Multiple Resolutions

51. Edward J. Delp  Texture Analysis  February 2000  Slide 51 Feature at Multiple Resolutions

52. Edward J. Delp  Texture Analysis  February 2000  Slide 52 Feature  f at Multiple Resolutions

53. Edward J. Delp  Texture Analysis  February 2000  Slide 53 A Simple Binary Tree Classifier

54. Edward J. Delp  Texture Analysis  February 2000  Slide 54 Advantages of Tree-Structured Approach Robust with respect to outliers and misclassified points in the training set The classifier can be efficiently represented Once trained, classification is very fast Provides easily understood and interpreted information regarding the predictive structure of the data Classifier used is described in a paper by Gelfand, Ravishankar, and Delp

55. Edward J. Delp  Texture Analysis  February 2000  Slide 55 Multiresolution Detection At each resolution, five features are used: the four features extracted at that resolution plus the feature  hist extracted from the next coarser resolution Detection starts from the second coarsest resolution A positive detection at a coarser resolution eliminates the need for both feature extraction and detection at the corresponding pixel locations at all finer resolutions A negative result at a coarser resolution will be combined with those at finer resolutions via weighted sum

56. Edward J. Delp  Texture Analysis  February 2000  Slide 56 Database MIAS database provided by the Mammographic Image Analysis Society in the UK 50 micron resolution A total of 19 mammograms containing spiculated lesions Smallest lesion extends 3.6mm in radius Largest lesion extends 35mm in radius

57. Edward J. Delp  Texture Analysis  February 2000  Slide 57 Half/half Training Methodology The 19 mammograms containing spiculated lesions together with another 19 normal mammograms are random split into two sets with approximately an equal number of lesion and normal mammograms in each set Each set was used separately as a training set to generate two BCTs A BCT trained by one set was used to classify mammograms in the other set, and vice versa

58. Edward J. Delp  Texture Analysis  February 2000  Slide 58 Detection Results A 35.0mm lesion detected at the coarsest resolution Automatic detectionGround truth

59. Edward J. Delp  Texture Analysis  February 2000  Slide 59 Detection Results Automatic DetectionGround truth A 12.4mm lesion detected at the second coarsest resolution

60. Edward J. Delp  Texture Analysis  February 2000  Slide 60 Detection Results A 6.6mm lesion detected at the finest resolution Automatic DetectionGround truth

61. Edward J. Delp  Texture Analysis  February 2000  Slide 61 FROC Analysis 100% TP detection at 2.2 FP per image 84.2% TP detection at less than 1 FP per image

62. Edward J. Delp  Texture Analysis  February 2000  Slide 62 Summary Multiresolution detection eliminates the problem of choosing a neighborhood size a priori to capture features of lesions of varying sizes Using features across resolutions simultaneously helps capture spiculated lesions of sizes that exist between the resolutions Top-down approach requires less computation by starting with the least amount of data and propagating detection results to finer resolutions