1. Introduction to Biomedical Image Analysis BMI 705 Winter 2009 Kun Huang Department of Biomedical Informatics Ohio State University

2. -Introduction to biomedical imaging -Imaging modalities -Components of an imaging system -Elements of image processing techniques -Machine learning and image analysis

3. -Why imaging? -Diagnosis X-ray, MRI, Ultrasound, microscopic imaging (pathology and histology) … -Functional analysis Functional MRI -Visualization (invasive and noninvasive) 3-D, 4-D -Phenotyping/Quantification Microscopic imaging for different genotypes, tissue microarray, cell count, volume rendering, Ca 2+ concentration …

4. -Why imaging? -Visualization (invasive and noninvasive) 3-D, 4-D

5. -Why imaging? -Phenotyping/Quantification Neuhaus’98 Calcium imaging (fluorescence) Invitrogen

6. -Why imaging? -Phenotyping/Quantification Perlman et. al., Science Nov. 2004 ~600,000 images, ~70,000,000 cells, ~10 9 data points

7. -Why imaging? -Phenotyping/Quantification

8. Structural Complexity

9. Multiscale Multimodal Approach The blind men and the elephant Individual imaging modality can only probe one aspect of the system. A comprehensive understanding calls for the integration of multiple modalities. Function  Physiology  Tissue  Cell  Molecular  Dynamics Meter to nanometer

11. Dataset Size: Systems Biology Future big science animal experiments on cancer, heart disease, pathogen host response Basic small mouse is 3 cm 3 1 μ resolution – very roughly 10 13 bytes/mouse Molecular data (spatial location) multiply by 10 2 Vary genetic composition, environmental manipulation, systematic mechanisms for varying genetic expression; multiply by 10 3 Total: 10 18 bytes per big science animal experiment

12. -How to do imaging? -Interdisciplinary research -Electrical engineering -Physics -Chemistry -Mathematics -Computer science -Statistics -… -Biomedical sciences -Pathology -Radiology -Pharmacology -Clinical study -Patient care -…

13. -Components of Imaging System -Instrumentation : -Electrical engineering, physics, histochemistry … -Image generation -Sensor technology (e.g., scanner), coloring agents … -Image processing and enhancement -Both software, hardware, or experimental (dynamic contrast) -Image analysis at all levels -Image processing, computer vision, machine learning, pattern recognition, knowledge computing … -Image storage and retrieval -Database/data warehouse

14. -Components of Imaging System -Instrumentation : From Dr. Petra Schmalbrock Philips 7T

15. -Components of Imaging System -Instrumentation/image generation : Dr. Raman

16. -Areas of Image Processing and Analysis -Image enhancement -Color correction, noise removal, contrast enhancement … -Feature extraction -color, point, edge (line, curves), area -cell, tissue type, organ, region -Segmentation -Registration -3-D reconstruction -Visualization -Quantization

17. Medical Imaging vs. Biological Imaging Medical imaging is for clinical use. It is to implemented to facilitate human decision (diagnosis). E.g., Computer Aided Diagnosis (CAD). It will never replace human being in decision making. Mostly in vivo imaging. Imaging informatics is an important component of medical informatics. The storage, retrieval and processing of the image involve many legal and policy related issues as well as economic concerns. There are existing standards and commercial systems in storing and formatting the images. Real application requires long term validation and FDA application (including algorithms).

18. Medical Imaging vs. Biological Imaging Biological imaging is for scientific discovery. Computer is used to replace human in performing tedious quantitative tasks. Algorithms are usually highly domain specific. Many projects are related to microscopic imaging. Small animal in vivo imaging is also emerging. Large amount of data is a big issue.

19. DICOM Image Standard Digital imaging and communication in medicine Groups information into a single data file (set) Contains information such as patient ID, acquisiton parameters and conditions Consists of a header with both standard and freeform fields and image data http://medical.nema.org/

20. In Vivo Imaging vs. In Vitro Imaging In vivo imaging Live sample (not always live animal) X-ray Computer Tomography (CT) Magnetic Resonance Imaging (MRI) Electron Paramagnetic Resonance (EPR) Positron Emission Tomography (PET) Ultrasound Bioluminescence Imaging Optical Coherent Tomography Microscopy Video microscopy …

21. In Vivo Imaging vs. In Vitro Imaging In vivo imaging (cont’d) Live sample (not always live animal) Good for longitudinal study Resolution of many modalities are low Multimodalities are usually combined

22. In Vivo Imaging vs. In Vitro Imaging In vitro imaging Mostly microcopy imaging Light microscope Fluorescent microscope Multiple photo microscope Confocal microscope Multispectral microscope Atomic force microscope Electron microscope Video microscopy … Large data size is an issue

23. -Introduction to biomedical imaging -Imaging modalities -Components of an imaging system -Elements of image processing techniques -Machine learning and image analysis

24. Digital Image

25. imss(:,:,1) = 17 36 39 51 70 60 75 110 79 52 58 44 36 38 32 65 129 74 134 80 20 53 46 57 32 42 103 94 80 76 51 33 25 23 81 44 23 76 40 90 77 27 36 42 61 60 44 112 86 59 75 98 87 87 48 89 97 54 145 29 98 44 5 86 15 39 43 76 78 88 108 31 51 50 15 52 36 131 46 92 170 144 61 39 72 73 72 21 26 32 141 120 153 55 36 75 26 20 45 49 imss(:,:,2) = 11 41 91 210 255 255 255 255 255 239 43 64 51 91 145 255 255 255 219 221 32 27 64 53 86 117 224 255 255 255 18 24 10 50 12 13 100 241 255 222 2 23 2 18 20 25 50 168 179 147 5 0 0 33 14 1 45 83 137 132 15 3 22 0 2 15 43 25 99 124 25 11 15 11 3 4 6 18 56 45 10 33 8 22 6 31 37 23 28 2 11 13 0 9 4 0 30 21 16 41 imss(:,:,3) = 3 0 8 4 36 45 60 81 65 65 7 3 3 1 21 31 58 63 63 71 4 6 3 3 6 9 27 32 63 61 7 2 15 10 10 11 11 35 54 66 1 4 7 5 6 9 15 22 43 52 5 3 2 0 10 5 3 13 24 19 10 2 11 7 1 4 11 7 15 27 4 3 5 6 3 10 7 3 7 23 7 2 12 7 0 2 1 8 7 18 10 7 1 5 2 3 4 9 20 21

26. Digital Image

27. Simplest segmentation: thresholding

28. -Image Enhancement -Denoise -Averaging -Median filter 1/9 20543 78322 115189200 43

29. Wavelet-based denoising

30. -Example Crystal detection Median filter removes “spikes” in the image. From M. Lee

31. Example -Crystal detection

32. -Image Enhancement -Color and intensity adjustment -Histogram equalization

34. -Image Enhancement -Color space transform RGB -> HSV, HSL, YCbCr, … R = 64 G = 31 B = 62 R = 125 G = 80 B = 147 H = 199 S = 117 V = 147 H = 214 S = 132 V = 64

35. More Sophisticated Operations -Color space transform RGB -> HSV, HSL, YCbCr, Lab, …

36. RGB  Lab K-means algorithm clusters the pixels in the new color space into three groups. Group merging. Mophorlogical operations.

37. -Feature Extraction -Region detection – morphology manipulation -Dilate and Erode -Open -Erode  dilate -Small objects are removed -Close -Dilate  Erode -Holes are closed -Skeleton and perimeter

38. − Example Cell detection

39. -Feature Extraction -Edge detection -Gradients -Canny edge detector -Gaussian smoothing -Gradients -Two thresholds -Thinning 11 11 11 11 1111 1111

40. -Feature Extraction -Edge detection

41. −Boundary detection and measurement −Active contour

42. −Boundary detection and measurement −Active contour

43. −Tools −PhotoShop −IrfanView −PaintShop −Metamorph −ImageJ −Matlab

44. -Example -Virtual Simulation of Temporal Bone Dissection