1. Segmentation In The Field Medicine Advanced Image Processing course By: Ibrahim Jubran Presented To: Prof. Hagit Hel-Or

2. What we will go through today A little inspiration. Medical image segmentation methods: -Deformable Models. -Markov Random Fields. Results.

3. Why Let A Human Do It, When The Computer Does It Better? “Image data is of immense practical importance in medical informatics.” For instance: CAT, MRI, CT, X-Ray, Ultrasound. All represented as images, and as images, they can be processed to extract meaningful information such as: volume, shape, motion of organs, layers, or to detect any abnormalities.

4. Why Let A Human Do It, When The Computer Does It Better? Cont. Here’s a task for you: Look at this image: could you manually mark the boundaries of the two abnormal regions? Answer: Maybe…

5. Not Bad...

6. And… What if I told you to do it in 3D? Answer? You would probably fail badly.

7. But… the computer, on other hand, dealt with it perfectly:

8. Common Methods: Deformable Models Deformable models are curves whose deformations are determined by the displacement of a discrete number of control points along the curve. Advantage: usually very fast convergence, depending on the predetermined number of control points. Disadvantage: Topology dependent: a model can capture only one ROI, therefore in images with multiple ROIs we need to initialize multiple models.

9. Deformable models A widely used method in the medicine field is the Deformable Models, which is divided into two main categories: -The Parametric Deformable Models. - The Geometric Deformable Models. We shall discuss each of them briefly.

10. Geometric Models Geometric Models use a distance transformation to define the shape from the n-dimentional to an n+1-dimentional domain (where n=1 for curves, n=2 for surfaces on the image plane…)

11. Example of a transformation Here you see a transformation from 1D to 2D.

12. Geometric Models cont. Advantages: 1) The evolving interface can be described by a single function even if it consists of more than one curve. 2) The shape can be defined in a domain with dimensionality similar to the dataset space (for example, for 2D segmentation, a curve is transformed into a 2D surface) -> more mathematically straightforward integration of shape and appearance.

13. In Other Words… We transform the n dimensional image into an n+1 dimensional image, then we try to find the best position for a “plane”, called the “zero level set”, to be in. We start from the highest point and descend, until the change in the gradient is below a predefined threshold.

14. And Formally…

15. Geometric Deformable Models Example

16. Geometric Models Results

17. Geometric Deformable Models Short demonstration Click to watch a demonstration of the MRF

18. Parametric Models Also known as “Active contours”, or Snakes. Sounds familiar? The following slides are taken from Saar Arbel’s presentation about Snakes. Five instances of the evolution of a region based deformable model

19. A framework for drawing an object outline from a possibly noisy 2D image. An energy-minimizing curve guided by external constraint forces and influenced by image forces that pull it towards features (lines, edges). Represents an object boundary or some other salient image feature as a parametric curve

20. External Energy Function Internal Energy Function A set of k points (in the discreet world) or a continuous function that will represent the points

22. Snakes are autonomous and self-adapting in their search for a minimal energy state They can be easily manipulated using external image forces They can be used to track dynamic objects in temporal as well as the spatial dimensions

23. Common Methods: Learned Based Classification Learning based pixel and region classification is among the popular approaches for image segmentation. Those methods use the advantages of supervised learning (training from examples) to assign a probability for each image site of belonging to the region of interest (ROI).

24. The MRF & The Cartoon Model A cartoon model

25. The Markov Random Field The name “Markov Random Field” might sound like a hard and scary subject at first… I thought so too when I started reading about it… Unfortunately I still do.

26. An unrelated photo of Homer Simpson Click to watch a demonstration of the MRF https://www.youtube.com/watch?v=hfOfAqLWo5c

27. The MRF & The Cartoon Model

29. The Cartoon Model Cont.

30. More Cartoon Model Examples Original labelled

31. The Probabilistic Approach For Finding The Model

32. The Probabilistic Approach cont.

33. Observation and Hidden Variables

34. Defining the Parameters needed

35. original

36. Example We want the regions to be more homogeneous.

37. Example cont.

38. Our Goal

39. An unrelated photo of Homer Simpson (again) Click to watch a demonstration of the MRF

40. A Lesson In Probability

41. Defining the Parameters needed Cont.

42. The MRF cont.

43. Feature extraction

44. Notes REMINDER: our features will be texture and color. We use the CIE-L*U*V color plane, so regions will be formed where both features are homogeneous while boundaries will be present where there is discontinuity in either color or texture.

45. CIE-L*u*v* VS. RGB CIELUV color histogram RGB color histogram

46. The Markov Random Field Segmentation Model Let’s call this SQUIRREL

47. Definitions

48. And now… the FUN part !! Don’t listen to me, just RUN!

49. The Image Process

50. The Image Process cont.

51. Intuition 1 2 3 4 5 6

52. Intuition cont.

53. The Image Process cont. Let’s call this CAT Let’s call this DOG

54. Fun Equations cont.

56. MINIMIZATION There are two main methods used to minimize our expression: 1) ICM (Iterated Conditional Modes). 2) Gibbs sampler. In some of the results we would be comparing those two methods.

57. Parameter estimation There are some parameters in our equations that should be estimated, with or without supervision: 1) If a training set is provided, then those parameters can be easily calculated based on the given data. 2) If we do not have such a training set, we would have to use an iterative EM algorithm.

58. Supervised Parameter Estimation cont.

59. Unsupervised Parameter Estimation cont.

60. The EM Algorithm E step: compute a distribution on the labels based on the current parameter estimates. M step: calculating the parameters again based on the new labels, very similar to the supervised case. We repeat those two steps until convergence. K-Means is a specific case of the EM algorithm. The EM approach is similar to the Gradient Descent.

61. MRF Results (Supervised) Texture Color Combined ICM Gibbs Sampler

62. MRF Results (Unsupervised) Texture Color Combined ICM Gibbs Sampler

63. The Finale Segmentation in the medicine field covers many topics and methods, today we covered 2 of them, saw some results and introduced a small estimation algorithm widely used in those topics.

64. References A Markov random field image segmentation model for color textured images. –Zoltan Kato, Ting-Chuen Pong. Medical Image Segmentation. –Xiaolei Huang, Gavriil Tsechpenakis. Deformable Model-Based Medical Image Segmentation. –Gavriil Tsechpenakis. http://en.wikipedia.org/wiki/Markov_random_field Saar Arbel’s presentation about snakes. http://en.wikipedia.org/wiki/Expectation%E2%80%93maximization_ algorithm