1. 8D040 Basis beeldverwerking Feature Extraction Anna Vilanova i Bartrolí Biomedical Image Analysis Group bmia.bmt.tue.nl

2. An image is just 2D? No! – It can be in any dimension Example 3D: Voxel-Volume Element

3. Reduction of dimensionality Why feature extraction ? Pixel level Image of 256x256 and 8 bits 256 65536 ~ 10 157826 possible images

4. Incorporation of cues from human perception Transcendence of the limits of human perception The need for invariance Why feature extraction ?

5. Apple detection …

6. Transformation (Rotation)

7. How do we transform an image? We transform a point P How do we transform an image f(P) ? How do we know which Q belongs to P ?

8. How do we transform an image? How do we transform an image f(P) ? We know T which is the transformation we want to achieve. How do we know which Q belongs to P ?

9. Apple detection …

10. Feature Characteristics Invariance (e.g., Rotation, Translation) Robust (minimum dependence on) Noise, artifacts, intrinsic variations User parameter settings Quantitative measures

12. We extract features from… Region of Interest Segmented Objects

13. Classification Features Texture Based (Image & ROI) Shape (Segmented objects)

14. Shape Based Features Object based Topology based (Euler Number) Effective Diameter (similarity to a circle to a box) Circularity Compactness Projections Moments (derived by Hu 1962) …

15. 4-neighbourhood of 8-neighbourhood of Adjacency and Connectivity – 2D  Notation: k -Neighbourhood of is

16. Adjacency and Connectivity – 3D 6-neighbourhood 18-neighbourhood 26-neighbourhood

17. Objects or Components (Jordan Theorem) In 2D – (8,4) or (4,8)-connectivity In 3D – (6,26)-,(26,6)-,(18,6)- or (6,18)-connectivity

18. Connected Components Labeling Each object gets a different label

19. Connected Components Labeling A B C Raster Scan Note: We want to label A. Assuming objects are 4-connected B, C are already labeled. Cortesy of S. Narasimhan

20. Connected Components Labeling 1 0 0  label(A) = new label 0 X X  label(A) = “background” 1 0 C  label(A) = label(C) 1 B 0  label(A) = label(B) 1 B C  If label(B) = label(C) then, label(A) = label(B) Cortesy of S. Narasimhan

21. 1 B C  What if label(B) not equal to label(C)? ? Connected Components Labeling Cortesy of S. Narasimhan

22. Connected Components Labeling Each object gets a different label

23. Topology based – Euler Number Euler Number E describes topology. C is # connected components H is # of holes.

24. Euler Number 3D Euler Number E describes topology. C is # connected components Cav is # of cavities G is # of genus E=1+0-1=0 E=1+1-0=2

25. Euler Number 3D E=2+0-0=2 E=1+1-0=2 Euler Number E describes topology. C is # connected components Cav is # of cavities G is # of genus

26. 3D Euler Number The Euler Number in 3D can be computed with local operations Counting number of vertices, edges and faces of the surfaces of the objects

27. Simple Shape Measurements 2D area - 3D volume Summing elements 2D perimeter - 3D surface area Selection of border elements Sum of elemets with weights Error of precision

28. Similarity to other Shape Effective Diameter Circularity (Circle C=1) Compactness – (Actually non-compactness) (Circle Comp= )

29. Projections x y

30. Moments Definition Order of a moment is Moments identify an object uniquely ? is the Area Centroid

31. Central Moments Moments invariant to position Invariant to scaling

32. Moments to Define Orientation Inertia Matrix – Covariance Matrix

33. Eigenanalysis of a Matrix Given a matrix S, we solve the following equation we find the eigenvectors and eigenvalues Eigenvectors and eigenvalues go in couples an usually are ordered as follows:

34. Eigenanalysis of the Inertia Matrix Eigenanalysis Sphere Flatness Elongated

35. Eigenanalysis of the Inertia Matrix Eigenanalysis Sphere Flatness Elongated

36. Orientation in 2D Using similar concepts than 3D Covariance or Inertia Matrix Eigenanalysis we obtain 2 eigenvalues and 2 eigenvectors of the ellipse

37. Moments Invariance Translation Central moments are invariant Rotation Eigenvalues of Inertia Matrix are invariant Scaling If moment scaled by (3D) (2D)

38. Moments invariant rotation-translation-scaling For 3D three moments (Sadjadi 1980) For 2D seven moments

39. Classification Features Texture Based (Image & ROI) Shape (Segmented objects)

40. Image Based Features Using all pixels individually Histogram based features −Statistical Moments (Mean, variance, smoothness) −Energy −Entropy −Max-Min of the histogram −Median …

41. Histogram L=9 bibi P(b i )

42. How do the histograms of this images look like?

43. Bimodal Histogram

44. Trimodal Features

45. Histogram Features Mean Central Moments

46. Histogram Features Mean Variance Relative Smoothness Skewness

47. Histogram Features Energy (Uniformity) Entropy

48. Examples of Energy and Entropy Energy=1 Entropy=0 Energy=0,111 Entropy=3,327 Energy=0,255 Entropy=2,018 Energy=0,0625 Entropy= 4

49. Examples TextureMeanstdR3rd momentEnergyEntropy 182.6411.790.002-0.1050.0265.434 2143.5674.630.079-0.1510.0057.783 399.7233.730.0170.7500.0136.374

50. Intensity Co-occurrance Matrix Operator Q defines the position between two pixels (e.g, pixel to the right) Co-occurance matrix G is ( L+1) x (L+1) (6x6). Counts how often Q occurs 0 0 1 1 4 4 0 0 4 4 5 5 2 2 3 3 1 1 2 2 3 3 4 4 4 4 4 4 1 1 1 1 6 6 3 3 5 5 1 1 1 1 6 6 5 5 5 5 1 1 0123456 00100100 10020101 20002000 30100100 42100011 50201000 60000010 Image G

51. Example L=256 Q “one pixel immediately to the right” Image G - Matrix

52. Features based on the co-ocurrence Matrix The elements of G (g ij ) is converted to probability (p ij ) by dividing by the amount of pairs in G Based on the probability density function we can use Maximum Energy (uniformity) Entropy

53. Features based on the co-ocurrence Matrix Homogenity – closeness to a diagonal matrix Contrast

54. Features based on the co-ocurrence Matrix Correlation – measure of correlation with neighbours

55. Example L=256 Q “one pixel immediately to the right” Image G - Matrix

56. Example Image G - Matrix CorrelationContrastHomogeneity 10.00006108380.0366 20.015005700.0824 30.0686013560.2048

57. Moments Definition Order of a moment is Moments identify an object uniquely Centroid

58. Central Moments Moments invariant to position Normalized central moments

59. Moments invariant rotation-translation-scaling For 3D three moments (Sadjadi 1980) For 2D seven moments (Hu’s 1962)

60. Moments invariant rotation-translation- scaling-mirroring (within minus sign) are all equal Mirroring