8D040 Basis beeldverwerking Feature Extraction Anna Vilanova i Bartrolí Biomedical Image Analysis Group bmia.bmt.tue.nl

An image is just 2D? No! – It can be in any dimension Example 3D: Voxel-Volume Element

Reduction of dimensionality Why feature extraction ? Pixel level Image of 256x256 and 8 bits ~ possible images

Incorporation of cues from human perception Transcendence of the limits of human perception The need for invariance Why feature extraction ?

Apple detection …

Transformation (Rotation)

How do we transform an image? We transform a point P How do we transform an image f(P) ? How do we know which Q belongs to P ?

How do we transform an image? How do we transform an image f(P) ? We know T which is the transformation we want to achieve. How do we know which Q belongs to P ?

Apple detection …

Feature Characteristics Invariance (e.g., Rotation, Translation) Robust (minimum dependence on) Noise, artifacts, intrinsic variations User parameter settings Quantitative measures

We extract features from… Region of Interest Segmented Objects

Classification Features Texture Based (Image & ROI) Shape (Segmented objects)

Shape Based Features Object based Topology based (Euler Number) Effective Diameter (similarity to a circle to a box) Circularity Compactness Projections Moments (derived by Hu 1962) …

4-neighbourhood of 8-neighbourhood of Adjacency and Connectivity – 2D  Notation: k -Neighbourhood of is

Adjacency and Connectivity – 3D 6-neighbourhood 18-neighbourhood 26-neighbourhood

Objects or Components (Jordan Theorem) In 2D – (8,4) or (4,8)-connectivity In 3D – (6,26)-,(26,6)-,(18,6)- or (6,18)-connectivity

Connected Components Labeling Each object gets a different label

Connected Components Labeling A B C Raster Scan Note: We want to label A. Assuming objects are 4-connected B, C are already labeled. Cortesy of S. Narasimhan

Connected Components Labeling  label(A) = new label 0 X X  label(A) = “background” 1 0 C  label(A) = label(C) 1 B 0  label(A) = label(B) 1 B C  If label(B) = label(C) then, label(A) = label(B) Cortesy of S. Narasimhan

1 B C  What if label(B) not equal to label(C)? ? Connected Components Labeling Cortesy of S. Narasimhan

Connected Components Labeling Each object gets a different label

Topology based – Euler Number Euler Number E describes topology. C is # connected components H is # of holes.

Euler Number 3D Euler Number E describes topology. C is # connected components Cav is # of cavities G is # of genus E=1+0-1=0 E=1+1-0=2

Euler Number 3D E=2+0-0=2 E=1+1-0=2 Euler Number E describes topology. C is # connected components Cav is # of cavities G is # of genus

3D Euler Number The Euler Number in 3D can be computed with local operations Counting number of vertices, edges and faces of the surfaces of the objects

Simple Shape Measurements 2D area - 3D volume Summing elements 2D perimeter - 3D surface area Selection of border elements Sum of elemets with weights Error of precision

Similarity to other Shape Effective Diameter Circularity (Circle C=1) Compactness – (Actually non-compactness) (Circle Comp= )

Projections x y

Moments Definition Order of a moment is Moments identify an object uniquely ? is the Area Centroid

Central Moments Moments invariant to position Invariant to scaling

Moments to Define Orientation Inertia Matrix – Covariance Matrix

Eigenanalysis of a Matrix Given a matrix S, we solve the following equation we find the eigenvectors and eigenvalues Eigenvectors and eigenvalues go in couples an usually are ordered as follows:

Eigenanalysis of the Inertia Matrix Eigenanalysis Sphere Flatness Elongated

Eigenanalysis of the Inertia Matrix Eigenanalysis Sphere Flatness Elongated

Orientation in 2D Using similar concepts than 3D Covariance or Inertia Matrix Eigenanalysis we obtain 2 eigenvalues and 2 eigenvectors of the ellipse

Moments Invariance Translation Central moments are invariant Rotation Eigenvalues of Inertia Matrix are invariant Scaling If moment scaled by (3D) (2D)

Moments invariant rotation-translation-scaling For 3D three moments (Sadjadi 1980) For 2D seven moments

Classification Features Texture Based (Image & ROI) Shape (Segmented objects)

Image Based Features Using all pixels individually Histogram based features −Statistical Moments (Mean, variance, smoothness) −Energy −Entropy −Max-Min of the histogram −Median …

Histogram L=9 bibi P(b i )

How do the histograms of this images look like?

Bimodal Histogram

Trimodal Features

Histogram Features Mean Central Moments

Histogram Features Mean Variance Relative Smoothness Skewness

Histogram Features Energy (Uniformity) Entropy

Examples of Energy and Entropy Energy=1 Entropy=0 Energy=0,111 Entropy=3,327 Energy=0,255 Entropy=2,018 Energy=0,0625 Entropy= 4

Examples TextureMeanstdR3rd momentEnergyEntropy

Intensity Co-occurrance Matrix Operator Q defines the position between two pixels (e.g, pixel to the right) Co-occurance matrix G is ( L+1) x (L+1) (6x6). Counts how often Q occurs Image G

Example L=256 Q “one pixel immediately to the right” Image G - Matrix

Features based on the co-ocurrence Matrix The elements of G (g ij ) is converted to probability (p ij ) by dividing by the amount of pairs in G Based on the probability density function we can use Maximum Energy (uniformity) Entropy

Features based on the co-ocurrence Matrix Homogenity – closeness to a diagonal matrix Contrast

Features based on the co-ocurrence Matrix Correlation – measure of correlation with neighbours

Example L=256 Q “one pixel immediately to the right” Image G - Matrix

Example Image G - Matrix CorrelationContrastHomogeneity

Moments Definition Order of a moment is Moments identify an object uniquely Centroid

Central Moments Moments invariant to position Normalized central moments

Moments invariant rotation-translation-scaling For 3D three moments (Sadjadi 1980) For 2D seven moments (Hu’s 1962)

Moments invariant rotation-translation- scaling-mirroring (within minus sign) are all equal Mirroring