1. Digital image processing Chapter 8 Image analysis and pattern recognition IMAGE ANALYSIS AND PATTERN RECOGNITION Introduction Feature extraction: - spatial feature extraction - transform feature extraction - edge feature extraction; edge detection Objects representation by their boundaries: - contour extraction - contour descriptors Objects representation by their regions: - region extraction - region representation Shapes and structures for region-based object representation: - object skeletons - binary morphology - shape descriptors (numerical shape descriptors) Textures; texture analysis Image segmentation Grey level based segmentation/color based segmentation Connected components analysis Contour-based segmentation Region-based segmentation Mixed techniques

2. Introduction Fig. 8.1 Image analysis system – block diagram Example image analysis tasks: Digital image processing Chapter 8 Image analysis and pattern recognition License plate recognition

3. Introduction Example image analysis tasks: Digital image processing Chapter 8 Image analysis and pattern recognition Text detection & recognition or removal Trafic sign recognition Calcite localization/assessmentDetect tissue on the slide

4. Introduction Example image analysis tasks: Ways to approach the image analysis task? - “straight forward” approach: analyze the content of the whole image => one will eventually find the entire information, including the information of interest – at too high computational complexity/cost - “smart” approach: focus strictly on the information you’re looking for! and extract only this information from the image  focus on the part of the image containing the information of interest. How? 1)describe somehow the main characteristics of the information of interest  describe the discriminative features of the region containing the information you look for, which are not present in the other regions => represent the image in the best feature space, by a feature map Digital image processing Chapter 8 Image analysis and pattern recognition Single object images; defect detection on the object

5. E.g. on the images with apples – only interested in the defect area (if there is), and for this defect – it appears darker grey than the normal apple coloration, but not black => design a grey scale slicing to set to black everything else 2)Analyze only the information in the region of interest, using the previously defined feature space/feature map e.g. Apples sort: no defects (defect area <2% of the apple); average defects; large defects… Digital image processing Chapter 8 Image analysis and pattern recognition New feature space

6. Feature extraction: Spatial features extraction: Amplitude features: e.g. the brightness levels can identify regions of interest in the image: Amplitude features may be discriminative enough if intensity is enough to distinguish wanted info from the rest of the scene => defining the best parameters of the transformation for feature extraction – most difficult => amplitude feature space representation is not necessarily binary; just that unwanted parts of the scenes should be represented uniquely (i.e. black) in the feature space => sometimes adaptive thresholding/adaptive grey scale slicing is needed. Tissue identification by color coding (e.g. violet)  Can measure afterwards the area, describe the shape, etc. Digital image processing Chapter 8 Image analysis and pattern recognition

7. Histogram based features: Local histogram = a local statistical description of the image; If u = an image pixel; x=a grey level => p u (x)=the probability of appearance of the grey level x in the image region = a value in the normalized histogram => One can compute: the standard deviation; the entropy; the median; percentiles, of p u (x). Region of interest (ROI) ROI histogram Measurements Digital image processing Chapter 8 Image analysis and pattern recognition Tissue of interest is well discriminated from the microscopic slide by the standard deviation of the local histogram

8. Transform features extraction Fig.8.2 Transform features extraction Digital image processing Chapter 8 Image analysis and pattern recognition

9. Edge features extraction. Edge detection Fig. 8.3 Edge detection with gradient operators (8.5) (8.6) Digital image processing Chapter 8 Image analysis and pattern recognition

10. Fig. 8.4 Edge detection by compass operators a b c d Fig. 8.5 Compass edge detectors (North direction) N NW W SW S SE E NE Fig. 8.6 Compass operators on different directions Digital image processing Chapter 8 Image analysis and pattern recognition

11. g k (m,n) –compass gradient on the direction k  0,...,7, The gradient in the spatial position (m,n) is defined as: (8.7) Laplace operators and the zero-crossings edge localization method: (8.8) Fig. 8.7 Edge detection by Laplace operators (the 1-D case) The Laplacian: Digital image processing Chapter 8 Image analysis and pattern recognition

12. The Laplacian of Gaussian operator (LoG) Discrete implementations: Gaussian filter The derivative of the Gaussian The 2 nd derivative (Laplacian of Gaussian) Digital image processing Chapter 8 Image analysis and pattern recognition

13. Edge detection by different operators – comparison: Original image Sobel edge detection LoG edge detection; sigma=5 Digital image processing Chapter 8 Image analysis and pattern recognition Roberts edge detection LoG edge detection; Sigma=10

14. Objects representation by their boundaries: Contour extraction: Fig. 8.8 4-connectivity; 8-connectivity The Hough transform: Digital image processing Chapter 8 Image analysis and pattern recognition s θ Φ s x y

15. The Hough transform of lines/random curves: Digital image processing Chapter 8 Image analysis and pattern recognition No convergence point => The points are not on a straight line ss Approx. convergence => approx. straight line Curve convergence on horizontal axis=> Straight line through the origin s

16. Applications to contour extraction formed by line segments: Digital image processing Chapter 8 Image analysis and pattern recognition Lower left object = Defined by the 3 line intersection points Input image Edge detection (gradient operator) + thresholding => Binary edge map Edge map; Contours not yet extracted/ not yet labeled Apply Hough transform: Extracting the contour of the triangle Contour extraction in 8-connectivity 8-connectivity: Correct labeling 4-connectivity: The contour is “broken” during labeling

17. Contour representation/contour descriptors: Goal: for a given object, described by its contour, find a compact description, by numerical attributes, able to: -Represent the contour with no significant loss of information (regenerative descriptors) -Generate (by a subset of attributes) descriptions of the contour/shape invariant to: scaling; rotation; translation; mirroring; projection distortions; small (limited) variations of shape (among different individual representations of the same shape)  Using the contour descriptors, one can recognize the shape by template matching or shape classification (contour descriptors classification) General assumption: single object contour (all edge pixels connected); 1-pixel width! Digital image processing Chapter 8 Image analysis and pattern recognition

18. Chain codes and polygonal approximations: Fig. 8.9 Contour representation by chain codes Fig. 8.10 Polygonal approximation of the contours Digital image processing Chapter 8 Image analysis and pattern recognition Original contour Contour approximation (“quantization”) on a rectangular grid Encode this contour by a chain code with 8 directions, Consider origin= the upper left point; contour following direction = arbitrary; the only possible positions are on the grid corners Start point

19. Fourier descriptors: (8.12) (8.13) (8.14) (8.15) (8.16) leading to a new contour x'(n) y'(n), given by: (8.17) where (8.18) Table 8.1 Digital image processing Chapter 8 Image analysis and pattern recognition

20. Shape reconstruction from its Fourier descriptors: OriginalReconstruction using the first 2 descriptors only Reconstruction using the first 6 descriptors Reconstruction using the first 10 descriptors Reconstruction using the first 20 descriptors … Digital image processing Chapter 8 Image analysis and pattern recognition

21. , then for a given translation u 0, the distance d is minimal when: and where a(k)  b*(k)  c(k)e j  k,  -2  n 0 /N and c(k) is a real valued term. Digital image processing Chapter 8 Image analysis and pattern recognition Shapes dictionary Model shape Macthing results: d<Thd

22. Objects representation by their regions: Region extraction: same as for contours (use connected components analysis, in 4- or 8- connectivity) Region representation: Fig. 8.11 Quad-tree region representation => the quad-tree code : gbgbwwbwgbwwgbwwb ; decoding: g(bg(bwwb)wg(bwwg(bwwb))) Digital image processing Chapter 8 Image analysis and pattern recognition

23.  Regions skeletons; medial axis transforms: 8.23)  (m,n) : u k (m,n)  u k (i,j),  (m,n;i,j)  1  (8.24) Fig. 8.12 Skeleton extraction Fig. 8.13 Examples skeletons Fig. 8.14 2-pass contour thinning: 2 logic rules: Fig. 8.15 Thinning result R1: P1==1 && N(P1)==1 && 2<=NT(P1)<=6 && P2·P4·P6==0 && P2·P4·P8==0 R2: P1==1 && N(P1)==1 && 2<=NT(P1)<=6 && P2·P6·P8==0 && P4·P6·P8==0 Shapes and structures for region-based object representation: Digital image processing Chapter 8 Image analysis and pattern recognition

24.  Morphological processing. Binary image morphology Fig. 8.16 Algoritmi de transformare morfologica Digital image processing Chapter 8 Image analysis and pattern recognition

25.  Syntactical representation Fig. 8.17 Syntactical representation of an object Digital image processing Chapter 8 Image analysis and pattern recognition

26. Shape descriptors Applications: shape recognition; quantitative measurements Def.: The shape of an object = the profile of the object + its physical structure => “shape descriptors” Classification: (1) regenerative descriptors (contours; regions; high order statistics; structural and syntactic descriptors) (2) geometrical shape descriptors (area, perimeter, max-min radii, eccentricity, corners, roundness, symmetry) (3) moments Digital image processing Chapter 8 Image analysis and pattern recognition

27. Geometrical features (geometrical descriptors):  Perimeter: t – some contour parameter - Discrete => T = count of contour pixels  Area: where: R and  R – the object region and the object contour - Discrete => A = count of pixels inside the object region  Min-max radii, R min and R max – the minimum and maximum distances from the center of mass of the object region to its contour (the R max / R min ratio – gives a measure of the eccentricity or elongation of the object) x y dx dy x(t)x(t+1) y(t) y(t+1) R max R min Digital image processing Chapter 8 Image analysis and pattern recognition

28.  Roundness or compactness: For a disc -  is minimum, =1. · Symmetry: 2 types of shape symmetry: rotational and mirror Moment-based features: · Center of mass: · The (p,q) order central moments: · Orientation = the angle of the axis of the smallest moment of inertia – found by minimizing: with respect to  : Digital image processing Chapter 8 Image analysis and pattern recognition compactnecompact CompactNon-compact

29. Textures  The texture = the periodic repetition of some basic structures in an image area; the basic image structure is called texel Texture analysis methods: statistical classification; structural classification  Statistical classification techniques:  The auto-correlation function (ACF): the spatial dimensions of the texels are proportional to the width of the auto- correlation functions:: where  Several measures used to evaluate the distribution of the ACF to describe the texture: Artificial textures Natural textures Digital image processing Chapter 8 Image analysis and pattern recognition

30.  Image transforms based approaches: (8.41) Fig. 8.18 Various masks in the frequency domain used for texture analysis ACF for “Sand”ACF for “Wool” Digital image processing Chapter 8 Image analysis and pattern recognition

31.  The edge density – as texture classification feature:  Histogram features for texture analysis: => the co-occurrence histogram: => various features can be extracted from the co-occurrence histogram: - Inertia: - The mean of the distribution: N 0 – the total number of possible orientations. - The variance of the distribution: - The spread of the distribution: Sobel SandRaffia Grass R  Digital image processing Chapter 8 Image analysis and pattern recognition

32.  Random texture models (8.47) Fig. 8.19 Texture analysis model Digital image processing Chapter 8 Image analysis and pattern recognition

33. Image segmentation Amplitude thresholding/ grey level window slicing Component labeling  Pixel labeling. Region growing 1) Local maxima detection 2) Local minima detection Objects are found between the local minima Object Digital image processing Chapter 8 Image analysis and pattern recognition

34. Boundary-based image segmentation Fig. 8.23 Segmentation algorithm based on boundary detection Region-based segmentation; segmentation based on regions and boundaries Fig. 8.24 Region merging Digital image processing Chapter 8 Image analysis and pattern recognition

35. Fig. 8.25 Segmentation by split and merge algorithm: a. input b. region splitting by quad-trees c. segmented regions Digital image processing Chapter 8 Image analysis and pattern recognition

36. a) Original image c) Main objects detected, marked on the original image by their contour b) Region-based image segmentation Digital image processing Chapter 8 Image analysis and pattern recognition