Machine Vision and Dig. Image Analysis 1 Prof. Heikki Kälviäinen C50A6100 Lectures 12: Object Recognition Professor Heikki Kälviäinen Machine Vision and Pattern Recognition Laboratory Department of Information Technology Faculty of Technology Management Lappeenranta University of Technology (LUT)

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A Motivation Task: To recognize an object based on a feature vector Image acquisition => digital image ↓ Preprocessing => better image ↓ Segmentation => basic features ↓ Representation and description => advanced features ↓ Object recognition (classification, clustering)

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A Content Classification. –Patterns, feature vectors. Clustering. –Pattern classes. Pattern recognition approaches: –Statistical approach. –Structural approach. –Neural approach. Other categorizations: for example, metric and non-metric approaches: –Decision trees, strings, grammars, rule-based methods (logic). Good reference: Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, Wiley, 2001.

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A Patterns and pattern classes A pattern is an arrangement of descriptors (also called features). A feature vector x = [x_1 x_2 … x_n]^T where x_i is a descriptor (feature). A pattern class is a family of patterns that share some common properties. Pattern classes ω_1, ω_2, …, ω_W where W is the number of classes.

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A Classification and clustering Feature selection and extraction. Clustering. –To find classes. Classification: –To find class ω _i to where x belongs. Building a classifier: –Learning (training): training set. –Testing: test set.

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A Statistical pattern recognition Bayesian classification strategies. Based on class samples and class parameters such as the mean and the covariance of the distribution of the features. Decision (discriminant) functions and decision surfaces. Normal (Gaussian) distribution. Parametric estimation: Maximum Likelihood (ML) Estimation, Bayesian estimation. Non-parametric estimation: Parzen windows, k-nn nonparametric estimation (k-nn classifier). Unsupervised learning and clustering: sequential algorithms, hierarchical algorithms, optimization based clustering (e.g., C- means), competitive learning (e.g., Self-Organizing Map, SOM).

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A Bayesian classification: example eye nostril eye Gaussian mixture model densities (EM estimation) Stability property guarantees approximately the Gaussian form of classes in the feature space. One class may still consist of several sub-clusters (open eye, closed eye, etc.). Bayesian classification of features

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A Syntactic pattern recognition Grammars: formal grammars, different kinds of grammars, examples how to apply grammars. G = (N, ∑, P, S) where P a set of rewriting rules. Parsing: the Cocke-Younger-Kasami (CYK) parsing algorithm, the CYK table. Tree grammars. Graphs: graph types, graph morphology, graph matching algorithms.

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A Neural pattern recognition Why do we need neural pattern recognition? Supervised and unsupervised learning: methods for classification and clustering. Supervised learning: MLP algorithm, Backpropagation. Unsupervised learning: the Kohonen net or Self- Organizing Map (SOM) algorithm.

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A Customer profiles by neural computing: Finnish supermarkets From many itemized receipts to categories of customers

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A Different customers as different nodes in SOM (similar ones near each other)

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A Visual inspection on wooden surfaces

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A Neural computing approach

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A Applications To be selected: Data. Features. Classifiers. considering the following requirement: Fast. Accurate/reliable. Not too expensive. Easy to implement and use.

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A Visual quality inspection in steel manufacturing

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A Vacuum chamber

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A Bubbling of argon gas in the melt Weak bubbling Good bubbling

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A Histograms of weak and good bubbling

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A Hydrogen prediction of vacuum degassing

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A The user interface of the system

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A Sorting ceramic tiles: RGB and spectral features

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A Sorting ceramic tiles: test material Class 1 Class 2 Class 3 …. Class 5 Tile 1Tile 2Tile 3Tile 4Tile 5

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A Sorting ceramic tiles: classification

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A Molecular computing: Bacteriorhodopsin-based color sensors Molecule structure Fotocycle

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A Molecular computing: bacterial camera Matrix elementCamera

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A Molecular computing: color sensors Responses Self-Organizing Map (SOM)

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A More examples of applications Medical image processing. –Diabetes and retinal image analysis using machine vision. Industrial machine vision. –Paper and board printability tests by machine vision in the paper making and printing industry. Biometrics. –Image-based biometric person authentication. For more see, e.g.,

Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A Summary Task: To recognize an object based on a feature vector Image acquisition => digital image ↓ Preprocessing => better image ↓ Segmentation => basic features ↓ Representation and description => advanced features ↓ Object recognition (classification, clustering)