Pattern Recognition NTUEE 高奕豪 2005/4/14

Outline Introduction Definition, Examples, Related Fields, System, and Design Approaches Bayesian, Hidden Markov Model, Neural Network, Decision Tree Modern Applications Face, Handwriting, Fingerprint, Speech Introduction Definition, Examples, Related Fields, System, and Design Approaches Bayesian, Hidden Markov Model, Neural Network, Decision Tree Modern Applications Face, Handwriting, Fingerprint, Speech

Introduction: What is Pattern Recognition? “The assignment of a physical object or event to one of several pre-specified categories” –Duda and Hart, author of Pattern Classification “Given some examples of complex signals and the correct decisions for them, make decisions automatically for a stream of future examples” – Ripley, Oxford University “The process of giving names ω to observations x”, –Schürmann “Pattern Recognition is concerned with answering the question ‘What is this?’ “ –Morse “The assignment of a physical object or event to one of several pre-specified categories” –Duda and Hart, author of Pattern Classification “Given some examples of complex signals and the correct decisions for them, make decisions automatically for a stream of future examples” – Ripley, Oxford University “The process of giving names ω to observations x”, –Schürmann “Pattern Recognition is concerned with answering the question ‘What is this?’ “ –Morse

Introduction: Typical Examples Machine vision Character recognition Computer aided diagnosis Speech recognition Machine vision Character recognition Computer aided diagnosis Speech recognition

Introduction: Related Field Adaptive Signal Processing Machine Learning Artificial Neural Networks Mathematical Statistics Fuzzy and Genetic systems Formal Languages Biological Cybernetics Computational Neuroscience And so on… Adaptive Signal Processing Machine Learning Artificial Neural Networks Mathematical Statistics Fuzzy and Genetic systems Formal Languages Biological Cybernetics Computational Neuroscience And so on…

Introduction: A particular example

Pattern Recognition System Sensing Segmentation Feature Extraction Classification Post Processing Sensing Segmentation Feature Extraction Classification Post Processing

Pattern Feature Any Distinctive aspect, quality, or characteristics.

Pattern Recognition System Design Cycle Collect Data Choose Features Choose Model Train Classifier Evaluate Classifier Collect Data Choose Features Choose Model Train Classifier Evaluate Classifier

Approach

Bayesian Decision Hidden Markov Model Multilayer Neural Network Decision Tree Bayesian Decision Hidden Markov Model Multilayer Neural Network Decision Tree

Bayesian Decision Provide all relevant probability and cost Bayes Formula: P(ω j |x) = P(x|ω j ) P(ω j ) / P(x) (posteriori = likelihood×prior÷evidence) Bayes Decision Rule: Decide ω 1 if P(ω 1 |x) > P(ω 2 |x), otherwise decide ω 2 Provide all relevant probability and cost Bayes Formula: P(ω j |x) = P(x|ω j ) P(ω j ) / P(x) (posteriori = likelihood×prior÷evidence) Bayes Decision Rule: Decide ω 1 if P(ω 1 |x) > P(ω 2 |x), otherwise decide ω 2

Bayesian Decision Example: Given P(ω 1 )=2/3, P(ω 2 )=1/3 P(x|ω)P(ω|x) Example: Given P(ω 1 )=2/3, P(ω 2 )=1/3 P(x|ω)P(ω|x)

Hidden Markov Model Useful for problems that have an inherent temporality Markov Model: A set of states with transition probability Useful for problems that have an inherent temporality Markov Model: A set of states with transition probability

Hidden Markov Model A state ω(t) may emit some visible symbol v(t) a ij =P(ω j (t+1)|ω i (t) b ij = P(v k (t)|ω j (t)) A state ω(t) may emit some visible symbol v(t) a ij =P(ω j (t+1)|ω i (t) b ij = P(v k (t)|ω j (t))

Hidden Markov Model Evaluation Problem Given a HMM, determine the probability that a particular sequence of visible states V T was generated by it Decoding Problem Given V T, determine the most likely sequence of hidden states ω T that led it Learning Problem Given the number of states and a set of visible symbols, determine a ij and b ij Evaluation Problem Given a HMM, determine the probability that a particular sequence of visible states V T was generated by it Decoding Problem Given V T, determine the most likely sequence of hidden states ω T that led it Learning Problem Given the number of states and a set of visible symbols, determine a ij and b ij

Hidden Markov Model Evaluation: Brute force Enumeration O(T C^T) Solution: Dynamic Programming Evaluation: Brute force Enumeration O(T C^T) Solution: Dynamic Programming

Hidden Markov Model State T-2 T-1 Viterbi Algorithm Time

Hidden Markov Model Search for “Yes”/”No”

Multilayer Neural Network Implement linear discriminants in a space where the inputs have been mapped nonlinearly The nonlinearity can be learned from training data Implement linear discriminants in a space where the inputs have been mapped nonlinearly The nonlinearity can be learned from training data

Multilayer Neural Network

Decision Tree A classification problem involves nominal data Property D-Tuple: Fruit: color, texture, shiny, taste Apple = { red, shiny, sweet, medium} String, DNA A classification problem involves nominal data Property D-Tuple: Fruit: color, texture, shiny, taste Apple = { red, shiny, sweet, medium} String, DNA

Decision Tree

Modern Applications Face Recognition Fingerprint Recognition Handwriting Recognition Speech Recognition Face Recognition Fingerprint Recognition Handwriting Recognition Speech Recognition

Face Recognition Recognition and Coding, MIT Media Lab

Face Recognition

FaceCheck, C-VIS

Face Recognition

Fingerprint Recognition Optical/Charge 10~40 feature points, transformed into feature vector Typically, 500 dpi FAR<25/1,000,000 FRR<3/ USD Optical/Charge 10~40 feature points, transformed into feature vector Typically, 500 dpi FAR<25/1,000,000 FRR<3/ USD

Fingerprint Recognition

Handwriting Recognition Optical Character Recognition: Printed, certain fonts Intelligent Character Recognition Constrained text entry Natural Handwriting Recognition Breakdown, check by linguistic rules Optical Character Recognition: Printed, certain fonts Intelligent Character Recognition Constrained text entry Natural Handwriting Recognition Breakdown, check by linguistic rules

Handwriting Recognition EverNote, CA, USA

Handwriting Recognition

Speech Recognition

Reference Pattern Classification, 2/e, Richard O. Duda, Peter E. Hart, David G. Stork ?CaseStudyID= Pattern Classification, 2/e, Richard O. Duda, Peter E. Hart, David G. Stork ?CaseStudyID=

Thank you for your attention.