1. Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley & Sons, with the permission of the authors and the publisher

2. Intro to Pattern Recognition Chapter 1: Sections 1.1-1.6
Machine Perception
An Example
Pattern Recognition Systems
The Design Cycle
Learning and Adaptation
Conclusion

3. Machine Perception Build a machine that can recognize patterns:
Speech recognition
Fingerprint identification
OCR (Optical Character Recognition)
DNA sequence identification

4. An Example “Sorting incoming Fish on a conveyor, according to species
using optical sensing”
Sea bass
Species
Salmon

6. Problem Analysis Take sample images … to extract features
Length
Lightness
Width
Number and shape of fins
Position of the mouth
etc…
= set of features to use in our classifier!

7. Preprocessing Use a segmentation operation to isolate
fish from one another
and from the background
Information from each single fish is sent to a feature extractor, whose purpose is to reduce the data by measuring certain features
The features are passed to a classifier

9. Classification
Select the length of the fish as a possible feature for discrimination

11. The length is a poor feature alone!
Select the lightness as a possible feature.

13. Threshold decision boundary and cost relationship
Move decision boundary toward smaller values of lightness, to minimize cost
reduce number of sea bass classified as salmon!
Task of decision theory

14. Adopt the lightness and add the width of the fish
Fish xT = [x1, x2]
Lightness
Width

16. Perhaps add other features
not correlated with current ones
Warning: “noisy features” will reduce performance
Best decision boundary == one that provides an optimal performance
Eg …

17. “Optimal Performance” ??

18. Objective: Dealing with Novel Data
Goal:
Optimal performance on NOVEL data
Performance on TRAINING DATA != Performance on NOVEL data
Issue of generalization!

20. Pattern Recognition Systems
Sensing
Using transducer (camera, microphone, …)
PR system depends of the bandwidth, the resolution sensitivity distortion of the transducer
Segmentation and grouping
Patterns should be well separated (should not overlap)

22. Feature extraction Classification Post Processing
Discriminative features
Want features INVARIANT wrt translation, rotation, scale.
Classification
Using feature vector (provided by feature extractor) to assign given object to a category
Post Processing
Exploit context info not from the target pattern itself
to improve performance

23. The Design Cycle Data collection Feature Choice Model Choice Training
Evaluation
Computational Complexity

25. Data Collection
How do we know when we have collected an adequately large and representative set of examples for training and testing the system?

26. Feature Choice Depends on characteristics of problem domain. Ideally…
Simple to extract
Invariant to irrelevant transformation
Insensitive to noise.

27. Model Choice If not satisfied with performance
(EG, of our fish classifier)
Consider another class of model

28. Training Use data to obtain good classifier
identify best model
determine appropriate parameters
Many procedures for training classifiers and choosing models

29. Evaluation Measure error rate
~= performance
May suggest switching from one set of features to another one

30. Computational Complexity
Trade-off between computational ease and performance?
How algorithm scales as function of number of features, patterns or categories?

31. Learning and Adaptation
Supervised learning
A teacher provides a category label or cost for each pattern in the training set
Unsupervised learning
The system forms clusters or “natural groupings” of the input patterns

32. Conclusion
Reader seems to be overwhelmed by the number, complexity and magnitude of the sub-problems of Pattern Recognition
Many of these sub-problems can indeed be solved
Many fascinating unsolved problems still remain