Chapter 4: Pattern Recognition

Classification is a process that assigns a label to an object according to some representation of the object’s properties. Classifier is a device or algorithm that inputs an object representation and outputs a class label. Reject class is a generic class for objects that cannot be placed in any of the designated known classes. Classification

Error & Reject rate Empirical error rate of a classification system is the number of errors made on independent test data divided by the number of classifications attempted. Empirical reject rate of a classification system is the number of rejects made on independent test data divided by the number of classifications attempted.

False Alarm & Miss Detection Two-class problem example: whether a person has disease or not False Alarm (False Positive): The system incorrectly says that the person does have disease. Miss Detection (False Negative): The system incorrectly says that the person does not have disease.

Receiver Operating Curve (ROC)

Precision & Recall Example: the objective of document retrieval (image retrieval) is to retrieve interesting objects and not too many uninteresting objects according to features supplied in a user’s query. Precision is the number of relevant documents retrieved divided by the total number of documents retrieved. Recall is the number of relevant documents retrieved by the system divided by the total number of relevant documents in the database.

Example Suppose an image database contains 200 sunset images. Suppose an automatic retrieval system retrieves 150 of those 200 relevant images and 100 other images.

Features used for representation Area of the character in units of black pixels Height and Width of the bounding box of its pixels Number of holes inside the character Number of strokes forming the character Center (Centroid) of the set of pixels Best axis direction (Orientation) through the pixels as the axis of least inertia Second moments of the pixels about the axis of least inertia and most inertia

Example Features

Classification using nearest mean

Euclidean Distance

Example

Classification using nearest neighbors A brute force approach computes the distance from x to all samples in the database and remembers the minimum distance. Then, x is classified into the same category as its nearest neighbor. Advantage  new labeled samples can be added to the database at any time. A better approach is the k nearest neighbors rule.

Structural Pattern Recognition Graph matching algorithm can be used to perform structural pattern recognition.

Two characters with the same global features but different structure Lid : a virtual line segment that closes up a bay. Left : specifies that one lid lies on the left of another. Right : specifies that one lid lies on the right of another.

Confusion Matrix Reject Rate = Error Rate =

Decision Tree 1

Decision Tree 2

Automatic Construction of a Decision Tree Information content I(C;F) of the class variable C with respect to the feature variable F is defined by The feature variable F with maximum I(C,F) will be selected as the first feature to be tested.

Example I(C,X) = I(C,Y) = I(C,Z) =

General Case At any branch node of the tree when the selected feature does not completely separate a set of training samples into the proper classes  the tree construction algorithm is invoked recursively for the subsets of training samples at each of the child nodes. At any branch node of the tree when the selected feature does not completely separate a set of training samples into the proper classes  the tree construction algorithm is invoked recursively for the subsets of training samples at each of the child nodes.

Bayesian Decision Making

Bayesian classifier Bayesian classifier classifies an object into the class to which it is most likely to belong based on the observed features. In other words, it makes the classification decision w i for the maximum p(x) is the same for all the classes, so compare p(x|w i )P(w i ) is enough. Poisson, Exponential, and Normal (Gaussian) distributions are commonly used for p(x|w i ).