1. SPECIAL ISSUE on Document Analysis, 5(2):1-15, 2005.
Combining Model-based and Discriminative Approaches
in a Modular Two-stage Classification System:
Application to Isolated Handwritten Digit Recognition
Jonathan Milgram – Robert Sabourin – Mohamed Cheriet

2. Motivations vs. Two categories of classification algorithms
discriminative
model-based
Two types of problems
ambiguous patterns
outliers

3. Proposition Combining the two types of approaches
in a two-stage classification system
First stage: Model-based approach
 Detect outliers
 Classify unambiguous patterns
Second stage: Discriminative approach
 Refine decision
 Reject ambiguous patterns

4. Overview Each class is modeled
by a hyperplane and the projection distance is used as similarity measure
In a first time the posterior probabilities are estimated by the first stage and only the highest are re-estimated by the second stage
Support Vector Classifiers
are trained using a pairwise coupling strategy

5. Experimental Validation
Isolated Handwritten
Digit Recognition
A classical problem:
A benchmark database:
Modified NIST
 28 x 28 grey-scale images
 60,000 learning samples
 10,000 test samples

6.   Some examples Ambiguous pattern Outlier Unambiguous pattern
Projection
on hyperplanes
Class 1 2 3 4 5 6 7 8 9
10
dj(x)
5.07
6.47
5.51
4.91
5.25
5.30
1.88
6.27
6.02
6.05
5.08
6.77
5.51
5.91
5.28
5.69
5.63
5.45
6.03
4.93
5.37
4.87
4.80
3.24
5.15
5.45
3.50
4.78
3.47
0.6641
0.1537
0.1818
0.0001
0.0000
0.0000
1.0000
All the projection distances are high  the pattern can be rejected
0.0000
0.9967
0.0029
0.0001

7. Summary of the results Error rate at 0% of reject Reject rate
at 0.1% of error
Complexity
(MFLOPs/pattern)
First stage
4.09 %
28.59 %
0.4
Second stage
1.48 %
9.55 %
26.2
Combination
1.50 %
9.85 %
3.0

8. Conclusions Speeding up the decision making of SVC
Making the outlier detection available
Using a modular architecture suited to
PR problems with large number of classes

9. Questions ?!?
Correspondence to: