1. Self-supervised adaptation for on-line text recognition
Loïc Oudot, Lionel Prevost

2. Overview Baseline system quick presentation Supervised adaptation
Self-supervised adaptation
Semi-supervised adaptation
Conclusions

3. Writer independent baseline
Based on the activation-verification cognitive model of Paap
Global encoding of the input stimulus
Activation of a list of hypothetic word in lexicon
Verification and selection of the best word candidate
Data
<x,y,p>
Base line
Segmentation
Blank detector
Shape
Encoding
Classifier
Hypothetic
Words list
Lexicon
Activation
Probabilistic
engine
ASCII
Verification
More information on poster session (P34 October 27)

4. Writer adaptation strategies
Causes of recognition rate reduction :
Missing grapheme : the grapheme is missing in the prototype data set and must be stored (added) in this set
Confusing grapheme : for a given writer, the prototype is confusing or erroneous and it must be removed from the prototype data set.

5. Character classifier Prototype based classifier (1-nn)
Omni-writer strokes prototype data set
3 000 prototypes for 62 classes (lower-case, upper-case and digit)
Computes a probabilistic vector of relevance for each classes
Easy to add and remove prototypes

6. Supervised adaptation
Comparison between the classification hypothesis and the label (segmentation known)
Word error rate
Data set size
Words 50
100
150
Baseline system
25 %
100 %
Text
1.3 %
1.1 %
0.6 %
+6 %
Line
0.7 %
0.4 %
+4 %
Good recognition rate
Needs labeled data

7. Self-supervised adaptation
Comparison between the classification hypothesis and the lexical hypothesis
Systematical activation (SA)
No needs of labeled data
Small improvement of the recognition rate
Word error rate
Data set size
Words 50
100
150
Baseline system
25 %
100 % SA
23 %
+6 %

8. Self-supervised adaptation
Conditional activation (CA)
To limit the storage of bad prototypes we use the probabilistic lexical reliability (PLR) of a word :
if a given word has a PLR >  (threshold) the mis-classified characters are added in the prototype data set.
Good recognition rate
No needs of labeled data
Growth of the prototype data set
Word error rate
Data set size
Words 50
100
150
Baseline system
25 %
100 % CA
22%
20 %
17 %
+2 %

9. Self-supervised adaptation
Dynamic management
Remove useless and erroneous prototypes :
Each prototypes have an initial adequacy Q0 = 1000
C : Reward (+) the prototype i when it performs good classification
I : Penalize (-) the prototype i when it performs mis-classification
N : Penalize (-) for all the useless prototypes of the class j
Qji(n+1) = Qji(n)+[C(n)-I(n)-N(n)]/Fj
When Qji=0 the prototype is remove from the data set
Huge reduction of the prototypes data base : -90 %
Speed up of the recognition time
Same recognition rate

10. Adequacy evolution
Omni-writer
User prototypes

11. Semi-supervised adaptation
Fusion between supervised and self-supervised adaptation
First, we train the system in a supervised way with N words.
Then, we use the dynamic management adaptation. N
Recognition rate
After enrolment
After 100 words
70 %
76 % 10
74 %
83 % 20
88 % 30
90 % 50
75 %
91 %
Best recognition rate : 99 %
Worst recognition rate : 70 %
Excellent recognition rate
Few labeled data

12. Conclusions
We propose semi-supervised strategies with better results than supervised adaptation.
Automatic reduction of a writer independent database into a writer-dependent database with great quality and compactness.
For ambiguous words (<PLR<’), ask the writer to enter the label