1. Multiple Agents for Pattern Recognition Louis Vuurpijl http://hwr.nici.kun.nl/~vuurpijl

2. Contents The problem: handwriting recognition PR1: The traditional solution –(some) solutions at the NICI PR2: Multiple classifiers –(some) solutions at the NICI PR3: Could MAPR be a solution? –our current achievements

3. Sources of variation (Schomaker, Plamondon et al ’00) Affine transforms Style Variations Neuro- biomechanichal Variations Order Variability

4. The more writers, the more….

5. Is this a problem? Shrihari ICDAR’01: “No problem” Handwriting is individual, so can be used in court (as a fingerprint)! IWFHRxx, ICDAR, IJDAR... “Yes!” We have 99+ digit recognition, 98+ character recognition and 90+ for isolated words........

6. Pattern Recognition (I) Raw data X(t),Y(t),P(t) Class labels

7. Pattern Recognition (I) Preprocessing Segmentation Feature Extraction Classification Class labels Raw data X(t),Y(t),P(t)

8. Solutions at the NICI More than 20 years of experience in –Handwriting production (Thomassen, van Galen, Meulenbroek, Maarse, Schomaker, et al) –Handwriting recognition (Schomaker, Teulings, Vuurpijl, et al) Keywords: –Use knowledge about human handwriting –Specialization and...... –Fusion

10. The basis of handwriting X(t),Y(t),P(t) Va(t)

11. UNIPEN data

12. Zooming in on writing styles UNIPEN styles: print, cursive, mixed

13. The lean recognition machine

14. Clustering on stroke- features (IWFHR’96) Specialization boosts recognition performance, while reducing computational and memory requirements

15. Finding structure in diversity (ICDAR’97)

16. Hierarchy in character shapes

17. Allograph prototypes

18. dScript: a MAPR system (’00)

19. dScript: 9 classifiers Neural networks (MLP, Kohonen) Nearest neighbour & clustering Structural/geometrical Support vector machines Hidden markov models Fusion through classifier combination and Multiple agents (IWFHR’98,’00,’02)

20. Pattern Recognition (II) Raw data Class labels Classifier 0 Classifier(i) Classifier Combination

21. Classifier combination (van Erp’00,’02)

22. Changing contexts.... - static architecture - what if “Go 551” was intended?

23. Dynamic PR Through extra heuristic information

24. Dynamic PR Through extra features (add PENUP) Features determine how you look at data

25. Dynamic PR Knowing when to use which feature

26. Dynamic PR Knowing when to use which feature

27. Dynamic PR Knowing when to use which feature/algorithm Through a knowledge base of PR Through a library of PR modules Through negotiation protocols Knowing how to use which PR module

28. Pattern Recognition (III) Raw data Class labels MAPR

29. What is an agent? (Wooldridge and Jennings, 1995) A software system with: Goals: ``What do you want?'' or ``What can you do?'' –I can solve 0-6 conflicts Beliefs and reasoning: ``How do you realize this goal?'' –I solve this 0-6 conflict using modules PR1 and PR2 –and features #84 and #96, extracted by FE(i) and FE(j) Assertions with confidence: –Based on my experience and these features I belief this input is a ``6'' with confidence 0.9. –I have been correct in 90% of the cases in the past.

30. Agent framework

31. MAPR: Our goal A distributed intelligent agent framework, with PR modules, symbolic equivalents and PR language. Driven by problem constraints and with learning capabilities.

32. Our current MAPR system Distributed processing over internet using sockets. Interfaces to KQML and Jatlite. Agents know about the environment. Agents know about the available PR modules and data. Agents interact with other agents. Agents to detect problems and conflicts

33. A proof of concept Trained recognition system hclus –15557 digits from UNIPEN –7778 train, 7779 test (95.9%) 1-7305-814 7-2271-812 5-3250-912 1-2238-011 1-2238-011 4-1224-810 4-614 1-7, 7-2, 4-6 conflicts solved  97%

34. But...... This is all hard-wired if confidence top[0] too low then solve(top[0],top[1],...) solve(1,7,2) = best(1-7,1-2,7-2)

35. Current research on MAPR Knowledge base PR language & implementation PR negotiation mechanisms

36. Conclusions Online HWR is still unsolved MCS can improve recognition rates, but..... –Hard-wired PR modules –Examples where dynamic PR is needed MAPR is a new paradigm that exploits knowledge about when to use which features or algorithms But how to implement shared access to knowledge? And how to perform agent-like negotiations ?