1. Features for handwriting recognition

2. The challenge
“Rappt JD 10 Feb no 175, om machtiging om af”

3. Short processing pipeline
Learning
“machtiging”
Feature extraction
Classification
82,34,66,…
“machtiging”
0.12

4. Processing pipeline
Preprocessing
Feature extraction
Classification

5. Input image types
Color:
Grayscale:
Binary:

6. Preprocessing
Goal: enhance the foreground while reducing other visual symptoms (stains, noise, pictures, ...)
Methods:
Contrast stretching
Highpass filtering
Despeckling
Change color representation (RGB, HSV, grayscale, black/white, …)
Remove selected connected components () …

7. Connected components

8. Processing pipeline Preprocessing Segmentation Feature extraction
Classification

9. Object of classification
Sentences
Words
Characters
(use grammar)
(use dictionary)
(use alphabet)

10. Object representations
Image
Unordered vectors (in a coco)
Contour vectors
On-line vectors
Skeleton image
Skeleton vectors
I(x, y)
(x, y)i
(x, y)k
(x, y)k
I(x, y)
(x, y)k

11. A full processing pipeline
Preprocessing
Segmentation
Normalization
Feature extraction
Classification

12. Invariance Luminance / contrast Position Size Rotation Shear
Writer style
Ink thickness …

13. Invariance by normalization
Contrast stretching
Luminance / contrast
Position
Size
Rotation
Shear
Writer style
Ink thickness …
Center on center of gravity
Scale to standard size

14. Invariance by trying many deformations
Luminance / contrast
Position
Size
Rotation
Shear
Writer style
Ink thickness …
Try different scale factors
Try different rotations
Try different deformations
… and use the best recognition result

15. Invariance by using invariant features
Luminance / contrast
Position
Size
Rotation
Shear
Writer style
Ink thickness …
Zernike invariant moments

16. A full processing pipeline
Preprocessing
Segmentation
Normalization
Feature extraction
Classification
82,34,66,…

17. Feature ROI types
Whole object
Zones
Windowing

18. Whole object (“holistic”)

19. Zones

20. Windowing

21. Feature types Image itself Statistical Structural Abstract
Image (off-line) features (1—20)
Contour / on-line features (21 – 28)

22. Feature 1 – 3 Connected component images Scaled image
Distance transform

23. Feature 4: density histogram

24. Feature 5: radon transform

25. Feature 6: run count pattern2 3 3 6

26. Feature 7: run length pattern
avg
stdev
avg
stdev

27. Feature 8: Autocorrelation

28. Feature 9: Polar zones

29. Feature 10: radial zones (tip!)

30. Feature 11: zone histograms

31. Feature 12: Hinge
(By Marius Bulacu)

32. Feature 13: Fraglets

33. Feature 14: J.C. Simon (1/2)
Singulariteiten
Regelmatigheden

34. Feature 14: J.C. Simon (2/2) "million" ==>
convex:concave:3(north:concave)
:(north:LOOP):concave:(north:LOOP)
:concave:north
:concave:HOLE
:2(convex:concave)
(J.-C. Simon, 1989)

35. Feature 15: Structure of background (1/3)

36. Feature 15: Structure of background (2/3)

37. Feature 15: Structure of background (3/3)

38. Feature 16: Structure of foreground + background

39. Feature 17: Fourier transform (1/2)
From:

40. Feature 17: Fourier transform (2/2)
Fig. 1 and 3 from:
Fig. 2 from:

41. Feature 18: Wavelet transform
From:

42. Feature 19: Hu invariant moments
Derived from moments
Moments describe the image distribution with respect to its axes
Works on (x, y) vectors
Invariant for scale, position and rotation
area of the object
center of mass
Slide adapted from:

43. Feature 20: Zernike moments
Invariant for scale, position and rotation
Reconstructing original
From: Trier, O. D., Jain, A. K., and Taxt, T. (1996). Feature extraction methods for character recognition - a survey. Pattern Recognition,29:641–662.

44. Feature 21 – 28: Contour features
(cos, sin) of running angle
(cos, sin) of running angular difference
Angular difference
Fourier transform
Ink density (horizontal or vertical)
Radon transform: (ink density, computed radially from the c.o.g.)
Angular histogram
Curvature scale space ()

45. Feature 28: Curvature scale space
iteration
pos
From:

46. End