1. Document Processing Methods for Telugu and other SE Asian Scripts
Authors: Atul Negi, VSR Sowri, K Mohan Rao
Presented by:
Atul Negi, Dept of CIS, University of Hyderabad

2. SE Asian Scripts Complex arrangement of connected components Problems
difficulty in identifying the words and text line boundaries
touching characters
Nature of scripts: consonants with vowels and large number of distinct symbols

3. SE Asian scripts-Contd.
SE Asian scripts such as Telugu, Kannada, Simhala are rounded in nature.
We base our work on Telugu Script which is orthographically similar to many SE Asian scripts.

4. About Telugu Script Acchulu Hallulu Maatra Voththulu
Vowel Sound Symbols (16)
Hallulu
Consonant Sound Symbols (38)
Maatra
Vowel Sound Modifying Symbols for Hallulu (16)
Voththulu
Core Consonant Sound Symbols
Consists of Rounded Shapes (no vertical strokes)
Characters may be basic vowel/consonant shapes or could be composed by compounding shapes ([NCK 01] shows examples)
Example above shows glyphs in bounded boxes in a word pronounced as “Maa-tru-gee-ta”

5. Some Features of Telugu Script
Telugu is a phonetic script with each character representing a spoken syllable.
Contains curved letters with no vertical linear strokes and shirorekha (head line).
16 Vowels, 36 consonants, Telugu OCR system [NCK 01] reduced possible 10,000 symbols to about 400 glyphs
Glyph represents a single connected component, but is NOT a character

6. More Features of Telugu Script
Orthography is compositional with vowel sound symbols (matraas) modifying basic consonants.
Pure consonants sounds can be symbolized as vottus and can be combined with other consonant/vowel modified consonant symbols.
A character is made from a combination of the above
Vottus and matraas can be positioned at locations surrounding the base character

7. Brief Review Recognition Approaches
OCR Efforts in Telugu
Brief Review Recognition Approaches
[RD 77] Rajasekharan and Deekshatulu 1977
[SSP 95] Sukhswami, Seetharamulu , and Pujari 1995
[NCK 01] Negi, Chakravarthy and Krishna 2001
[NCS 02] Negi, Chakravarthy and Suresh Kumar 2002
[ P 02] Pujari et al 2002
[C R M N] Chakravarthy et al. 2002
[VP 02] Vasantha and Patvardhan 2002
[ NKC 03] Negi, Kasinadhuni, Chandrakanth 2003

8. Focus on Text Line and Character Segmentation Issues
In this presentation our contribution is focussed on
Text line Extraction: By clustering of connected components based upon their spatial properties.
Character segmentation- Drop Fall method and White stream method

9. Text Line Segmentation

10. Motivation
Text-line and text column extraction are crucial in PLA (Text Line Segmentation)
Affects the word and character level analysis.
Helps in logical grouping of individual glyphs into characters.
Simplifies the determination of logical sequence of characters.
Can be used to reduce the search space of OCR.

11. Overview of Text Line Segmentation Approaches
Approach as shown in [NKC 03] very complex, high time complexity
Pixel Projection Profile Approaches
Simpler, but do not work well with complex layouts and overlapping lines, or presence of skew
Bounding Box Projection Approaches
More efficient, work well in certain conditions
Limitations due to unevenness of white spacing
Bounding Box Co-ocurrances (this work)

12. Text Line Segmentation Using BB Projections
Heuristics Based on BB Projections
Concept is to extract adjacent zero BB count scan lines between BB peak lines
White space in between text-lines is broken, uneven and not contiguous because of the vottus and maatras in between text lines.
Touching characters from adjacent text-lines
More heuristics to improve results by estimating interfering characters from BB projections but results are not very good due to difficulty of estimation

13. Co-occurrence
“A measure of OVERLAP between different connected components.”
It is based on the spatial relationships of connected components.
It’s symmetric in nature.
Two types:
Horizontal co-occurrence
Vertical co-occurrence
Co-occurrence defines 3 different spatial relationships between components.

14. Horizontal Co-occurrence
Total Inclusion

15. Horizontal Co-occurrence
Partial-inclusion

16. Horizontal Co-occurrence
No Relation

17. Vertical Co-occurrence

18. Vertical Co-occurrence

19. Text-line extraction using co-occurrence
Text-line extraction problem is formulated as:
Identifying all the connected components which belong to the same text-line and obtaining the boundaries of text-lines by considering the bounding boxes of components.
Two major steps:
Computation of horizontal co-occurrence matrix for each pair of components.
Clustering of connected components based on the h-cooccurrence matrix.

20. Text-line extraction - Clustering
Let P,Q be two CC in the document image. P<Q, P,QC, PTk, Q  ?
If h-co-occurrence(P,Q) = total inclusion add Q
h-cooccurrence(P,Q) = partial inclusion, add Q *
h-cooccurrence(P,Q) = no relation, check next
*-conditional to overlap being greater than ½(height)
Post processing step

21. Text-line extraction - Results

22. Experimental Results – Hand Written Document Image

23. Experimental Results – Kannada Document Image

24. Experimental Results – Tamil Document Image

25. Character Segmentation

26. Character Segmentation
Is an operation that seeks to decompose an image of sequence of characters into sub images of individual symbols.

27. Character Segmentation methods

28. How can we segment characters?
Successful segmentation mainly involves two steps:
1.Locating a segmentation point
2.Generating a segmentation path
Drop Fall Methods attempt to do both

29. Hybrid Drop Fall Method
Segments the characters by following the contour of the image.
Advanced version of Hit and Deflect strategy.
Follows a set of rules that maximizes the chances that it will hit and deflect its way to an accurate path.

30. Drop Fall builds a path by mimicking an object falling or rolling in between the two characters
There are 8 varieties of Drop Fall methods which differ in directions, starting points and set of rules.
Path generated by a drop fall can be seen in fig given below

31. Locating the segmentation point
Pixels are scanned row-by-row until a black boundary pixel with another black boundary pixel to the right of it is detected, where the two pixels are seperated by atleast one white space.
This white pixel is then used as the starting point from which the marble is rolled down

32. Starting point for Drop fall

33. Incorrect segmentation of touching characters can be seen in the figure shown below.
Incorrect starting points leads to incorrect segmentation path.

34. Drop Fall Path Generation
The algorithm first looks out for a white pixel in its surroundings and if unable to find a white pixel then only cuts through the black pixel.
The directions that the algorithm will move in according to the present pixel positions and its surroundings is shown below

35. Top Left Drop Fall Input: Image 1.Binarize the input image
2.Locate the Segmentation point (x, y) using drop fall
3. Generate the segmentation path using the rules specified in the previous slide.
Output: Segmented Image

36. Characters segmented using top left Drop Fall: (standard drop fall)

37. Top left fails to segment the touching characters when the first character contains a Talakattu or is of concave shape.
Eg :Incorrect segmentation of Touching characters using Top left drop fall

38. Top Right Drop Fall
Identical to Top left drop fall except that it initiates from the top-right of the connected component.
Input: Touching character Image
Binarise the input image
Flip the image vertically
Locate the segmentation point
Generate the segmentation path
Re-flip the Image and obtain the segmented image.

39. Top Right Drop Fall

40. Bottom Left Drop Fall
Identical to standard drop fall except that it initiates from the bottom left drop fall
Input:Touching characters
Binarise the input image
Flip the image horizontally
Locate the segmentation point
Generate the segmentation path
Re-flip the Image horizontally and obtain the segmented image.

41. Bottom Left Drop Fall Method
Touching Characters segmented using Bottom left drop fall

42. Bottom left Drop Fall
Fails to segment the touching characters when the bottom half of the first character consists of curves or grooves

43. Bottom right drop fall Method

44. Characters segmented using Bottom right drop fall
Cases where Bottom right drop fall fails to segment the touching characters

45. Advanced Drop fall methods
Similar to Drop fall method in locating the segmentation point but while generating the segmentation path follows different set of rules.
While generating the segmentation path it will be look out for white pixels and when unable to find a while pixel it will move for black pixels and when it is on black pixels it will only look for black pixels.

46. Difference between drop fall and Advanced drop fall segmentation paths

47. Advanced Top left Drop Fall
Characters using Advanced top left drop fall
Incorrectly segmented characters using Advanced top left drop fall

48. Advanced Top right drop fall
Identical to Top right drop fall except the segmentation path generated is different.
Characters segmented using Advanced top right drop fall

49. Advanced bottom left drop fall
Characters segmented using Advanced bottom left drop fall
Incorrectly segmented characters using Advanced bottom left drop fall

50. Advanced bottom right drop fall (ABRD)
Characters segmented using ABRD
Incorrect segmented characters using ABRD

51. White Stream Method Used for identifying correct segmentation point
Input: Touching characters
1.Generate the contour
2Generate a bounding box
3.Count the number of white pixels in each column until a black pixel is encountered starting from the bottom of image
4.find column C with maximum count of white pixel.
5.output: Segmented characters

52. White stream Drop Fall Segmentation
In this method depending on the aspect ratios the segmentation is done. Horizontally touching characters are segmented using white stream method for locating the segmentation point and drop fall method to generate the segmentation path
Vertical touching characters are segmented using the column anding and projection profile

53. White stream drop fall segmentation
Characters segmented using white stream DF

54. Objective Function Drop Fall Segmentation (OFDS)
Is a Hybrid method (Column Anding + DF)
Input: Touching characters
Calculate the aspect ratio of C.C
Perform Column anding and find the column to be segmented
Generate the segmentation path using DF

55. OFDS
Characters segmented using OFDS

56. Results

57. Future work
Segmenting characters which consists of double and triple touchings.
Find the best path among various drop fall segmentation paths.
Finding the cavities and their positioning in order to segment the touching characters.

58. THANKS
Questions/comments?