Document Processing Methods for Telugu and other SE Asian Scripts Authors: Atul Negi, VSR Sowri, K Mohan Rao Presented by: Atul Negi, Dept of CIS, University of Hyderabad atulcs@uohyd.ernet.in

SE Asian Scripts Complex arrangement of connected components Problems difficulty in identifying the words and text line boundaries touching characters Nature of scripts: consonants with vowels and large number of distinct symbols

SE Asian scripts-Contd. SE Asian scripts such as Telugu, Kannada, Simhala are rounded in nature. We base our work on Telugu Script which is orthographically similar to many SE Asian scripts.

About Telugu Script Acchulu Hallulu Maatra Voththulu Vowel Sound Symbols (16) Hallulu Consonant Sound Symbols (38) Maatra Vowel Sound Modifying Symbols for Hallulu (16) Voththulu Core Consonant Sound Symbols Consists of Rounded Shapes (no vertical strokes) Characters may be basic vowel/consonant shapes or could be composed by compounding shapes ([NCK 01] shows examples) Example above shows glyphs in bounded boxes in a word pronounced as “Maa-tru-gee-ta”

Some Features of Telugu Script Telugu is a phonetic script with each character representing a spoken syllable. Contains curved letters with no vertical linear strokes and shirorekha (head line). 16 Vowels, 36 consonants, Telugu OCR system [NCK 01] reduced possible 10,000 symbols to about 400 glyphs Glyph represents a single connected component, but is NOT a character

More Features of Telugu Script Orthography is compositional with vowel sound symbols (matraas) modifying basic consonants. Pure consonants sounds can be symbolized as vottus and can be combined with other consonant/vowel modified consonant symbols. A character is made from a combination of the above Vottus and matraas can be positioned at locations surrounding the base character

Brief Review Recognition Approaches OCR Efforts in Telugu Brief Review Recognition Approaches [RD 77] Rajasekharan and Deekshatulu 1977 [SSP 95] Sukhswami, Seetharamulu , and Pujari 1995 [NCK 01] Negi, Chakravarthy and Krishna 2001 [NCS 02] Negi, Chakravarthy and Suresh Kumar 2002 [ P 02] Pujari et al 2002 [C R M N] Chakravarthy et al. 2002 [VP 02] Vasantha and Patvardhan 2002 [ NKC 03] Negi, Kasinadhuni, Chandrakanth 2003

Focus on Text Line and Character Segmentation Issues In this presentation our contribution is focussed on Text line Extraction: By clustering of connected components based upon their spatial properties. Character segmentation- Drop Fall method and White stream method

Text Line Segmentation

Motivation Text-line and text column extraction are crucial in PLA (Text Line Segmentation) Affects the word and character level analysis. Helps in logical grouping of individual glyphs into characters. Simplifies the determination of logical sequence of characters. Can be used to reduce the search space of OCR.

Overview of Text Line Segmentation Approaches Approach as shown in [NKC 03] very complex, high time complexity Pixel Projection Profile Approaches Simpler, but do not work well with complex layouts and overlapping lines, or presence of skew Bounding Box Projection Approaches More efficient, work well in certain conditions Limitations due to unevenness of white spacing Bounding Box Co-ocurrances (this work)

Text Line Segmentation Using BB Projections Heuristics Based on BB Projections Concept is to extract adjacent zero BB count scan lines between BB peak lines White space in between text-lines is broken, uneven and not contiguous because of the vottus and maatras in between text lines. Touching characters from adjacent text-lines More heuristics to improve results by estimating interfering characters from BB projections but results are not very good due to difficulty of estimation

Co-occurrence “A measure of OVERLAP between different connected components.” It is based on the spatial relationships of connected components. It’s symmetric in nature. Two types: Horizontal co-occurrence Vertical co-occurrence Co-occurrence defines 3 different spatial relationships between components.

Horizontal Co-occurrence Total Inclusion

Horizontal Co-occurrence Partial-inclusion

Horizontal Co-occurrence No Relation

Vertical Co-occurrence

Vertical Co-occurrence

Text-line extraction using co-occurrence Text-line extraction problem is formulated as: Identifying all the connected components which belong to the same text-line and obtaining the boundaries of text-lines by considering the bounding boxes of components. Two major steps: Computation of horizontal co-occurrence matrix for each pair of components. Clustering of connected components based on the h-cooccurrence matrix.

Text-line extraction - Clustering Let P,Q be two CC in the document image. P<Q, P,QC, PTk, Q  ? If h-co-occurrence(P,Q) = total inclusion add Q h-cooccurrence(P,Q) = partial inclusion, add Q * h-cooccurrence(P,Q) = no relation, check next *-conditional to overlap being greater than ½(height) Post processing step

Text-line extraction - Results

Experimental Results – Hand Written Document Image

Experimental Results – Kannada Document Image

Experimental Results – Tamil Document Image

Character Segmentation

Character Segmentation Is an operation that seeks to decompose an image of sequence of characters into sub images of individual symbols.

Character Segmentation methods

How can we segment characters? Successful segmentation mainly involves two steps: 1.Locating a segmentation point 2.Generating a segmentation path Drop Fall Methods attempt to do both

Hybrid Drop Fall Method Segments the characters by following the contour of the image. Advanced version of Hit and Deflect strategy. Follows a set of rules that maximizes the chances that it will hit and deflect its way to an accurate path.

Drop Fall builds a path by mimicking an object falling or rolling in between the two characters There are 8 varieties of Drop Fall methods which differ in directions, starting points and set of rules. Path generated by a drop fall can be seen in fig given below

Locating the segmentation point Pixels are scanned row-by-row until a black boundary pixel with another black boundary pixel to the right of it is detected, where the two pixels are seperated by atleast one white space. This white pixel is then used as the starting point from which the marble is rolled down

Starting point for Drop fall

Incorrect segmentation of touching characters can be seen in the figure shown below. Incorrect starting points leads to incorrect segmentation path.

Drop Fall Path Generation The algorithm first looks out for a white pixel in its surroundings and if unable to find a white pixel then only cuts through the black pixel. The directions that the algorithm will move in according to the present pixel positions and its surroundings is shown below

Top Left Drop Fall Input: Image 1.Binarize the input image 2.Locate the Segmentation point (x, y) using drop fall 3. Generate the segmentation path using the rules specified in the previous slide. Output: Segmented Image

Characters segmented using top left Drop Fall: (standard drop fall)

Top left fails to segment the touching characters when the first character contains a Talakattu or is of concave shape. Eg :Incorrect segmentation of Touching characters using Top left drop fall

Top Right Drop Fall Identical to Top left drop fall except that it initiates from the top-right of the connected component. Input: Touching character Image Binarise the input image Flip the image vertically Locate the segmentation point Generate the segmentation path Re-flip the Image and obtain the segmented image.

Top Right Drop Fall

Bottom Left Drop Fall Identical to standard drop fall except that it initiates from the bottom left drop fall Input:Touching characters Binarise the input image Flip the image horizontally Locate the segmentation point Generate the segmentation path Re-flip the Image horizontally and obtain the segmented image.

Bottom Left Drop Fall Method Touching Characters segmented using Bottom left drop fall

Bottom left Drop Fall Fails to segment the touching characters when the bottom half of the first character consists of curves or grooves

Bottom right drop fall Method

Characters segmented using Bottom right drop fall Cases where Bottom right drop fall fails to segment the touching characters

Advanced Drop fall methods Similar to Drop fall method in locating the segmentation point but while generating the segmentation path follows different set of rules. While generating the segmentation path it will be look out for white pixels and when unable to find a while pixel it will move for black pixels and when it is on black pixels it will only look for black pixels.

Difference between drop fall and Advanced drop fall segmentation paths

Advanced Top left Drop Fall Characters using Advanced top left drop fall Incorrectly segmented characters using Advanced top left drop fall

Advanced Top right drop fall Identical to Top right drop fall except the segmentation path generated is different. Characters segmented using Advanced top right drop fall

Advanced bottom left drop fall Characters segmented using Advanced bottom left drop fall Incorrectly segmented characters using Advanced bottom left drop fall

Advanced bottom right drop fall (ABRD) Characters segmented using ABRD Incorrect segmented characters using ABRD

White Stream Method Used for identifying correct segmentation point Input: Touching characters 1.Generate the contour 2Generate a bounding box 3.Count the number of white pixels in each column until a black pixel is encountered starting from the bottom of image 4.find column C with maximum count of white pixel. 5.output: Segmented characters

White stream Drop Fall Segmentation In this method depending on the aspect ratios the segmentation is done. Horizontally touching characters are segmented using white stream method for locating the segmentation point and drop fall method to generate the segmentation path Vertical touching characters are segmented using the column anding and projection profile

White stream drop fall segmentation Characters segmented using white stream DF

Objective Function Drop Fall Segmentation (OFDS) Is a Hybrid method (Column Anding + DF) Input: Touching characters Calculate the aspect ratio of C.C Perform Column anding and find the column to be segmented Generate the segmentation path using DF

OFDS Characters segmented using OFDS

Results

Future work Segmenting characters which consists of double and triple touchings. Find the best path among various drop fall segmentation paths. Finding the cavities and their positioning in order to segment the touching characters.

THANKS Questions/comments? atulcs@uohyd.ernet.in