Language-Independent Text Line Extraction from Historical Document Images Presented by: Abed Asi First International workshop on Historical Document.

Slides:



Advertisements
Similar presentations
Patient information extraction in digitized X-ray imagery Hsien-Huang P. Wu Department of Electrical Engineering, National Yunlin University of Science.
Advertisements

Image Processing Ch2: Digital image Fundamentals Part 2 Prepared by: Tahani Khatib.
Word Spotting DTW.
November 12, 2013Computer Vision Lecture 12: Texture 1Signature Another popular method of representing shape is called the signature. In order to compute.
Robust statistical method for background extraction in image segmentation Doug Keen March 29, 2001.
Prénom Nom Document Analysis: Document Image Processing Prof. Rolf Ingold, University of Fribourg Master course, spring semester 2008.
Image Segmentation Image segmentation (segmentace obrazu) –division or separation of the image into segments (connected regions) of similar properties.
電腦視覺 Computer and Robot Vision I Chapter2: Binary Machine Vision: Thresholding and Segmentation Instructor: Shih-Shinh Huang 1.
DIGITAL IMAGE PROCESSING
Texture Segmentation Based on Voting of Blocks, Bayesian Flooding and Region Merging C. Panagiotakis (1), I. Grinias (2) and G. Tziritas (3)
An article by: Itay Bar-Yosef, Nate Hagbi, Klara Kedem, Itshak Dinstein Computer Science Department Ben-Gurion University Beer-Sheva, Israel Presented.
Image Indexing and Retrieval using Moment Invariants Imran Ahmad School of Computer Science University of Windsor – Canada.
Segmentation (2): edge detection
Chapter 5 Raster –based algorithms in CAC. 5.1 area filling algorithm 5.2 distance transformation graph and skeleton graph generation algorithm 5.3 convolution.
December 5, 2013Computer Vision Lecture 20: Hidden Markov Models/Depth 1 Stereo Vision Due to the limited resolution of images, increasing the baseline.
Agenda Seam-carving: finish up “Mid-term review” (a look back) Main topic: Feature detection.
Surface Reconstruction from 3D Volume Data. Problem Definition Construct polyhedral surfaces from regularly-sampled 3D digital volumes.
Contents Description of the big picture Theoretical background on this work The Algorithm Examples.
Prénom Nom Document Analysis: Data Analysis and Clustering Prof. Rolf Ingold, University of Fribourg Master course, spring semester 2008.
CS 376b Introduction to Computer Vision 02 / 27 / 2008 Instructor: Michael Eckmann.
Survey of Text Line Segmentation Methods of Historical Documents
Objective of Computer Vision
Stereo Computation using Iterative Graph-Cuts
Robust estimation Problem: we want to determine the displacement (u,v) between pairs of images. We are given 100 points with a correlation score computed.
E.G.M. PetrakisBinary Image Processing1 Binary Image Analysis Segmentation produces homogenous regions –each region has uniform gray-level –each region.
Radial-Basis Function Networks
Chapter 10: Image Segmentation
Dimensionality Reduction: Principal Components Analysis Optional Reading: Smith, A Tutorial on Principal Components Analysis (linked to class webpage)
CS 551/651 Advanced Computer Graphics Warping and Morphing Spring 2002.
EE 492 ENGINEERING PROJECT LIP TRACKING Yusuf Ziya Işık & Ashat Turlibayev Yusuf Ziya Işık & Ashat Turlibayev Advisor: Prof. Dr. Bülent Sankur Advisor:
S EGMENTATION FOR H ANDWRITTEN D OCUMENTS Omar Alaql Fab. 20, 2014.
Digital Image Fundamentals II 1.Image modeling and representations 2.Pixels and Pixel relations 3.Arithmetic operations of images 4.Image geometry operation.
September 23, 2014Computer Vision Lecture 5: Binary Image Processing 1 Binary Images Binary images are grayscale images with only two possible levels of.
Digital Image Processing CCS331 Relationships of Pixel 1.
Chapter 10, Part II Edge Linking and Boundary Detection The methods discussed in the previous section yield pixels lying only on edges. This section.
Introduction Image geometry studies rotation, translation, scaling, distortion, etc. Image topology studies, e.g., (i) the number of occurrences.
Data Extraction using Image Similarity CIS 601 Image Processing Ajay Kumar Yadav.
Image gradients and edges Tuesday, September 1 st 2015 Devi Parikh Virginia Tech Disclaimer: Many slides have been borrowed from Kristen Grauman, who may.
Kshitij Padalkar 13 th Oct Contents: In one Line… Need Solutions and Other Work Definition and Maths Energy, Seam Pixel Removal Criteria Other Energy.
Mathematical Morphology Mathematical morphology (matematická morfologie) –A special image analysis discipline based on morphological transformations of.
CS654: Digital Image Analysis Lecture 24: Introduction to Image Segmentation: Edge Detection Slide credits: Derek Hoiem, Lana Lazebnik, Steve Seitz, David.
CVPR2013 Poster Detecting and Naming Actors in Movies using Generative Appearance Models.
Digital Image Processing Lecture 16: Segmentation: Detection of Discontinuities Prof. Charlene Tsai.
CS654: Digital Image Analysis Lecture 5: Pixels Relationships.
Mathematical Morphology
Image Segmentation Image segmentation (segmentace obrazu)
Digital Image Processing Lecture 16: Segmentation: Detection of Discontinuities May 2, 2005 Prof. Charlene Tsai.
Course 3 Binary Image Binary Images have only two gray levels: “1” and “0”, i.e., black / white. —— save memory —— fast processing —— many features of.
Chapter 6 Skeleton & Morphological Operation. Image Processing for Pattern Recognition Feature Extraction Acquisition Preprocessing Classification Post.
Color Image Segmentation Mentor : Dr. Rajeev Srivastava Students: Achit Kumar Ojha Aseem Kumar Akshay Tyagi.
Visible-Surface Detection Methods. To identify those parts of a scene that are visible from a chosen viewing position. Surfaces which are obscured by.
Computer Graphics CC416 Lecture 04: Bresenham Line Algorithm & Mid-point circle algorithm Dr. Manal Helal – Fall 2014.
April 21, 2016Introduction to Artificial Intelligence Lecture 22: Computer Vision II 1 Canny Edge Detector The Canny edge detector is a good approximation.
Application 2 Detect Filarial Worms SourceBTTRemove NoisesThreshold Skeleton Eliminate short structures ReconstructionFinal result.
Digital Image Processing Lecture 16: Segmentation: Detection of Discontinuities Prof. Charlene Tsai.
Image gradients and edges
Computer Vision Lecture 9: Edge Detection II
Image gradients and edges April 11th, 2017
Computer Vision Lecture 16: Texture II
CSSE463: Image Recognition Day 23
Lecture 5 Image Characterization ch. 4 of Machine Vision by Wesley E
Pixel Relations.
Computer and Robot Vision I
CSC 381/481 Quarter: Fall 03/04 Daniela Stan Raicu
Seam Carving Project 1a due at midnight tonight.
Handwritten Characters Recognition Based on an HMM Model
CSSE463: Image Recognition Day 23
EE 492 ENGINEERING PROJECT
CSSE463: Image Recognition Day 23
ECE/CSE 576 HW 1 Notes.
Presentation transcript:

Language-Independent Text Line Extraction from Historical Document Images Presented by: Abed Asi First International workshop on Historical Document Imaging and Processing 2011, Beijing , China

Motivation Historical handwritten manuscripts are valuable cultural heritage Providing insights into both tangible and intangible cultural aspects from the past  Efforts to understand, manipulate and archive historical manuscripts Digitization increases accessibility and allows automatic processing *Courtesy: - wadod.com - Genizah Project

Outline Background Challenges Seam Carving Text line representation by seams Energy Map Seam Generation Experimental Results Summary

Image representation N x M (Matrix)

Binarization # pixels intensity

Connectivity & Components We can define 4- or 8-paths depending on the type of connectivity specified A set of pixels S is a Connected Component if for each pixel pair (x1,y1) є S and (x2,y2) є S there is a path between them such that every two successive pixels in the path are in S and are X-neighbors. (X = 4, 8). 4-Neighborhood 8-Neighborhood

Connected Component One word, but 3connected components

Distances Given 2 points P = (u,v) , Q = (x,y) Euclidean Distance City Block Distance Chessboard Distance In example: P = (1,8); Q = (4,1)

Distance transform Given a set of pixels S, calculate the distance of other pixels to S The pixels in the set S will be considered as reference pixels Let . We scan the image by a pre-defined connectivity : First pass: Consider Green pixels (N1)

Distance transform In reverse scan, consider Blue pixels (N2) First scan Distance transform

Distance transform – (cont’d) 3 2 1 4 1 Alef Letter - Arabic Printed Handwritten Binary Representation Distance transform Chessboard metric = Reference pixels

Sign Distance transform 3 2 1 -1 4 Alef Letter Printed Handwritten Sign Distance transform chessboard metric

Sign Distance transform – (cont’d) The brighter the color the larger the distance from reference pixels Original Document Image Sign Distance transform (SDT)

Gradient is the derivative of the image in the horizontal direction A gray-scale image I is defined as a two-dimensional function I(x,y)=gray The gradient of the image (I ) is given by the formula : Where: is the derivative of the image in the horizontal direction is the derivative of the image in the vertical direction The magnitude of the gradient is defined by:

Gradient

*Courtesy: Islamic manuscript, Leipzig University Library, Germany Background Pre-Processing De-noising Binarization Page Layout Analysis Text-line and word Segmentation Indexation and Recognition Segmentation Original *Courtesy: Islamic manuscript, Leipzig University Library, Germany

*Courtesy: Juma Al-majid Center for Culture and Heritage, Dubai. Text-line Extraction Assigning the same color to each text line ب ت ث يــ جـ خـ حـ Original Manuscript Processed Manuscript *Courtesy: Juma Al-majid Center for Culture and Heritage, Dubai.

Outline Background Challenges Seam Carving Text line representation by seams Energy Map Seam Generation Experimental Results Summary

Challenges Looser layout format Line Proximity Multi-Oriented lines Historical handwritten documents pose different challenges than those in machine-printed. Looser layout format Line Proximity Multi-Oriented lines Touching components Different slope (within the same line) Delayed strokes Overlapping components A 19th century master thesis – SAAB medical Library, American University of Beirut

Outline Background Challenges Seam Carving Text line representation by seams Energy Map Seam Generation Experimental Results Summary

Seam Carving Content-aware image resizing An energy function defines energy value for each pixel A seam is an optimal 8-connected path of low energy pixels Original Image Calculated seams Gradient Image Resized

Seam Carving – (cont’d) let I be an n x m size image. Define a vertical seam to be: where x is a mapping x : [1, . . . ,n] [1, . . . ,m]. Seam contains one, and only one, pixel in each row of the image, otherwise a distorted image might be obtained. The pixels of the path of a seam will therefore be : one can change the value of K in the constraint, and get either a simple column for k = 0 , or even completely disconnected set of pixels.

Seam Carving – (cont’d) Given an energy function e, the cost of a seam is: We look for the optimal seam s* that minimizes this cost : The optimal seam can be found using Dynamic programming

Outline Background Challenges Seam Carving Text line representation by seams Energy Map Seam Generation Experimental Results Summary

Text line representation by seams Human perception of text lines Tracks text lines by ink concentration and in-between line spaces Two types of seams have been defined *Courtesy: Wadod Center for masnuscripts.

Text line representation by seams -(cont’) The medial seam crosses the text area of a text line. A Separating seam is a path that passes between two consecutive text lines. Original Document Image Seam Seed Medial Seam Separating Seam Processed *Courtesy: Wadod Center for masnuscripts.

Outline Background Challenges Seam Carving Text line representation by seams Energy Map Seam Generation Experimental Results Summary

Energy Map We use the Sign distance transform (SDT) as an energy map In SDT, pixels values are assigned according to their distance from the nearest reference pixel Recall, distance values are negative inside connected components and positive in- between Intuition: Local minima and maxima points determine the medial and separating seams, respectively Original Document Image Sign Distance Transform (SDT) *Courtesy: Wadod Center for masnuscripts

Outline Background Challenges Seam Carving Text line representation by seams Energy Map Seam Generation Experimental Results Summary

Seam Generation – (cont’d) The SDT is traversed horizontally to compute a cumulative energy map - Seam Map - for all possible connected seams for each entry (i,j): SDT is traversed with two passes to enhance text line patterns Sign distance transform Bi-linearly interpolate the resulting two maps Left-to-right pass Interpolated map Right-to-left pass

Seam Generation – (cont’d) The minimal entry of the last column is detected. Backtrack from the minimal entry to find the medial seam. Original Document Image Seam Map – One pass Seam Map – Two passes

Seam Generation – (cont’d) Iteratively, all text lines will be extracted

Seam Generation – (cont’d) Then, why separating seams are needed? Avoid recalculation of energy and seam maps after each line extraction Avoid additional strokes classification (post processing)

Seam Generation – (cont’d) Separating seams define the boundaries of text lines Generated with respect to the medial seam of the corresponding text line Grown from seam seeds toward the two sides of the image guided by the SDT

Seam Generation – (cont’d) Seam fragment is a connected group of pixels defined as the closest local maxima along the vertical direction Seam fragments with low priority are discarded Seeds candidate set is constructed The seed that generates the optimal (maximal cost) seam was chosen Medial Seam Seam Map Sign Distance Transform

Seam Generation – (cont’d) The separating seams may diverge from the medial seam due to the fork of ridges A spring force anchored at the medial seam guides the separating seams Before After

Touching/Overlapping Components Usually, crossing overlapping components is avoided gracefully Touching components are split too, but not necessarily in the optimal position Processed Processed

Outline Background Challenges Seam Carving Text line representation by seams Energy Map Seam Generation Experimental Results Summary

Overlapping Components Experimental Results Language Overlapping Components Lines Description Dataset Arabic and Spanish 516 1050 Wadod Center for Manuscripts Wadod Arabic 258 900 Al-Majid Center for Culture and Heritage, Dubai Al-Majid English 485 420 American University of Beirut AUB 317 150 Congress Library 1576 2520

Experimental Results- (cont’d) Correctness (%) Dataset Line Lower Upper Medial 98 97 99 Wadod 96 Al-Majid 95 94 AUB 94.25 93 Congress library Stroke Crossing (%) Overlapping Components Dataset 9 516 Wadod 2 258 Al-Majid 485 AUB 10 317 Congress library Table 1: correctness of text line extraction Table 2: crossed components

Experimental Results- (cont’d)

Outline Background Challenges Seam Carving Text line representation by seams Energy Map Seam Generation Experimental Results Summary

Summary Summary Language independent approach Dynamic programming was used to find text lines Saves energy map re-computing after text line extraction Post processing steps are avoided Crossing overlapping components was avoided in most cases Still need more research to split touching components optimally

Thank you