Recognition Of Textual Signs Final Project for “Probabilistic Graphics Models” Submitted by: Ezra Hoch, Golan Pundak, Yonatan Amit

Introduction Input: 600x480 image Output: 6X6 pixels region labeling (text /no text) Motivation: narrow OCR search area for urban / road signs.  Can be used by various ‘Vehicle Vision’ applications.

Loopy BP Project Overview Classifier Features extractor

Contribution of Graphical Model The % of agreement between hand-label and graphic inference The % of agreement between hand-label and classifier This graph portraits the results of the naïve classification vs. the results after graphic inference. It is clear that using inference improves the results. We also note the importance of quality classification to our results Classifier After inference

Initial Labeling & Classification 250 pictures were taken in jerusalem 20% of pictures were hand labeled for three categories  Text, Sign Background, General Scene  10% were used for training, the rest for testing. Feature Extraction  Images were divided to blocks of 6x6  Each block was considered with a 10x10 surrounding  19 Features were selected, attempting to apprehend spatial information relevant to textual areas Using a Back-Propagation network as classifier  Three level neural network Classifaier output Hand labelingOriginal image

Learning the Potential Functions Our Model  80x100 nodes, Grid topology  Assume movement invariance Horizontal VS. Vertical Psi.  Sensitive to changes of scale and rotations. (Improvement requires better features) Our Input:  10 Images used as input  Local PSI’s were fixed by classifier The Algorithm  Using IPF as learning algorithm  Empirical Distribution calculated from hand-labeled instances.  Current Estimation calculated by approximate inference, namely Loopy BP Output:  Horizontal and Vertical Psi

Model Variations Using uniform Psi for Vertical Neighbours and Horizontal Neighbours.  Results were poorer than different Psi A second regional grid was tested  Creating a decent classifier for regional nodes has proven a difficult task  Has shown little improvement if any  Was not used in latest results Classifier Features extractor Regional Classifier Features extractor Regional Nodes Local Nodes

Second Layer Contribution Original image Classifier result Single layer output 2nd layer output We attempted to integrate a 2 nd layer of regional nodes each “looking” at a collection of 10x10 local nodes. Uniform Local Psi was given to the 2 nd layer. Intuitively selected Psi were used between the first and second layer (no learning was performed). On some cases this yields better results. Arched text was blurred by single layer and preserved with 2 layers

Results

More Results

Conclusions Using Graphical Inference allows seamless combination of local classification and global properties Variations of the Graphical Model improves our ability to perceive the underling world However, there’s no substitute for a proper local classifier and feature-extractor Robust feature extraction Complete second level regional nodes Improve hand labeling, enabling more accurate training Future Work