Smart Traveller with Visual Translator

What is Smart Traveller? Mobile Device which is convenience for a traveller to carry Mobile Device which is convenience for a traveller to carry E.g. Pocket PC, Mobile Phone E.g. Pocket PC, Mobile Phone

What is Visual Translator? Recognize the foreign text and translate it into native language Recognize the foreign text and translate it into native language Detect the face and recognize it into name Detect the face and recognize it into name

Requirements Simple (Computational low power) Simple (Computational low power) Lightweight (Low Storage) Lightweight (Low Storage) User Friendly User Friendly

Core Pattern Recognition Model Image Segmentation Feature Extraction Classification Input ImageObject Image Feature VectorObject Type Find Each Object from the Image Quantify the object by some characteristics Assign Label for each object

Character Recognition Language: Korean Language: Korean Target: Sign, Guidepost Target: Sign, Guidepost –Contrast in Color –Printed Character

Image Segmentation Binarization Binarization –Using Color Histogram to binarize the image for the background and the character Text Region Segmentation Text Region Segmentation –User Define Method –Edge Detection with horizontal and vertical projections Stroke Extraction Stroke Extraction –Labeling of connected component Algorithm

Feature Extraction Stroke Features Stroke Features –Number of Junctions, Corners –Any Hole Gabor Features Gabor Features

Recognition Minimum Euclid Distance Minimum Euclid Distance Learn the Decision Tree by training examples Learn the Decision Tree by training examples

Demo

Face Detection Outline Find Face Region Find Face Region Find the potential eye region Find the potential eye region Locate the iris and eyelids Locate the iris and eyelids

Find Face Region - Color-based model We used this method because of its simplicity and robustness. We used this method because of its simplicity and robustness. Usually RGB color model will be transformed to other color modes such as YUV (luminance-chrominance) and HSB (hue, saturation and brightness) Usually RGB color model will be transformed to other color modes such as YUV (luminance-chrominance) and HSB (hue, saturation and brightness)

YUV We use YUV or YC b C r color model. We use YUV or YC b C r color model. Y component is used to represent the intensity of the image Y component is used to represent the intensity of the image C b and C r are used to represent the blue and red component respectively. C b and C r are used to represent the blue and red component respectively.

YC b C r Image Y, Cb,Cr component image Y, Cb,Cr component image Y C b C r

Representation of skin color We just use a simple ellipse equation to model skin color. We just use a simple ellipse equation to model skin color. Cb Cr

Representation of skin color The white regions represent the skin color pixels

Color segmentation We distribute some agent in the image uniformly. We distribute some agent in the image uniformly. Then each agent will check whether the pixel is a skin-like pixel and not visited by the other agent. Then each agent will check whether the pixel is a skin-like pixel and not visited by the other agent. If yes, it will produce 4 more agents at its four neighboring points. If yes, it will produce 4 more agents at its four neighboring points. If no, it will move to one of four neighboring points randomly and decrease its lifespan by 1. When its lifespan becomes zero, it will be removed from the image. If no, it will move to one of four neighboring points randomly and decrease its lifespan by 1. When its lifespan becomes zero, it will be removed from the image.

Color segmentation This agent produce 4 more agents

Color segmentation The advantage of this algorithm is that we need not to search the whole image. The advantage of this algorithm is that we need not to search the whole image. Therefore, it is fast. Therefore, it is fast.

Color segmentation of pixels is searched (about 18.7%) of pixels is searched (about 18.7%) There are 37 regions There are 37 regions Each color regions represent each regions searched by a father agent Each color regions represent each regions searched by a father agent

Eye detection After the segmentation of face region, we have some parts which are not regarded as skin color. After the segmentation of face region, we have some parts which are not regarded as skin color. They are probably the region of eye and mouth They are probably the region of eye and mouth We only consider the red component of these regions because it usually includes the most information about faces. We only consider the red component of these regions because it usually includes the most information about faces.

Eye detection We extraction such regions. We extraction such regions. The red region represent the region which is not skin color. The red region represent the region which is not skin color.

Eye detection We do the following on the regions of potential eye region 1. Histogram equalization 2. Threshold 3. Template matching

Eye detection Histogram equalization Threshold with < 49 Template Matching

Locating the iris and eyelids We plan to use the following methods to improve the face detection We can use these methods to locate the iris and eyelid precisely. Template matching –Correlation variance filter –Deformable template

END