LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition Supervised by Prof. LYU, Rung Tsong Michael Prepared by: Wong Chi Hang Tsang Siu Fung Department of Computer Science & Engineering The Chinese University of Hong Kong

Outline Introduction System Architecture Korean OCR Friend Reminder Conclusion Acknowledgement

Introduction – What is VTT? Smart Traveller with Visual Translator (VTT)  Mobile Device which is convenient for a traveller to carry Mobile Phone, Pocket PC, Palm, etc.  Recognize and translate the foreign text into native language  Detect and recognize the face into name

Introduction – Objective Two main features:  Korean to English Visual Translation  Remind Somebody’s Information with Face Image

Introduction – Objective (Cont.) Real Life Examples  Sometimes we lose the way, we need to know where we are.  Sometimes we forget somebody we met before.

System Architecture GUI Camera API Camera Korean OCRFace Recognizer Face Database Stroke Database & Dictionary Request Data RequestOutput User QueryResultQueryUpdateResult Request Response

Korean OCR (KOCR) Usage  Visual Translator from Korean to English Procedure for using KOCR  Text Area Detection  Character Identification  Translation

KOCR – Program Flow Initialization Capture Image Text Segmentation Recognition Translation

KOCR – Text Area Detection Edge Detection using Sobel Filter Horizontal Projection and Vertical Projection Find Potential Text Area by threshold Hor izon tal Proj ecti on Threshold Vertical Projection

KOCR – Text Area Detection (Cont.)

KOCR – Character Identification Features on Stroke  Extracted by Labeling Connected Component algorithm Proposed Feature Extraction  Five rays each side  Difference of adjacent rays (-1 or 0 or 1)  Has holes (0 or 1)  Dimension ratio of Stroke (width/height) (-1 or 0 or 1)

KOCR – Character Identification (Cont.)

KOCR – Translation Dictionary  Korean to English  About 1000 Korean Words Matching  Longest Match from left to right

KOCR – Translation (Cont.)

KOCR – Evaluations OCR Correctness  Training Set (3327 – 30% of all Character)  Testing Set (7845 – Others)  Result (64%)  Suggestion Train all Korean characters

KOCR – Evaluations (Cont.) Text Segmentation Correctness  45 Captured Images  99 Characters  Result Segment 83% characters correctly Segment 71% image correctly  Acceptable Result

KOCR – Evaluations (Cont.) OCR Correctness  45 Captured Images  99 Characters  Result 79% Characters correctly Recognized 69% Images correctly Recognized

Friend Reminder – Program Flow Initialization Capture Image Face Segmentation Recognition Show Profile

Friend Reminder (FR) Usage  Show the Profile of Friend by capturing a photo Procedure for using FR  Face Segmentation  Face Identification  Friend’s Profile

FR – Face Segmentation Eye Detection  Algorithm Gabor Wavelet Feature Log-Polar Sampling  Manual Selected (Suggest) Selected Eyes and Mouth Positions

FR – Face Segmentation

FR – Face Identification EigenFace  By using Principal Component Analysis (PCA)  Project the input face into the eigenvectors that pre-learned  Find the difference between the projection and the faces in database  Face determined to be ‘NEW’ if the difference is larger than a threshold

FR – Friend’s Profile

FR – Evaluations Eye Detection Correctness  40 Images  Result 22.5% Image Successfully Detected  Non-acceptable  Suggestion Manually Select Eyes and Mouth Positions

FR – Evaluations Face Identification  Evaluation Information 26 Test Persons’ Faces  16 faces is in database  10 faces is not in database 3 faces Trained per person 8 persons in face database  Result 77% Successfully Identified  63% Successfully Identified as person in database  100% Successfully Identified as person not in database

Conclusion Combined Modern Equipments  Digital camera  Personal Data Assistant (PDA) Techniques Learned  Image Processing  Optical Character Recognition  Face Recognition Techniques VTT Integrated  VTT for Korean to English OCR  VTT for Friend Reminder

Acknowledgement Thanks Professor Michael Lyu, Project Supervisor  Give us valuable advice  Provide us necessary equipments Thanks Edward Yau, Technical Manager of VIEW project  Give us many ideas

~The End~