Arabic Sign Language Recognition Mohamed Mohandes King Fahd University of Petroleum and Minerals

OUTILNE Introduction Sign Language Recognition Sign Language Recognition Translation of text to sign language Conclusion and future work

Importance of Sign Language Arabic sign language (ARSL) is different from spoken Arabic language in terms of grammars, vocabulary, and delivery. ARSL is the natural language for deaf like spoken language to vocal Sign language is different from country to other (Australia, UK, USA) 100,000 deaf and hearing impaired in KSA

Objectives Using Computers to make life of deaf easier and integrating them in the society : Translating ARSL to spoken language Translating ARSL to spoken language Translating Arabic speech to ARSL Translating Arabic speech to ARSL

ARSL Recognition Image based Requires special set up for camera Requires special set up for camera Heavy computational load to extract hands Heavy computational load to extract hands Electronic-Glove based Inconvenience of gloves Inconvenience of gloves Ease of signal extractions Ease of signal extractions

Translating ARSL to Speech سلمان

System Components

CyberGlove CyberGlove

CyberGlove 22 sensors Light weight Flexible

Tracking System

Sensor signals for two different words

Coordinates of word الله Coordinates of word مع السلامة

المنظمة العربية للتربية والثقافة والعلوم الاتحاد العربي للهيئات العاملة في رعاية الصم الاشارات المعتمدة

1300 signs 344 single handed

Data Collection 6880 Samples6880 samples 20 samples from every sign: 15 for training and 5 for testing 344 single handed signs

Support Vector Machine

SVM

System Performance Time segments and Principle Component Analysis for feature extraction Correct recognition rate of 98.33%

Analysis of Misclassified Signs Frames of sign of letter “س” Frames of sign of letter “ش”

Analysis of Misclassified Signs Frames from the sign “employee” Frames from the sign “Down syndrome”

Analysis in Feature space Hand shape of “employee” Hand shape of “Down syndrome”

Translating Speech to ARSL سلمان ســـــــــــلمان

Conclusions and Future work Developed a Real-time single-handed Arabic sign language recognition system with accuracy of 98.33% Working on two-handed signs recognition Developing our own smart glove for Arabic Sign Language Mapping of signs to roots for speech to sign translation