A Bangla Predictive Keyboard For People With Neuro-Motor Disorders Presented By Animesh Mukherjee Research Scholar Department of Computer Science and Engineering IIT Kharagpur

The Reality Suppose you are asked to use a computer which has The Mouse Unplugged The Keyboard Unplugged

The Divide Nevertheless there is a big population in India (14.56 million approx) that experiences such a difficulty every day These are people suffering from neuro-motor disorders For them the presence or the absence of a mouse or a keyboard is always synonymous to its absence

Neuro-Motor Disorder – What is it ? These disorders are caused by -  Faulty development of motor areas in the brain, or,  Total damage of these motor areas. Produces Nerve Cells that Causes Movements of the Body Parts Serves to Modify the Movements

Consequences … Severe difficulty with fine motor tasks (like writing, stitching, using computer peripherals, and various other such tasks.) Severe difficulty with any kind of communication. In a nutshell,  Access to the computers is almost a “dream come true”  The presence/absence of the peripherals are irrelevant for them.

Can Computers Help Certainly computers can help this population by being  An easy medium of communication (which they find very difficult)  An intelligent companion by understanding the needs and thereby reducing the communication efforts

The Impetus: Something Indian!! Mainly the Indian scenario  Present systems are tuned to foreign socio- cultural context  All of them are imported – no local support  Costly for an average Indian user ( E Z Keys - $1400, Gyro-HeadMouse - $1495, CameraMouse TM - $695 + costly video camera)  Lack of Adaptation in existing systems

Contributions Implementation of a virtual adaptive mouse – SweepSticks  This work was a joint effort of one of my fellow researchers Mr. Koushik Chakraborty and myself Design and Implementation of the prediction support for a virtual keyboard (both Hindi and Bengali) – SulekhA Field Testing and analysis of both SweepSticks and SulekhA

The Prelims: Special Access Mechanisms Hardware Component – Depending upon the degree of their motor control the disabled people can use either one or at most two switches (specially designed for them) in order to access the computer. Courtesy IICP, Kolkata

Special Access Mechanisms (contd…) Software Component  Scanning Mechanisms – Guided / periodic focusing and defocusing of screen elements.  Shift of focus – Shift operation (needs one switch)  Selection of a particular screen element – Register operation (needs another switch)

Methods of Scanning Co-ordinate ScanMatrix Scan (3D, 2D, 1D) CartesianPolar

SulekhA: A Demo

Learning User Preferences ROOT of BST (-1000,-1000) [0] User clicks the point (25,40) on the screen (25,40) [1] (15,25) [1] User clicks the point (15,25) on the screen (30,45) [1] User clicks the point (30,45) on the screen (50,80) [1] User clicks the point (50,80) on the screen (15,20) [1] User clicks the point (15,20) on the screen

Forming Paths Now if the user clicks the point (30,45) once again ??? (-1000,-1000) [0] (25,40) [1] (15,25) [2] (15,20) [2] (30,45) [2] (50,80) [2] fq = 1 User clicks the point (15,25) once again fq = 1 User clicks the point (50,80) once again fq = 1 User clicks the point (15,20) once again fq = 1

Statistics SulekhA uses  Bigram Prediction Strategy for Word Level  The training corpus at present contains approximately 1 million words and 0.12 million distinct bi-grams.  The format of the corpus is shown below,  Unigram Prediction Strategy for Character Level  The training corpus at present contains approximately 1.3 million words and 0.05 million distinct unigrams.  The format of the corpus is shown below,

The Strategies Word Level Character Level

Shradha Writes with SulekhA

Assessments: SweepSticks  Presently tested for four subjects at IICP Kolkata  Testing is done by measuring success and failure  Success - Each user is asked to open a particular application using SweepSticks. If the user is able to do the same in one go, with the minimum number of shift and register operations actually required to open the application, then it is a success, else failure.  Success and failures are measured for all the users for a number of sessions both in absence and presence of the adaptive help %Success =

Assessments: SweepSticks Steady growth in success rate for Barsha Initial fumbling followed by growth in success rate for Shradha

 Typing rate (number of characters typed per minute) was measured  Measurements were taken when the prediction was not in use and also when in use Assessments: SulekhA Typing Rate of BarshaTyping Rate of Shradha

Usability 5 – Excellent, 4 – Good, 3 – Average, 2 – Difficult, 1 – Very Difficult

References [1] Hufschmidt-Schneider M., Kuhme Thomas and Malinowski U., Adaptive User Interfaces, Principles and Practice. [2] Ahmed Seffah and Homa Javahery, Multiple User Interfaces, Cross-Platform Applications and Context-Aware Interfaces. [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] Kaul Sudha and Warrick A., Their Manner of Speaking, Indian Institute of Cerebral Palsy, Kolkata, India, 1997.

Questions