A Virtual Keyboard with Multi Modal Access for people with disabilities Vijit Prabhu 1, Girijesh Prasad 2 1 Computer Science & Engineering, Indian School of Mines, Dhanbad, Jharkhand , India 2 Intelligent Systems Research Centre (ISRC), University of Ulster, Magee Campus, Derry, N. Ireland, UK References [1] Blankertz et al., The Berlin Brain-Computer Interface Presents The Novel Mental Typewriter Hex-O-Spell, In: Proc. of the 3rd International Brain-Computer Interface Workshop and Training Course 2006, September , Verlag der Technischen Universität Graz.. [2] Dasher-Information-Efficient Text Entry-Hanna Wallach, University of Cambridge/University of Pennsylvania Introduction Persons with speech and motor disorders face problems in expressing themselves in an easy and intelligible way. Computer based augmentative and alternate communication (AAC) systems are developed to assist them. A Virtual Keyboard (VK), also called as On Screen Assistive Keyboard is a commonly used AAC system. A VK is characterized by the input modalities and the layout. Background “Hex-o-Spell” [1] virtual keyboard is an EEG based Brain Computer Interface that uses machine learning techniques to identify brain signals. Dasher[2] is an information-efficient text- entry interface, driven by natural continuous pointing gestures via a access switch, touch screen, or eye-tracker. We revisited the problem to incorporate a number of input modalities and optimize the design layout to achieve optimum performance from the Virtual Keyboard. Figure 1: Hex-o-Spell Virtual Keyboard 1. Working Model Figures2(a) and 2(b) show the working of the Virtual Keyboard. The pointer can be rotated either by the user or it can be rotated at a fixed speed by the computer itself. The pointer rotated till it reaches the desired sub circle. A trigger is used to select the sub circle which expands [figure 2(b)] The pointer is again rotated to point to the desired character The trigger is used to select the character and it is typed in to the corresponding application. Figure 2 (a) Virtual Keyboard: pointer pointing at the sub circle Figure 2(b) Expanded Sub Circle: pointer points at the character L The Virtual Keyboard 2. Modes of Access Any one or the combination of three incorporated technologies can be used for access. The Brain Computer Interfacing (BCI) based on EEG uses ‘vivid imagination’ of a motor activity as a trigger. The Eye Tracker technology uses prolonged gaze and/or eye blink as a trigger. Access switches (also called as soft switches) use any active body part such as hand, foot, mouth or head as a trigger 3. Layout Design Different characters have different frequency of occurrence in English Text. [3] [4] Different positions of characters in the layout, require different amount of access activity. Design Principle- higher the frequency of occurrence of character, lower should be the amount of activity required to access it. Clashes in positioning of character were resolved based on the probability of the character being the first letter of the word. [5] Analysis and Results x i : access activity required (including rotation and selection) to access the i th character p(x i ): probability distribution function based on relative frequency of occurrence in text in [3][4] The expected value [6] or mean of access activity required per character is given by: E(x)=∑ (x i. p(x i )) A similar VK with ‘alphabetical ordering’ has expected activity per character = 6.55 Our layout has the expected activity per character = 4.55 Position Access Activity Layout Alphab- etical Position Access Activity Layout Alphab- etical 1,12spacea3,14io 1,23eb3,25rp 1,34ac3,36uq 1,45nd3,47yr 1,56de3,58bs 1,67mf3,69xt 1,78gg3,710ju 2,13th4,15hv 2,24oi4,26cw 2,35sj4,37wx 2,46lk4,48vy 2,57fl4,59qz 2,68pm4,610zspace 2,79kn4,711del Figure 3 : Comparison of our layout against the alphabetical ordering Position (m,n) refers to m th sub-circle; n th location in the sub-circle Conclusion The analysis confirms that our layout has better performance than alphabetical ordering as in [1] The multiple modes of access allows same the VK to be used by a wider spectrum of people with different levels of speech and motor disorders. [[3] Lewand, Robert (2000). Cryptological Mathematics. The Mathematical Association of America. p. 36. ISBN [4] Lee, E. Stewart; Essays about Computer Security; University of Cambridge Computer Laboratory, p. 181 [5] Letter Frequency, Wikipedia, [6] Expected Value, Wikipedia, Figure 2 (c) : Subject using Eye tracker as mode of access.