1. BCI Keyboards: towards mind writing
P. Perego, A. Alamia, L. Maggi, G. Andreoni, Politecnico di Milano (Milan, Italy)
This work has been partially supported by IIT
Abstract: Our applications consists of a smart dynamic virtual keyboard developed with the BCI++ framework proposed by the SensibiLab Laboratory (Politecnico di Milano) and interfaced with a four-commands SSVEPs BCI system. We compared this new keyboard with respect to a standard static virtual keyboard (both QWERTY and not) to assess the performance differences between the two solutions, both driven through BCI system. Four subjects participated to the study executing an ad hoc protocol. The first part of the protocol consisted in the calibration of BCI system; the second one was for the comparison of the two input methods. The results in terms of bit-rate, characters per minute and learning curve showed that the new dynamic keyboard allowed to use a Brain Computer Interface for text input with an average rate of about 11 characters/minutes, a result that is comparable with the performance of an eye-tracking used on ALS (Amyotrophic Lateral Sclerosis) patient (16 characters/minute).
The BCI System
It is based on the BCI++ Framework [1]. That is composed by two modules running onto two different personal computers communicating through a TCP/IP socket based connection:
The Hardware Interface Module (HIM) is dedicated to the acquisition, recording and visualization of the signal, the communication with the user interface and the real-time processing through the use of algorithms developed using both C/C++ and Matlab® (MathWorksTM, USA).
The User Interface module (Aenima) is a flexible tool for the implementation of new operating protocols for BCI based user applications.
We used already tested algorithms [2] for SSVEP based BCI and we developed new user interface (for AEnima).
The visual stimulation system consisted of external stimulators into four cubic boxes with sides of 3 cm; each cube was made by a matt film and a thin semitransparent material paper to avoid direct exposition to the light and diffuse the beam. Each box included a green LED and was placed on a specific side of a standard 19’’ LCD monitor. Each stimulator was associated to a 2D direction (up, down, left and right) as shown in Fig. 1.
Two different AEnima protocols were developed.
Each protocol consisted of a virtual keyboard. A virtual keyboard is a keyset displayed on monitor through which the user can interact with personal computer. They were composed by:
6x6 matrix with 36 characters
alphabetical letters and some special symbols (accented, delete button, enter button…).
cursor to highlight the current selection
The four different commands of the BCI
system were used respectively for:
UP  select the character;
DOWN  move the cursor downward;
RIGHT  move the cursor rightward;
LEFT  delete the last character inserted.
The user couldn’t move the cursor
upward or leftward.
The features of the two different virtual Keyboards were the following:
In the static keyboard, the character was fixed in the 6x6 matrix.
In the dynamic keyboard, the characters’ position configuration changed continuously depending on the previously selected char. The character repositioning followed the probability of finding the next character of the word near the default position of the cursor: the most likely character was near the cursor default position, while the most unlikely was faraway. The probability of each character to follow another one changes from language to language, for this reason in this study it was calculated for Italian, analyzing a text of approximately 200 words.
Fig. 2
The user interface
19” Screen with four stimulators associated with four directions
Subjects and Protocol
Four healthy university students (age 22 – 26) tested the entire system. Each subject:
None had eye impairment
None ever used a BCI system before.
All the test were carried out at the Sensibilab laboratory of Politecnico di Milano into a dedicated room with controlled light and noise. The test duration was about one hour with two short pauses in order not to overexert the subject.
The complete experimental test can be divided into five sessions:
Screening: to identify the best stimulation frequencies for SSVEP
Training: to train the classifier
Testing: to test the trained classifier
Static keyboard: to test the subject writing three given sentences with BCI with static keyboard.
Dynamic keyboard: to test the subject writing three given sentences with BCI with dynamic keyboard.
Fig. 3
Three different keyboard configurations writing the word “can”
Results and Conlcusion
Tab. 1 shows the increase of performance using virtual dynamic keyboard, results comparable with the char-rate of an eye/gaze tracker used with SLA patient [3].
Future works will include in the virtual dynamic keyboard these additional features and will use it with SSVEP and P300 based BCI on advanced ALS patient, in order to compare the performance of our dynamic keyboard with the two kind of BCI in severe condition usage with very low bit-rate.
Subject
BR [bit/min]
[1]
Characters/min static Keyboard
Characters/min dynamic Keyboard AA
40,81
3,50
9,54 FF
34,68
4,74
10,61 MA
55,13
4,40
1,19 PP
50,77
4,48
10,95
Tab. 1
References
[1] P. Perego, L. Maggi, S. Parini, G. Andreoni “BCI++: a new frame work for Brain computer Interface application”, Proc. of SEDE 2009, June 21-24, Las Vegas (NV, USA).
[2] S. Parini , L. Maggi, A. C.Turconi , G. Andreoni “A robust and self-paced BCI system based on a four class SSVEP paradigm: algorithms and protocols for a high transfer-rate direct brain communication”, Computational Intelligence and Neuroscience – 2009.
[3] A. Kiyohiko, O. Shoichi, O. Minoru “An Eye-Gaze Input System Using Information on Eye Movement History”, Universal Access in Human-Computer Interaction. Ambient Interaction – 2007.
[4] J. Kronegg, S. Volshyovskiy, T. Pun “Analysis of bit-rate definitions for brain-computer interfaces”, Proc of 11th int. Conference on HCI Las Vegas (NV,USA).