Shyamanta M. Hazarika, Adity Saikia, Simanta Bordoloi, Ujjal Sharma And Nayantara Kotoky Department Of Computer Sc. & Engineering, Tezpur University Tezpur, India Brain Computer Interface as Sensor for Ambient Intelligent Living: A Position Paper

Biomimetic and Cognitive TU conducts research in the area of Cognitive Robotics and Knowledge Representation & Reasoning. We are particularly interested in Qualitative Spatial and Temporal Reasoning. This translates into interest in Cognitive Vision and Rehabilitation Robotics. Our research within Cognitive Robotics and KR &R is driven by biomimetics i.e., examination of nature particularly human intelligence and skills, its models, systems, processes, and elements to emulate or take inspiration from these designs and processes. For development of prostheses and assistive devices within Rehabilitation Robotics we undertake biomimetic design, which is NOT JUST A COPY of the geometry! For us biomimetic design is biomimetic geometry together with functional biomimesis.

Brain Computer Interfaces Brain computer interfaces –Use computers to sense human thoughts and enable the users to control external devices –Infer a user’s intentions using only brain activity –Provide a non-muscular avenue for communication Applications –BCIs are aimed at assisting, augmenting, or repairing human cognitive or sensory-motor functions. e.g. locked-in syndrome (cognitively unimpaired, but no motor control)

Brain Computer Interfaces Depending on application, BCI can be classified as Cognitive Sensory Motor Motor BMI seeks to translate brain activity from the central or peripheral nervous system into useful commands to external devices. Drive Prosthetics Functional electrical stimulation Motor BMI can be categorized as Invasive Partially Invasive Non-Invasive

What is an EEG? An electroencephalogram is a measure of the brain's voltage fluctuations as detected from scalp electrodes. It is an approximation of the cumulative electrical activity of neurons. Brain –set of interconnected modules –performs information processing operations at various levels sensory input analysis memory storage and retrieval reasoning feelings consciousness Neurons –basic computational elements –heavily interconnected with other neurons

Beta Rhythm Alpha & Mu Rhythm Grounding Electrode Placement Standard 10:20 System

Experiment Protocol

Bispectrum of EEG Signal Bispectrum is the expectation of three frequencies; two direct frequency components and the third the conjugate frequency of the sum of those two frequencies. Knowing the Fourier frequency components X(f) the bispectrum B(f 1, f 2 ) can be estimated using the Fourier-Stieltjes representation. B(f 1, f 2 ) = E(X(f 1 )X(f 2 )X*(f 1 + f 2 )) Where X*(f) is the complex conjugate of X(f) and E( ) is the statistical expectation operator

Bispectrum Analysis

Bispectrum analysis provide a way to evaluate mental representation during observation and imagination of hand movement Prior visual representation of motor acts make difference during motor imagination.

Another experiment Aim is to classify four different motor imagery, namely, – Both Hands Up – Tighten Both Fists – Left Hand Up – Right Hand Up The Protocol Start Audio Cue Action Audio Cue Stop Audio Cue Relax and keep your eyes closed. Imagine the action. End the task and relax.

The Architecture EEG Unit Noise & Amplitude Normalization Feature Extraction Unit K-fold cross validation SVM Filtration & Normalization Unit Motor Imagery Types Classification Unit

Hybrid Features of Bispectrum Here we do not make use of the bispectrum feature directly rather we use following two hybrid features of bispectrum in order to retain the temporal as well as frequency information within the EEG data. Sum of Logarithmic Amplitudes (SLA) to characterizes temporal bispectral information. θ gives the principal domain. First Order Spectral Moment (FOSM) to characterizes frequency information of the bispectrum. N is the number of diagonal elements of Bispectrum

Figure : Bispectrum Estimation of the EEG Signals. Top-left: left hand motor imagery; top-right: right hand motor imagery; bottom-left: both hands motor imagery & bottom-right: both fists motor imagery. Bispectrum Analysis

Classification We have used RBF kernel SVM for classification of the MIs. The original SVM algorithm was proposed by Vladimir Vapnik in The result is cross-validated through 10-Fold Cross Validation.

Confusion Matrix

BCI Based Maze Game With an aim of developing a non-invasive BCI to be used as an intelligent assistive system, we have designed and developed a simple maze game, where a player plays the game in real time by using his brain signals.

Mapping of Motor Imageries with Game Moves Motor ImageryGame Move Both Hands UpMove Forward Right Hand UpMove Right Left Hand UpMove Left Tight Both FistsMove Backward

What we have done so far?

BCI Integrated Collaborative Control The idea is to integrate a BCI with a cognitive architecture for collaborative control of a smart wheelchair. The cognitive architecture mediates based on the extent of automatic vs. manual control to be achieved. AIM… – To help people with mobility disability (with or without cognitive impairment) to achieve a level of independence so that carryout their daily activities.

BCI Integrated Collaborative Control Architecture Automatic control Module Adaptation Module Mediator Manual control Module SensingControl Sensor RoleActor Role Brain Computer Interface Assistive Device Intelligent/Smart Wheelchair

BCI Integrated Collaborative Control Architecture Three layered control architecture – BCI; Superior Control and Local Control. The BCI plays a dual role that of an actor as well as a sensor. It not only does provide control commands to drive the wheelchair but also monitor the cognitive state of the user - his confidence, cognitive workload and wellbeing, depending on which BCI could provide assistance range from partial control of navigation to complete autonomous mode.

Final Comments Over the years AI has drifted away from its main aim. This work is an attempt to focus on integrated systems rather than component algorithms. The cognitive systems paradigm needs to have its source of ideas in human cognition. This position paper describes work done at the Biomimetic and Cognitive Robotics Lab at Tezpur University for development of a BCI Integrated Collaborative Controller for an intelligent wheelchair.