1. SIE 515 Brain-Computer Interfaces (BCI) and Sensory Substitution
Class 25

2. Today’s Schedule BCI function BCI input devices
Neuroprostheses (retinal, cochlear, cortical implants)
HCI challenges with BCI
Output BCI devices
Sensory substitution (various flavors)
Ethics and BCI

3. Overview
What is BCI?
Most HCI principles = indirect connection to the brain
Direct BCI devices
Replace sensory organ (implant)
Monitor brain activity (EEG device)

4. How Does BCI Work? Two categories based on information flow:
BCI input devices: Feed-forward processing of external signals
BCI output devices: Measuring electrical activity from the brain
Goal of BCI input
Neuroprosthetics: Artificial device aims to restore function of damaged sense organ

5. Figure of Retinal Implants
Goal = Transduction of stimulus energy into neural signal

6. Figure of Cochlear Implant

7. Cortical Implants

8. HCI Considerations Problems with neuroprosthetics
What are two important HCI factors when designing BCI devices?

9. BCI as Output
Using electrical signals from the brain to control an interactive system
Facilitates execution of movements without motor output
Works by determining correspondence between electrical signals and motor response codes
Role of population vectors and neural ensembles
Two types of BCI output
Implanted electrodes
External device

10. Monkey Figure

11. External BCI Devices

12. How Do External BCI Output Devices Work?
Works by measuring electrical activity of brain from outside the head
Problems and challenges
Requires brain wave entrainment
p300 signal most important
Future BCI devices could use blood oxygen changes (fMRI)

13. Neural Motion Detection
Left panel: Voxels in both hemispheres did not reveal significant differences between rightward and leftward stimuli, both for moving and for static sounds.
Right panel: The graph shows the classification accuracy for moving and for static sounds separately, the grey line indicates chance performance. In contrast to the conventional ROI analysis, the classifier was able to reliably differentiate between leftward and rightward moving stimuli.

14. Sensory Substitution What is the goal of sensory substitution?
Difference between sensory substitution and neuroprosthetics
Direct vs. indirect interface
Intra-modal vs. inter-modal communication
Assumptions underlying effective sensory substitution

15. How Does Sensory Substitution Work?
Brain plasticity: Adapt to new stimuli/function
Types of plasticity

16. Visual Tactile Tongue Display
News in Science, Vol. 299, 14 March 2003

17. Seeing With Sound Uses three translation dimensions
Horizontal position in image = Time
Vertical position in image = Frequency
Brightness of image = Loudness

18. vOICe Demo

19. vOICe Demo

20. vOICe Demo

21. vOICe Demo

22. Vibro-audio Interface
Used Samsung tablet with 7-inch touchscreen
Contact with onscreen visual elements synchronously triggered vibration and auditory cues
Focal vibration to the finger allows for vibro-tactile perception of points, lines, and regions
Vibro-Audio cues were optimized based on psychophysical studies (Raja, 2011)

23. Ethical Considerations of BCI
Tapping our inner mental world
Many mental states we would not want to be recorded or broadcasted
Other potential issues:
Animal research, mind control, electro-viruses…

24. For Next Class Assignment 22: HCI Advancement Themes
Task 1: Write down two major themes that you think will be important to the advancement of HCI research and technology development moving forward. In doing so, think about how the changing trajectory of technology will impact future user demographics and interaction styles.
Task 2: For each overarching theme, write about questions or thought experiments to discuss with the class about how future developments will change our current notions of human-computer interaction. What problems do you foresee with these developments and whether they are ultimately helpful or hindering. This will be the primary topic of the class, so everyone should read each other’s blog posts. 
Please post your response to the blog by 11:59PM on 4/27/16.