Brain Computer Interface Presenter : Jaideo Chaudhari CS575:Human Issues in Computing Instructor: Dr. B. Rao

BCI Outline BCI Concept: rebuilding instead of repairing BCI overview: actions from thoughts BCI Story: from fiction to reality BCI apps: Silent Talk

BCI Concept: direct communication pathway between a brain and an external device. Often aimed at assisting, augmenting or repairing human cognitive or sensory-motor functions. A brain–computer interface (BCI), or a brain–machine interface, is a direct communication pathway between a brain and an external device. Research on BCIs began in the 1970s at the University of California Los Angeles (UCLA) under a grant from the National Science Foundation, followed by a contract from DARPA. The papers published after this research also mark the first appearance of the expression brain–computer interface in scientific literature. The field of BCI took momentum after that, mostly toward neuroprosthetics applications that aim at restoring damaged hearing, sight and movement. Signals from implanted prostheses can, after adaptation, be handled by the brain like a natural sensory organ. Following years of animal experimentation, the first neuroprosthetic devices implanted in humans appeared in the mid-nineties.

BCI motivation: In USA, more than 200,000 patients live with the motor sequelae (consequences) of serious injury. There are two ways to help them restore some motor function: Repair the damaged nerve axons Build neuroprosthetic device

BCI Principle: (a) In healthy subjects, primary motor area sends movement commands to muscles via spinal cord. (b) But in paralyzed people this pathway is interrupted. (c) A Computer based decoder is used, which translates this activity into commands for muscle control.

BCI versus neuroprosthetics: uses artificial devices to replace the function of impaired nervous systems or sensory organs. connect the nervous system to a device E.g. cochlear implants, retinal implants. BCI: connect the brain (or central nervous system) with a computer system. E.g. EEG, Neuroprosthetics is an area of neuroscience concerned with neural prostheses—using artificial devices to replace the function of impaired sensory organs. The most widely used neuroprosthetic device is the cochlear implant, which, as of 2006, has been implanted in approximately 100,000 people worldwide. There are also several neuroprosthetic devices that aim to restore vision, including retinal implants. The differences between BCIs and neuroprosthetics are mostly in the ways the terms are used: neuroprosthetics typically connect the nervous system to a device, whereas BCIs usually connect the brain (or nervous system) with a computer system. Practical neuroprosthetics can be linked to any part of the nervous system—for example, peripheral nerves—while the term "BCI" usually designates a narrower class of systems which interface with the central nervous system. The terms are sometimes used interchangeably, and for good reason. Neuroprosthetics and BCIs seek to achieve the same aims, such as restoring sight, hearing, movement, ability to communicate, and even cognitive function. Both use similar experimental methods and surgical techniques.

Working of BCI: Every time we think, move, feel or remember something, our neurons are at work. That work is carried out by small electric signals that zip from neuron to neuron as fast as 250 mph some of the electric signal escapes, which can be detected, interpret and use them to direct a device of some kind. The reason a BCI works at all is because of the way our brains function. Our brains are filled with neurons, individual nerve cells connected to one another by dendrites and axons. Every time we think, move, feel or remember something, our neurons are at work. That work is carried out by small electric signals that zip from neuron to neuron as fast as 250 mph [source: Walker]. The signals are generated by differences in electric potential carried by ions on the membrane of each neuron. Although the paths the signals take are insulated by something called myelin, some of the electric signal escapes. Scientists can detect those signals, interpret what they mean and use them to direct a device of some kind. It can also work the other way around. For example, researchers could figure out what signals are sent to the brain by the optic nerve when someone sees the color red. They could rig a camera that would send those exact signals into someone's brain whenever the camera saw red, allowing a blind person to "see" without eyes.

BCI Input and Output: f Consider the potential to manipulate computers or machinery with nothing more than a thought. It isn't about convenience -- for severely disabled people, development of a brain-computer interface (BCI) could be the most important technological breakthrough in decades.

Monkey thinks, Robot does! Experiments with monkey operating a robotic arm with its mind at the Pittsburgh University Medical Center http://www.makeahistory.com/index.php/your-details/222-the-worlds-first-commercial-brain- computer-interface- There has been rapid development in BCIs since the mid-1990s. Several groups have been able to capture complex brain signals using recordings from groups of neurons and use these to control external devices. They have managed to record signals from monkey and rat cerebral cortices in order to operate BCIs to carry out movement. Monkeys have navigated computer cursors on screen and commanded robotic arms to perform simple tasks simply by thinking about the task and without any motor output. The monkeys were trained to reach and grasp objects on a computer screen by manipulating a joystick while corresponding movements by a robot arm were hidden. The monkeys were later shown the robot directly and learned to control it by viewing its movements.

Ratbot! Reverse BCI What if we want to do the exact reverse of what we saw in the earlier experiment?

BCI System:

BCI System: Non-Invasive BCI Non-Invasive BCIs: easy to wear produce poor signal resolution because the skull dampens signals Signals recorded in this way have been used to power muscle implants and restore partial movement E.g. EEG. http://www.youtube.com/watch?v=G0cz- q1g6go&feature=player_embedded Signals recorded in this way have been used to power muscle implants and restore partial movement in an experimental volunteer. Although they are easy to wear, non-invasive implants produce poor signal resolution because the skull dampens signals, dispersing and blurring the electromagnetic waves created by the neurons. Although the waves can still be detected it is more difficult to determine the area of the brain that created them or the actions of individual neurons. video clip: A brain-computer interface based on EEG signals, developed by MMSPG. It allows to play a video game without any device like joystick.

BCI System: Partially-Invasive BCI Electrocorticography (ECoG) is a very promising intermediate BCI modality. higher spatial resolution, better signal-to-noise ratio, wider frequency range first trialed on humans in 2004 on a teenage boy suffering from epilepsy to play Space Invaders. http://www.braingate2.org/systemOverview.html ECoG is a very promising intermediate BCI modality because it has higher spatial resolution, better signal-to-noise ratio, wider frequency range, and lesser training requirements than scalp-recorded EEG, and at the same time has lower technical difficulty, lower clinical risk, and probably superior long-term stability than intracortical single-neuron recording. This feature profile and recent evidence of the high level of control with minimal training requirements shows potential for real world application for people with motor disabilities.

BCI System: Invasive BCI Dobelle implant "had permitted him to see rough images of large objects BCI containing 68 electrodes was implanted onto Jen’s brain and succeeded in producing phosphenes Initially, allowed Jen to see shades of grey in a limited field of vision at a low frame-rate. able to use his imperfectly restored vision to drive slowly Cost indication: $125,000 It can also work the other way around. For example, researchers could figure out what signals are sent to the brain by the optic nerve when someone sees the color red. They could rig a camera that would send those exact signals into someone's brain whenever the camera saw red, allowing a blind person to "see" without eyes.

Human BCI Research: sighted person: image resolution of some 32 by 32 pixels Image resolution of the brain implant: In 2002 resolution of 144 pixels In 2003, 250 pixels i.e. 15 by 16 pixel matrix Now its around 625 pixels.

Cell-culture BCIs: Aim of the experiment: to study how brain cells function as a network and to learn more about one of the most complex devices in the known universe: the human brain. to find out exactly how the neurons do what they do and extract those rules and apply them in software or hardware for novel types of computing By watching the brain cells interact, scientists hope to understand what causes neural disorders, such as epilepsy. The research may also help the researchers in their quest to build "living" computers that combine neural and silicon systems. Computers lack the flexibility and adaptability of the human brain and perform poorly at pattern recognition tasks. A Florida scientist has developed a "brain" in a glass dish that is capable of flying a virtual fighter plane and could enhance medical understanding of neural disorders such as epilepsy. "We're interested in studying how brains compute," "If you think about your brain, and learning and the memory process, I can ask you questions about when you were five-years-old and you can retrieve information. That's a tremendous capacity for memory. In fact, you perform fairly simple tasks that you would think a computer would easily be able to accomplish, but in fact it can't." ............ Pattern recognition. Although computers can perform certain tasks extremely quickly, they lack the flexibility and adaptability of the human brain and perform particularly poorly at pattern recognition tasks. If we extract the rules of how these neural networks are doing computations like pattern recognition we can apply that to create computing systems

Cell-culture BCIs: Experiment at Univ. of Florida 25,000 neurons taken from the brain of a rat that are connected to a computer via 60 electrodes. rapidly began to reconnect themselves to form a living neural network. To put the experimental brain to the test, it is connected to a jet flight simulator via the electrode grid and a desktop computer. If you take these cells out of the cortex and you put them into one of these dishes, you remove all of the inputs—sensory systems like vision or hearing—that they would normally have. The only thing that's going on is the spontaneous activity of reconnecting. Experiment: The "living computer" was grown from 25,000 neurons extracted from a rat's brain and arranged over a grid of 60 electrodes in a Petri dish. There's a lot of data out there that will tell you that the computation that's going on here isn't based on just one neuron. The computational property is actually an emergent property of hundreds of thousands of neurons cooperating to produce the amazing processing power of the brain. It's essentially a dish with 60 electrodes arranged in a dish at the bottom. The brain cells then started to reconnect themselves, forming microscopic interconnections. If you take these cells out of the cortex and you put them into one of these dishes, you remove all of the inputs—sensory systems like vision or hearing—that they would normally have," DeMarse said. "The only thing that's going on is the spontaneous activity of reconnecting.. But as the neurons begin to receive information from the computer about flight conditions—similar to how neurons receive and interpret signals from each other to control our bodies—the brain gradually learns to fly the aircraft. The neurons will analyze data from the computer, like whether the plane is flying level or is tilted to one side," DeMarse said. "The neurons respond by sending signals to the plane's controls to alter the flight path. New information is sent back to the neurons, creating a feedback system.

Cell-culture BCIs: Experiment at Univ. of Florida neurons begin to receive information from the computer about flight conditions The neurons analyze data from the computer, like whether the plane is flying level or is tilted to one side Information is sent back and forth creating a feedback system Are we creating Living Computers or Hybrid computers here? Silicon-based computers are very accurate and fast at processing some kinds of information, but they have none of the flexibility of the human brain. Brains can easily make certain kinds of computations that computers are unable to do, such as answering open-ended questions about what happened sometime in the past. To do a search like that in silicon is pretty difficult, unless you program [a computer] to specifically answer that question," DeMarse said. "Yet these neurons are able to do this in rats and in humans all the time." Understanding how neurons distribute information and encode it would allow scientists to take those rules and develop a silicon system that operates similar to the neurons, yet has the retention capacity of a silicon computer.

Military applications: Pentagon’s mad-science division Darpa to start program named Silent Talk – synthetic telepathy. Budget is $4 million. nerve signals are used as the mechanism for information transfer two major groups - passive and active As of 2010, research is being driven by military for "covert speech" technology, which would allow troops to communicate over radio without having to physically speak.

Simulated reality: In the futuristic vision of movies like The Matrix and Avatar, humans dive into a virtual world by connecting their brains directly to a computer…… Simulation hypothesis – are we living in a simulated reality? A simplified version of his argument proceeds as such: i. It is possible that an advanced civilization could create a computer simulation which contains individuals with artificial intelligence (AI). ii. Such a civilization would likely run many, billions for example, of these simulations (just for fun, for research or any other permutation of possible reasons.) iii. A simulated individual inside the simulation wouldn’t necessarily know that it is inside a simulation — it is just going about its daily business in what it considers to be the "real world."

Simulated reality: In the futuristic vision of movies like The Matrix and Avatar, humans dive into a virtual world by connecting their brains directly to a computer…… Simulation hypothesis – are we living in a simulated reality? It is possible that an advanced civilization could create a computer simulation which contains individuals with AI. Such a civilization would likely run many, billions for example, of these simulations (just for fun, for research or any other permutation of possible reasons.) A simulated individual inside the simulation wouldn’t necessarily know that it is inside a simulation — it is just going about its daily business in what it considers to be the "real world."

Simulated reality: Then the ultimate question is — if one accepts that the above premises are at least possible— which of the following is more likely? We are the one civilization which develops AI simulations and happens not to be in one itself? We are one of the many (billions) of simulations that has run? (Remember point 3!) Then the ultimate question is — if one accepts that the above premises are at least possible— which of the following is more likely? a. We are the one civilization which develops AI simulations and happens not to be in one itself? b. We are one of the many (billions) of simulations that has run? (Remember point iii.) In greater detail, his argument attempts to prove the trichotomy, either that: 1. intelligent races will never reach a level of technology where they can run simulations of reality so detailed they can be mistaken for reality (assuming that this is possible in principle); or 2. races who do reach such a sophisticated level do not tend to run such simulations; or 3. we are almost certainly living in such a simulation.

Commercialization and companies: Ciberkinetics Honda Neural Signals Starlab NASA A few companies are pioneers in the field of BCI. Most of them are still in the research stages, though a few products are offered commercially. Neural Signals is developing technology to restore speech to disabled people. An implant in an area of the brain associated with speech (Broca's area) would transmit signals to a computer and then to a speaker. With training, the subject could learn to think each of the 39 phonemes in the English language and reconstruct speech through the computer and speaker [source: Neural Signals]. NASA has researched a similar system, although it reads electric signals from the nerves in the mouth and throat area, rather than directly from the brain. They succeeded in performing a Web search by mentally "typing" the term "NASA" into Google [source: New Scientist]. Cyberkinetics Neurotechnology Systems is marketing the BrainGate, a neural interface system that allows disabled people to control a wheelchair, robotic prosthesis or computer cursor [source: Cyberkinetics]. Japanese researchers have developed a preliminary BCI that allows the user to control their avatar in the online world Second Life [source: Ars Technica].

BCI Challenges: The brain is incredibly complex. There are chemical processes involved as well. The signal is weak and prone to interference i.e. reading brain signals is like listening to a bad phone connection. There's lots of static. The equipment is less than portable. 1. The brain is incredibly complex. To say that all thoughts or actions are the result of simple electric signals in the brain is a gross understatement. There are about 100 billion neurons in a human brain [source: Greenfield]. Each neuron is constantly sending and receiving signals through a complex web of connections. There are chemical processes involved as well, which EEGs can't pick up on. 2. The signal is weak and prone to interference. EEGs measure tiny voltage potentials. Something as simple as the blinking eyelids of the subject can generate much stronger signals. Refinements in EEGs and implants will probably overcome this problem to some extent in the future, but for now, reading brain signals is like listening to a bad phone connection. There's lots of static. 3. The equipment is less than portable. It's far better than it used to be -- early systems were hardwired to massive mainframe computers. But some BCIs still require a wired connection to the equipment, and those that are wireless require the subject to carry a computer that can weigh around 10 pounds. Like all technology, this will surely become lighter and more wireless in the future.

BCI video http://www.youtube.com/watch?v=I7lmJe_EXEU&feature =related In this short talk, Dr. Shenoy describes how his team of Stanford researchers has built a system that achieves typing at 15 words-per-minute, just by "thinking about it". http://www.youtube.com/watch?v=I7lmJe_EXEU&feature =related

References: http://mlpr.wikidot.com/brain-computer-interface http://www.makeahistory.com/index.php/your-details/222- the-worlds-first-commercial-brain-computer-interface- http://www.wireheading.com/ http://news.nationalgeographic.com/news/2004/11/1119_0 41119_brain_petri_dish_2.html http://www.wired.com/dangerroom/2009/05/pentagon- preps-soldier-telepathy-push http://en.wikipedia.org/wiki/Brain-computer_interface http://www.wired.com/wired/archive/9.08/assist_pr.html http://www.youtube.com/watch?v=I7lmJe_EXEU&feature =related