1. Alan L. Yuille. UCLA. Dept. Statistics and Psychology. www.stat.ucla/~yuille Neural Prosthetic: Mind Reading. STATS 19 SEM 2. 263057202. Talk 6. Neural Prosthetic: Mind Reading. STATS 19 SEM 2. 263057202. Talk 6.

2. Neural Prosthetic. Basic Idea: Basic Idea: Injury to spinal cord prevents actions. Injury to spinal cord prevents actions. But patient can still think, or plan, the actions. But patient can still think, or plan, the actions. If these plans can be decoded by a microchip, then the patient can control a robot limb, or other device. If these plans can be decoded by a microchip, then the patient can control a robot limb, or other device.

3. Mind Reading. Basic Assumption: Mind = Brain. Basic Assumption: Mind = Brain. Brain consists of neurons transmitting information by electrical activity. Brain consists of neurons transmitting information by electrical activity. All thoughts have neuronal correlates. All thoughts have neuronal correlates. If you can decode the activity of neurons, then you can read people’s thoughts. If you can decode the activity of neurons, then you can read people’s thoughts.

4. Neural Activity Decoding neural activity corresponds to mind reading. Decoding neural activity corresponds to mind reading. But how do neurons encode information? But how do neurons encode information? Two classic viewpoints: Two classic viewpoints: 1. The grandmother cell that responds whenever you see your grandmother. 2. Activity encoded by populations of neurons. Neural networks (holistic).

5. Decoding Neurons. It seems clear that some brain activity is localized in specific areas – e.g. face recognition. It seems clear that some brain activity is localized in specific areas – e.g. face recognition. Also some neurons respond to specific stimuli – e.g. to faces but not to dogs. Also some neurons respond to specific stimuli – e.g. to faces but not to dogs. There might even be a Clinton cell… There might even be a Clinton cell… There is a lot of plasticity. There is a lot of plasticity.

6. Motion Planning Motion Planning : experiments (1990) showed that populations of neurons in the motor cortex predicted the direction of movement. Motion Planning : experiments (1990) showed that populations of neurons in the motor cortex predicted the direction of movement. Better ways to interpret neuronal activity – get monkeys to perform simple tasks, measure neuronal activity, perform Bayesian inference. (Brown University). Better ways to interpret neuronal activity – get monkeys to perform simple tasks, measure neuronal activity, perform Bayesian inference. (Brown University).

7. Plasticity and Feedback Helps The decoding is used to drive a device. The decoding is used to drive a device. The monkey gets feedback by observing that the device is not doing what it wanted. The monkey gets feedback by observing that the device is not doing what it wanted. The monkey can somehow correct its thought patterns, so that the decoding works. The monkey can somehow correct its thought patterns, so that the decoding works. Like learning to ride a bicycle. Like learning to ride a bicycle.

8. Movement Planning These movement plans, or thoughts, need not be conscious. These movement plans, or thoughts, need not be conscious. Novel actions usually require conscious thought. Novel actions usually require conscious thought. But most common actions do not. But most common actions do not. Neuronal correlates of consciousness. Neuronal correlates of consciousness.

9. Neural Prosthetics. Only a few Universities have serious neural prosthetics programs. Only a few Universities have serious neural prosthetics programs. This talk describes work by Richard Andersen’s Laboratory at Caltech. Figures copied from the lab’s webpage. This talk describes work by Richard Andersen’s Laboratory at Caltech. Figures copied from the lab’s webpage. Michael Black at Brown University. (Decoding). Michael Black at Brown University. (Decoding).

10. Goal of Neural Prosthetic System. Read the plans from the cortex. Read the plans from the cortex. Activate an external device. Activate an external device.

11. Implanting Electrodes. Measurement is done by inserting electrodes into the cortex. These transmit to an external device. Measurement is done by inserting electrodes into the cortex. These transmit to an external device. Will the implanted electrodes stay in the right place? Will they cease functioning over time? Will the implanted electrodes stay in the right place? Will they cease functioning over time? But the electrodes only need to measure activity (passive). They do not need to stimulate neurons (epilepsy). But the electrodes only need to measure activity (passive). They do not need to stimulate neurons (epilepsy).

12. How to Implant Electrodes? Neurosurgery: Neurosurgery: Deep implants of electrodes to simulate (treatment for Parkinson’s disease). Deep implants of electrodes to simulate (treatment for Parkinson’s disease). No pain receptors in the brain. No pain receptors in the brain.

13. Miniaturized Chip. Chip must measure neuronal activity. Chip must measure neuronal activity. Action potentials. Action potentials.

14. Where the Output Goes. Chip implanted inside the brain. Chip implanted inside the brain. Wires go out and activate external devices. Wires go out and activate external devices.

15. Posterior Parietal Cortex. Earliest place where the thought to make a movement is made. Earliest place where the thought to make a movement is made. Size of little fingernail. Size of little fingernail. About 50,000,000 neurons. About 50,000,000 neurons. Others work with motor cortex. But this can atrophy with disuse. Others work with motor cortex. But this can atrophy with disuse. Plans made in PPC. Sent to Motor Cortex. Transmitted to Spinal Cord.

16. Cognitive Plans Planning and Visual Coordinates. Planning and Visual Coordinates. Advantage of PPT. Advantage of PPT.

17. Parietal Reach Region. Reach, Move Eyes, Grasp. Reach, Move Eyes, Grasp.

18. Spatial Representation Known. Knowing spatial representations makes it easier to read the neuronal code. Knowing spatial representations makes it easier to read the neuronal code.

19. Neural activity as Monkey Thinks. Monkey is trained to perform tasks. Monkey is trained to perform tasks. Waiting period when monkey plans. Waiting period when monkey plans.

20. Close the Loop by Giving Feedback. Monkey has feedback. Monkey has feedback. Monkey can adjust neural activity to improve performance. Monkey can adjust neural activity to improve performance.

21. Improve Recordings. Moveable Probe Moveable Probe Need to adjust Need to adjust Electrodes to take tissue movement into account.

22. Computer Probe. Adjust Probes. Adjust Probes. 1,000 electrodes. 1,000 electrodes. Automatically optimize positions. Microfluidic drug delivery. To remove scar tissue. Microfluidic drug delivery. To remove scar tissue.

23. Decoding Neural Activity. Local Field Potentials. Local Field Potentials.

24. Local Field Potentials. Measure Field Potentials. Measure Field Potentials.

25. Tuning of Local Field Potentials. Local Field Potentials: Local Field Potentials:

26. Decoding LFP’s and Spikes Decoding. Decoding.

27. Informatics. Real Time Decoding. Real Time Decoding. Analysis is similar to that for speech recognition. Analysis is similar to that for speech recognition. Information in space and time. Information in space and time. Similar techniques can be used. Similar techniques can be used.

28. Summary. Planned movements are represented in the Parietal Reach Region. Planned movements are represented in the Parietal Reach Region. Neural decoding enables these plans to be read. Neural decoding enables these plans to be read. Plasticity of neurons makes this easier, training by feedback. Plasticity of neurons makes this easier, training by feedback. Implanting device by deep brain surgery. Implanting device by deep brain surgery. Output wires control external device. Output wires control external device.