1. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 1 L. Itti: CS564 - Brain Theory and Artificial Intelligence University of Southern California Lecture 15. Saccades 1 Reading Assignments: TMB 6.2 The Modular Design of the Oculomotor System in Monkeys Peter Dominey, Michael Arbib, and Amanda Alexander Sections: Description, Architecture Supplementary Reading: Crowley-Arbib Saccade Model M. Crowley, E. Oztop, and S. Marmol

2. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 2 From Schemas to Neural Networks - A Case Study: Modeling the Control of Saccades Classical Model of Reflex Saccades Expanding the Model to Bring in Cortical Functions

3. Filling in the Schemas: Neural Network Models Based on Monkey Neurophysiology Peter Dominey & Michael Arbib: Cerebral Cortex, 2:153-175 Develop hypotheses on Neural Networks that yield an equivalent functionality: mapping schemas (functions) to the cooperative cooperation of sets of brain regions (structures)

4. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 4 Experimental Protocols and Simulation Interfaces An experimental protocol defines a class of experiments based on:  Preparation used  External stimuli  Experimental manipulations : electrical stimulation, drug application, etc.  Measurements and observations The protocol can be translated into a simulation interface for “stimulating” a simulation and displaying the results.

5. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 5 Experiment - Double Saccade

6. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 6 Double Saccade -- Neural Array Activity Snapshot

7. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 7 Brain Stem Saccade Burst Generator LLBN-Long Lead Burst Neurons, MLBN-Medium Lead Burst Neurons, EBN- Excitatory Burst Neurons, PN-Omni-Pause Neurons, RI-Resettable Integrator, TN- Tonic Neurons, MN-Oculomotor Neurons. In the present Chapter/Lecture, this is treated as an unanalyzed module (available from the NSL Library) so that attention may focus on the other brain regions, which are described next.

8. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 8 Bringing in SC and FEF SC - Superior Colliculus: The midbrain’s path from vision (and other senses) to eye movement. FEF - Frontal Eye Field: The cortex’s path from vision to eye movement. Data point: A cat or monkey can still make saccades if one of SC or FEF is lesioned.

9. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 9 Simple Saccade Task time +

10. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 10 Experimental Setup

11. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 11 Timing Diagram for Simple Saccade Experiment "visinP3M3" is the stimulus for the first target, "fixation" is the fixation timing, "verticalTheta" is the vertical eye movement response, and "horizontalTheta" is the horizontal eye movement response. Notice that the eyes do not move until the "posteriorParietalCenter", a specific neuron in the model of LIP, goes low.

12. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 12 Memory Saccade Experiment time + + +

13. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 13 Timing Diagram for Memory Saccade Experiment "visinM2P2" is the stimulus for the target, "fixation" is the fixation timing, "verticalTheta" is the vertical eye movement response, and "horizontalTheta" is the horizontal eye movement response

14. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 14 Double Saccade Experiment time +

15. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 15 Timing Diagram for the Double Saccade Experiment "visinM3P0" is the stimulus for the first target and "visinM3P3" is the stimulus for the second target. "fixation" is the fixation timing, "verticalTheta" is the vertical eye movement response, and "horizontalTheta" is the horizontal eye movement response.

16. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 16 Timing Diagram for Lesioning of SC Experiment Output of SC is disabled. "visinP3M3" is the stimulus for the first target, "fixation" is the fixation timing, "verticalTheta" is the vertical eye movement response, and "horizontalTheta" is the horizontal eye movement response.

17. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 17 FEF Stimulation with SC Lesioned time + FEF Stimulation

18. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 18 Compensatory Saccade: Stimulation of FEF with Lesioning of SC A visual target is briefly presented and removed before a saccade can begin. Before the visual saccade can begin, an electrical stimulus is applied to the FEF. The monkey will first saccade to the stimulated location and then to the real target even though timewise the real target appeared first. This is due to the fact that the the visual signal takes time to get from the retina to the FEF.

19. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 19 The Full Dominey Model DCEP-Damped Change in Eye Position 7a/LIP-Oculomotor Region of Posterior Parietal Cortex

20. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 20

21. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 21

22. Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 22