Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall Lecture 20. Schemas 1 1 Michael Arbib: CS564 - Brain Theory and Artificial Intelligence University of Southern California, Fall 2001 Lecture 20. Saccades 1 Reading Assignments: TMB2, Section 6.2. Prepare for Next Time: The NSL Book The Modular Design of the Oculomotor System in Monkeys Peter Dominey, Michael Arbib, and Amanda Alexander Supplementary Reading in the NSL Book: Crowley-Arbib Saccade Model M. Crowley, E. Oztop, and S. Marmol

Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall Lecture 20. Schemas 1 2 Neurophysiological Data on Control of Saccades Saccades — lasting from 15 to 100 msec. — bring the eyes rapidly to foveate on some target. David A. Robinson 1964 showed that a saccade involved  an initial pulse of force to drive the eye to its new position Excitatory Burst Neuron (EBN): About 5 msec before a saccade in its on-direction, the activity EBN(t) of an EBN jumps to a high frequency which is maintained until about 10msec before the end of the saccade, when it drops to the resting level appropriate to its new . During a saccade in the off-direction, EBN(t) is 0.  followed by a maintained force to hold the eye in its new position When the gaze is fixed, a Tonic Neuron (TN): has firing rate TN(t) = K[  o ] + where [x] + = max(x,0). K varies from neuron to neuron. Motor neuron activity sums burst and toinc activity:

Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall Lecture 20. Schemas 1 3

4 Excitatory burst neurons (EBNs) are thought to drive the burst of activity in ipsilateral motoneurons. Inhibitory burst neurons (IBNs) inhibit the contralateral motor neurons, thus seeming to control the pause in motoneuron firing during movements in the off direction. Tonic neurons (TNs) exhibit regular firing at a rate that is related (in a nonlinear way) to eye position during and after saccades. The gain of burst cells is very large, to keep the duration of saccades as small as possible by generating a high velocity movement when any appreciable motor error exists. Omnipause neurons (OPNs) fire continuously save that they pause during saccades. They inhibit burst cells and prevent them from generating a saccade until released. Another group of burst cells - trigger cells (TRIGs) - is thought to play a role in triggering a saccade.

Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall Lecture 20. Schemas 1 5 Hypotheses about saccade control When the omnipause neurons fire (between saccades), they inhibit the burster neurons (both EBNs and IBNs), thus blocking the burst drive to the motoneurons. For a saccade, the trigger cells silence the omnipause neurons; while the excitatory burst neurons receive a signal to fire, and thus  raise the firing rate of ipsilateral motoneurons and  via the IBNs, silence the contralateral motor neurons for the appropriate duration for the length of the saccade. Robinson 1975 suggested that the tonic neurons (TN) integrate the firing of the EBNs: TN(t) = k  t EBN(t)dt, to obtain the desired eye position to maintain the gaze after the saccade has occurred. plausible Cannon et al.offer a plausible neural-network model of the neural integrator but the model has not yet been verified experimentally.

Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall Lecture 20. Schemas 1 6 Control Systems Models of Saccade Generation van Gisbergen, Robinson, and Gielen 1981 model the saccade generator by a servomechanism. Desired eye position (E d ) has subtracted an efference copy/corollary discharge signal (E') replicating eye position (supplied by a neural integrator NI) to yield a motor error signal e m which is input to the burst generator (B).

Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall Lecture 20. Schemas 1 7 Burst neurons B send their burst directly to the motoneurons (MN), and also indirectly through an integrator path (T, the neural integrator NI) which has a recurrent collateral for the "positive feedback" which provides the integration. B is also coupled in a loop of mutual inhibition with the omnipause neurons (P). Between saccades, firing of P inhibits all activity in B. However, trigger input can inhibit P, thus freeing B from the tonic inhibition. And once B is activated, it can "latch" P off for the duration of the saccade. The saccade ends when e m is reduced to 0, and inhibitory bias from the pause neurons is restored. E d (and thus e m ) plays no further role until another saccade is triggered. During a saccade, the MN output exhibits a pulse step in its discharge, R(t), caused by the transient burst B(t) riding atop the eye position signal E'. The eye is held in position by the output from the neural integrator T to the motor neuron MN.

Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall Lecture 20. Schemas 1 8 Push-pull nature of the control of burst neurons See text for details.

Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall Lecture 20. Schemas 1 9 The Many Oculomotor Systems The vestibulo-ocular reflex (VOR) uses the semicircular canals to monitor accelerations of the head; relayed via the neurons of the vestibular nucleus to brainstem circuitry which causes the eyes to compensate for head movements thus yielding (approximate) constancy of gaze. Such a system is of great importance in its own right. It also plays an important role in eye movement. Each can be studied in isolation, but in normal conditions several of these systems will work together to ensure coordinated movements: cooperative computation at the level of motor control. Hierarchical control: information is passed to several maps to apportion the contributions of different movements to the overall adaptive shift of gaze as if there were "lookup tables" for the relative contributions of eye and head.

Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall Lecture 20. Schemas 1 10 The pursuit system has the task of matching the velocity of the target. The saccade system must work with the pursuit system to track an object adjusting position with a saccade whenever the eye moves too far off target. Retinal velocity even affects the saccadic system, allowing the eye to jump into the extrapolated position of the target. Conversely, there may be saccadic suppression to tell the pursuit system to ignore the apparent target velocity during the saccade.

Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall Lecture 20. Schemas 1 11 Dynamic Remapping in superior colliculus

Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall Lecture 20. Schemas 1 12 Deeper layers of monkey superior colliculus contain cells that only fire before saccades. It is probably only the "center of gravity" of a peak of SC activity which is important in coding saccadic size and direction, though burst frequency of the ensemble of active collicular neurons affects saccade velocity. McIlwain 1982 found that microstimulation in the intermediate gray layer of cat superior colliculus yields widespread synaptic activation of the layer. Yet such stimuli evoke saccades whose metrics seem to depend primarily on the location of the stimulating e lectrode. McIlwain postulates that the spatial densities of the cells projecting to vertical and horizontal generators of the saccadic system vary systematically beneath the retinotopic collicular map.

Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall Lecture 20. Schemas 1 13 Sensor Fusion in Superior Colliculus There are superior colliculus neurons that burst prior to saccades in response to visual targets that also burst before saccades in response to auditory targets. Somatosensory input is also mapped into superior colliculus. The somatosensory input depends on the position of body and limbs, while position in auditory input is time-coded via the difference of arrival of signals at the two ears.

Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall Lecture 20. Schemas 1 14 From Schemas to Neural Networks - A Case Study: Modeling the Control of Saccades Classical Model of Reflex Saccades Expanding the Model to Bring in Cortical Functions

Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall Lecture 20. Schemas 1 15 Remapping of Maps in Superior Colliculus Existing visual targets in superficial layers get remapped to deep layers when the eye moves. Mays and Sparks 1980, using trials in which an intervening saccade changed the position of the eyes after a brief visual target had been extinguished, discovered quasi-visual (QV) cells the location of whose activity, even if the eyes had moved after the target disappeared, represented the current retinotopic position of the remembered target. Two processes of remapping are required  One transforms the auditory position code into a retinotopic map of the auditory input in superficial layers of the auditory cortex. This remapping depends on eye (and, in cat and monkey, pinna) position.  The second translates the remembered input map by the extent of the saccade. Dominey and Arbib (1992) adapted a model of Droulez and Berthoz (1991) to use corollary discharge from eye movements to control the remapping

Filling in the Schemas: Neural Network Models Based on Monkey Neurophysiology Peter Dominey & Michael Arbib: Cerebral Cortex, 2: Develop hypotheses on Neural Networks that yield an equivalent functionality: mapping schemas (functions) to the cooperative cooperation of sets of brain regions (structures)

Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall Lecture 20. Schemas 1 17 Work at USC with Nicolas Schweighofer Modeling the role of cerebellum in adapting the size of saccades to compensate for  muscle linearities  effects of prisms Relates to experiments of Mickey Goldberg Builds on earlier studies with Peter Dominey (discussed later in these lectures) on  Dynamic remapping and working memory  Role of basal ganglia in learning conditional and sequential behavior.

Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall Lecture 20. Schemas 1 18 for the saccade generator addresses the issue of spatial coding in SC: SC is intimately related with the pulse generators:  the projection from SC to long lead bursters is monosynaptic,  while the latency from deep SC activity to motoneurons is polysynaptic. The output of SC is integrated by a neural integrator composed of long lead burster neurons (LLBN)  the integration is shown schematically by self- excitation with unit weight The retinotopic code of saccade size in SC is converted to an intensity code by differentially weighting the output of SC neurons according to the eccentricity of the neuron's movement field. Scudder's model

Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall Lecture 20. Schemas 1 19 The discharges of cells simulated by this model: The first event is the onset of the superior colliculus discharge which simultaneously begins to charge the LLBN/integrator and to inhibit the OPN by a long latency indirect pathway. As excitation from the LLBN/integrator discharge increases and the inhibition from the OPN decreases, the net EBN input goes from inhibitory to excitatory. As the EBN fires, the IBN fires and further inhibits the OPN, causing a regenerative increase in EBN and IBN discharge rate. The firing of the IBN also inhibits the LLBN/integrator, causing it to begin to be discharged. Progressively, the LLBN/Integrator firing rate decreases until the inhibition of the OPN by the IBN no longer exceeds the excitatory bias of the OPN. The OPN begins firing, and the burst ends. The model matches the physiological data for saccades up to 15°.

Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall Lecture 20. Schemas 1 20 A problem with standard models of saccade generators: when a movement is required in an oblique direction the durations of the saccades generated by uncoupled horizontal and vertical saccade generators are different, whereas the duration of both components of an oblique saccade are equal in the animal. Scudder showed how to couple a horizontal and vertical burst generator pair via the omnipause neurons alone to obtain a model which yields realistic oblique saccades.