1. Why do we move our eyes? - Image stabilization in the presence of body movements. - Information acquisition - bring objects of interest onto high acuity region in fovea.

2. Retinal structure

3. Cone Photoreceptors are densely packed in the central fovea

4. Visual Acuity matches photoreceptor density

5. Oculomotor Muscles Muscles innervated by oculomotor, trochlear, and abducens (cranial) nerves from the oculomotor nuclei in the brainstem. Oculo-motor neurons: 100-600Hz vs spinal motor Neurons: 50-100Hz

6. Types of Eye Movement Information GatheringStabilizing Voluntary (attention)Reflexive Saccadesvestibular ocular reflex (vor) new location, high velocity (700 deg/sec), body movements ballistic(?) Smooth pursuitoptokinetic nystagmus (okn) object moves, velocity, slow(ish) – typically whole field image motion up to 35 deg/sec Vergence change point of fixation in depth slow, disjunctive (eyes rotate in opposite directions) (all others are conjunctive) Note: link between accommodation and vergence Fixation: period when eye is relatively stationary between saccades.

7. Acceleration Depth-dept gain, Precision in natural vision Velocity Acuity – babies

8. https://www.youtube.com/watch?v=KSJksSA6Q-A

9. Latency of vestibular-ocular reflex=10msec

10. It is almost impossible to hold the eyes still. Demonstration of VOR and its precision – sitting vs standing Miniature eye movements Slow drift Micro-saccades tremor

12. Saccade latency approx 200 msec, pursuit approx 100 – smaller when there is a context that allows prediction. Step-ramp allows separation of pursuit (slip) and saccade (displacement)

14. “main sequence”: duration = c Amplitude + b Min saccade duration approx 25 msec, max approx 200msec

15. Factors That Control Gaze. - TASK Defines behavioral goals, what information is relevant. - REWARDS Oculomotor circuitry sensitive to reward/subjective value of those goals. - UNCERTAINTY Get information. Peripheral resolution/ working memory REDUCTION decay etc - PRIORS/ Memory Gaze targeting reflects stored knowledge. - IMAGE Salient properties eg high contrast/ spatial outliers

16. 1. Neural activity related to saccade 2. Microstimulation generates saccade 3. Lesions impair saccade Brain Circuitry for Saccades

17. Brain Circuitry for Pursuit

18. Eye Tracking Methods

19. Developments in Eye Tracking Head fixed /restricted: Contact lenses: mirror / magnetic coils Early infra-red systems Dual Purkinje Image tracker Head Free: Head mounted IR video-based systems Remote systems with head tracking Scene camera Difficulty: optical power of eye + observer movement

20. Why eye movements are hard to measure. 18mm 0.3mm = 1 deg visual angle xa tan(a/2) = x/d a = 2 tan - 1 x/d Visual Angle d 1 diopter = 1/focal length in meters 55 diopters = 1/.018 A small eye rotation translates into a big change in visual angle

21. Early Methods: “ Barlow photographed a droplet of mercury placed on the limbus. Translations of the head were minimized by having subjects lie on a stone slab with their heads wedged tightly inside a rigid iron frame ” Kowler, 1990 Measuring Eye Movements

22. Early methods: “ The eye is first cocainized, then the lids should be propped apart by some form of eye-lid fastener, of which the best is probably that in form of a wide- opening spring with tortoise-shell grooves for the lids. ” Delabarre, 1898

23. Monitoring Eye Movements; Yarbus Mirror mounted on eye using suction. Light bounces off mirror and is recorded on film

24. Non image-based eye trackers –Electrical/analog –Limbus –Magnetic search coil Non image-based Eye Trackers

25. EOG The eye is a ‘dipole’ with ~millivolts voltage difference between the retina and the cornea.

26. ElectroOculoGram (EOG) Use in clinic – head not fixed

27. By monitoring the ‘whites of the eye’ below the iris, it is possible to determine eye position. Vertical eye movements cause both signals to increase (up) or decrease (down). Horizontal eye movements cause differential illumination between the right and left sensors. Limbus Trackers

29. Limbus

30. EOG and Limbus trackers Good temporal resolution. Lousy spatial resolution High noise, drift Mostly useful in clinic

31. Skalar search coils Magnetic Search Coils Used for much animal work, though less so recently. Very high precision and accuracy (few minutes of arc). Used in older human em literature. Can use similar methodology for head and hand (see Hayhoe lab)

32. Image-based eye trackers –Dual Purkinje –Video based Image-based Eye Trackers

33. Dual Purkinje Trackers The ‘gold standard’ in eye trackers Multiple reflections from the cornea and lens vary in a very well- defined way as the eye moves. By tracking the 1 st and 4 th reflections, the tracker can determine eye position with very high precision. Bill Geisler lab has a binocular tracker.

34. Dual Purkinje Trackers Precision: 500 Hz

35. Dual Purkinje Trackers Usually requires bite bar but theoretically can get away with head rest.

36. Video-based eye trackers: –Head mounted –Remote Video-based Eye Trackers

37. Head mounted Camera on head views scene, another camera views eye.

38. Video-based Eye Trackers Infra-red video camera finds center of pupil and corneal reflection. Advantages: unconstrained viewing. Disadvantages: temporal resolution may be as low as 30 Hz Accuracy never better than 0.5 deg.

39. RIT Wearable Eyetracker

43. Build-up neurons in the intermediate layers of the SC are active prior to a saccade. Cell in the superifical layers get input from the retina. This may mediate Very fast saccades – sometimes called “express saccades” Extent of buildup neuron activity reflects stimulus probability. Express saccades might also reflect activity in buildup neurons.

44. LIP: Lateral Intra-parietal Area Target selection for saccades: cells fire before saccade to attended object Posterior Parietal Cortex reaching grasping Intra-Parietal Sulcus: area of multi-sensory convergence Visual stability

45. Model of saccade generation: target selection depends on expected value Trommershauser, Glimcher, Gegenfurtner, 2009 Area LIP contains a reward expectation signal which modulates the gain of visual neurons in LIP. Reward modulation of saccadic eye movements originates from dopaminergic input to caudate nucleus.

46. Relation between saccades and attention. Saccade is always preceded by an attentional shift However, attention can be allocated covertly to the peripheral retina without a saccade. Pursuit movements also require attention.

47. Figure 8.18 The comparator Visual Stability

49. A cross seen through an aperture that moves clockwise around the boundary. Alternatively, the aperture may be stationary, and the cross move behind it. Individual views, shown on the right, are ambiguous. Observers have no trouble with this if they have an “internal model” or schema that readily allows interpretation of the sequence.

51. -Saccades/Smooth Pursuit -Planning/ Error Checking -relates to behavioral goals Supplementary eye fields A subset of SEF neurons and LFPs exhibited strong modulation following erroneous saccades to a distractor. Altogether, these results suggest that SEF plays a limited role in controlling ongoing visual search behavior, but may play a larger role in monitoring search performance. Nearby Anterior Cingulate also involved in performance monitoring.

52. Motor neurons for the eye muscles are located in the oculomotor nucleus (III), trochlear nucleus (IV), and abducens nucleus (VI), and reach the extraocular muscles via the corresponding nerves (n. III, n. IV, n. VI). Premotor neurons for controlling eye movements are located in the paramedian pontine reticular formation (PPRF), the mesencephalic reticular formation (MRF), rostral interstitial nucleus of the medial longitudinal fasciculus (riMLF), the interstitial nucleus of Cajal (IC), the vestibular nuclei (VN), and the nucleus prepositus hypoglossi (NPH). Motor neurons Pre-motor neurons Oculomotor nucleus Trochlear Abducens H V

53. Pulse-Step signal for a saccade

54. Brainstem circuits for saccades. Omnipause neurons (OPN) in the nucleus raphe interpositus (RIP) tonically inhibit excitatory burst neurons (EBN) located in the paramedian pontine reticular formation (PPRF). When OPNs pause, the EBNs emit a burst of spikes, which activate motor neurons (MN) in the abducens nucleus (VI) innervating the lateral rectus muscle. The burst also activates interneurons (IN) which activate motor neurons on the oculomotor nucleus (III) on the opposite side, innervating the medial rectus. Inhibitory burst neurons (IBN) show a pattern of activity similar to EBNs, but provide inhibitory inputs to decrease activation in the complementary circuits and antagonist muscles. Long-lead burst neurons (LLBN) show activity long before movement onset, and provide an excitatory input to EBNs.

55. Brain areas involved in making a saccadic eye movement Behavioral goal: make a sandwich (learn how to make sandwiches) Frontal cortex. Sub-goal: get peanut butter (secondary reward signal - dopamine - basal ganglia) Visual search for pb: requires memory for eg color of pb or location (memory for visual properties - Inferotemporal cortex; activation of color - V1, V4) Visual search provides saccade goal. LIP - target selection, also FEF Plan saccade - FEF, SEF Coordinate with hands/head Execute saccade/ control time of execution: basal ganglia (substantia nigra pars reticulata, caudate) Calculate velocity/position signal oculomotor nuclei Cerebellum?

56. RF reticular formation VN vestibular nucle PN, pontine nuclei i Cerebellum OV oculomotor vermis VPF ventral paraflocculus FN fastigial nucleus

57. otoliths Rotational (semi-circular canals) translational (otoliths)

58. target selection signals to muscles (forces) inhibits SC saccade decision saccade command (where to go) monitor/plan movements Function of Different Areas H V

60. Smooth pursuit & Supplementary Brain Circuitry for Pursuit Velocity signal Early motion analysis

61. Eye Movement Research Consequences of image motion on visual acuity: stabilized images Metrics of saccades/ pursuit/ vergence/vor Constancy of visual direction Eye movements in reading/ Cognitive role of eye movements Active Vision/Natural tasks: Fixation patterns, eye/head/hand coordination Language comprehension in visual context

62. Paradigm Differences Natural Tasks: Study small segments of behavior Multiple visual operations: transitions between operations S in control of agenda complexity of scene Standard approach: Repeated observations of a small time slice Single visual operation or movement Limited complexity

63. Remote Bright pupil Dark pupil

64. Video-based Eye Trackers

66. Dark pupil Bright pupil Bright-pupil; coaxial illumination

67. Two signals; pupil and corneal reflection - fairly robust to tracker movement wrt head. Lower temporal and spatial resolution than eg coils/DPI 1 deg, 60-120 Hz Embed eye image in video record to monitor quality of track.