Jochen Triesch, UC San Diego, 1 Eye Movements and Eye Tracking Why move the eyes? see the same thing better (stabilize image, but not too well!) see something else (overcome low peripheral resolution) S. Anstis only small part of visual field is sampled at full resolution

Jochen Triesch, UC San Diego, 2 Cortical Magnification in V1 Eric Schwartz Or in complex form: Approximately:

Jochen Triesch, UC San Diego, 3 Types of eye movements Major Distinctions: conjugate vs. disjunctive abrupt vs. slow Types: vestibuocular reflex: counteract head rotation optokinetic reflex: counteract large field retinal motion smooth pursuit: counteract object motion (<30 deg/s) saccades: rapidly shift gaze direction (<600 deg/s) vergence movements: adjust vergence angle accomodation: adjust focus microsaccaes: counteract drift during fixation (1-2 Hz, <5’) microtremor:.5’, Hz

Jochen Triesch, UC San Diego, 4 Saccades Yarbus (1950s) alternations of fixations and saccades, 3 per second

Jochen Triesch, UC San Diego, 5 Yarbus, 1967: examine the picture at will estimate the economic level of the people estimate the people’s ages guess what the people were doing before the arrival of the visitor remember the people’s clothing remember the people’s (and objects’) position in the room estimate the time since the guest’s last visit

Jochen Triesch, UC San Diego, 6 Saccade Metrics from Becker, 1991 approximately linear relationship between saccade amplitude A and saccade duration D: 20 ms < D 0 < 30 ms 2 ms/° < d < 3 ms/° Example: for 20° saccade display needs to be changed ~75ms after saccade onset

Jochen Triesch, UC San Diego, 7 Anthropomorphic Robot Head

Jochen Triesch, UC San Diego, 8 System overview Gigabit Router Desktop ADesktop BDesktop C

Jochen Triesch, UC San Diego, 9 Motor System 9 DoF (hobby grade servo motors) 2 Mini SSC II interface boards Four bar linkage connection Eye/neck system is redundant

Jochen Triesch, UC San Diego, 10 Vision System 2 Point Grey Research Firefly cameras (Firewire) 640x480 resolution up to 30 fps 2, 4, 6, and 8mm focal length lenses

Jochen Triesch, UC San Diego, 11 Saccade Performance 1

Jochen Triesch, UC San Diego, 12 Saccade Performance 2

Jochen Triesch, UC San Diego, 13 Demo video

Jochen Triesch, UC San Diego, 14 Why Eye tracking Basic Neuroscience: - measuring brain output - understanding neural control architecture - psychophysics: how do we use gaze during natural tasks Applications: - user interface design, human factors: e.g. eye tracking for driver of a car - advertising: do people look where I want them to look (in my web page, my newspaper ad, etc.) Note 1: now several conferences solely devoted to this Note 2: can also be done in fMRI Note 3: possible in the real world: “portable eye trackers”

Jochen Triesch, UC San Diego, 15 Eye tracking techniques Eye tracking techniques (Review in Duchowski&Vertegaal, 2000) Contact lenses: with mirrors or induction loops - fast, accurate, uncomfortable (often used with bite bars) Video based: - temporal accuracy limited to camera frame rate (60Hz) - less accurate (~1 deg) - typically with infrared illumination of the eye (dark pupil vs. bright pupil) - can be done remotely or camera can be attached to head Electro-oculogram: - exploits electric dipole property of eye ball - fast but imprecise Limbus tracking: - predecessor of the video based tracker - imprecise - small operating range

Jochen Triesch, UC San Diego, 16 Electro-oculogram eye is electric dipole electric field of moving dipole induces measurable voltages provides analog voltage output that can be digitized and processed extremely fast, but technique is not accurate electrode

Jochen Triesch, UC San Diego, 17 Limbus Tracking utilize difference in reflective properties of iris vs. sclera infrared LED photo diode sclera limbus iris

Jochen Triesch, UC San Diego, 18 Search Coil gold standard for speed and accuracy (5-10’’), but quite uncomfortable and head movement restrained

Jochen Triesch, UC San Diego, 19 Video based tracking most widely used today, good accuracy and speed, easy-to-use dual purkinje tracker bright pupil image

Jochen Triesch, UC San Diego, 20 inside of head mounted display U. of Rochester

Jochen Triesch, UC San Diego, 21 Gaze contingent display changes Idea: decide what is being displayed contingent on where observer is looking Saccade contingent updating (SCU): a special case of this: make display changes while a saccade is progressing (pioneering work by McConkie and colleagues) powerful technique for studying visual perception frequently used in e.g. change blindness studies

Jochen Triesch, UC San Diego, 22 Combination of video and limbus tracker inside HMD for gaze contingent display analog limbus tracker’s sensor with infrared LED, photo diodes (horizontal) video based pupil tracker’s sensor LCD displays inside the HMD video based tracker for good accuracy limbus tracker for low latency saccade detection

Jochen Triesch, UC San Diego, 23 System Overview 4 processor high-end graphics computer as backbone images rendered in V8 helmet (Virtual Research) Three sensors: magnetic tracking device (Polhemus Fastrak) limbus tracker (ASL 210), sampled at 1.25 kHz pupil tracker (ASL 501), sampled at 60 Hz Sensors send data via separate serial lines

Jochen Triesch, UC San Diego, 24 Latency Comparison in this example: limbus tracker has 26 ms advantage over video tracker on average: limbus tracker has 37±13 ms advantage (15 trials)

Jochen Triesch, UC San Diego, 25 chess pieces (dis-)appear contingent on saccade (~25-30°) with 50% probability