Eye-Based Interaction in Graphical Systems: Theory & Practice Part III Potential Gaze-Contingent Applications

G: Applications—Introduction Wide variety of eye tracking applications exist, each class increasingly relying on advanced graphical techniques Wide variety of eye tracking applications exist, each class increasingly relying on advanced graphical techniques PsychophysicsHuman Factors AdvertisingDisplays Virtual RealityHCI & Collaborative Systems Two broad categories: diagnostic or interactive Two broad categories: diagnostic or interactive

H: Psychology, Psychophysics, and Neuroscience Applications range from basic research in vision science to investigation of visual exploration in aesthetics (e.g., perception of art) Applications range from basic research in vision science to investigation of visual exploration in aesthetics (e.g., perception of art) Examples: Examples: psychophysics: spatial acuity, contrast sensitivity,... psychophysics: spatial acuity, contrast sensitivity,... perception: reading, natural scenery,... perception: reading, natural scenery,... neuroscience: cognitive loads, with fMRI,... neuroscience: cognitive loads, with fMRI,...

H: Psychology, Psychophysics, and Neuroscience (cont’d) Fig.48: Perception of art small but visible differences in scanpaths small but visible differences in scanpaths similar sets of fixated image features similar sets of fixated image features (a) aesthetic group (b) semantic group

I: Ergonomics and Human Factors Applications range from usability studies to testing effectiveness of cockpit displays Applications range from usability studies to testing effectiveness of cockpit displays Examples: Examples: evaluation of tool icon groupings evaluation of tool icon groupings comparison of gaze-based and mouse interaction comparison of gaze-based and mouse interaction organization of click-down menus organization of click-down menus testing electronic layout of pilots’ visual flight rules testing electronic layout of pilots’ visual flight rules testing simulators for training effectiveness testing simulators for training effectiveness

I: Ergonomics and Human Factors (cont’d) Fig.49: Virtual aircraft cargo-bay environment examination of visual search patterns of experts during aircraft inspection tasks examination of visual search patterns of experts during aircraft inspection tasks 3D scanpaths: gaze/wall intersection points 3D scanpaths: gaze/wall intersection points

J: Marketing / Advertising Applications range from assessing ad effectiveness (copy testing) in various media (print, images, video, etc.) to disclosure research (visibility of fine print) Applications range from assessing ad effectiveness (copy testing) in various media (print, images, video, etc.) to disclosure research (visibility of fine print) Examples: Examples: eye movements over print media (e.g., yellow pages) eye movements over print media (e.g., yellow pages) eye movements over TV ads, magazines,... eye movements over TV ads, magazines,...

J: Marketing / Advertising (cont’d) Fig.50: Scanpaths over magazine ads

K: Displays Applications range from perceptually- based image and video display design to estimation of corrective display functions (e.g., gamma, color spaces, etc.) Applications range from perceptually- based image and video display design to estimation of corrective display functions (e.g., gamma, color spaces, etc.) Examples: Examples: JPEG/MPEG (no eye tracking per se, but perceptually based, e.g., JPDs) JPEG/MPEG (no eye tracking per se, but perceptually based, e.g., JPDs) gaze-contingent displays (e.g., video-telephony, …) gaze-contingent displays (e.g., video-telephony, …) computer (active) vision computer (active) vision

K: Displays (cont’d) Fig.51: Gaze-based foveo-peripheral image coding 2 Regions Of Interest (ROIs) 2 Regions Of Interest (ROIs) smooth degradation (wavelet interpolation) smooth degradation (wavelet interpolation) (a) Haar HVS reconstruction (b) wavelet acuity mapping

L: Graphics and Virtual Reality Applications range from eye-slaved foveal Region Of Interest (ROI) VR simulators to gaze-contingent geometric modeling Applications range from eye-slaved foveal Region Of Interest (ROI) VR simulators to gaze-contingent geometric modeling Examples: Examples: flight simulators (peripheral display degradation) flight simulators (peripheral display degradation) driving simulators (driver testing) driving simulators (driver testing) gaze-based dynamic Level Of Detail modeling gaze-based dynamic Level Of Detail modeling virtual terrains virtual terrains

L: Graphics and Virtual Reality (cont’d) Fig.52: Gaze- contingent Martian terrain subdivided quad mesh subdivided quad mesh per-block LOD per-block LOD resolution level based on viewing direction and distance resolution level based on viewing direction and distance

L: Graphics and Virtual Reality (cont’d)

M: Human-Computer Interaction and Collaborative Systems Applications range from eye-based interactive systems to collaboration Applications range from eye-based interactive systems to collaboration Examples: Examples: intelligent gaze-based informational displays (text scroll window synchronized to gaze) intelligent gaze-based informational displays (text scroll window synchronized to gaze) “self-disclosing” display where digital characters responded to user’s gaze (e.g., blushing) “self-disclosing” display where digital characters responded to user’s gaze (e.g., blushing) multiparty VRML environments multiparty VRML environments

M: Human-Computer Interaction and Collaborative Systems (cont’d) Fig.53: GAZE Groupware display Fig.54: GAZE Groupware interface multiparty tele- conferencing and document sharing system multiparty tele- conferencing and document sharing system images rotate to show gaze direction (who is talking to whom) images rotate to show gaze direction (who is talking to whom) document “lightspot” (deictic “look at this” reference) document “lightspot” (deictic “look at this” reference)

Eye-Based Interaction in Graphical Systems: Theory & Practice For further information: SIGGRAPH course notes SIGGRAPH course notes Eye Tracking Research & Applications Symposium Eye Tracking Research & Applications Symposium

Eye Tracking Research & Applications Symposium 2000 November 6th-8th 2000, Palm Beach Gardens, FL, USA Sponsored by... With corporate sponsorship from … Don’t forget to attend: LC TechnologiesUnilever Corp.SensoMotoric Instr.

Eye-Based Interaction in Graphical Systems: Theory & Practice Demonstration GAZE Groupware System