Generic Gaze Interaction Events for Web Browsers Using the Eye Tracker as Input Device 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Benjamin Wassermann, Media University Stuttgart Agenda Project Learning Environment Eye Tracking Framework Gaze Events Sample Applications Future Work 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Benjamin Wassermann, Media University Stuttgart Project Adaptable and Adaptive Multimedia Systems Cluster 4 of the ScienceCampus Tübingen Part of the Knowledge Media Research Center (kmrc) Research line: Design of Interactive Informational Environments Includes 4 interdisciplinary Subprojects Participating: Computer Science (Stuttgart) and Psychology (Tübingen, Freiburg) Names of Subprojects: IT Foundations of Adaptable and Adaptive Multimedia Systems (Stuttgart) User-Driven Access to Representations and Self-Regulation in Multimedia Learning (Freiburg) Procedure- and Knowledge-Related Adaptation of Prompts in Multimedia Learning Environments: An Eye-Movement-Based Approach (Freiburg) Using Eye-Movement Parameters for the Design of an Adaptive Access to Multimedia Representations (Tübingen) 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Benjamin Wassermann, Media University Stuttgart Learning Environment Implemented as Web application Based on ILIAS (eLearning CMS) Two main parts: Dynamic learning view Authoring environment 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Benjamin Wassermann, Media University Stuttgart Learning View 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Benjamin Wassermann, Media University Stuttgart Drag & Drop 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Benjamin Wassermann, Media University Stuttgart Interactive Help 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Benjamin Wassermann, Media University Stuttgart Dynamic Content 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Benjamin Wassermann, Media University Stuttgart Dynamic Content 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Benjamin Wassermann, Media University Stuttgart Templates 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Authoring Environment Integrated in ILIAS Conventions from ILIAS Courses Folders Containers 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Authoring Environment Integrated in ILIAS Conventions from ILIAS Courses Folders Containers 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Authoring Environment Content creation Content manipulation 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Authoring Environment Content creation Content manipulation 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Authoring Environment Content creation Content manipulation 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Authoring Environment Layout and navigation configuration Import and Export 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Benjamin Wassermann, Media University Stuttgart Motivation Requirements for Learning Environment Adaptable User Interface, Adaptable Content by User, Interactive Help, Assessments, …, Eye Tracking Requirements for Eye Tracking Access to data in real-time Correlate data to content elements Data has to be accessible within Learning Environment 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Benjamin Wassermann, Media University Stuttgart Eye Tracking Tracking user gaze on screen Analysis of data Scanpath Heatmap Statistics Applied in Marketing-, Usability- and Psychology-Research Can also be used for User Interaction Selecting Elements Activating Elements 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Benjamin Wassermann, Media University Stuttgart Framework iTrackServer Connect Eye Tracker to Web browser Custom protocol Generic Eye Tracker support JavaScript API Control Eye Tracker Access gaze samples Generate gaze events 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Benjamin Wassermann, Media University Stuttgart Gaze Events Generate events on DOM-Level in Web browser Using ElementFromPoint to get target objects for events Events derived from mouse events: gazeOver, gazeMove, gazeOut, gazeEnter and gazeLeave Additional events fixationStart and fixationEnd Uses jQuery event system 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Implementation Quirks Screen to Web browser client coordinate conversion for ElementFromPoint Trick with mouse button click Special treatment for area tags within image maps 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Coordinate Evaluation Validated with SMI Experiment Center Compared gaze position within Web browser with global position on screen 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Coordinate Evaluation Validated with SMI Experiment Center Compared gaze position within Web browser with global position on screen 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Benjamin Wassermann, Media University Stuttgart Sample Applications Interactive highlighting Quiz Pong 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Benjamin Wassermann, Media University Stuttgart Sample Applications Interactive highlighting Quiz Pong 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Benjamin Wassermann, Media University Stuttgart Sample Applications Interactive highlighting Quiz Pong 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Learning Environment: Highlighting of Relations 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Learning Environment: Assessments & Prompts LU1 RA1 LU2 prompt B RA2 RA3 false Pretest true Analysis prompt C prompt A 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Learning Environment: Assessments & Prompts 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Learning Environment: Assessments & Prompts 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Benjamin Wassermann, Media University Stuttgart Future Work Calibration within Web browser coordinate adaptation is no longer required Gaze sample filter API + filters High-level gaze events User tools for marking and annotating content in learning view User tools to share enriched data Optimizations for mobile devices 16 April, 2012 Benjamin Wassermann, Media University Stuttgart
Benjamin Wassermann, Media University Stuttgart Thank you. Contact: Benjamin Wassermann Media University Stuttgart Nobelstraße 10 70569 Stuttgart - Germany wassermann@hdm-stuttgart.de 16 April, 2012 Benjamin Wassermann, Media University Stuttgart