1. Generic Gaze Interaction Events for Web Browsers
Using the Eye Tracker as Input Device
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

2. Benjamin Wassermann, Media University Stuttgart
Agenda
Project
Learning Environment
Eye Tracking Framework
Gaze Events
Sample Applications
Future Work
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

3. Benjamin Wassermann, Media University Stuttgart
Project
Adaptable and Adaptive Multimedia Systems
Cluster 4 of the ScienceCampus Tübingen
Part of the Knowledge Media Research Center (kmrc)
Research line: Design of Interactive Informational Environments
Includes 4 interdisciplinary Subprojects
Participating: Computer Science (Stuttgart) and Psychology (Tübingen, Freiburg)
Names of Subprojects:
IT Foundations of Adaptable and Adaptive Multimedia Systems (Stuttgart)
User-Driven Access to Representations and Self-Regulation in Multimedia Learning (Freiburg)
Procedure- and Knowledge-Related Adaptation of Prompts in Multimedia Learning Environments: An Eye-Movement-Based Approach (Freiburg)
Using Eye-Movement Parameters for the Design of an Adaptive Access to Multimedia Representations (Tübingen)
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

4. Benjamin Wassermann, Media University Stuttgart
Learning Environment
Implemented as Web application
Based on ILIAS (eLearning CMS)
Two main parts:
Dynamic learning view
Authoring environment
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

5. Benjamin Wassermann, Media University Stuttgart
Learning View
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

6. Benjamin Wassermann, Media University Stuttgart
Drag & Drop
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

7. Benjamin Wassermann, Media University Stuttgart
Interactive Help
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

8. Benjamin Wassermann, Media University Stuttgart
Dynamic Content
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

9. Benjamin Wassermann, Media University Stuttgart
Dynamic Content
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

10. Benjamin Wassermann, Media University Stuttgart
Templates
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

11. Authoring Environment
Integrated in ILIAS
Conventions from ILIAS
Courses
Folders
Containers
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

12. Authoring Environment
Integrated in ILIAS
Conventions from ILIAS
Courses
Folders
Containers
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

13. Authoring Environment
Content creation
Content manipulation
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

14. Authoring Environment
Content creation
Content manipulation
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

15. Authoring Environment
Content creation
Content manipulation
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

16. Authoring Environment
Layout and navigation configuration
Import and Export
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

17. Benjamin Wassermann, Media University Stuttgart
Motivation
Requirements for Learning Environment
Adaptable User Interface, Adaptable Content by User, Interactive Help, Assessments, …, Eye Tracking
Requirements for Eye Tracking
Access to data in real-time
Correlate data to content elements
Data has to be accessible within Learning Environment
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

18. Benjamin Wassermann, Media University Stuttgart
Eye Tracking
Tracking user gaze on screen
Analysis of data
Scanpath
Heatmap
Statistics
Applied in Marketing-, Usability- and Psychology-Research
Can also be used for User Interaction
Selecting Elements
Activating Elements
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

19. Benjamin Wassermann, Media University Stuttgart
Framework
iTrackServer
Connect Eye Tracker to Web browser
Custom protocol
Generic Eye Tracker support
JavaScript API
Control Eye Tracker
Access gaze samples
Generate gaze events
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

20. Benjamin Wassermann, Media University Stuttgart
Gaze Events
Generate events on DOM-Level in Web browser
Using ElementFromPoint to get target objects for events
Events derived from mouse events:
gazeOver, gazeMove, gazeOut, gazeEnter and gazeLeave
Additional events
fixationStart and fixationEnd
Uses jQuery event system
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

21. Implementation Quirks
Screen to Web browser client coordinate conversion for ElementFromPoint
Trick with mouse button click
Special treatment for area tags within image maps
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

22. Coordinate Evaluation
Validated with SMI Experiment Center
Compared gaze position within Web browser with global position on screen
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

23. Coordinate Evaluation
Validated with SMI Experiment Center
Compared gaze position within Web browser with global position on screen
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

24. Benjamin Wassermann, Media University Stuttgart
Sample Applications
Interactive highlighting
Quiz
Pong
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

25. Benjamin Wassermann, Media University Stuttgart
Sample Applications
Interactive highlighting
Quiz
Pong
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

26. Benjamin Wassermann, Media University Stuttgart
Sample Applications
Interactive highlighting
Quiz
Pong
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

27. Learning Environment: Highlighting of Relations
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

28. Learning Environment: Assessments & Prompts
LU1
RA1
LU2
prompt B
RA2
RA3
false
Pretest
true
Analysis
prompt C
prompt A
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

29. Learning Environment: Assessments & Prompts
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

30. Learning Environment: Assessments & Prompts
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

31. Benjamin Wassermann, Media University Stuttgart
Future Work
Calibration within Web browser
coordinate adaptation is no longer required
Gaze sample filter API + filters
High-level gaze events
User tools for marking and annotating content in learning view
User tools to share enriched data
Optimizations for mobile devices
16 April, 2012
Benjamin Wassermann, Media University Stuttgart

32. Benjamin Wassermann, Media University Stuttgart
Thank you.
Contact:
Benjamin Wassermann
Media University Stuttgart
Nobelstraße 10
70569 Stuttgart - Germany
16 April, 2012
Benjamin Wassermann, Media University Stuttgart