1. Eye tracking: principles and applications 廖文宏 Wen-Hung Liao whliao@gmail.com 12/10/2009

2. Outline Eye Tracking Device Application areas State-of-the-art technology Eye tracker @NCCUCS 1.0 (wearable) Eye tracker @NCCUCS 2.0 (remote) Gaze-based HCI Demo Conclusions

3. Eye Tracking Device An eye tracker is a device for measuring eye positions and eye movements.eye movements The most popular variant uses video images from which the eye position is extracted. Input source: visible spectrum vs. infrared

4. Eye Movements Eye movements are typically divided into fixations (when the eye gaze pauses in a certain position) and saccades (when it moves to another position). Eye movements The resulting series of fixations and saccades is called a scanpath.scanpath

6. Web Design (U. of Manchester) Heat mapGaze plot

7. Usability Study: Google Search Evaluation

8. Gaming

9. Driving Behavior [Andrew T. Duchowski]

10. Character Input EyeWrite [Andrew T. Duchowski]

11. Media Research The Poynter Institute : http://www.poynter.orghttp://www.poynter.org Published first eye track study in 1991. More results published in 2000, 2004 and 2007. Eyetrack ’07: http://www.poynter.org/content/content_view. asp?id=105035 http://www.poynter.org/content/content_view. asp?id=105035

12. Research Communities ACM SIG/CHI Eye Tracking Research and Applications (ETRA) Symposium: http://www.e-t-r-a.org/http://www.e-t-r-a.org/ COGAIN: Communication by Gaze Interaction http://www.cogain.orghttp://www.cogain.org References: http://www.cogain.org/downloads/ http://www.cogain.org/downloads/

13. State-of-the-art Eye tracking Technology Hi-Speed (SMI iView X™ Hi-Speed) Head-Mounted System Remote tracking Integrated with LCD monitor

14. Mobile Eye Tracker 1.0 @ NCCUCS eye camera scene camera

15. Key Objectives Low-cost (NTD 10,000 vs. 1,000,000) Mobility Easily customized for specific applications On-line processing Sampling rate? Accuracy?

16. System Architecture Eye image Preprocessing Pupil detection Gaze point projection Scene image Calibration 9 pairs of points Calibration process

17. Pupil Detection (I): extracting feature points

18. Pupil Detection (II): Ellipse fitting using RANSAC

19. Pupil Detection (III): checking fitness measure Ellipse model: Number of bright vs. dark points inside and out the perimeter

20. Results

21. HCI Application: Eye Scrolling

22. HCI Application: Eye Gaming

23. HCI Application: Eye Typing

24. Mobile Eye Tracker 2.0 @ NCCUCS Improve the pupil detection algorithm to alleviate corneal reflection problem. Enhance the accuracy by compensating for head movement. Construct and test a remote eye tracker. More HCI applications using the remote eye tracker. Use the eye tracking device to assist mobile user interface design.

25. Improved Pupil Detection

26. Allowing Head Movement

27. Remote Eye Tracker

28. Experimental Results (Wearable) Calibration point Original error Compensate for head movement (error/standard deviation) 198.1732.91 (12.7) 2137.5455.89 (10.94) 3108.2029.69 (9.57) 476.3028.79 (21.19) 5113.3022.49 (7.79) 6117.8634.83 (10.96) 7116.5333.31 (8.01) 8112.5616.03 (6.78) 9146.7326.49 (9.66) 1 cm = 38 Pixels

29. Accuracy (Wearable)

30. Experimental Results (Remote) Calibration pointOriginal error Compensate for head movement (error/standard deviation) 1238.7339.69 (24.6) 2227.5163.56 (22.82) 3132.9744.30 (21.49) 4222.9751.66 (30.94) 5306.5831.15 (16.71) 6280.7951.66 (21.83) 7311.8969.03 (41.57) 8344.2469.88 (31.71) 9347.0364.66 (24.41)

31. Accuracy (Remote Eye Tracker)

32. Demo: Web Browsing

33. Demo: Photo Viewing

34. Demo: Interactive Story Telling

35. Demo: Tic-Tac-Toe

36. Summary Eye tracking device Low cost (<$10,000 NTD) Head-mounted with mobile functionality Remote eye tracking Allow slight head movements Accuracy: suitable for some HCI applications, image viewing tasks, not high enough for reading HCI applications

37. http://www.cs.nccu.edu.tw/~whliao/dct/

38. More materials

39. Gaze Path Analysis Recursive intersection Find the similarity between two scan paths Order of scanning is irrelevant Suitable for processing fixation data Modified dynamic time warping Order of scanning is taken into account Can handle both fixation and saccade data

40. Recursive Intersection Path 2 Path 1

41. Recursive Intersection: Example

42. Dynamic Time Warping (DTW)

43. Modified DTW (MDTW) X-axis Y-axis time

44. MDTW Result