1. Eye Detection and Gaze Estimation
Ryland Fallon, ECE172a

2. Motivation
Develop a technique to extract a position onscreen that the user’s eyes are looking at.
Could be used as a user interface, if scaled up in speed and accuracy.

3. Process Detect face Find eyes in upper half of face
Use template eyes
Find pupils in found eyes
Clean false positives, average correct
Estimate gaze from face, eye, pupil positions

4. Face Detection Binary Threshold
Close, open, open, close with large strel (disk, R=25)
Form box from edges (seen in output)

5. Find Eyes Search upper half of face box
Perform Correlation with template image, maximum corresponds to eye location
Extract position

6. Find Pupils Find candidates
Threshold removes bright inner eye corner false positive if present
Determines position within eye image.

7. Gaze Estimation
Screen region displayed for each eye (if pupil detected)
Very inaccurate, needs more samples, training
However, does get the general idea

8. Go show the videos
Now, preferably.

9. Performance Slow as molasses on a hot day
Slight speed improvement made by limiting search area in the correlation step
Finishes processing a frame in 1-2 seconds, as opposed to 3-5 seconds
Again, gaze estimation step needs a better model, more training images
Pupils not always detected, but between the two eyes, can usually extract some region onscreen.

10. Good/Bad when… Good Bad
Background, lighting, head position, held constant
Head oriented vertically, with good view of both eyes
Near-infrared light source near camera
Background significantly darker than head region
Bad
Most other cases
Gaze estimation always pretty bad
Currently untested on other people

11. Future Improvements
Add robustness to accommodate different head positions, lighting conditions
Better eye gaze estimation system
More precise eye gaze estimation
Speed (algorithms, change to C)
Hardware less awkward