Eye Detection and Gaze Estimation Ryland Fallon, ECE172a

Motivation Develop a technique to extract a position onscreen that the user’s eyes are looking at. Could be used as a user interface, if scaled up in speed and accuracy.

Process Detect face Find eyes in upper half of face Use template eyes Find pupils in found eyes Clean false positives, average correct Estimate gaze from face, eye, pupil positions

Face Detection Binary Threshold Close, open, open, close with large strel (disk, R=25) Form box from edges (seen in output)

Find Eyes Search upper half of face box Perform Correlation with template image, maximum corresponds to eye location Extract position

Find Pupils Find candidates Threshold removes bright inner eye corner false positive if present Determines position within eye image.

Gaze Estimation Screen region displayed for each eye (if pupil detected) Very inaccurate, needs more samples, training However, does get the general idea

Go show the videos Now, preferably.

Performance Slow as molasses on a hot day Slight speed improvement made by limiting search area in the correlation step Finishes processing a frame in 1-2 seconds, as opposed to 3-5 seconds Again, gaze estimation step needs a better model, more training images Pupils not always detected, but between the two eyes, can usually extract some region onscreen.

Good/Bad when… Good Bad Background, lighting, head position, held constant Head oriented vertically, with good view of both eyes Near-infrared light source near camera Background significantly darker than head region Bad Most other cases Gaze estimation always pretty bad Currently untested on other people

Future Improvements Add robustness to accommodate different head positions, lighting conditions Better eye gaze estimation system More precise eye gaze estimation Speed (algorithms, change to C) Hardware less awkward