Eye/gaze tracking in video; identify the user’s “focus of attention” oMihaela Romanca – Technical University of Cluj-Napoca oPeter Robert - Technical University of Cluj-Napoca oVilius Matiukas - Vilnius Gediminas Technical University oBrigitta Nagy – University of Debrecen

Introducing the team SSIP

Mihaela Romanca SSIP 2009 Student from Technical University of Cluj-Napoca Hobbies: Sports and ecology 3

Robert Peter SSIP 2009 Masters student from Technical University of Cluj-Napoca Hobbies: PC games, football and movies/music 4

Vilius Matiukas SSIP 2009 PhD student from Vilnius Gediminas Technical University, Faculty of Electronics, Department of Electronic Systems Hobbies: Image Processing and fishing 5

Brigitta Nagy SSIP 2009 Student from University of Debrecen, Faculty of Informatics Hobbies: Image processing, Wing- Tsun Kung-fu, Reading and Puzzles 6

Test subject SSIP 2009 Uneducated peace of paper Hobbies: Staring at the same direction. Address: computer laboratory 7

SSIP 2009 Problem Description Input: video of a user sitting in front of the computer Goal: Detect the focus of attention and the modification of the region of interest of the user. 8

Equipment and software Genius Slim 321c webcamera. Language: C# IDE: Microsoft Visual Studio 2005 EMGU CV: Wrapper for C# of OpenCV SSIP

Tasks to do 1.Face Detection 2.Detection of the eyeregion 3.Pupil Detection 4.Eye Corner Detection 5.Determine the focus of attention 10

SSIP Face Detection We used Haar-like features for face detection. Haar-like features are digital image features used in object recognition. Then we reduced the face region and split it to region of eyes. 11

Example SSIP 2009 Our test subject 12

SSIP Detection the region of the eye To detect the region of eyes we used also Haar-like features. Then contrast enchancement on the detected eye region was applied. 13

Example The test subject SSIP 2009 ApproximationWith Haar-like features 14

SSIP Pupil detection Circular Hough transformation was applied for detection of the pupil. The Hough transform is a feature extraction technique. The classical Hough transform was concerned with the identification of lines in the image, but later the Hough transform has been extended to identifying positions of arbitrary shapes, most commonly circles or ellipses. 15

SSIP Eye Corner Detection We choose the one closest corner to the nose. 16

Calibration for gaze detection Wait until the user sits in a position, where 80% of the frames detect the iris center and the corner also. Put circles in the center and the four extremities of the screen, and wait until at least 15 pupil and eye corners are detected in both region of eye. Calculate the average of eye corner and center coordinates in all the positions (center, topleft, topright…). SSIP

SSIP Focus of attention As the users moves the eyes the mouse cursor moves in the corresponding direction. 18

Statistics Spot Variance (pixel^2) Test1 Variance (pixel^2) Test2 Variance (pixel^2) Test3 Variance (pixel^2) Test4 Variance (pixel^2) Test5 Left pupil Left corner Right pupil Right corner SSIP 2009 The numbers represent the variance of coordinates during calibration. 19

Future development Imitating left and right mouse clicks with blink detection Recognition even when face is in different angle Expression detection for different focus regions Higher precision for full control for people with disabilities SSIP

Thank you! SSIP