1. Detecting Eye Contact Using Wearable Eye-Tracking Glasses

2. Eye contact is important aspect in face to face interaction In case of children Atypical pattern of gaze and eye contact Early sign of autism

3. Goal: Eye contact detection between two Individuals Single wearable eye tracking device Clinical settings

19. Multiple static camera-distant, problem with frontal view Mutual eye tracking – Use EOG

26. Face analysis Problem of finding and analysing faces in video Computer vision Localize the face and facial parts (eyebrow, eye, nose, mouth etc) Gaze estimation using 2D appearance of eye Estimate the 3D gaze direction based on a single image of an eye Key idea: Learn the appearance model of the eye for different gaze directions Large number of samples

27. OMRON OKAO Commercial vision library Detect and analyse child’s face Takes the video as input Localizes all facial parts in video Estimates 3D head pose If eyes in the correct position, Gaze direction Promising result for frontally presented faces

28. Face analysis result by OKAO Bounding box Facial parts Head pose Gaze direction

29. Experimental setup Objectives –Record the video and gaze data with minimum obtrusiveness for children –Allow the analysis of the data for eye contact detection Protocol –Interactive session 5 to 8 min Examiner wears the eye tracker glass Interact with the child Sitting in front

30. Provide online annotation by pressing foot pedal----error prone

31. Experimental setup During interaction –Eye gaze was tracked and egocentric video was recorded OKAO vision library is applied –Obtain face information of the child Location/orientation of face 3D gaze direction –Adult gaze information Provided by eye tracker

32. Experimental setup Participants Female subject of age 16 months Recoded session – 7 min

33. Method Combines –Eye gaze of the examiner –Face information of the child (gaze direction) Extract features from gaze and face information Train the classifier –Detect existence of eye contact in a specific frame

34. Features Relative location (RL) –Examiner’s gaze point with respect to child’s eye center 3D Gaze direction of the child (GD) –With respect to image plane Head orientation (HO) –3D head position of the child Confidence of eye detection (CE)

35. Method Detect Eye contact –Binary classification with ground truth For each frame in the video and given feature –Detect the eye contact Simple rule works –Fix threshold for RL and GD Examiner’s gaze point is close to child’s eye Child’s gaze facing towards examiner

36. Gaze direction Face orientation

38. Results Randomly select 60% as training; rest as testing 5 trees ---- depth 6

39. 1.RL is more reliable 2.Vertical is more frequent than horizontal

40. Performance

41. OKAO vision library fails to detect correct gaze direction