Gaze Awareness for Videoconferencing: A Software Approach Nicolas Werro

Table of contents Introduction Main problems The software approach Rendering Eye manipulation Altering head pose Computer vision Conclusion

Introduction Importance of gaze awareness in face-to- face communication Most video conferincing systems make it impossible for participants to make eye contact

Main problems People stare into their displays rather than into the camera In addition to inhibiting eye contact, participants have no sense of spatial relationship

The software approach The aim is to develop an inexpensive videoconferencing system that work with cheap, commonly available hardware. The architecture of the video subsystem :

Rendering On the receiving end, the system uses the vision information to : extract the head from the video frame correct the gaze This is achieved in two steps: the system replaces the eyes with synthetic eyes to aim the gaze the head pose is adjusted

Eye manipulation New synthetic eyes focused on the desired direction are rendered The system uses segmentation data to decide, for each pixel, whether to use the pixel from the original face image or from the eyeball image

Altering head pose The head orientation is changed using texture mapping: First, the system creates a 3D model in the shape of the subject’s head Then, it executes texture mapping, projecting the face image from the video frame onto the model  The system can rotate the model to any desired orientation

Altering head pose

Computer vision The vision component must track the head pose, segment the eyes, and determine gaze direction

Computer vision Computer vision in general is very difficult, but several factors make the problem more tractable in this problematic : We’re looking only for a head, not arbitrary objects We deal only with head poses that permit the subject to gaze directly at the screen The system only needs to detect gaze that is directed at the screen

Conclusion The results of this work appear promising: Accurate vision data can be extracted from each video regarding head pose, eye segmentation and gaze direction The pose of the head can be arbitrarily oriented The eyes can be synthesized with the appropriate gaze direction The resulting videoconferencing system supports a sense of space and gaze awareness