User Attention Tracking in Large Display Face Tracking and Pose Estimation Yuxiao Hu Media Computing Group Microsoft Research, Asia

Scenario  When using a large scale display or multiple monitors, a user often “lost” the mouse cursor / input cursor;  When working with multiple windows, a user often need to move the mouse cursor for long distance to click a window and activate it.

Our Solution  Track the user’s face with real-time face detector and tracker;  Estimate the user’s head pose according to the face and facial components detection results;  Switch the mouse/keyboard cursor among multiple monitors/windows according to the user’s attention direction

System Framework

Key Issues  Robust: detect and track the user’s face with arbitrary pose, expression under unconstrained illumination  Accurate: estimate the 3D pose of the user’s head precisely;  Fast: detect, track the user’s face and estimate its pose in real-time  Convenient: easy initialization and no special operations  Light: use typical CPU, web camera, and low system resource cost

Preliminary Test Results  Pose Coverage: yaw ±60°, roll: ±30°  Precision in yaw pose: error < ±10 ° (can support up to 5 monitors)  Speed: 15 frame/sec  Typical USB Web Camera: 320*240 resolution  CPU Usage: 20% average and 80% max

Performance Comparison MethodFeatureAccuracySpeed Ellipsoidal model, edge density feature point Wide-Range, person and Illumination- Insensitive 19 º for different person, about 10 º if initialization for person 200ms~30 0ms on PII 450M Hz PCA + SVRKernel-based learning 10 º for different person N/A KPCA+ SVCKernel-based learning, multi-class classification 99% classification accuracy, within 20 º,97% within 10 º N/A Confidence of Feature Point + Regression (our method) Fast, accurate, small change 6.54 º in yaw angle and 5.35 º in tilt angle if initialization for person <100 ms on P4 1.4GH z

For More Details  Related Publication: Y.X. Hu, L.B. Chen, Y. Zhou, H.J. Zhang, "Estimating Face Pose by Facial Asymmetry and Geometry",FGR2004  Website:  Contacts: Yuxiao Hu,

Let track it for you ! Cannot find your mouse cursor ? User Attention Area Tracking in Large Display