Head Tracking Using Video Analytics Xuan Wang 1, Yuhen Hu 1, Robert G. Radwin 2, John D. Lee 2 University of Wisconsin – Madison 1 Dept. Electrical and Computer Engineering 2 Dept. Industrial and Systems Engineering GLOBALSIP 2015, Orlando, FL

Backgrounds Related works Proposed approach –Spurious face elimination –Head tracking Experiment result Conclusion 2 Outline

Driver Distraction: –Texting, phone calling, dozing, passenger interruption, eating, drinking, maneuvering gadgets, etc. A major cause of highway accidents! 3 Driver Distraction

Program: Naturalistic Driving Study (NDS) Sponsor: US Federal Highway Administration Goal: To study driver distraction and attentiveness during driving using continuous video monitoring. Tool: Over two petabytes of video recordings of 3,400 people driving vehicles has been collected A typical video frame: 4 Naturalistic Driving Study

Big data Identify factors of distraction-related crashes Manually coding cannot scale up Task –Video analytic algorithms Goals 5 Video Analytics: Quantifying Driver Distraction Distraction Level: 9 Causal relation between distraction & accidents Driver status: Head turned back

Camera fixed: Challenge: Video quality is often low! 6 Specific Task: Driver’s Head Tracking

Related Works “Visual Tracking Decomposition” – Junseok Kwon, 2010 “Structured Output Tracking” – Sam Hare, 2011 “Sparsity-based Collaborative Model” – Wei Zhong, 2012 “Tracking by Sampling Trackers” – Kyoung Mu Lee, 2011 “HyHope System”- Murphy-chutorian, 2008 The prior observation, requirement of video quality, computation load required by these methods motivate us to propose our video analytics approach. 7

Features of our approach: –Uses statistical information of head detections of whole video –Uses temporal correlations between successive frames Implementation of our approach: –A modular based, multi-step video analytic work-flow 8 Proposed Approach Video Face Detection Head tracking Spurious Face Elimination Anno- tation 2 Anno- tation 1 Anno- tation 3 Head Pose Estimation

Viola-Jones Frontal Face Detection Video Face Detectio n Head tracking Spurious Face Eliminatio n Annot ation 2 Annot ation 1 Annot ation 3 Head Pose Estimatio n 9

Approach: –It adopts adaboost to construct Haar feature classifier and build cascaded classifiers from training dataset to detect objects. Advantages: –Good performance –Existing implementation Shortcomings: –Only detect frontal face –Performance not easily estimated An example of different types of failures: 10 Viola-Jones Face Detector

Spurious Face Elimination Video Face Detectio n Head tracking Spurious Face Eliminatio n Annot ation 2 Annot ation 1 Annot ation 3 Head Pose Estimatio n 11

Spurious faces: An un-supervised learning problem Clustering is based on 2 key features: –Distribution model of head position of the whole video –Gaussian distribution of head positions displacement 12 Automated Spurious Face Elimination

Driver’s Head Position Analysis Head position distribution –Clustered –Not circular symmetric –k-means not suitable –DBSCAN feasible Changes of head positions –Gaussian distributed 13

Density-Based Spatial Clustering of Applications with Noise Basics of algorithm: Advantages: –No requirement of k –No requirement of shape "DBSCAN-Illustration" by Chire - Own work. Licensed under CC BY-SA 3.0 via Commons - Illustration.svg#/media/File:DBSCAN-Illustration.svg 14 DBSCAN

15 Spurious Face Elimination: DBSCAN vs Kmeans

Head Tracking Video Face Detectio n Head tracking Spurious Face Eliminatio n Annot ation 2 Annot ation 1 Annot ation 3 Head Pose Estimatio n 16

Video Analysis –Head position displacement –Template changes little between successive frames 17 Head Tracking Template Matching Approach -- A Bayesian Estimation – Search range (Prior) – Template matching (max. likelihood)

24 challenging-quality video clips are collected. 18 Experiment Data

Experiment Evaluation Criteria Precision = TP/(TP + FP) –(a.k.a. positive predicted value) –It measures fraction of positively identified driver’s head positions being true (correctness). Recall = TP/(TP + FN) –(a.k.a. sensitivity) –It measures fraction of driver’s head positions detected out of all true driver’s head positions (Completeness). 19

Experiment Results P GT HeadNot Head Head Not Head 2391 P GT HeadNot Head Head Not Head 0114 P GT HeadNot Head Head Not Head 0114 Face Detection Spurious Face Elimination Head Tracking Precision: 99.27% Recall: 28.09% Precision: 100% Recall: 31.50% Precision: 100% Recall: 88.24% Confusion Matrix of selected 24 challenging-quality video clips. P is for prediction, GT is for ground truth. 20

Conclusion Our algorithm handles most of challenges of the NDS video. Computationally efficient for statistical inference in a batch mode for video clips with variable quality. This algorithm is new and outstanding to handle big video databases. Further work: improve recall by modified template matching. 21

22 Thank you!