AHED Automatic Human Emotion Detection

I am Retshidisitswe Lehata Hello! I am Retshidisitswe Lehata Supervisor: Mehrdad Ghaziasgar Co-Supervisor: Reg Dodds

Background Overview of the System Training the System SVM Optimization Feature Optimization Test Parameters Test Dataset Test Results Tools Used References Demo Add optimization

Customer service is a large revenue stream for some companies Background Customer service is a large revenue stream for some companies Not all customers readily express their emotions verbally A dissatisfied customer is more likely to walk away do their business elsewhere than complain

Real time emotion classification Emoji – Dominant emotion Overview of the System Live video stream Real time emotion classification Emoji – Dominant emotion Probability of all 7 emotions

Training the System

Mean Cross-Validation score: 84% SVM Optimization Grid Search: RBF kernel C and Gamma Linear kernel C C – (2-5,215) Gamma - (2-15, 23) Cross-Validation: Stratified K fold 3 Folds Mean Cross-Validation score: 84% Where C determines the extent to which the SVM model should avoid misclassication in the training. Gamma? RBF? Linear?

Accuracy of HOG Parameters Feature Optimization Accuracy of HOG Parameters Cell (8; 8) Block (3; 3) Cell (4; 4) Block (3; 3) OpenCV HOG 86% 88.2% AHED HOG 77.9% 75.7% Cell (8; 8) Block (2; 2) Cell (4; 4) Block (2; 2) 83.8% 82.3%

(0o-180o) Unsigned Gradients Test Parameters Parameter Size Type Input Image Image (56, 56) Pixels HOG Parameters Cell (4, 4) Block (3, 3) Cells Overlap 66.66% Blocks Bins 9 (0o-180o) Unsigned Gradients Feature Vector Size: 11664

40% of CK+ Dataset 136 test cases 83 test subjects 7 emotions Test Dataset 40% of CK+ Dataset 136 test cases 83 test subjects 7 emotions

Different genders and races Test Dataset Sample of CK+ Dataset Different genders and races

Figure: Confusion Matrix Test Results Sample Correct Predictions F1-Score (accuracy) Angry 17 14 78% Disgust 24 22 96% Fear 10 6 71% Happy 27 Neutral 69% Sad 11 9 Surprise 33 32 98% 136 120 88% F1 – Score – accuracy of the predictions Recall – True positive rate Precision – Positive prediction rate Figure: Confusion Matrix

Tools Used

References P. Viola and M. J. Jones. Robust real-time face detection. International Journal of Computer Vision, 2(57):137-154, 2004. N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference, 1:886-893, 2005. Chih-Wei Hsu, Chih-Chung Chang, Chih-Jen Lin, et al. A practical guide to support vector classification. 2003.

Demo Angry Disgust Fear Happy Neutral Sad Surprise