1. Project 10 Facial Emotion Recognition Based On Mouth Analysis SSIP 08, Vienna 1 http://www.we-hope-project10-will-win.info

2. The Project Objective : To recognize emotional state / expression using mouth information Input: Mouth images (no make-up) Output: Emotional State/ Expression Happy, Neutral, Sad 2 http://www.we-hope-project10-will-win.info

3. The Team 3 Kornél programmer Kornél programmer Péter Web programmer Péter Web programmer Kamal programmer Kamal programmer Naiem researcher Naiem researcher Sofia programmer Sofia programmer http://www.we-hope-project10-will-win.info

4. The Tasks Create facial expressions photographic database Segment the mouth in the input image Use suitable features for expression characterization Design a reliable classifier to distinguish between different mouth expressions 4 http://www.we-hope-project10-will-win.info

5. SSIP Lips database Happy, Neutral and Sad Photos of SSIP students and lecturers Thank you all!!! Happy Neutral Sad 5 http://www.we-hope-project10-will-win.info

6. Mouth Segmentation 6 Input ImageHSV Space - Hue Thresholding Morphological Operations http://www.we-hope-project10-will-win.info

7. Segmentation Results… And Segmentation Problems… 7 http://www.we-hope-project10-will-win.info

8. Lips Features Extraction Detect the leftmost and rightmost lip points Normalize images (rotation, translation and scaling) Calculate features Eccentricity Convex Area Minor Axis Ratio of Upper to Lower Lip 8 http://www.we-hope-project10-will-win.info

9. Expression Classification SVM Classifier Two Stage Classification Mouth Features  ☺ 9 http://www.we-hope-project10-will-win.info

10. Results 1 Differences between different classes were found to be statistically significant (p<0.01) Classification Accuracy Stage 1 (Sad / Not Sad)  88% Stage 2 (Happy/ Neutral)  62% 10 http://www.we-hope-project10-will-win.info

11. Results 2 11 http://www.we-hope-project10-will-win.info

12. Future Work Acquire larger database for training and testing Test different facial expressions (such as anger and disgust) Other classifiers: NN, FIS Conclusion Mouth information is often insufficient for recognizing facial expression / emotional state Other face features such as eyes and eyebrows can contribute in emotional state recognition 12 http://www.we-hope-project10-will-win.info

13. GUI 13 http://www.we-hope-project10-will-win.info

14. References M. Gordan, C. Kotropoulos, I. Pitas, “ Pseudoautomatic Lip Contour Detection Based on Edge Direction Patterns” J. Kim, S. Na, R. Cole, “ Lip Detection Using Confidence-Based Adaptive Thresholding” F. Tang, “ Facial Expression Recognition using AAM and Local Facial Features” M. Pantic, M. Tomc, L. Rothkrantz, “ A Hybrid Approcah to Mouth Features Detection ” 14 http://www.we-hope-project10-will-win.info

15. Thank you for your attention!!! 15 http://www.we-hope-project10-will-win.info