Under Guidance of Mr. A. S. Jalal Associate Professor Dept. of Computer Engineering and Applications GLA University, Mathura Presented by Dev Drume Agrawal M.Tech. (CSE) GLA University Mathura

Content: Abstract Introduction Motivation Literature Review Issues & Challenges Objective and Proposed Framework Summary References

INTRODUCTION  FERS is an automated system, works based on training data.  It is very useful tool for human and machine interaction.  The FERS works on: Image Sequence of Image (or a segment of video)  It works on 2D images.

Introduction (Cont….) The FERS recognize six basic emotions [1] represented as ‘universal facial expressions’ by Ekman, et al [2]. happiness, anger sadness surprise disgust fear Fig 1.1: Six universal emotions (anger, happiness, sadness, surprise, fear and disgust) [f1]

MOTIVATION The FERS is the fast growing tool with different more or less functionalities [3] such as  Real time automated system,  Gesture recognition,  Recognition from video  recognition of facial emotion affected by audio input etc.

Motivation (Cont…) FERS can be used as a tool in various integrated systems which are useful for the human and machine interaction like  Robots  Investigations by intelligence agencies etc.

LITERATURE SURVEY Ekman & Friesen, (1978) [4] A technique “facial action coding system (FACS)” for measuring facial movements in behavioral science, was developed. Ekman & Rosenberg (1997) [5] FACS uses 46 defined action units to correspond into each independent motion of the face

Literature Survey (Cont…) Kwang-Eun Ko & Kwee-Bo Sim, (2009) [6] The advance active appearance model was developed for emotion recognition and used with Bayesian network. Kwang-Eun Ko & Kwee-Bo Sim, (2010) [7] The AAM (Active appearance model) and DBN (Dynamic bayesian network) was combined for emotion recognition.

Literature Survey (Cont…) Yoshihiro Miyakoshi, and Shohei Kato, (2011) [8] A human facial emotion recognition system was developed using Fuzzy logic. Kumar at. el., (2009) [9] A system was developed using bayesian network with partial occluding problem handling.

Literature Survey (Cont…) Jia-Jun Wong & Siu-Yeung Cho, (2009) [10] A model was developed for facial emotion recognition using local expert organization (LEO) technique. B. Fasel & Juergen Luettinb, (2003) [11] This survey paper addresses the techniques and approaches used for facial expression detection.

ISSUES AND CHALLENGES In the all above systems/tools and used algorithms some issues are there, which are the major factor, responsible for inaccuracy in the performance and desired result of the systems. Following are some major issues:  Pose  Illumination  Partial Occlusion  Facial Texture Deformation (Caused by age or physical disability

Issues and Challenges (Cont…) 1. POSE Fig 4.1: Pose variation [f2]

Issues and Challenges (Cont…) 2. ILLUMINATION Fig 4.2: Illumination variation [f3]

Issues and Challenges (Cont…) 3. PARTIAL OCCLUSION Fig 4.3: Partial Occlusion [f4]

Issues and Challenges (Cont…) 4. Facial Texture Deformation (Caused by age or physical disability Fig 4.2: Facial Texture Deformation [f5]

OBJECTIVE AND PROPOSED METHODOLOGY/FRAMEWORK Based on the previous works we are preparing an automated system for emotion recognition of facial images, having three major modules:  Face detection  Facial features extraction  Facial emotion recognition

Development & Implementation Framework Data set Test images Training images Face Detection Image Segmentation Facial Feature Extraction Classification Training By a classifier Capture Image Emotion Fig 5.2: Methodology of the system

REFERENCES [1] [2] P. Ekman, W.V. Friesen, Constants across cultures in the face and emotion, J. Personality Social Psychol. 17 (2) (1971) 124–129. [3] [4] Ekman and W. Friesen, Facial Action Coding System: A Technique for the Measurement of Facial Movement, Consulting Psychologists Press, Palo Alto, [5] Ekman, P., & Rosenberg, E. L. (1997). What the face reveals: Basic and studies of spontaneous expression using the facial action coding system (FACS). New York: Oxford University Press. [6] Kwang-Eun Ko, Kwee-Bo Sim, “Development of Advanced Active Appearance Model for Facial Emotion Recognition”, IEEE International Symposium on Industrial Electronics (ISlE 2009). [7] Kwang-Eun Ko, Kwee-Bo Sim,(2010) “Development of a Facial Emotion Recognition Method based on combining AAM with DBN”, IEEE International Conference on Cyberworlds, 2009.

REFERENCES (Cont…) [8] Yoshihiro M., and Shohei K. (2011) ”Facial Emotion Detection Considering Partial Occlusion of Face Using Bayesian Network”, IEEE Symposium on Computers & Informatics, [9] Kumar U. & Chakraborty A. (2009), “Emotion Recognition From Facial Expressions and Its Control Using Fuzzy Logic”, IEEE Transaction on Systems, Man & Cybernetics- part A: Systems and Humans, Vol. 39 [10] Jia-Jun W. & Siu-Yeung C., (2009)“A local experts organization model with application to face emotion recognition”, Elsevier Transaction on Expert System with Application’, vol. 36, pp [11] Fasel B. & Juergen L.,(2003) “Automatic facial expression analysis: a survey”, Elsevier Transaction on ‘Pattern Recognition’ vol. 36, pp [f1] Jia-Jun W. & Siu-Yeung C., (2009)“A local experts organization model with application to face emotion recognition”, Elsevier Transaction on Expert System with Application’, vol. 36, pp [f2] [f3] [f4] [f5]

THANK YOU