Emotion Recognition from Electromyography and Skin Conductance Arturo Nakasone (University of Tokyo) Helmut Prendinger (National Institute of Informatics, Tokyo) Mitsuru Ishizuka (University of Tokyo) 1. Emotion Recognition - Introduction 6. Architecture of Emotion Recognition Component Embodied Conversational Agents (ECA) are being developed to enhance communication in a natural way between humans and computer applications. In this context, emotions are considered one of the key components to increase the believability of ECAs. By analyzing emotional state inputs, ECAs may be able to adapt their behaviors, allowing users to experience the interaction in a more sensible way. Applications like Affective Gaming are making use of emotion recognition through physiological signal analysis in order to control several aspects of the gaming experience 2. Objective of Research Develop a real time Emotion Recognition Component (ERC) based on the analysis of two physiological signals : Electromyography Skin conductance 3. Experimental Gaming Environment The ERC was integrated to a game where the user plays a card game called “Skip-Bo” against the ECA Max. The perceived emotion from the ERC allows Max to adapt his own emotional behavior expressed by his facial expressions and game play Initialization parameters are provided to control data sampling rates, data file storage and queue sizes for retrieved values. The Device Layer retrieves the data from the Procomp Infiniti unit and store them in separate queues corresponding to each of the sensors attached to the unit. Prompted by the ECA Max, the mean of the current values stored in the queues are calculated and compared to the baselines in order to search for meaningful changes in the valence/arousal space. “happy” “surprised” 7. Emotion Resolution through Bayesian Networks “angry” “reproach” EMG and SC signal values 4. Relation between Emotions and Physiological Signals In his research, P.J. Lang claimed that emotions can be characterized in terms of judged valence (pleasant or unpleasant) and arousal (calm or aroused) The relation between physiological signals and arousal/valence is established due to the activation of the autonomic nervous system when emotions are elicited Meaningful changes in EMG and/or SC are categorized into discrete levels in the Categorization Layer In our network, the value from the categorized skin conductance signal is used to determine arousal directly. Since the value from the categorized electromyography signal cannot completely determine the sign of the valence component, a non-physiological node was introduced to discriminate this value based on the current outcome of the game (i.e. game status). Probability values have been set according to psychophysiology literature. 8. Conclusions and Future Work Skin Conductance (SC) and Electromyography (EMG) have been chosen because of their high reliability. Skin Conductance determines arousal level through linear relation Electromyography has been shown to correlate with negatively valenced emotions Emotions are key components in the development of truly believable ECAs. Even if people do not perceive them as humans, some suspension of disbelief is possible when emotions come into play. Empathic behavior contributes to a better interaction in terms of user experience. In some cases, the use of only two signals may not be enough to properly handle the emotion recognition process. Therefore, other kind of information like gaze and pupil dilation will be included in our ERC to further enhance the emotion recognition network. 5. Issues in Real Time Assessment of Physiological Data Baseline values calculations were performed by inducing an initial relaxation period on the subject of approx. 3 minutes. These values were used for comparison purposes. Properly detection of emotional activity required sampling every 50 milliseconds and using a 5 second window of data values