Audio-based Emotion Recognition for Advanced Information Retrieval in Judicial Domain ICT4JUSTICE 2008 – Thessaloniki,October 24 G. Arosio, E. Fersini, E. Messina, F. Archetti Dipartimento di Informatica, Sistemistica e Comunicazione Università degli Studi di Milano-Bicocca
Affective Computing Learning the emotional state of a human being Learning from: Vocal signals Facial expressions Biometric signals Multimodal sources Applications Games (personal robots) Call centers Automotive JUMAS: Emotion Recognition in Judicial Domain for Semantic Retrieval Emotion Recognition
JUMAS Project Audio&Video Document Current Scenario Manual Transcription Manual Retrieval Manual Information Extraction Automatic Recording Manual Retrieval Manual Information Extraction Automatic Recording Audio Stream Analogical / Digital Acquisition Video Stream Future Scenario Audio&Video Document Digital Acquisition Automatic Audio Transcription Automatic Audio&Video Annotation Automatic Information Extraction Automatic Semantic Retrieval Audio&Video Stream Emotion Annotation for
Neutral Fear Emotion Recognition Output: XML Searchable Tags Neutral
Challenges: What features are able to describe and discriminate different emotional states? Which kind of environment influences emotional state recognition? Which kind of learning models produces the optimal performance? Emotion Recognition
Italian DB: 391 samples Sentences from movies 5 emotional states: Anger Happiness Sadness Neutral Fear Step 1 – Vocal Signature Acquisition Emotion Recognition from vocal signatures German DB: 531 samples Acted sentences: emotion on request 7 emotional states Anger Fear Happiness Sadness Neutral Disgust Boredom
Preliminary Experimental Results Flat Models Learning Models are biased by: Language Gender Neutral emotional state
Multi-Layer Support Vector Machines Hierarchical Classification: Multi-Layer Support Vector Machines
Experimental Results Consclusion Multi-layer SVMs outperform traditional learning techniques Fututre Work Dynamic Techniques Integration with Semantic Information Retrieval System Cooperation with Deception Recognition Flat vs Multi-Layer Multi-Layer Flat German DB Italian DB