PROGRESS ON EMOTION RECOGNITION J G Taylor & N Fragopanagos King’s College London.

Slides:

Advertisements

Similar presentations

Applications of one-class classification

Advertisements

Answering Approximate Queries over Autonomous Web Databases Xiangfu Meng, Z. M. Ma, and Li Yan College of Information Science and Engineering, Northeastern.

National Technical University of Athens Department of Electrical and Computer Engineering Image, Video and Multimedia Systems Laboratory

A Computational Model of Emotional Influences on Visual Working Memory Related Neural Activity Nienke Korsten Nikos Fragopanagos John Taylor OFC DLPFC.

Pseudo-Relevance Feedback For Multimedia Retrieval By Rong Yan, Alexander G. and Rong Jin Mwangi S. Kariuki

Cognitive Systems, ICANN panel, Q1 What is machine intelligence, as beyond pattern matching, classification and prediction. What is machine intelligence,

Active Appearance Models

Centre for Design Research © 2008 Benedict Singleton and Dr. Kev Hilton The Emotional Spectrum Analyser.

Punctuation Generation Inspired Linguistic Features For Mandarin Prosodic Boundary Prediction CHEN-YU CHIANG, YIH-RU WANG AND SIN-HORNG CHEN 2012 ICASSP.

Data preprocessing before classification In Kennedy et al.: “Solving data mining problems”

Facial expression as an input annotation modality for affective speech-to-speech translation Éva Székely, Zeeshan Ahmed, Ingmar Steiner, Julie Carson-Berndsen.

Speech Group INRIA Lorraine

Modeling Human Reasoning About Meta-Information Presented By: Scott Langevin Jingsong Wang.

A saliency map model explains the effects of random variations along irrelevant dimensions in texture segmentation and visual search Li Zhaoping, University.

1 Learning to Detect Objects in Images via a Sparse, Part-Based Representation S. Agarwal, A. Awan and D. Roth IEEE Transactions on Pattern Analysis and.

Un Supervised Learning & Self Organizing Maps Learning From Examples

CONTENT BASED FACE RECOGNITION Ankur Jain 01D05007 Pranshu Sharma Prashant Baronia 01D05005 Swapnil Zarekar 01D05001 Under the guidance of Prof.

Multimodal emotion recognition recognition models- application dependency –discrete / dimensional / appraisal theory models theoretical models of multimodal.

VESTEL database realistic telephone speech corpus:  PRNOK5TR: 5810 utterances in the training set  PERFDV: 2502 utterances in testing set 1 (vocabulary.

Visual Expertise Is a General Skill Maki Sugimoto University of California, San Diego November 20, 2000.

Smart Traveller with Visual Translator for OCR and Face Recognition LYU0203 FYP.

Latent Semantic Analysis (LSA). Introduction to LSA Learning Model Uses Singular Value Decomposition (SVD) to simulate human learning of word and passage.

HTA as a framework for task analysis Presenter: Hilary Ince, University of Idaho.

Face Recognition Using Neural Networks Presented By: Hadis Mohseni Leila Taghavi Atefeh Mirsafian.

TOWARDS A CONTROL THEORY OF ATTENTION by John Taylor Department of Mathematics King’s College London, UK s:

Slide Image Retrieval: A Preliminary Study Guo Min Liew and Min-Yen Kan National University of Singapore Web IR / NLP Group (WING)

Comparison of Boosting and Partial Least Squares Techniques for Real-time Pattern Recognition of Brain Activation in Functional Magnetic Resonance Imaging.

GUI: Specifying Complete User Interaction Soft computing Laboratory Yonsei University October 25, 2004.

Presentation on Neural Networks.. Basics Of Neural Networks Neural networks refers to a connectionist model that simulates the biophysical information.

Recognition of meeting actions using information obtained from different modalities Natasa Jovanovic TKI University of Twente.

Kumar Srijan ( ) Syed Ahsan( ). Problem Statement To create a Neural Networks based multiclass object classifier which can do rotation,

Chapter 9 Neural Network.

Analysis of a Neural Language Model Eric Doi CS 152: Neural Networks Harvey Mudd College.

Eric H. Huang, Richard Socher, Christopher D. Manning, Andrew Y. Ng Computer Science Department, Stanford University, Stanford, CA 94305, USA ImprovingWord.

Multimodal Information Analysis for Emotion Recognition

Methodology of Simulations n CS/PY 399 Lecture Presentation # 19 n February 21, 2001 n Mount Union College.

EE459 Neural Networks Examples of using Neural Networks Kasin Prakobwaitayakit Department of Electrical Engineering Chiangmai University.

Designing multiple biometric systems: Measure of ensemble effectiveness Allen Tang NTUIM.

Cognition Through Imagination and Affect Murray Shanahan Imperial College London Department of Computing.

Multimodality, universals, natural interaction… and some other stories… Kostas Karpouzis & Stefanos Kollias ICCS/NTUA HUMAINE WP4.

Lexical Affect Sensing: Are Affect Dictionaries Necessary to Analyze Affect? Alexander Osherenko, Elisabeth André University of Augsburg.

State Equations BIOE Processes A process transforms input to output States are variables internal to the process that determine how this transformation.

ENTERFACE’08 Multimodal Communication with Robots and Virtual Agents mid-term presentation.

WP6 Emotion in Interaction Embodied Conversational Agents WP6 core task: describe an interactive ECA system with capabilities beyond those of present day.

Deep Belief Network Training Same greedy layer-wise approach First train lowest RBM (h 0 – h 1 ) using RBM update algorithm (note h 0 is x) Freeze weights.

Neural networks (2) Reminder Avoiding overfitting Deep neural network Brief summary of supervised learning methods.

Interpreting Ambiguous Emotional Expressions Speech Analysis and Interpretation Laboratory ACII 2009.

The problem. Psychologically plausible ways of

Today’s Lecture Neural networks Training

Compact Bilinear Pooling

making certain the uncertainties

Final Year Project Presentation --- Magic Paint Face

Hu Li Moments for Low Resolution Thermal Face Recognition

Neuropsychology of Vision Anthony Cate April 19, 2001

Lecture 22 Clustering (3).

Neural Networks Advantages Criticism

Neural Networks and Their Application in the Fields of Coporate Finance By Eric Séverin Hanna Viinikainen.

Capabilities of Threshold Neurons

Family History Technology Workshop

Phonological Priming and Lexical Access in Spoken Word Recognition

Math I Quarter I Standards

Face Detection Gender Recognition 1 1 (19) 1 (1)

Computer Vision Lecture 19: Object Recognition III

Deep Learning Authors: Yann LeCun, Yoshua Bengio, Geoffrey Hinton

Human-object interaction

Word representations David Kauchak CS158 – Fall 2016.

Lecture 16. Classification (II): Practical Considerations

Bug Localization with Combination of Deep Learning and Information Retrieval A. N. Lam et al. International Conference on Program Comprehension 2017.

Week 7 Presentation Ngoc Ta Aidean Sharghi

Data-Driven Approach to Synthesizing Facial Animation Using Motion Capture Ioannis Fermanis Liu Zhaopeng

Presentation transcript:

PROGRESS ON EMOTION RECOGNITION J G Taylor & N Fragopanagos King’s College London

KCL WORK IN ERMIS Analysis of emotion v cognition in human brain ( →simulations of emotion/attention paradigms) → emotion recognition architecture ANNA ANNA hidden layer = emotion state, + feedback control for attention (= IMC) Learning laws for ANNA developed ANNA fuses all modalities or only one HUMAINE: WP3 +WP4

BASIC BRAIN EMOTION CIRCUIT Valence in amygdala & OBFC Attention in parietal & PFC Interaction in ACG SC Parietal A Thal ACG SFG NBM

SIMPLIFIED ARCHITECTURE OF EMOTIONAL/COGNITIVE PROCESSING IN THE BRAIN:

DETAILED ARCHITECTURE FOR FACES CLASSIFICATION gender

BASIC ERMIS EMOTION RECOGNITION ARCHITECTURE: Attention control system: Feature vector Inputs: Emotion state as hidden layer Output as recognised emotional state ↑ →

ANNA:Assume linear output: Hidden layer response: IMC node response: Then solve self- consistent equations for (y, z) for each training input by relaxation

NATURE OF ANNA Handles both unimodal and multimodal data (input vector x of arbitrary dimension, not too large) Needs consistent input and output data {x(t), OUT(t)}, with t specified for both x & OUT=(activat, evaluat) Uses SALAS date-base (450 tunes) from QUB (Roddie/Ellie/Cate)

UNIMODAL RESULTS Can use numerous representations of emotion: extreme, continuous in n dimensions, … ANNA → FEELTRACE output (continuous 2-D) Trained on unimodal for prosody First look at word content

Text Post-Processing Module Prof. Whissell compiled ‘Dictionary of Affect in Language (DAL)’ Mapping of ~9000 words → (activation- evaluation), based on students’ assessment Take words from meaningful segments obtained by pause detection → (activation-evaluation) space But humans use context to assign emotional content to words

Text Post-Processing Module (SALAS data) Table 1. Quadrant match for normal text (full DAL). ParticipantP1P2P3P4P9P12All Quadrant match (%) Table 2. Quadrant match for scrambled text (full DAL). ParticipantP5P6P7P8P10P11All Quadrant match (%) Table 3. Standard deviation of participants’ assessments for normal and scrambled text (average over all passages assessed). NormalScrambled Evaluation Activation Table 4. Quadrant match averaged over participants’ groups for normal text and scrambled text when threshold for DAL range* is varied. Threshold Normal text Scrambled text *The higher the threshold the higher emotionally rated words are spotted only. Conclude: need further context/semantics

Correlational analysis of ASSESS features Correlational analysis between ~450 ASSESS features and FeelTrace => –ASSESS features correlate more highly with activation –Similar top ranking features for 3 out of 4 FeelTracers (but still differences) –Different top ranking features for different SALAS subjects ->Is there a male/female trend? Difficult to say - insufficient data

ANNA on top correlated ASSESS features Quadrant match using top 10 activation features + top 10 evaluation features and activation – evaluation output space: Feeltracerjdccdrem Avg Quad Match Std Dev

ANNA on top correlated ASSESS features Half-plane match using top 10 activation features and activation only output space: Feeltracerjdccdrem Avg Quad Match Std Dev

PRESENT SITUATION OF ANNA: MULTIMODAL Time-stamped data now becoming available for lexical (ILSP) & face streams (NTUA) Expect to have results in about 1 month for recognition for fused modalities (faces/prosody/words)

CONCLUSIONS UNIMODAL: ANNA on prosody OK (especially activation) MULTIMODAL: Soon to be done On semi-realistic data (SALAS QUB) Future work: 1) analysis of detailed results 2) insert temporality in ANNA

QUESTIONS How to handle variations across experiencers and across FEELTRACERS? How to incorporate expert knowledge? How combine recognition across models? Coding of emotions: as dimensional reps or as dissociated states (sad AMYG v angry OBFC)? Nature of emotions as goal/reward assessment (frustration → anger; impossible →sadness, etc: brain-based)?