Processing Facial Emotion

Darwin (1872) argued that facial expression evolved from expressions in other animals and are therefore innate.

Universality of Facial Emotion Ekman & Oster (1979) review evidence for cross-cultural similarities in emotion processing.

Disgust Anger Sadness Happiness Fear Surprise

Literate Cultures Show Distinctive Facial Expression For: Happiness Sadness Disgust Anger Fear ) Preliterate cultures show Surprise ) confusion over fear and surprise

But where might emotion be processed in the brain? Are there separable brain regions for processing each of these different emotions?

Laterality and Emotion Carmon and Nachshon (1973): Left ear advantage for discriminating emotion eliciting sounds (crying, laughing etc.). Damage to RH more likely to result in impaired emotion recognition (Bowers et al., 1985) and loss of prosody in voice (emotional expression). LH generally has positive affect - RH negative affect. Damage to LH leave patients feeling anxious or angry, damage to RH leaves them with flattened affect or indifference toward failure. Emotional processing different across the hemispheres, but perhaps emotion more right than left.

Laterality and Emotion Original Face Sackheim & Gur (1978) showed subjects chimeric faces and asked which ones showed the strongest emotion. Left halves more expressive than right halves (Left half perceived by RH). Left Half Composite Right Half

System for Reward, Motivation, Arousal, Emotion etc. Electrode Implanted in Septum Area The Limbic System Rat presses lever to stimulate the brain - Yeah baby! Rat falls asleep exhausted but happy! System for Reward, Motivation, Arousal, Emotion etc. Rat wakes and immediately starts to press the lever again. Rats will even neglect babies.

The Amygdala There are two routes to the Amygdala: 1: Subcortical Route Via the superior colliculus and pulvinal thalamus. Cortical Route Via the visual cortex. Subliminally presented fearful faces activate the amygdala in normal participants. In fact unattended presentation gives highest level of activation. Blindsight patients and those with hemispatial neglect can discriminate emotional expression.

The Amygdala Lesion studies find impaired recognition of emotional faces when bilateral damage has occurred. This is mostly associated with fear (Calder et al., 1996, Morris et al., 1996), but may also be found in other “negative” emotions such as anger, sadness and disgust (Adolphs et al., 1991; Adolphs, 2002 Cur. Biol). Some have argued that the amygdala is particularly important in processing emotion related to perceived threat. Individuals with bi-lateral lesions also appear to be indiscriminately friendly and have problems judging trust in others. Elderly people also show reduced amygdala activation - they also get conned more by people?

fMRI Evidence A. Comparing Fearful Faces with Neutral Faces shows bi-lateral Amygdala activation. B. Comparing Angry Faces with Neutral Faces also shows Amygdala activation. Whalen et al. (2001) But so does comparing happy faces with neutral faces! (Somerville et al., 2004)

But Why? What is it doing? Heberlein et al. (1998) Impaired attribution of social meanings to abstract dynamic visual patterns following damage to the amygdala.

The Amygdala as an Ambiguity Detector Whalen et al. (2001) argue that the pattern of activation found for the amygdala may not reflect the processing of negative emotion/threat per se, but more about detecting ambiguity in the face and how that relates to predictibility of the persons actions. Both convey perceived threat but ANGRY face conveys information about the source of the threat.

Are you looking at me? Emotion and Gaze Information Ambiguous Who’s he angry at? Unambiguous He’s angry at me!

Something threatening to my left! Emotion and Gaze Information Are you looking at me? Ambiguous Where is the threat? Unambiguous Something threatening to my left!

Emotion and Gaze Information Are you looking at me? Ambiguous

Summary of Amygdala Function in Emotion Recognition Part of rapid system for processing highly salient stimuli. Responds most to autonomic/unconscious stimuli. More responsive to fearful than angry faces. Also active for ascribing social meaning to dynamic representations. May actually be important in prediction of threat in ambiguous situations (Whalen et al., 2001; Adams et al., 2003).

The Orbito-Frontal Cortex (OFC) and Negative Emotion OFC unlike amygdala responds more to explicitly encoded aversive stimuli perhaps with a view to generating a response to them. Responds more to fear than happiness (Kawasaki et al., 2001) and to anger over sadness (Blair et al., 1999) Response latency in OFC 120-150ms. Thus OFC like amygdala can make rapid responses to emotionally salient stimuli. Can modulate early aspects of perception

The Insular Cortex & Basal Ganglia and Disgust The anterior insula is connected to the ventro–posterior–medial thalamic nucleus, and has been identified in primates as gustatory cortex (Rolls, 1994), containing neurons that respond to pleasant and unpleasant tastes (Yaxley et al., 1988).

The Insular Cortex & Basal Ganglia and Disgust Calder et al. (2000) report a patient with damage to left insula and left basal ganglia who was impaired at recognizing disgust in facial expression and in the ability to experience disgust himself. Adolphs et al. (2002) report a patient with bi-lateral insula damage who was severely impaired at experiencing and recognizing disgust from all types of stimuli. Insula Basal Ganglia

The Insular Cortex & Basal Ganglia and Disgust 75% 150% 150% > 75% activates right anterior insula Phillips et al. (1997)

The Insular Cortex & Basal Ganglia and Disgust Diseases that specifically attack the basal ganglia such as Huntingdon’s Disease lead to disproportionate impairment in disgust processing (Sprengelmeyer et al., 1996)

The Insular Cortex & Basal Ganglia and Disgust Summary Recognition of disgust appears to occur in a region associated with experiencing the feeling of disgust.

Somatosensory Cortex Lundqvist & Dimberg (1995) Recorded facial muscle movements while participants viewed emotional faces. They noted that the muscles used to produce facial expressions were activated simply by viewing them. Thus recognition may occur from simulation of the emotion by the perceiver.

Somatosensory Cortex Do we therefore recognize facial emotions in others by simulating them ourselves?

Somatosensory Cortex The Simulation Hypothesis Support from other literatures…. Rizzolatti and colleagues (Gallese et al., 1996; Rizzolatti et al., 1996) report a “mirror system” in premotor cortex.

When observing the actions of others, brain activity is also observed in the motor cortex of the perceiver. Thus we simulate the movements of others. Referred to as the “Mirror System” or “Mirror Neurons.

In emotion there is also evidence of a causal link between the production of emotional expressions and changes in emotional state (Cacioppo et al., 1992). Thinking happy thoughts can also enable you to fly! (JM Barrie) Viewing the facial expressions of others can lead to changes in the emotional experience of the perceiver (Wild et al., 2001).

So do we simulate to recognize emotion in others?

Möbius Syndrome Rare congenital disorder. Infant has problems feeding and exhibits a mask-like expression. Diagnosis reserved for palsies of the VIth and VIIth cranial nerves and skeletal defects (missing fingers, toes, club foot). Also problems with eye movements in horizontal plane.

Möbius Syndrome

Möbius Syndrome So can Möbius individuals perceive emotions in others in the absence of simulation themselves? Giannini et al. (1984). MS individual asked to watch video of face of someone playing a slot machine and to guess what value “jackpot” they were playing for. Found that MS was unable to perform this task. Also informed experimenters that she could not interpret facial expression.

Möbius Syndrome Calder et al. (2000) tested 3 MS individuals and found no impairment in recognizing basic types of facial affect (anger, fear, sad, happy, disgust). Did show mild impairment in face recognition - may be due to problems with eye movements. Suggests that simulation not necessary for facial emotion recognition. Damasio (1994, 1999) argues that one can bypass the generation stage of producing the expression and utilize the direct modulation of somatic (feeling) structures to simulate.

The Time Course and Network for Emotion Processing Core system engaged in person perception (IOG, STS, FG). Processing in areas like FG can take ~170ms (N170). But… Discrimination of emotion can occur at ~80 to 110ms (Pizzagali et al., 1999; Halgren et al., 2000). Amygdala responds to emotion at ~120ms (Halgren et al., 1994), but activation is reduced when subjects asked to report the emotion exhibited by the face. Suggestion that activation in amygdala inhibited by the frontal cortices here. OFC: Damage affects emotion processing. OFC activation occurs ~150ms (Marinkovic et al., (2000). OFC activation also found ~120ms (Kawasaki et al., 2001) so perhaps OFC can respond rapidly to salient stimuli also.

Different Neural Systems At Different Time Points: Amygdala & OFC Early processing of salient stimuli. Subcortical route feed-back to OC. Bottom-Up Process Detailed perception through Core system and emotional reaction (OFC). Interaction between visual and somatosensory areas in recognition of facial emotion, and possible simulation. Top-Down Process Adolphs (2002)

Summary Multiple distinct brain areas involved in the perception and recognition of emotion and in the resultant observer expression of emotion. Different systems reflect different requirement for information extraction at different times, and bottom-up vs. top-down processes. Perception-Reaction-Conceptual Processes

Recommended Reading Adams R (2003) Science, 300. Adolphs R (2002) Current Opinion in Neurobiology, 12. Adolphs R (2002) Beh. & Cog. Neuro. Revi., 1. Adolphs R (2003) Nat. Reviews. Neuro., 4. Calder et al. (2000) Cognitive Neuropsychology, 17. Kawasaki et al. (2001) Nature Neuro., 4. Phillips et al. (1997) Nature, 389. Whalen PJ et al (2001) Emotion, 1. Whalen et al. (1998) Curr. Dir. Psych. Science, 7.