1. Systemizing Emotions: Using Interactive Multimedia as a Teaching Tool
by Ofer Golan and Simon Baron-Cohen
Chapter 10

2. Nonverbal Communication
Fluent social and communication skills are dependent on the ability to infer emotional and other mental states based on:
Facial expressions
Vocal intonation
Context

3. Facial Emotion Recognition
Individuals with autism have more difficulty than neurotypicals or those with cognitive delays in recognizing the six basic emotions expressed in the face:
Happiness
Sadness
Fear
Anger
Surprise
Disgust

4. Facial Emotion Recognition
Individuals with Asperger Syndrome or HFA may recognize the basic emotions, but have trouble with more complex emotions such as:
Embarrassment
Insincerity
Intimacy

5. Processing Faces Differently
Individuals with ASDs tend to process faces by looking at individual features (mouth, nose, or eyes)
Neurotypicals tend to process faces as a whole

6. Processing Facial Emotions Differently
When individuals with ASDs process facial emotions, there is less activation (than with neurotypicals) in regions of brain that process
Faces (fusiform gyrus)
Emotions (amygdala)

7. Vocal Recognition of Emotion
Less research in this area, but several studies have reported a deficit in high-functioning adults with ASDs, compared to neurotypicals
One neuroimaging study showed less intensive/extensive activation in the medial frontal area of the brain (used for judging others’ mental states) during an vocal emotion recognition task

8. Contextual Recognition of Emotion
Individuals w/ ASDs have more difficulty inferring emotional/mental state based on contextual information (i.e., “reading between the lines”)
Neuroimaging study showed reduced activation of the left medial prefrontal cortex during the Strange Stories Test*
*assesses ability to interpret nonliteral statements made by story characters, based on context

9. Multimodal Recognition of Emotion
In real life, we have to interpret and integrate facial, vocal, body language, and contextual information simultaneously
Individuals with ASDs show deficit in this area, as evidenced in studies that use life-like social situations (e.g., movies)

10. Multimodal Recognition of Emotion
Individuals with ASDs may have difficulty because they don’t pick up the correct cues, AND/OR
They don’t integrate the cues into a whole (i.e., weak central coherence), and focus too much on the details
This could be due to under-connectivity among frontal brain regions

11. Systemizing in Autism
Individuals with ASDs show good, and sometimes superior, skills in systemizing
The drive to analyze or build systems in order to understand and predict the behavior of rule-based events

12. Systemizing in Autism
Individuals with ASDs show superior abilities in tasks that require
Searching for detail
Analyzing and manipulating systems
Could high-functioning individuals w/ ASDs use their systemizing skills to compensate for their difficulties recognizing emotions?

13. Early Attempts to Teach Emotion Recognition
Either focused on just the basic emotions, or
Were included as part of social skills group curriculum, but didn’t focus on systemizing training

14. Advantages of Using Computers to Teach Emotion Recognition
Computerized environment is predictable, consistent, and free from social demands
Many individuals w/ ASDs enjoy using computers
Lessons can be repeated over and over until mastery is achieved
Interest and motivation can be maintained through different and individually selected computerized rewards

15. Mind Reading Software Interactive guide to emotions and mental states
Designed by Baron-Cohen, Golan, Wheelwright & Hill (2004)
Teaching tool for emotion recognition for learners with ASDs

16. Mind Reading’s Systemizing Approach
412 emotions and mental states, each defined and demonstrated by:
6 silent films of faces
6 voice recordings
6 written examples
18 x 412 = 7,416 units of emotion info
Grouped by:
24 emotion groups
6 developmental levels (ages 4-adult)

17. Emotions Database Applications
Emotions Library
Allows users to freely browse, compare, and add their own notes
Learning Center
Structured approach to lessons, quizzes, rewards
Game Zone
5 educational games to play while studying emotions

18. Controlled Study of Adults Using Mind Reading Software
3 groups matched on age, verbal and performance IQ, handedness, and gender
Intervention Group: 19 adults with AS or HFA each used software at home for hours over 10 weeks
AS/HFA Control Group: 22 adults participated only in pre- and post- assessments, with no intervention
Typical Control Group: 28 participants

19. Intervention Group
Allowed to use Emotions Library and Learning Center as they wished, but told to limit games to no more than 1/3 of usage time
Encouraged to analyze facial and vocal stimuli systematically
e.g., Distinguish “insincere” from “interested” by looking not just at smiling mouth but at eyes as well

20. Findings
Intervention group significantly improved their ability to recognize complex emotions/mental states from both faces and voices
Improvement was limited to taught stimuli, (i.e., faces and voices taken from Mind Reading)

21. Generalization Issues
Participants found it difficult to generalize to other tasks of emotion recognition from voices and eyes
No improvement on integration of facial, vocal, and contextual cues
Software usage time was correlated with higher film task scores, so perhaps using software longer would increase generalization

22. Qualitative Data
Participants reported paying greater attention to faces and emotions, as well as improved eye contact
A year later, participants who continued to use Mind Reading reported having improved friendship skills

23. Weak Generalizing or Strong Systemizing?
Authors’ view is that reduced generalization is not so much a deficit as a reflection of the strong drive to systemize in people with ASDs
But socio-emotional world is flexible, context-dependent, and unpredictable, making it difficult--if not impossible--to systemize

24. Potential Problem
Attempting to systematically teach about emotions to individuals with ASDs may lead to rigid application of the system, and eventually to limited generalization

25. Mind Reading’s Role in an Intervention Plan
Authors recommend that Mind Reading be viewed as a first step in a training program
Examples demonstrate what emotions look like and sound like
Next steps should deal with context and integration of socio-emotional cues into one (flexible) picture

26. Question for Future Research
When participants improve their emotion recognition by using Mind Reading software, are regions of their brains activating in patterns more like those of neurotypicals?
Or are they using compensatory strategies in other parts of the brain?

27. Conclusions
Complex emotion recognition can be improved over a relatively brief length of time using Mind Reading’s systemizing approach, but …
Additional methods are required to improve generalization.