1. The Development of Emotional Interactions Across the Senses:
Interactions between visual and auditory emotional information in children versus adults
Keri Swenson, Hiu Mei Chow, Sarah Izen and Vivian Ciaramitaro
Psychology Department, University of Massachusetts, Boston, MA
INTRODUCTION
EXPECTED RESULTS
RESULTS – PSE shift
Our senses allow us to receive a wealth of information. In a social environment, voices and faces act as primary means of communicating social information (Ghazanfar, 2005). Together, multisensory input from visual and auditory information present a cross-modal interaction that should influence affective facial perception. Known as perceptual aftereffects, visual or auditory adaptation to one affective category (e.g. angry) will cause subsequent ambiguous facial affect to appear opposite from the adapted category (e.g. happy). Though this effect is well studied in adults, less is known about its development. Specifically, there is a discrepancy in the literature as to whether multisensory integration is innate at birth or modified by experience (Brandwein et. al, 2011). In our study, we examine audiovisual affective correspondence in participants aged We hypothesize that (1) the strength of adaptation in both children and adults will be stronger for congruent (matching) vs. incongruent (non-matching) affect and 2) adaptation to happy affective stimuli will be more robust across conditions for both children and adults.
Adapt Happy
Adapt Happy
Appears Happy (%)
100
Baseline
Post- Adapt
PSE at baseline is a 0% morph. After adaptation, the same morph appears 25% angrier. 50
}Effect
Adaptation % 10 20
40 80
Face Morphs (% emotion)
METHOD: AFFECTIVE ADAPTATION
In our baseline condition, participants judged a series of faces on a morphed emotional continuum as happy or angry via button press, in a two alternative forced choice paradigm. From this data we establish each participant’s point of subjective equality (PSE). In the subsequent adaptation condition, participants were adapted to one of four possible affective conditions: Congruent Happy, Incongruent Happy, Congruent Angry, Incongruent Angry. After adaptation, participants again judge the morphed faces (24 trials). Adaptation is quantified as the change in PSE prior to, and after, adaptation.
Adapt Angry
Adapt Angry
Place conditions table here
24 trials
24 trials
CONCLUSIONS
Visual Stimuli All face stimuli were selected from the NimStim database (Tottenham et al.). For adaptation 30 unique faces (15 female, 15 male) were displayed, either 100% happy or angry. MorphMan software was used to create probe face images included a subset of 4 unique adaptation faces (2 female, 2 male) depicted along an emotional continuum of angry  to neutral to happy,  including 80%,  40%,  20%  10%  or 0%  of a  given  emotion.
RESULTS – Demographics
Title here
Adapt Happy
Summary of findings: These findings show interesting results between the adults and children.
Congruency effect: In adults, the strength of adaptation is similar when congruent vs. incongruent affect whereas in children, the strength of adaptation is stronger in the congruent vs. incongruent in the adapt happy condition; the opposite is true in the adapt angry condition.
Emotion effect: In both adults and children, adaptation strength is similar across emotions. In happy affective adaptation, we see a positive shift; in angry affective adaptation, we see a negative shift .
Given these differences, our findings support that multisensory integration and the perceptual processing of emotional information are processes that develop through childhood and adolescence into adulthood.
Adapt Happy
5 Male,
5 Female,
2 Unknown
Congruent
4 Male,
8 Female
4 Male,
3 Female
2 Male, 3Female
Incongruent
Angry 80%
40%
10%
10%
40%
80% Happy
Auditory Stimuli
Auditory adaptation included 30 (15 positive and 15 negative) unique crowd sounds.
Participants
Adapt Angry
This research was supported in part by the UMass Boston Dean’s Office and an RTF grant from the UMass Boston Psychology Department.
7 Male,
3 Female,
4 Unknown
Congruent
10 Male, 4 Female
3 Male,
4 Female
1 Unknown
2 Male,
4 Female
REFERENCES
26 children (6-12 years of age) and 52 adults ( years of age) were recruited from the Living Laboratory Museum of Science in Boston, Massachusetts. Before presenting the task to a participant, we first gathered consent or assent and measured their present emotional state using adult and child versions of the Positive and Negative Affect Schedule (PANAS). Before debriefing, we again presented the PANAS to capture the participants’ post affective measure.
Incongruent
Dionne-Dostie, E., Paquette, N., Lassonde, M., & Gallagher, A. (2015). Multisensory Integration and Child Neurodevelopment. Brain Sciences. doi: /brainsci
Ghazanfar, Maier, Hoffman, & Logothetis. (2005). Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex. Journal of Neuroscience. Retrieved from
Adult
Children