Charles Spence Department of Experimental Psychology, Oxford University New Perspectives from the Human Sciences Consumer Focus Workshop (November, 2001)

The Execution of Lady Jane Grey

We are aware of only a small amount of the sensory information available at any one time For example…...

Do people detect changes in ‘real- world’ interactions? Implications for: - eye-witness testimony - head-up displays in cars & planes

Wickens (1980, 1984, 1988, ) Structure of human processing resources

Same-Side Condition Different-Sides Condition Relevant Visual Irrelevant AuditoryRelevant Auditory Irrelevant Visual Relevant Visual Relevant AuditoryIrrelevant Auditory Irrelevant Visual

Shadowing Performance 1) Lip-reading facilitates shadowing 2) Better performance when auditory & visual information from same location Same Side Different Sides 1) 2) % Correct

Don’t Dial & Drive?

Spence & Reid (2000)

Ergonomics: Design based on Human Capabilities- Multisensory Warning Signals

In the Realm of the Senses

Interactions between smell & vision Speeded left/right response. Menthol to either nostril. Olfactory & visual stimuli from same locations. Cuing (80% valid, 20% invalid).

Spence, Kettenmann, Kobal, & McGlone (2000, 2001a, b) Attending to smell affects vision Implications for: advertising, alerting, & pain management Target Modality

Multisensory Perception Most products are multimodal: Skincare products have smell, colour, tactile qualities Foodstuffs: taste, colour, & smell It’s important to know what aspects of a product people attend to, to determine what they will perceive

Multisensory Perception The McGurk effect

Crossmodal Illusions - McGurk effect McGurk effect shows that what we see alters what we hear Manipulating information in one modality allows us to change perception in other modalities predictably

Vision can also dominate taste DuBose et al. (1980) Cherry-flavoured carbonated drinks perceived as ‘orange-flavoured’ when coloured orange

Sound dominates touch ‘Parchment Skin’ Illusion Jousmaki & Hari (1998) People estimated skin roughness & wetness, while rubbing hands together Sound presented over headphones, & frequency content manipulated Changing sound dramatically altered perception of skin roughness

Changing perception of skin through sound? Dry Hydrated ?

1) Selective use of subjects 2) Results rely on subjective report, & hence open to task demands Solution: Develop experimental paradigms that rely on more objective measures (eg psychophysics, neuroimaging) But, there are problems...

Changing perception of products through sound?

Superadditivity: Neurophysiology Stein & Meredith (1993)

Multisensory Integration “The integration of inputs from different sensory modalities not only transforms some of their individual characteristics, but does so in ways that can enhance the quality of life” “Integrated sensory inputs produce far richer experiences than would be predicted from their simple coexistence or the linear sum of their individual products” Stein & Meredith (1993)

Superadditivity: Taste & Smell (Dalton et al., 2000)

Cognitive Neuroscience Orbitofrontal Cortex (Reward) Neutral TouchPleasant Touch

Improving the quality of life for the physically challenged / brain damaged

Conclusions Attention & cognition critically determine perception & behavior A better understanding of multisensory interactions will enhance quality-of-life Converging methodologies approach provides powerful & increasingly subtle tools to understand both brain function & behavior (consumer science)