1. Attention and emotion: From data to conceptual issues Luiz Pessoa Department of Psychology University of Maryland, College Park

2. Background 1990s: work showing limitations of visual processing and the need for attention  Change blindness  Attentional blink

3. Background Processing of emotion-laden information is prioritized  Independent of awareness R L Morris et al. (1998) Whalen et al. (1998)

4. LeDoux “Automatic” Processing Amygdala

5. Research goal Understand the role of attention and awareness during the processing of emotional visual items  Employ strong attentional manipulations  Evaluate awareness with Signal Detection Theory

6. Role of spatial attention Is activity evoked by emotional faces automatic? OR Does activity evoked by emotional faces require attention?

7. 200 ms Same/different Difficult: 64% correct Not drawn to scale 200 ms Attended Faces Unattended Faces Spatial attention Male/female Easy: 91% correct

8. Attention is required for the expression of valence (N = 21) Strong valence X attention interaction: Effect of valence depends on attention Fear UNATT Happy UNATT Neutral UNATT Happy ATT Neutral ATT Fear ATT -0.1 024681012 -0.05 0 0.05 0.1 0.15 0.2 0.25 Right Amygdala Response Amplitude Seconds Pessoa et al. (2002): PNAS X LR

9. Emotional perception requires attention Attention parametrically manipulated within the same task Affective significance enhanced via conditioning Lim et al. (2008): Neuropsychologia Task: find X EasyHard Pessoa et al. (2005): Neuroimage Hsu and Pessoa (2007): Neuropsychologia

10. Role of visual awareness Pessoa et al. (2005): Emotion Target Mask Yes/No 1-3 scale Confidence?

11. 67 ms 33 ms Visual awareness: Signal Detection

12. 67 ms Y = -4 AWARE LR 33 ms Y = -4 UNAWARE LR Amygdala responses VS. Pessoa et al. (2006): Cerebral Cortex

13. Many participants can detect fearful faces even at 17 ms Behavioral results: Individual differences Szczepanowski and Pessoa et al. (2007): Journal of Vision 17 ms

14. Fear stimulus > Neutral stimulus Amygdala Pessoa et al. (2006): Cerebral Cortex 67 ms “Normals” N = 19 Y = -4 AWARE LR “Detecters” N = 8 Y = -6 AWARE LR 33 ms Y = -4 UNAWARE LR Y = -6 AWARE LR x.

15. Role of temporal attention/awareness... 100 ms Lim, Padmala, and Pessoa (2009): PNAS 2 s T2 T1 CS+ vs. CS–

16. Attentional blink: Behavior (N = 30) Enhanced perception of CS+: Reduced blink T1 T2 CS+ CS– Building or House?

17. Role of attention/awareness... Parahippocampal gyrus T2

18. Miss trials... T2 Parahippocampal gyrus

19. Role of attention/awareness Miss trials: no differences observed between CS+ and CS- trials Visual ctx % signal change Amygdala Time % signal change CS+ CS-

20. Conceptual issues

21. Impasse While a great deal has been learned about the extent and limits of affective visual processing, two camps have opposing and entrenched views

22. Impasse While a great deal has been learned about the extent and limits of affective visual processing, two camps have opposing and entrenched views Capacity-limited Capacity-Unlimited

23. Impasse While a great deal has been learned about the extent and limits of affective visual processing, two camps have opposing and entrenched views Capacity-limited Capacity-Unlimited

24. Not too surprising… Emotional stimuli are sufficiently potent that they exhibit a host of properties that do not appear to occur with neutral items  They are processed when unattended Affective processing is subject to capacity limitations, as revealed by several experimental manipulations  Attentional blink

25. Impasse: will it go away? Advocates of limited processing can claim that processing resources have not been consumed  “If the manipulation were stronger, the impact of affective items would go away…”

26. Impasse Showing that the emotional effect has disappeared is always subject to the “null problem”  Arguing for the absence of an effect

27. Power vs. strength of manipulation Fear unatt Happy unatt Neutral unatt Happy att Neutral att Fear att 024681012 Right Amygdala Seconds Response Amplitude Left Amygdala 024681012 -0.1 -0.05 0 0.05 0.1 0.15 0.2 0.25 Seconds Pessoa et al. (2002): PNAS X

28. processing resources performance Processing resources Given the limited capacity of mental processes, performance is impaired if demands are greater than available resources Easy/efficient Hard/inefficient Norman and Bobrow (1975)

29. Processing resources processing resources performance processing resources performance “automatic” Nakayama and Joseph (1998)

30. Processing resources processing resources performance processing resources performance Capacity limitation Dual-tasks Nakayama and Joseph (1998)

31. Processing resources Moors and De Houwer (2006): Every process is uncontrolled, efficient, unconcious, and fast

32. Processing resources Moors and De Houwer (2006): Every process is uncontrolled, efficient, unconcious, and fast, to some degree… Relative to what? Affective processing: relative to neutral not enough  Fine comparisons needed (e.g., abrupt onsets, search, etc.)  Broad set of comparison tasks

33. Two camps Capacity-limited Capacity-Unlimited

34. Preattentive-attentive model t1t1 t2t2 Stage 1: Preattentive Stage 2: Attentive boundary

35. Preattentive-attentive model Some features are processed pre-attentively in virtue of the fact that they are optimally matched to properties of the early visual system (e.g., orientation) Affective processing: Sub-cortical pathway  Superior colliculus  pulvinar  amygdala

36. Dynamic model Multiple interactive “stages”

37. Dynamic model Processing is not pre-attentive or attentive, but a gradient of processing efficiency is hypothesized to exist Gradient based on the properties of early visual areas But critically, gradient is dynamically configured based on task demands Configuring is suggested to depend on parietal and frontal cortex

38. Dynamic model Multiple “gates” Variable permeability Less susceptible to capacity limitations More susceptible to capacity limitations “bottlenecks”

39. Dynamic model Hierarchical and “short-cut” connections

40. Multiple waves Initial processing of visual information proceeds simultaneously along parallel channels “Multiple waves” of activation across visual cortex and beyond The multiple waves are engaged dynamically based on task requirements

41. Subcortical processing

42. Pessoa and Adolphs, Nat. Rev. Neurosci (2010) “passive” “integrative”

43. Subcortical processing Pessoa and Adolphs, Nat. Rev. Neurosci (2010)

44. Processing architecture and attention Task 1 Task 2

45. Collaborators  Ralph Adolphs  Jan Engelmann  Shruti Japee  Shen-Mou Hsu  Seung-Lark Lim  Srikanth Padmala  Remik Szczepanowski  Leslie Ungerleider National Institute of Mental Health emotioncognition.org