Interactive Activation: Behavioral and Brain Evidence and the Interactive Activation Model PDP Class January 10, 2011

Overview Modular approaches: Marr and Fodor A critique of modular approaches in vision The word superiority effect and the interactive activation model Interactivity in the brain –Anatomy and neurophysiology

Marr’s Modular Approach Divide and conquer approach to vision: –E.g., Marr suggests studying shape from shading, shape from motion, shape from depth as separate computations. –A key motivation is that sometimes just one type of information is enough (crumpled newspaper example). –Forces close attention to just how much can be done with each source of information alone, and a careful consideration of how it might be done. Similar to Fodor’s modular approach in Modularity of Mind. –Fodor assumes that specific input and output systems are encapsulated (insensitive to inputs from other sources) and reflex-like. –The suggestion is that in order for them to be reflex-like they must be very narrow in the range of considerations they take into account. –If in fact the brain were modular this would be nice, for then function would be considerably easier to analyse.

Critique of Fodor’s appeal to ‘Reflexes’ Reflexes are not encapsulated. Instead they involve at least a synapse between the sensory and motor neuron or even an interneuron (as shown). Additional inputs either to the interneuron or the motor neuron can modulate the response. The cat’s paw reflex makes clear that context can modulate reflexes. They are still fast and (pretty) direct but still highly context sensitive. Influence of subsequent context requires more complex mechanisms. Illustration from McGraw Hill Online Lecture on Motor Control.

Critique of the Marr’s Modular Approach in Vision (Bulthoff and Yuille, 1996) Provides no insight into what to do when different modules both provide inconclusive (possibly conflicting) evidence. More generally, it punts on how we successfully integrate multiple sources of information, as we clearly do much of the time. Ignores the fact that one source of information can change the way we use information from another source (see next slide).

Gilcrest, A. L. Perceived lightness depends on perceived spatial arrangement. Science, 1977, 195, Experiment shows that subjects assign a surface a ‘color’ based on which other surfaces they see it as co-planar with. Thus color depends on perceived depth, violating modularity.

Findings Motivating the IA Model The word superiority effect (Reicher, 1969) –Subjects identify letters in words better than single letters or letters in scrambled strings. The pseudoword advantage –The advantage over single letters and scrambled strings extends to pronounceable non- words (e.g. LEAT LOAT…) The contextual enhancement effect –Increasing the duration of the context or of the target letter facilitates correct identification. Reicher’s experiment: –Used pairs of 4-letter words differing by one letter READ ROAD –The ‘critical letter’ is the letter that differs. –Critical letters occur in all four positions. –Same critical letters occur alone or in scrambled strings _E__ _O__ EADR EODR W PW Scr L Percent Correct

READ _E__ O

The Contextual Enhancement Effect

The Interactive Activation Model Feature, letter and word units. Between-layer connections were + or -; only inhibitory connections within. Activation follows the ‘iac’ function. Response selected from the letter units in the cued location according to the Luce choice rule:

How the Model Works: Words vs. Single Letters

Word and Letter Level Activations for Words and Pseudowords Idea of ‘conspiracy effect’ rather than consistency with rules as a basis of performance on ‘regular’ items.

Simulation of Contextual Enhancement Effect

Role of Pronouncability vs. Neighbors Three kinds of pairs: –Pronounceable: SLET-SPET –Unpronouncable/good: SLCT-SPCT –Unpronouncable/bad: XLQJ-XPQJ

Autonomous vs. Interactive Approaches visual or auditory feature level Letter/phoneme identification

Can the Models be Distinguished? Attempts to support IA models seek to demonstrate ‘knock-on’ effects influencing the phoneme level inputs to the word level. –Lexically-Triggered Compensation for Coarticulation –Selective Adaptation  Tuning Phoneme Boundaries Basic logic: –Use context to determine identity of an ambiguous segment –Show that the contextually-determined segment identity triggers a phenomenon that affects phoneme identification (on the way to lexical access) See TiCS paper in readings for details

Tuning Phoneme Boundaries Present ambiguous (s/f) segment in a context where lexical information determines its identity as ‘f’, while presenting normal ‘s’ segments: –Consider / Con(s/f)use Later, test for identification of the ambiguous segment, and identification of words where both interpretations are possible. –(f/s)ear Participants identify ambiguous segment and the word containing it as though they hear it now as an ‘f’. This can be explained by assuming participants use the top-down signal to adjust the connection weights mapping features onto the ‘f’ sound.

Interactivity in the Brain Bidirectional Connectivity Interactions between V5 (MT) and V1/V2: Bullier Subjective Contours in V1: Lee and Nguyen MEG Evidence: Bar et al (2006)

Hupe, James, Payne, Lomber, Girard & Bullier (Nature, 1998, 394, ) Investigated effects of cooling V5 (MT) on neuronal responses in V1, V2, and V3 to a bar on a background grid of lower contrast. MT cooling typically produces a reversible reduction in firing rate to V1/V2/V3 cells’ optimal stimulus (figure) Top down effect is greatest for stimuli of low contrast. If the stimulus is easy to see when it is not moving, top-down influences from MT have little effect. Concept of ‘inverse effectiveness’ arises here and in many other related cases. *

Lee & Nguyen (PNAS, 2001, 98, ) They asked the question: Do V1 neurons participate in the formation of a representation of the illusory contour seen in the upper panel (but not in the lower panel)? They recorded from neurons in V1 tuned to the illusory line segment, and varied the position of the illusory segment with respect to the most responsive position of the neuron.

Response to the illusory contour is found at precisely the expected location.

Temporal Response to Real and Illusory Contours Neuron’s receptive field falls right over the middle of the real or illusory line defining the bottom edge of the square

Feedback loop between OFC and Fusiform Gyrus indicates top-down contribution to object recognition Bar, M., Kassam, K., Ghuman, A., Boshyan, J., Dale, A., Hämäläinen, M., Marinkovic, K., Schacter, D.L., Rosen, B., and Halgren, E. (2006). Top-down facilitation of visual recognition. Proceedings of the National Academy of Sciences, 103(2),