There’s never just one reinforcer Hmm…what to do?

Studying Choice Experimentally Hernstein “concurrent schedules” VI 10VI 30

The Matching Law R A = rate of response A r A = payoff rate for response A RARA R A + R B rArA r A + r B =

Matching (R.J. Herrnstein) responding

Mechanisms of Matching: Melioration “Meliorate” = to make better, improve Animals try to make the local rates of reinforcement equivalent

Matching law with simple schedules RARA R A + R O rArA r A + r O = “other” behavior

ReinforcementPunishment Positive contingency Negative contingency Chocolate BarElectric Shock Excused from Chores No TV privileges – omission training Effect on Behavior

Professor Drew

Negative Reinforcement - Escape

Negative reinforcement - Avoidance

Avoidance: Experimental Paradigm The shuttle box Light = CS Light  Shock Shuttling stops shock

Two-Process Theory of Avoidance Light  Shock ( = Pavlovian Conditioning) -Light elicits fear Shuttling  Reduction of Fear (= negative reinforcement) Now, what happens with continued training?

Learned Helplessness Paradigm “Triadic” Design Group A: Escapable Shock Group B: Yoked Inescapable Shock Group C: Exposure to apparatus only Phase 1Phase 2 Escape/Avoidance training (For Group A shock can be terminated by rotating a wheel.)

Phase 2 Results Inescapable shock

Possible Explanations Learned Helplessness: Organisms learn that their behavior is ineffectual Poverty of activity: inescapable shock reduces the variability in behavior that is so crucial for operant conditioning Inattention: animals stop attending to their own behavior

LH in the Spinal Cord Recall: many reflexes are mediated within the spinal cord. Operant conditioning can occur within SC (escape/avoidance of leg shock after SC transection) Grau: Experience with inescapable legshock will prevent subsequent avoidance learning.

LH in Humans LH produced by… insoluble logic problems living in a crowded dorm

LH in the Spinal Cord Recall: many reflexes are mediated within the spinal cord. Operant conditioning can occur within SC (escape/avoidance of leg shock after SC transection) Grau: Experience with inescapable legshock will prevent subsequent avoidance learning.

Extinction

Session 1Session 2 Session 3

Spontaneous recovery occurs as a function of time 8-day break no break no CS

Theories of Extinction Forgetting? Associative loss? (= “reverse acquisition”)

Extinction  Associative Loss “Renewal” TrainExtinguishTest Tone  Shock Context A Context B Tone - Tone: CR Bouton & King (1983)

In contrast, acquisition is not context-specific TrainTest Tone  Shock Context A Context B Tone: CR

Extinction  Associative Loss “Reinstatement” TrainExtinguishReinstatementTest Tone  Shock Tone - Shock alone -- Tone: CR

Extinction  Associative Loss Post-extinction sensitivity to outcome devaluation Rescorla 1996

So, what is learned in extinction? An inhibitory S  R association? SR Context

Inhibitory S  R Associations Theory In extinction, the context effectively becomes a conditioned inhibitor. Why? Just like in normal CI, there is the violation of expectations of reinforcement But is this true?

Inhibitory S  R Associations Summation test Retardation test Does extinction produce them? TrainExtTest A+/B+A- Test: AB TrainExtTrain A+A-AB+ Does A inhibit responding to B? Does A inhibit acquisition to B?

So, what is learned in extinction? An inhibitory S  R association? SR Context

Paradoxical Effects of Reward Overtraining extinction effect: more training leads to faster extinction Reinforcement magnitude effect: Big rewards lead to faster extinction And, of course, the partial reinforcement extinction effect (PREE)

Paradoxical effects of reward: Why? Frustration hypothesis (Amsel): animals learn to make response as a reaction to nonreward. Discrimination hypothesis: Nonreinforcement is easier to detect after CRF than PRF. Sequential theory (Capaldi): The memory of nonreinforcement becomes a cue that elicits responding.