2. There’s never just one reinforcer Hmm…what to do?

3. Studying Choice Experimentally Hernstein “concurrent schedules” VI 10VI 30

4. The Matching Law R A = rate of response A r A = payoff rate for response A RARA R A + R B rArA r A + r B =

5. Matching (R.J. Herrnstein) responding

6. Mechanisms of Matching: Melioration “Meliorate” = to make better, improve Animals try to make the local rates of reinforcement equivalent

7. Matching law with simple schedules RARA R A + R O rArA r A + r O = “other” behavior

8. ReinforcementPunishment Positive contingency Negative contingency Chocolate BarElectric Shock Excused from Chores No TV privileges – omission training Effect on Behavior

9. Professor Drew

12. Negative Reinforcement - Escape

13. Negative reinforcement - Avoidance

14. Avoidance: Experimental Paradigm The shuttle box Light = CS Light  Shock Shuttling stops shock

15. Two-Process Theory of Avoidance Light  Shock ( = Pavlovian Conditioning) -Light elicits fear Shuttling  Reduction of Fear (= negative reinforcement) Now, what happens with continued training?

16. Learned Helplessness Paradigm “Triadic” Design Group A: Escapable Shock Group B: Yoked Inescapable Shock Group C: Exposure to apparatus only Phase 1Phase 2 Escape/Avoidance training (For Group A shock can be terminated by rotating a wheel.)

17. Phase 2 Results Inescapable shock

18. Possible Explanations Learned Helplessness: Organisms learn that their behavior is ineffectual Poverty of activity: inescapable shock reduces the variability in behavior that is so crucial for operant conditioning Inattention: animals stop attending to their own behavior

19. LH in the Spinal Cord Recall: many reflexes are mediated within the spinal cord. Operant conditioning can occur within SC (escape/avoidance of leg shock after SC transection) Grau: Experience with inescapable legshock will prevent subsequent avoidance learning.

20. LH in Humans LH produced by… insoluble logic problems living in a crowded dorm

21. LH in the Spinal Cord Recall: many reflexes are mediated within the spinal cord. Operant conditioning can occur within SC (escape/avoidance of leg shock after SC transection) Grau: Experience with inescapable legshock will prevent subsequent avoidance learning.

22. Extinction

23. Session 1Session 2 Session 3

24. Spontaneous recovery occurs as a function of time 8-day break no break no CS

25. Theories of Extinction Forgetting? Associative loss? (= “reverse acquisition”)

26. Extinction  Associative Loss “Renewal” TrainExtinguishTest Tone  Shock Context A Context B Tone - Tone: CR Bouton & King (1983)

27. In contrast, acquisition is not context-specific TrainTest Tone  Shock Context A Context B Tone: CR

28. Extinction  Associative Loss “Reinstatement” TrainExtinguishReinstatementTest Tone  Shock Tone - Shock alone -- Tone: CR

29. Extinction  Associative Loss Post-extinction sensitivity to outcome devaluation Rescorla 1996

30. So, what is learned in extinction? An inhibitory S  R association? SR Context

31. Inhibitory S  R Associations Theory In extinction, the context effectively becomes a conditioned inhibitor. Why? Just like in normal CI, there is the violation of expectations of reinforcement But is this true?

32. Inhibitory S  R Associations Summation test Retardation test Does extinction produce them? TrainExtTest A+/B+A- Test: AB TrainExtTrain A+A-AB+ Does A inhibit responding to B? Does A inhibit acquisition to B?

33. So, what is learned in extinction? An inhibitory S  R association? SR Context

34. Paradoxical Effects of Reward Overtraining extinction effect: more training leads to faster extinction Reinforcement magnitude effect: Big rewards lead to faster extinction And, of course, the partial reinforcement extinction effect (PREE)

35. Paradoxical effects of reward: Why? Frustration hypothesis (Amsel): animals learn to make response as a reaction to nonreward. Discrimination hypothesis: Nonreinforcement is easier to detect after CRF than PRF. Sequential theory (Capaldi): The memory of nonreinforcement becomes a cue that elicits responding.