1. Instrumental Conditioning II

2. Delay of Reinforcement Start DelayChoice Correct Incorrect Grice (1948) Goal Reward or No Reward

3. Grice (1948) Results

4. Overcoming the effects of delay Secondary reinforcers “Marking” procedure

5. Lieberman, McIntosh & Thomas (1979)

6. Delayed reinforcement

7. ReinforcementPunishment Positive contingency Negative contingency Chocolate BarElectric Shock Excused from Chores No TV privileges Effect on Rate Behavior

11. Professor Drew

12. Anticipatory Contrast - Crespi (1942) Rats run down maze to find food pellets in goal arm.

13. What is a reinforcer? Operational Definition (behaviorists): That which increases the probability of the response that preceded it. Thorndike: A stimulus that produces a “satisfying state of affairs”

14. Drive Reduction Theory Amt of H2O in body Compare with Set Point Seek water/ don’t seek water drives

15. Drive Reduction Considered: Are reinforcers necessary for survival? –Eating to excess –Drugs of Abuse –“Pleasure centers” of the brain

17. Behavioral Regulation View: The Premack Principle Behaviors are reinforcing, not stimuli To predict what will be reinforcing, observe the baseline frequency of different behaviors Highly probable behaviors will reinforce less probable behaviors

18. Premack Revised: The Response Deprivation Hypothesis Low frequency behaviors can reinforce high frequency behaviors (and vice versa) All behaviors have a preferred frequency = the behavioral bliss point Deprivation below that frequency is aversive, and organisms will work to remedy this Timberlake & Allison (1974)

19. Response deprivation hypothesis.25.5.75 The ice cream scale (in pints) 1.01.251.51.752.02.252.5 Bliss point (1.0 pints/night) Will work to avoid ice cream Will work to obtain

20. The behavioral bliss point and motivation

21. Contiguity versus Contingency in operant conditioning

22. Degraded Contingency Effect = bar press = food Perfect contingency Strong Responding Degraded contingency Weak Responding

23. G.V. Thomas (1983) Contiguity pitted against contingency “Free” reinforcers given every 20s Lever press advances delivery of pellet, but cancels pellet for next 20-s interval So if you press at second 2, you get a pellet immediately, but you get no pellet during seconds 3-20 and 21-40. 20s40s60s

24. G.V. Thomas (1983) Contiguity pitted against contingency So if you press at second 2, you get a pellet immediately, but you get no pellet during seconds 3-20 and 21-40. 20s40s60s Lever press here Lose this pellet

25. “Superstitious Behavior” Suggested that temporal contiguity more important than contingency 15-s FT, no response requirement “adventitious reinforcement” “In 6 out of 8 cases the resulting responses were so clearly defined that two observers could agree perfectly in counting instances. One bird was conditioned to turn counter-clockwise about the cage, making 2 or 3 turns between reinforcements. Another repeatedly thrust its head into one of the upper corners of the cage….”

26. Orienting toward feeder Pecking near feeder Moving along wall ¼ turn

27. “Misbehavior” and the limits of operant conditioning

28. Limits of Operant Conditioning Some behaviors can’t be conditioned –Yawning –Scratching Belongingness –Presentation of a female won’t reinforce biting “Misbehavior”

29. Marian Breland Bailey – How to train a chicken

30. The famous dancing chicken

31. What is learned in operant conditioning?

32. SR What is learned? Edwin Guthrie: mere contiguity of a stimulus and a behavior stamps in that S-R; reinforcement is not necessary

33. SR What is learned? Thorndike: Reinforcement “stamps in” this connection

34. SR O What is learned? ?

35. SR O 2-Process Theory operant Pavlovian

36. SR CR 2-Process Theory operant Pavlovian

37. Evidence for 2-process theory Pavlovian-Instrumental Transfer Phase 1Phase 2Test Lever  FoodLight  FoodLight: #Presses? No Light: #Presses? # Presses LightNo CS The presence of the CS intensifies operant responding

38. SR O ? ? What is learned? Does the Pavlovian S-O association activate a vague emotional state or a specific mental representation of the outcome?

39. Specific Outcome Representations Trapold Phase 1Phase 2Test (operant)(classical) R Lever  PelletTone  PelletTone:Left? Right? L Lever  SucroseLight  SucroseLight:Left? Right? # Presses LightNoise Left Right

40. RORO Colwill & Rescorla (1986) Phase 1DevaluationTest Push Left  PelletPellet+LiClRight? Push Right  SucroseSucrose+LiClLeft? # Pushes Pellet Devalued Sucrose Devalued Right Pushes Left Pushes