1. Chapter 6 Operant Conditioning Schedules

2. Schedule of Reinforcement Appetitive outcome --> reinforcement –As a “shorthand” we call the appetitive outcome the “reinforcer” –Assume that we’ve got something appetitive and motivating for each individual subject Fairly consistent patterns of behaviour Cumulative recorder

3. Cumulative Record Cumulative recorder Flat line Slope

4. paper strip pen roller Cumulative Recorder

5. Recording Responses

6. The Accumulation of the Cumulative Record VI-25

7. Fixed Ratio (FR) N responses required; e.g., FR 25 CRF = FR1 Rise-and-run Postreinforcement pause Ratio strain no responses reinforcement responses “pen” resetting

8. Variable Ratio (VR) Varies around mean number of responses; e.g., VR 25 Short, if any postreinforcement pause Never know which response will be reinforced

9. Fixed Interval (FI) Depends on time; e.g., FI 25 Postreinforcement pause; scalloping Clock doesn’t start until reinforcer given

10. Variable Interval (VI) Varies around mean time; e.g., VI 25 Don’t know when time has elapsed Clock doesn’t start until reinforcer given

11. Response Rates

12. Duration Schedules Continuous responding for some time period to receive reinforcement Fixed duration (FD) –Set time period Variable duration (VD) –Varies around a mean

13. Differential Rate Schedules Differential reinforcement of low rates (DRL) –Reinforcement only if X amount of time has passed since last response –Sometimes “superstitious behaviours” Differential reinforcement of high rates (DRH) –Reinforcement only if more than X responses in a set time

14. Noncontingent Schedules Reinforcement delivery not contingent upon passage of time Fixed time (FT) –After set time elapses Variable time (VT) –After variable time elapses

15. Choice Behaviour

16. Choice Two-key procedure Concurrent schedules of reinforcement Each key associated with separate schedule Distribution of time and behaviour

17. Concurrent Ratio Schedules Two ratio schedules Schedule that gives most rapid reinforcement chosen exclusively

18. Concurrent Interval Schedules Maximize reinforcement Must shift between alternatives Allows for study of choice behaviour

19. Interval Schedules FI-FI –Steady-state responding –Less useful/interesting VI-VI –Not steady-state responding –Respond to both alternatives –Sensitive to rate of reinforcemenet –Most commonly used to study choice

20. Alternation and the Changeover Response Maximize reinforcers from both alternatives Frequent shifting becomes reinforcing –Simple alternation –“Concurrent superstition”

21. Changeover Delay COD Prevents rapid switching Time delay after “changeover” before reinforcement possible

22. Herrnstein’s (1961) Experiment Concurrent VI-VI schedules Overall rates of reinforcement held constant –40 reinforcers/hour between two alternatives

23. KeyScheduleRft/hrRsp/hrRft rateRsp rate 1 VI-3min 20 2000 0.5 0.5 2 VI-3min 20 2000 0.5 0.5 1 VI-9 6.7 250 0.17 0.08 2 VI-1.8 33.3 3000 0.83 0.92 1 VI-1.5 40 4800 1.0 1.0 2 Extinction 0 0 0 0 1 VI-4.5 13.3 1750 0.33 0.31 2 VI-2.25 26.7 3900 0.67 0.69 Proportional Rate of Response B 1 = resp. on key 1 B 2 = resp. on key 2 Proportional Rate of Reinforcement R 1 = reinf. on key 1 R 2 = reinf. on key 2 R 1 R 1 +R 2 = 0.5 20 B 1 B 1 +B 2 = 2000 2000+2000 =0.5 2000 20 20+20 =0.5 R 1 R 1 +R 2 = 6.7 6.7+33.3 =0.17 0.83 6.7 33.3 0.08 0.92 250 3000 B 1 B 1 +B 2 = 250 250+3000 =0.08

24. The Matching Law The proportion of responses directed toward one alternative should equal the proportion of reinforcers delivered by that alternative.

25. Bias Spend more time on one alternative than predicted Side preferences Biological predispositions Quality and amount

26. Varying Quality of Reinforcers Q 1 : quality of first reinforcer Q 2 : quality of second reinforcer

27. Varying Amount of Reinforcers A 1 : amount of first reinforcer A 2 : amount of second reinforcer

28. Combining Qualities and Amounts

29. Extinction

30. Disrupt the three-term contingency Response rate decreases

31. Stretching the Ratio/Interval Increasing the number of responses e.g., FR 5 --> FR 50, VI 4 sec. --> VI 30 sec. Extinction problem Shaping; gradual increments “Low” or “high” schedules

32. Extinction Continuous Reinforcement (CRF) = FR 1 Intermittent schedule: everything else CRF easier to extinguish than any intermittent schedules Partial reinforcement effect (PRE) Generally: –High vs. low –Variable vs. fixed

33. Discrimination Hypothesis Difficult to discriminate between extinction and intermittent schedule High schedules more like extinction than low schedules e.g., CRF vs. FR 50

34. Frustration Hypothesis Non-reinforcement for response is frustrating On CRF every response reinforced; no frustration Intermittent schedules always have some non- reinforced responses –Responding leads to reinforcer (pos. reinf.) –Frustration = S + for reinforcement Frustration grows continually during extinction –Stop responding --> stops frustration (neg. reinf.)

35. Sequential Hypothesis Response followed by reinf. or nonreinf. Intermittent schedules: nonreinforced responses are S + for eventual delivery of reinforcer High schedules increase resistance to extinction because many nonreinforced responses in a row leads to reinforced Extinction similar to high schedule

36. Response Unit Hypothesis Think in terms of behavioural “units” FR1: 1 response = 1 unit --> reinforcement FR2: 2 responses = 1 unit --> reinforcement –Not “response-failure, response-reinforcer” but “response-response-reinforcer” Says PRE is an artifact

37. Mowrer & Jones (1945) Response unit hypothesis More responses in extinction on higher schedules disappears when considered as behavioural units 300 250 200 150 100 50 Number of responses/units during extinction FR1FR2FR3FR4 absolute number of responses number of behavioural units

38. Economic Concepts and Operant Behaviour Similarities Application of economic theories to behavioural conditions

39. The Economic Analogy Responses or time = money Total responses or time possible = income Schedule = price

40. Consumer Demand Demand curve –Price of something and how much is purchased –Elasticity of demand Amount Purchased Price Elastic Inelastic

41. Three Factors in Elasticity of Demand 1. Availability of substitutes –Can’t substitute complementary reinforcers e.g., food and water –Can substitute non-complementary reinforcers e.g., Coke and Pepsi 2. Price range –e.g., FR3 to FR5 vs. FR30 to FR50

42. 3. Income level –Higher total response/time…the less effect cost increases have –Increased income --> purchase luxury items –Shurtleff et al. (1987) Two VI schedules; food, saccharin water High schedules: rats spend most time on food lever Low schedules: rats increase time on saccharin lever

43. Behavioural Economics and Drug Abuse Addictive drugs Nonhuman animal models Elasticity –Work for drug reinforcer on FR schedule –Inelastic...up to a point lowmediumhighvery high Price (FR schedule) Response rate

44. Elsmore, et al. (1980) –Baboons –Food and heroin –Availability of substitutes 2 minutes12 minutes Frequency of Choice Response rate Food Heroin