“Stimulus, response! Stimulus, response! Don’t you ever think?” Outline B.F. Skinner Biography Theoretical notions Respondent and Operant Behaviour Operant Conditioning Principles The Skinner Box Shaping and Extinction Superstitious behavior Discriminative responding Secondary reinforcers “Stimulus, response! Stimulus, response! Don’t you ever think?” 2

B. F. Skinner Biography Wanted to be writer B.A. in English Literature (1926) Entered graduate school at Havard (1928) Mentored by the Chair of Physiology (W. Crozier) Who studied the “animal as a whole” without appealing to internal processes. Obtained his Ph.D. in 1931 Taught at University of Minnesota (1936 - 1945) Published “The Behaviour of Organisms” (1938) Affiliated with Havard until he died (1990) 3

B. F. Skinner Biography Inventions The air-crib Project pigeon Easily-cleaned, temperature and humidity-controlled Somewhat controversial (but effective) Commercially manufactured Project pigeon Received a 25K grant to develop a cruise missile Guided by trained pigeons U.S. Navy passed on it (but retested the idea in the 1980s) 3

B. F. Skinner Biography Inventions The Skinner Box (AKA, operant chamber) Animal can respond multiple times Operant response: Bar pressing Operant conditioning: Increased bar pressing when food is delivered following the response. 3

B. F. Skinner Biography Inventions Cumulative recorder Keeps track of the animal’s responding Time is recorded on the ‘X’ axis Total number of responses is recorded on the ‘Y’ axis Faster rates of responding lead to steeper slopes 3

Major Theoretical Notions Respondent and operant behaviour B. F. Skinner B. F. Skinner (1904-1990) Major Theoretical Notions Respondent and operant behaviour Respondent behaviour - Behaviour elicited by a known stimulus E.g., Unconditioned responses Elicited by unconditioned stimuli Reflexive Operant behaviour - Behaviour not elicited by a known stimulus E.g., Most of our everyday behaviour Occurs spontaneously 3

Major Theoretical Notions Type S and Type R conditioning B. F. Skinner B. F. Skinner (1904-1990) Major Theoretical Notions Type S and Type R conditioning Two kinds of conditioning Type S (respondent conditioning) - classical (Pavlovian) conditioning ‘S’ to emphasize the role of the eliciting Stimulus Strength is measured by the magnitude of the CR Type R (operant conditioning) - learning that involves operant behaviour ‘R’ to emphasize Response Strength is measured by the response rate 3

Major Theoretical Notions Type S and Type R conditioning B. F. Skinner B. F. Skinner (1904-1990) Major Theoretical Notions Type S and Type R conditioning Comparison with Thorndike’s approach Thorndike’s puzzle box Dependent variable was time-to-solution -> I.e., how long it took to learn a (single) response Skinner Dependent variable was rate of responding -> I.e., What variables affect the rate of responding Edward Lee Thorndike (1874 - 1949) 3

Major Theoretical Notions Operant conditioning principles B. F. Skinner B. F. Skinner (1904-1990) Major Theoretical Notions Operant conditioning principles Two general principles Any response that is followed by a reinforcing stimulus tends to be repeated A reinforcing stimulus is anything that increases the rate with which an operant response occurs I.e., anything that increases the probability of a response’s re-occurring 3

Major Theoretical Notions Operant conditioning principles B. F. Skinner B. F. Skinner (1904-1990) Major Theoretical Notions Operant conditioning principles Contingent reinforcement Emphasis on behaviour and its consequences Gaining reinforcement depends (i.e., is contigent) on making the appropriate response Culture as a set of reinforcement contigencies Different cultures reinforce different behaviour patterns Controlling reinforcement -> controls behaviour E.g, child rearing 3

Major Theoretical Notions Conditioning the lever-pressing response B. F. Skinner B. F. Skinner (1904-1990) Major Theoretical Notions Conditioning the lever-pressing response Three steps 1. Deprivation Food/water deprived for 23 hours per day Animal is held at 80% of its free-feeding body weight 2. Magazine training Food pellets are delivered by the experimenter Animal learns to associate the sound of the delivery mechanism with food 3

Major Theoretical Notions Conditioning the lever-pressing response B. F. Skinner B. F. Skinner (1904-1990) Major Theoretical Notions Conditioning the lever-pressing response Three steps 3. Lever pressing Animal is placed in the box Eventually hits the lever (operant response) Delivery of food pellet reinforces the response. 3

Major Theoretical Notions Conditioning the lever-pressing response B. F. Skinner B. F. Skinner (1904-1990) Major Theoretical Notions Conditioning the lever-pressing response Shaping A faster method of teaching the rat to lever press Two components 1. Differential reinforcement -> Some responses are reinforced, others are not 2. Successive approximations -> Only reinforce responses that become progressively closer to the desired response 3

Major Theoretical Notions Extinction and spontaneous recovery B. F. Skinner B. F. Skinner (1904-1990) Major Theoretical Notions Extinction and spontaneous recovery Extinction - removing the reinforcer removes the operant response Spontaneous recovery - The reoccurrence of a response that had been extinguished, with no additional training. Cumulative responses Time Rest Extinction Spontaneous recovery 3

Major Theoretical Notions Superstitious behavior B. F. Skinner B. F. Skinner (1904-1990) Major Theoretical Notions Superstitious behavior What if we delivered pellets noncontingently? Random behaviour would get reinforced E.g., Dog running in circles while waiting to get fed Humans are susceptible to similar conditioning E.g., Athletes/ coaches’ game rituals 3

Major Theoretical Notions Discriminative operant B. F. Skinner B. F. Skinner (1904-1990) Major Theoretical Notions Discriminative operant Skinner box is set so that reinforcement is only available when the light is on. The light is the discriminative stimulus I.e., indicates that reinforcement is available SD = light on, SD = light off, SR = reinforcing stimulus A discriminative operant is symbolized as: SD -> R -> SR Skinner was interested in the SD -> R association -> Cf. respondent conditioning Stimulus control of behaviour 3

Major Theoretical Notions Secondary reinforcement B. F. Skinner B. F. Skinner (1904-1990) Major Theoretical Notions Secondary reinforcement A neutral stimulus paired with a reinforcer can take on reinforcing qualities of its own To test this notion Lever press -> light -> food Extinguish the response Neither light nor food is delivered Allow lever - press to deliver light (not food) Response rate increases 3

Major Theoretical Notions Secondary reinforcement B. F. Skinner B. F. Skinner (1904-1990) Major Theoretical Notions Secondary reinforcement A secondary reinforcer can be used to reinforce other responses Clicker training for dogs Warning: secondary reinforcers can be extinguished! Money for humans Generalized reinforcer 3