The basic design: CS > US > UR bell food salivation \ | ----> CR: salivation important variables: –CS = conditioned stimulus: arbitrary stimulus that does not automatically evoke the response –UCS or US = unconditioned stimulus: –nonarbitrary stimulus that does automatically evoke the response –UCR or UR = unconditioned response: the response that is automatically evoked by the US –CR = conditioned response: response that the CR evokes (what learned): May or may not be identical to UR Crucial aspect for learning: Pairing of CS and US predicts an event

Important (critical) things to note about classical conditioning: the CS MUST precede the US the CS MUST predict the US if the CS does not predict the US, no conditioning occurs the CR does not have to be identical to the UR – E.g., subtle differences even Pavlov noticed) –may even be opposite: Morphine studies Any response is a classically conditioned response if it occurs to a CS after that CS has been paired with a US but does NOT occur to a randomly presented CS-US pairing

Rescorla: 6 types of control groups CS-alone –present CS alone with no US pairing –problem: not have same number of US trials as experimental animals do, may actually be extinction effect Novel CS group: – looks at whether stimulus is truly "neutral" –may produce habituation- animal doesn't respond because it "gets used to it" US-alone –present US aloine with no CS pairing –problem: not have same number of CS trials explicitly unpaired control –CS NEVER predicts US –that is- presence of CS is really CS-, predicts NO US – animal learns new rule: if CS, then no US Backward conditioning: –US precedes CS –assumes temporal order is important (but not able to explain why) –again, animal learns that CS predicts no US Discrimination conditioning (CS+ vs CS-) –use one CS as a plus; one CS as a minus –same problem as explicitly unpaired and backward- works, but

What is it that's important about CC? Rescorla's ideas CS-US correlation vs contiguity: –Typically in conditioning arrangements- CS always followed by the US in a perfect correlation –p(US|CS) = 1.0 –p(US|no CS) = 0.0 but: life not always a perfect correlation Problem: how to prove this beyond a reasonable doubt- MUST use truly random control –Must be absolutely no prediction –CS does not either predict or not predict US

Probabilities interact to determine size of CS CS = 2 min tone; presented at random intervals (X = 8 minutes) –E.g., for: Group 1: p(shock|CS) = 0.4 during 2 min presentation p(shock|no CS) = 0.2 only information that CS provided = whether probability of shock was high or low –used 10 groups of rats, all with different values of p(US|CS) and p(US|no US) whenever p(US|CS) > p(US|NO CS): –TONE = EXCITATORY CS –that is, response suppression occurred (CER) amount of suppression depended on size difference between p(US|CS) and p(US|no CS) and vice versa Most predictive stimulus was what attended to

Relation of CS to US appears to be the CORRELATION between the CS and US, not the contiguity (closeness in time) that is important that is: – correlation (r) carries more information – if r = + then excitatory CS – if r = - then inhibitory CS –if r = 0 then neutral CS (not really even a CS)

Blocking and overshadowing Overshadowing: –use one "weak" and one "strong" CS –reaction to weaker stimulus is blotted out by stronger CS Blocking: –One stimulus “blocks” learning to second CS

Kamin’s investigations Wanted to study role of attention in classical conditioning Usual set up: neutral stimulus becomes CS predictive of a US Note: used CER Wanted to know about “nonneutral” stimuli –Compound stimuli –Stimuli with a history

How measure in classical conditioning? look at change in an operant behavior as a result of a CS-US pairing –teach the rat to bar press for food –shock rat- rat naturally freezes –incompatible response- can't bar press and freeze at same time – suppression ratio: –baseline of A –intro CS condition B suppression ratio = B/A+B –no effect = 0.5 –complete suppression = 0.0 –(disinhibition = 1.0 or oops!)

Kamin’s blocking experiment used multiple CS's and 4 groups of rats the blocking group receives –series of L+ trials which produce strong CR –series of LT+ trials –then tested to just the T control group receives – SAME TOTAL NUMBER OF TRIALS AS BLOCKING GROUP – no first phase – LT+ in phase 2 (totaling phase 1 and 2 above)

Data are “surprising”!.prediction: –since both received same # of trials to the tone- –should get equal conditioning to the tone results quite different: –Blocking group shows no CR to the tone- –the prior conditioning to the light "blocked" any more conditioning to the tone directly contradicts frequency principle Group Phase I Phase II Test Phase Result Group ALNNTest L L elicits small CR.25 Group BNLNTest L L elicits no CR.45 Group C--LNTest L L elicits CR.05 Group B2N----Test L L elicits no CR.45

Second experiment: 1 st training2 nd 3rd Group Y:N (16x w/Sr)LN (no sr) (8)N (non sr) Group Z:N (16x w/Sr)N (no sr) (12) Result: –For first 16 trials: identical treatment: 0.02 on average –Group Y: presented with compound, ratio increased to 0.41 –Group Z: presented with noise only, ratio = 0.33 (EXT) –Goup Y: noise only slight decrease to about 0.35 Conclusion: superimposed element provided NEW information –not only notice cue –respond to cue because it carries info!

Things we know about blocking the animal does "detect" the stimulus: –EXT of CER with either N alone or with NL is slower than EXT for compound NL appears to be independent of: –length of CS –number of trials of conditioning to compound CS influenced by: –use of CER measure (not the best) –nature of CS may be important- e.g. modality –intensity of stimuli important –depends on amount of conditioning to blocking stimulus which already occurred constancy of US from phase 1 to 2 important. change in either US or CS can prevent/overcome blocking –change the intensity of the CS from one situation to another –this is why spent so much time on overshadowing- strong vs weak stimulus is change intensity of the stimulus- presents a different learning situation and no blocking same is true if change the intensity of US –(although generally must be stronger, not weaker) –e.g. experiments when changed from 1 ma to 4 ma shock –quickly condition to compound stimulus –little or no overshadowing or blocking

Theoretical Explanations? Perceptual gating theory: –tone never gets processed –tone not informative –data not really support this Kamin's Surprise theory: –to condition requires some mental work on part of animal –animal only does mental work when surprised –bio genetic: prevents having to carry around excess mental baggage thus only learn with "surprise" situation must be different from original learning situation Alternative explanation: Rescorla Wagner model: –particular US only supports a certain amount of conditioning –if one CS hogs all that conditioning- none is left over for another CS to be added –question- how do we show this?

Assumptions of R-W model helpful for the animal to know 2 things about conditioning: –what TYPE of event is coming –the SIZE of the upcoming event Thus, classical conditioning is really learning about: –signals (CS's) which are PREDICTORS for –important events (US's) model assumes that with each CS-US pairing 1 of 3 things can happen: –the CS might become more INHIBITORY –the CS might become more EXCITATORY –there is no change in the CS how do these 3 rules work? –if US is larger than expected: CS = excitatory –if US is smaller than expected: CS= inhibitory –if US = expectations: No change in CS The effect of reinforcers or nonreinforcers on the change of associative strength depends upon: –the existing associative strength of THAT CS –AND on the associative strength of other stimuli concurrently present

More assumptions Explanation of how an animal anticipates what type of CS is coming: –direct link is assumed between "CS center" and "US center": e.g. between a tone center and food center –assumes that STRENGTH of an event is given and that the conditioning situation is predicted by the strength of this connection – THUS: when learning is complete: the strength of the association relates directly to the size or intensity of the CS The change in associative strength of a CS as the result of any given trial can be predicted from the composite strength resulting from all stimuli presented on that trial: –if composite strength is low, the ability of reinforcer to produce increments in the strength of component stimuli is HIGH –if the composite strength is low; reinforcement is relatively less effective (LOW)

More assumptions: Can expand to extinction, or nonreinforced trials : –if composite associative strength of a stimulus compound is high, then the degree to which a nonreinforced presentation will produce a decrease in associative strength of the components is LARGE –if composite associative strength is low- nonreinforcement effects reduced Yields an equation: V i =α i ß j (Λ j -V AX ) Here is an easier way to write it: V T =α i ß j (Λj-V sum )

First example: rat is subjected to conditioned suppression procedure: –CS (light) ---> US (1 mA shock) –what is associative strength? –1 = associative strength that a 1mA shock can support at asymptote ( Λ j ) – V L = associative strength of the light (strength of the CS-US association) thus: Λ 1 = size of the observed event (actual shock) V L = measure of the Subjects current "expectation" about the size of the shock V L will approach Λ 1 over course of conditioning

Second example: Same rat, same procedure but 2CS's: CS (light+tone) --> 1 mA shock –Determine associative strength when Λ 1 is constant – V sum = V L + V T = assoc. strength of the 2 CS's – V sum = α i ß j (Λ) –if V L and V T equally salient: V L = 0.5α i ß j; V T = 0.5α i ß j – V T = if not equally salient: V L > V T or V L < V T now can restate the 3 rules of conditioning: –Λ j > Vsum = excitatory conditioning –Λ j < Vsum = inhibitory conditioning –Λ j = Vsum = no change

Now have the Rescorla-Wagner Model: Model makes predictions on a trial by trial basis For each trial: predicts increase or decrement in associative strength for every CS present The equation: Vi =αißj(Λ j -Vsum) –V i = change in associative strength that occurs for any CS, i, on a single trial –Λ j = associative strength that some US, j, can support at – asymptote –V sum = associative strength of the sum of the CS's (strength of – CS-US pairing) –α i = measure of salience of the CS (must have value between 0 – and 1) –ß j = learning rate parameters associated with the US (assumes –that different beta values may depend upon the particular US employed)

Assumptions of the formal model: General Principle: as Va increases with repeated reinforcement of j, the difference between Λa and Va decreases –increments of Va then decrease –produce negatively accelerated learning curve with asymptote of Λ j Reinforcement of compound stimuli: lots of Va trials, then give trials of compound Vax –Va increases toward Λa as a result of a-alone presentations –Vax then exceeds Λa –result: reinforced aX trial results in DECREMENT to the associative strength of a and X components as a and aX are reinforced: – increments to A occur on the reinforced A trials –increments to A and X occur on reinforced AX trials –result: transfer to A of whatever associative strength X may have

The equation: Vi =αißj(  j-Vsum) Vi = change in associative strength that occurs for any CS, i, on a single trial αi = stimulus salience (assumes that different stimuli may acquire associative strength at different rates, despite equal reinforcement) ßj = learning rate parameters associated with the US (assumes that different beta values may depend upon the particular US employed) Vsum = associative strength of the sum of the CS's (strength of CS-US pairing) Λ j = associative strength that some CS, i, can support at asymptote In English: How much you learn on a given trial is a function of the value of the stimulus x value of the reinforcer x (the absolute amount you can learn minus the amount you have already learned).

Acquisition first conditioning trial: CS = light; US= 1 ma Shock –Vsum = Vl; no trials so Vl = 0 –thus: Λ j -Vsum = = 100 –-first trial must be EXCITATORY BUT: must consider the salience of the light: αi = 1.0 and learning rate: ßj = 0.5 Plug into the equatio: for TRIAL 1 –Vl = (1.0)(0.)(100-0) = 0.5(100) = 50 –thus: V only equals 50% of the discrepancy between Aj an Vsum for the first trial TRIAL 2: –V1 = (1.0)(0.5)(100-50) = 0.5(50) = 25 –Vsum = (50+25) = 75 TRIAL 3: –V1 = (1.0)(0.5)(100-75) = 0.5(25) = 12.5 –Vsum = ( ) = 87.5 TRIAL 4: –V1 = (1.0)(0.5)( ) = 0.5(12.5) = 6.25 – Vsum = ( ) = TRIAL 10: Vsum = 99.81, etc., until reach 100 on approx. trial 14 When will you reach asymptote?

Overshadowing Pavlov: compound CS with 1 intense CS, 1 weak – after a number of trials found: strong CS elicits strong CR –weak CS elicits weak or no CR Rescorla-Wagner model helps to explain why: assume – αL = light = 0.2; αT = tone = 0.5 –ßL = light = 1.0 ; ßt = tone = 1.0 Plug into equation: –Vsum = Vl + Vt = 0 on trial 1 –Vl = 0.2(1)(100-0) = 20 –Vt = 0.5(1)(100-0) = 50 –after trial 1: Vsum = 70 TRIAL 2: –Vl = 0.2(1)(100-(50+20)) = 6 –Vt = 0.5(1)(100-(50+20)) = 15 –Vsum = (70+(6+15)) = 91 TRIAL 3: –Vl = 0.2(1)(100-(91)) = 1.8 –Vt = 0.5(1)(100-(91)) = 4.5 –Vsum = (91+( )) = 97.3 and so on –thus: reaches asymptote (by trial 6) MUCH faster w/2 CS's NOTE: CSt takes up over 70 units of assoc. strength CSl takes up only 30 units of assoc. strength

Blocking similar explanation to overshadowing: –no matter whether VL more or less salient than Vt, because CS has basically absorbed all the assoc. strength that the CS can support give trials of A-alone to asymptote: –reach asymptote: VL = Λ j =100 =Vsum – αL =1.0 – ß =0.2 –First Vt Trial: Vt= αß(Λ j -Vsum) Vt=0.2*1.0*( )=? No learning!

How could one eliminate blocking effect? increase the intensity of the US to 2 mA with Λ j now equals = 160 –then: Vsum still equals 100 (learned to 1 mA shock) plug into the equation: (assume Vl and Vt equally salient) –Vt = 0.2(1)( ) = 0.2(60) = 12 –Vl = 0.2(1)( ) = 0.2(60) = 12 on trial 2: – Vsum = 124 – Vt = 0.2(1)( ) = 0.2(36) = 7.2 –Vl = 0.2(1)( ) = 0.2(36) = 7.2 – Vsum now = ( ) = 138. could also play around with ß

Critique of the Rescorla-Wagner Model: R-W model really a theory about the US effectiveness: –says nothing about CS effectiveness – states that an unpredicted US is effective in promoting learning, whereas a well-predicted US is ineffective Fails to predict the CS-pre-exposure effect: – two groups of subjects (probably rats) –Grp ICS-US pairingsControl –Grp II CS aloneCS-US pairings PRE-Expos pre-exposure group shows much less rapid conditioning than the control group R-W model doesn't predict any difference, because no conditioning trials occur when CS is predicted alone: Vsum = 0 – BUT: may be that salience for the CS is changing: –habituation to CS Original R-W model implies that salience is fixed for any given CS –R-W assume CS salience doesn't change w/experience –these data strongly suggest CS salience DOES change w/experience Newer data supports changes salience –data suggest that Si DECREASES when CS is repeatedly presented without consequence –NOW: appears that CS and US effectiveness are both highly important Model has stood test of time, now widely used in neuroscience

Can deal with variety of other issues Compound CSs: –When two CSs are conditioned together –How much conditioning occurs to one or other depends on previous exposure and salience of each stimulus. Time alone as CS –Time can serve as a CS; as long as it is predictive! Difference between CS and no CS

Can also explain why probability of reward given CS vs no CS makes a difference: π = probability of US given the CS or No US given No CS can make up three rules: – if πax > πa then Vx should be POSITIVE –if πax < πa then Vx should be NEGATIVE –if πax = πa then Vx should be ZERO modified formula: (assume Λ1 =1.0; Λ2 =0; ß1 =.10; ß2=.05; α1=.10; α2=.5) Va = πaß πaß1 - (1-πa)ß2 Vax = πaxß πaxß1 - (1-πax)ß2 Vx = Vax - Va

PLUG IN: Probability of CSa then US = 0.2; Probability of CSax then US = 0.8 Va = (0.2)(1.0) = -10 ((.2)(.10)) - (1-.2)(.05) Vax = (0.8)(1.0) = ((.8)(.10)) - (1-.8)(.05) Vx = Vax - Va or (-10) = probability of US given AX greater than probability of US given X)

PLUG IN: Probability of CSa then US = 0.8; Probability of CSax then US =0.2 Va = (0.8)(1.0) =11.43 ((.8)(.10)) - (1-.8)(.05) Vax = (0.2)(1.0) =-10 ((.2)(.10)) - (1-.2)(.05) Vx = Vax - Va or = probability of US given AX is less than probability of US given A

PLUG IN: Probability of CSa then US = 0.5 Probability of CSax then US = 0.5 Va = (0.5)(1.0) =20 ((.5)(.10)) - (1-.5)(.05) Vax = (0.5)(1.0) =20 ((.5)(.10)) - (1-.5)(.05) Vx = Vax - Va or = 0 (probability of AX = A)