1. Other Models of Choice

2. Molar Maximizing Idea that animals maximize their responding – Most reinforcement for least effort – Maximize over a long time horizon can explain why subjects switch with concurrent VI-VI schedules but not with concurrent ratio schedules

3. Molar Maximizing molecular and molar theories – molecular theories- animals choose whichever response alternative is most likely to be reinforced at that time – Your grade in an individual class – best alternative at the moment molar theories- overall strategy – Animals choose responses using long term strategy – Your overall GPA

4. Melioration To meliorate means to make better Proposes that subjects shift responding toward one or other alternative until both alternatives are equally attractive in terms of reinforcement per unit of effort or time Produces matching on concurrent VI VI schedules Predicts exclusive responding on more favorable schedule on concurrent VR VR

5. Temporal discounting

6. Temporal Discounting Len Green; Mark Dixon; Tom Critchfield Theory or model which examines time frame under which people choose longer/later vs. Shorter/smaller How long are you willing to wait for a reward Assumes that the value of a reward is discounting (decreases) as time between response and reinforcer increase http://www.youtube.com/watch?v=G7LN96jEXHc

7. Temporal Discounting (Rachlin & Green, 1972; Rachlin and Logue, 1976; Mazur, 1987, 1988)

9. Temporal Discounting as a Measure of Impulsivity in Rats: Pitts & McKinney (2005) Compared effects of: – Ritalin (1.0, 3.0, 5.6, 10.0, or 17.0 mg/kg) – Morphine (0.3, 1.0, 5.6, 10.0, or 17.0 mg/kg) Reinforcers: – SS = 3s access to 0.02cc of sugar water – LL = two different delay schemes, and was four 3s accesses to 0.02cc of sugar water Rats were exposed to a delay discounting task where: – Group 1: exposed to delay scheme 1 for the LL – Group 2: exposed to delay scheme 1 & 2 for the LL: – Delay scheme 1 = 0, 5, 10, 20, 30, 40, or 50s – Delay scheme 2 = 0, 2, 4, 6, 8, 16, or 20s

10. Pitts & McKinney (2005) cont. Results: – Rats chose the LL over the SS only for the 3.0 mg/kg and the 5.6 mg/kg doses of Ritalin. – Remained impulsive at low or very high doses – Morphine data showed never learned to choose LL over SS Ritalin, which is often prescribed in lieu of D-AMPH as an ADHD medication, effects discounting in a manner much like D-AMPH

11. Animal Data consistent Animals tend to be impulsive: choose SS over LL Some drugs can alter this pattern; typically these are drugs that increase the tonic level of DA Green, et al. (2013) showed that time delay is critical: – Animals choosing NOW vs. later are impulsive – If choose “a little later” vs. “later”- less impulsivve – Seem to respond to “A bird in the hand is better than 2 in the bush” – This makes sense ecologically

12. But what about humans? Use more “cognitive” task Ask subjects to choose between SS and LL – Do you want $10 now or $100 in 10 minutes $100 in 1 day $100 in 2 days, etc. – Ascending and descending series using constant “smaller” – Also vary the probability of the later reward – Look at “switching point” At what point is the human no longer willing to wait?

13. Problems with human studies Real vs. hypothetical rewards – Some slight differences in data – But data seem as good for pretend as for real! Discounting functions look just like animal functions, with similar fits to themodel Group vs. individual analyses Stability of temporal discounting patterns – Population differences: gamblers vs. non gamblers – Experience plays a role – K is context-sensitive and experience driven measure of impulsiveness Look at an example

14. Temporal Discounting as a Measure of Impulsivity in Humans Shiels et al., 2009 – Measured delay discounting in children with ADHD Children were measured both on and off medication – Three groups More innatentive ADHD More hyperactive ADHD Normal control – All three groups were tested on and off both doses of ADHD medication Low dose Concerta (39 mg) High dose Concerta (73 mg)

15. Shiels et al., 2009 cont. Children were exposed to a discounting task – Choose 5 cents now (SS) – Choose 30 cents later (LL) Delayed option was manipulated probabilistically – 7s, 14s, or 28s delay – With a constant 35% of being delivered, 65% not being delivered Results concluded that: – Children with ADHD significantly chose SS over LL – Each dose of Concerta significantly lowered preference for SS in ADHD children Resulted in similar responding for ADHD children and normal controls

16. Functional Characteristics of Psychological Disorders Topographical features of a disorder does not necessarily provide answer as to function of the behavior: e.g., Self- injurious Behavior (SIB) – Escape from demand – Attention – To get treats/toys, etc. – Self stimulation But: Behavior analysis has shown that groups of individuals can be differentiated by the function of their behavior: – Violent vs. nonviolent criminals and aggression – ADHD vs. typical kids – Addicts vs. nonaddicts

17. Does looking at function of behavior help improve diagnosis and treatment - and how does this tie in the Temporal discounting! Examine ADHD: – Can describe topography of behaviors – But vast differences among those diagnoses What if examine sensitivity to reinforcement and temporal discounting? – Must define each term – Use models to drive definitions

18. Explaining ADHD Individuals with ADHD are impulsive because: – Look at context of immediate and delayed consequences – Immediate reinforcers unusually potent: competing reinforcers heavily discounted due to delay – Relatively unaffected by delayed punishment – Thus: should see differences in temporal discounting and reward sensitivity between typical and ADHD individuals

19. Why is this important? Applied data on temporal discounting are informative in 2 ways: – sensitivity to consequences =problem of choice Provin context of reinforcement settings, all behavior occurs at expense of another, competing behavior Competing behaviors loom large Response strength = function of strength of alternative behaviors! – Choice occurs across extended temporal contexts Delayed consequences have different effect on behavior than immediate consequences With passage of time, responses and consequences aggregate, and relation between them grows more complex May not be able to parcel out effect of individual response- consequence, but must take in context

20. Molar vs. Molecular Molar sequences of responses may be just as important as sequences of consequences – Again, context and control over pattern – Variables such as time and effort to change a pattern of responding are important A shift in an established response pattern may have its own value that must be taken into consideration (oh, but this will be behavioral momentum theory in a few weeks!)

21. Example: Drug Addiction Socially important aspects of abuse behavior are molar rather than molecular patterns of responding: – Abuse – Recovery – Relapse Amount of responding maintained by drug reinforcers depends on characteristics and availability of alternative reinforcers – Oxycontin abusers do not use illegal drugs or engage in high rates of illegal behavior – Until they lose access to prescription drugs!

22. Why an impulsive addict? Substance abuse behaviors produce favorable immediate consequences but delay unfavorable consequences Non drinking behaviors facilitate favorable delayed consequences but less tangible immediate benefits Abuse behaviors compete for time and resources with non-abuse behaviors

23. Can explain some drinking behavior: Research examined 2 specific issues: – Why do habitual heavy drinkers occassionally abstain from drinking independently or by seeking treatment? – Why do alcohol abusers who have received treatment start drinking again (relapse)? Must track drinking episodes and environmental events that might contribute to them – Over weeks, months, years – Must use self report, but data are quite reliable! Vuchinich and Tucker, 1995: Examined temporal patterns of drinking and its relation to a variety of environmental events – Assumed that these events reflected the reinforcing value of engaging drinking vs. not drinking alternative behavior

24. Vuchinich and Tucker (1995) Indiviudals who ceased drinking had experienced variety of drinking related outcomes that threatened availability of non- drinking related reinforcers – These events accumulated until sufficiently different to change value of drinking vs. not drinking- then sought treatment – Hitting “bottom” – Cost of drinking became temporally acute

25. Vuchinich and Tucker (1995) Individuals who started drinking had experienced adverse social and vocational events which preceded onset of drinking Again, these events accumulated until sufficiently different to change value of drinking vs. not drinking- then sought treatment – Hitting “bottom” for alternative sources of reinforcement – Cost of “not drinking” became temporally acute

26. Why important to understand in these terms? Suggest a molar relation between overall patterns of drinking and aggregated consequences for alternative nondrinking behavior – Similar to data from laboratory drug data, suggesting may be same underlying process Understanding this relationship guides development of intervention protocols – Must build behavioral resistance for alternative nondrinking; make resistance to behavioral disruption Suggests focus must be on boosting reinforcement for alternative behaviors, not focusing on “not drinking”

27. What else might be explained via this model? Any choice behavior, really Eating vs. exercise Other addictive behaviors Mental illness and medication compliance Saving vs. spending Studying vs. not studying! But is there a biological explanation that could support this? Yes!

28. Dopamine and Reward A brief review

29. Reinforcement and Dopamine? Olds and Milner: – Brain stimulation = lots of behavior – Animals would work until death to gain access to this brain stimulation – Thought had discovered pleasure center: Nucleus accumbens Mesolimbic pathway Dopamine (DA) was neurotransmitter involved in these areas

30. Reinforcement and DA EBS = releasing LOTS of dopamine (DA) – Results in lots of locomotion or exploratory behavior – Salamone and Schultz’ modern work has shown this release DA release modulates “appetitive behavior” – Occurs in modes or modules related to terminal event – E.g., food modes, sex modes – Which mode depends on context of environment Search, capture, prepare, consume DA not affect consummatory behavior

31. DA is regulated in two ways: DA is released in pulses (phasic) and has an overall tone in synapse (tonic) – Result is feedback system DA release in response to stimuli in environment – Motivated or energize appropriate behavior – Feedback system follows Rescorla Wagner model

32. DA regulation DA also has a constant level or overall tone in the synapse: – More released when “surprised above what expected – Less when “surprised” below what expected – When get what expected- behavior is “well learned” and appears to become habit (not sure how this works yet) Thus, fluctuations in DA as learn, and then serve as feedback regarding state of environment

33. Temporal Discounting and DA Changes in phasic and tonic DA may help explain temporal discounting DA tone decays across time – Thus background by which DA phasic spikes are related is changes – Value of a reward changes across time With short delay: – DA spike for small reinforcer is obviously different than DA spike for larger/more valuable reinforcer With long delay: – DA tone for larger/more valuable reinforcer decays with time; this makes it less valuable – This may alter the relative value of the immediate reinforcer compared to the delayed reward – Smaller sooner looks better!

34. Self-Control A way to teach LL choices?

35. Self-Control Process by which individual deliberately alters or changes his or her behavior to achieve a specific goal – Sets own goals – Monitors own behavior – Self Administration of own reward

36. Self Control Learning to choose delayed but larger consequences over shorter but smaller consequences – Very difficult for animals to do – May be biological: take what can get, not wait for “unknown” Important human behavior – We can learn to delay reward – Question is: how long are we willing to wait

37. Causes of Self-Control Problems: Problems of Behavioral Excesses: engage in too much of a behavior for too little reward Problems of Behavioral Deficits: engage in too little behavior to earn reward

38. Problems of Behavioral Excesses Immediate reinforcers versus delayed punishers for a behavior Immediate reinforcers versus cumulatively significant punishers for a behavior Immediate reinforcers for problem behavior versus delayed reinforcers for alternative desired behavior

39. Problems of Behavioral Deficiencies Immediate small punishers for a behavior versus reinforcers that are cumulatively significant Immediate small punisher for behavior versus immediate but highly improbable major punisher if the behavior does not occur Immediate small punisher for a behavior versus a delayed major punisher if the behavior does not occur

40. A Model for Self-Control Two parts 1.Clear specification of the problem as a behavior to be controlled 2.Application of behavioral techniques to manage the problem

41. A Model for Self-Control Emitting a controlling behavior to effect a change in a behavior to be controlled

42. Steps in Self-Control Program 1.Specify problem and set goals – What do you want to change and how will you know success? 2.Make a commitment to change – Statements and actions that indicate that you want to change Write out advantages to change and post in visible space Make it public Incorporate frequent reminders of your commitment in your environment 3.Conduct assessment – When, where, and how does a problem behavior occur?

43. Design and implement a program Manage antecedents - what are the causes? Instructions – use self-instructions – Modeling – find someone who is good at behavior and observe – Physical guidance – may use physical barriers to avoid some behaviors Manage Immediate surroundings – may need to change – Other people – may need to change – Time of day – may need to change what you do at certain times Manage the behavior – for complex behaviors, need to focus on the behavior itself; may need to conduct task analysis and have mastery criteria. Manage consequences – eliminate reinforcers of negative behaviors

44. Prevent relapse and make gains last Be sure to define behavior clearly If have long-term goals, have intermittent short-term goals to check progress – Do not over do it – Try to avoid setback situations If can’t recognize setback situations in advance, have coping strategies – Do not overreact to setbacks – Avoid counterproductive self-talk Incorporate everyday rewards into the program In programs where consequences are only cumulatively significant, – plan postchecks – have specific strategies to follow if the postchecks are unfavorable Practice self-control strategies to improve additional behaviors Involve supportive others Sign a behavioral contract with supportive others

45. Behavioral Contract Serves at least four important stimulus control functions: – Ensures all parties agree to goals and procedures – Ensures that all parties agree on how close they are to reaching the goals – Provides the client with a realistic estimate of the cost of the program to him or her in time, effort, and money – Signatures help ensure that all parties with follow procedures

46. Contract Troubleshooting Guide The Contract – Was the target behavior specified clearly? – If the target behavior was complex, did the contract ask for small approximations to the desired behavior? – Were specific deadlines identified for the target behavior? – Did the contract clearly identify situations where the target behavior should occur?

47. Contract Reinforcement Analysis – Did the contract provide for immediate reinforcement? Are the reinforcers still important and valuable? – Could reinforcers be earned often? – Did the contract call for and reward accomplishment rather than obedience? – Was the contract phrased in a positive way? – Do you consider the contract to be fair and in your best interests?

48. Contract Troubleshooting Guide The Mediator (your cosigner) – Did the mediator understand the contract? – Did the mediator dispense the kind and amount of reinforcement specified in the contract? – Did the mediator meet with you on the dates specified in the contract? – Is a new mediator required?

49. Contract Troubleshooting Guide Measurement – Are the data accurate? – Is your data collection system too complex or too difficult? – Does your data collection system clearly reflect your progress in achieving the target behavior? – Do you need to improve your data collection system?

50. What do the data say about self-control? Too often we are unable to control our own behavior unless there are external contingencies that drive our behavior If you don’t want to eat the cookies: arrange your contingencies carefully – Don’t by them – Get them out of the house – Let others know your plans Strengthen alternative responses and consequences – Focus on what ELSE you could be doing, not the thing you want to stop doing – Learning “coping strategies” There isn’t really any “self” control- you don’t stop wanting the thing you are avoiding, you just learn to find something better!!!!!