1. James R. McKay, Ph.D. University of Pennsylvania Philadelphia VAMC
Sequential multiple assignment randomized trial (SMART) adaptive studies for SUD
James R. McKay, Ph.D.
University of Pennsylvania
Philadelphia VAMC

2. Problems in SUD treatment
High non-response rate
Clients’ mixed reactions to “standard care” in the treatment system:
Behavioral interventions
Group counseling
12-step model (i.e., AA approach)
Currently, treatment seekers with substance use disorders (SUD) really do not have many TX options

3. Treatment as usual Intensive Outpatient Program (IOP)
Total of 9 hours of treatment per week, typically spread over 3 days
Primarily group counseling and group didactic sessions (e.g., films, lectures)
12-step, abstinence-oriented approach
Little to no availability of addiction medication
Standard Outpatient (OP)
Same as above, 1-2 hours/week

4. Evidence-based SUD treatments
Behavioral
Cognitive-Behavioral Therapy (CBT)
Motivational Enhancement Therapy (MET) and Motivational Interviewing (MI)
Behavioral Couples Therapy (BCT)
12-Step Facilitation Therapy (TSF)
Contingency Management (CM)
Medications
Naltrexone/vivitrol (alcohol/opioids)
Methadone/buprenorphine (opioids)

5. Limits of the evidence base
Effects are often relatively small
Studies tend not to support predicted moderation and mediation effects
As more studies are done, effects tend to get smaller (failure to replicate)
In most recent meta-analysis, CBT was no better than other active interventions
Therefore, rates of non-response to even the best “evidence-based” treatments are often surprisingly high

6. Hypothetical example
Trial comparing “BestShot” (BS) intervention to standard care (SC) for cocaine dependence
Rigorous study design with high follow-up rate and biologically confirmed cocaine outcomes assessed over 12 months finds:
BS yields a cocaine abstinence rate that averages 15 percentage points higher than SC across all follow-ups (50% vs. 35%, p< .05)
Authors conclude that BS should replace SC
But we still have half of the participants in BS failing to achieve cocaine abstinence at any given follow-up– what do we do for them??

7. Possible solution: Adaptive treatment
Develop an algorithm in which:
Client progress is monitored using standardized procedures and instruments
When non-response is apparent, modify treatment according to a predetermined set of decision rules
Methods for developing algorithm
Prior research/clinical judgment
Experimental procedures

8. Example
Cocaine dependent clients who do not achieve abstinence early in treatment are likely to have bad outcomes– what can be done to help them?? (i.e., what is the best “Plan B”?)
Does augmenting standard care with extended recovery support help?
Does also providing incentives to attend extended recovery support further improve outcomes by increasing sustained attendance?

9. Example, cont.
Run a trial in which non-responders (i.e., those who continue to drink/use cocaine) are randomized to:
Standard care only
SC plus augmentation with Telephone Monitoring and Counseling extended recovery support (TMC)
SC plus augmentation with extended recovery support plus incentives (TMC+)

10. Effects of Cocaine Use Early in Treatment
Baseline cocaine use p< TX x cocaine use interaction p< .07
TMC > TAU OR=1.95 p= TMC+ >TAU OR=1.58 p=.14

11. Effects of Alcohol Use Early in Treatment
Alcohol use p< TX x alcohol use interaction p=.04
TMC > TAU OR=2.47 p= TMC+>TAU OR=1.71 p= .09

12. Limitations of such studies
Rates of non-response may still be too high even in the best TX identified with the first randomization
Treatment resistant disorder
Moderately effective TX
Client response heterogenity
Need to identify the best “Plan C” for persistent non-responders

13. Possible solution!
Study design in which non-responders to the first adaptive modification are re-randomized to:
Another shot at “Plan B” vs. standard care
A new intervention– “Plan C”– vs. standard care or Plan B
Entirely new set of interventions– “Plan C vs. Plan D”
Referred to as Sequential Multiple Assignment Randomized Trial (SMART)

14. Using SMART design to address high dropout rate in SUD TX
The reality is that almost all available SUD TX is 12-step oriented interventions provided in outpatient programs via group counseling
Dropout rate is high– why?
Low efficacy
Considerable response heterogeneity
Many clients do not like this approach
What can be done to improve outcomes within this system?

15. Research questions
Does offering patients who do not engage in standard outpatient treatment a choice of other interventions improve outcomes?
Does offering patients who initially engage but then drop out a choice of other interventions improve outcomes?
Does a second attempt to offer TX choice to persistent non-engagers improve outcomes?

16. Tailoring Variable
We are tailoring on IOP attendance (rather than substance use)
Definition of “disengaged” was derived through an expert consensus process
At 2 weeks: failure to attend any treatment in the second week following intake
During weeks 3-7: failure to attend any IOP sessions for two consecutive weeks
At 8 weeks: failure to attend any IOP sessions in prior two weeks

17. Treatment Sites and Patients
Participants recruited from IOPs in publicly funded and VA programs
Participants enrolled at intake
Two studies:
Cocaine dependent (N=300), 80% with past or current alcohol dependence
Alcohol dependent (N=200), 40% with past or current cocaine dependence
Typical participant: African-American male in Philadelphia, around 40yo

18. Adaptive Protocol With Patient Choice
Week 2
Week 8
Intake to
Specialty
Care
(IOP)
Engaged
Patients
Telephone MI
For IOP
Engagement
Now
Engaged
Monitor for
Self-Selection
Two weeks
Randomization
Still
Non-Engaged
Non-Engaged
Patients
Telephone MI
With Choice
of TX Option
Second
Randomization
TEL MI
W/Choice
No Further
MI Calls
CBT
Medical
Management
Stepped
Care
IOP

19. Monthly Outcome Measures
Alcohol Use (for alcohol dependent Pts)
Any use and any heavy use
Frequency of any and heavy use
Cocaine Use (for cocaine dependent Pts)
Any use
Frequency of use
Urine toxicology

20. Study Participation Engaged/Disengaged at Week 2:
Study 1– 188 (63%) / 112 (37%) of 300
Study 2– 123 (62%) / 77 (38%) of 200
Disengaged Weeks 3-7:
Study 1—43 (23%) of 188 engaged at W2
Study 2—24 (20%) of 123 engaged at W2
Still disengaged at Week 8:
Study 1—66 (59%) of 112 disengaged W2
Study 2—43 (56%) of 77 disengaged W2

21. Treatment Options in MI-PC
Intensive outpatient program (IOP)
Cognitive-behavioral therapy (CBT)
Telephone stepped care (telephone)
Medication plus medical management (medication)

22. What non-engaged MI-PC PTs select in weeks 2-7:

23. What non-engaged MI-PC PTs select at week 8: (at re-randomization)

24. Alcohol Use in Patients Disengaged at 2 weeks
Main Effects Analyses
Alcohol Use in Patients Disengaged at 2 weeks

25. Any Alcohol Use in Month
Study 1
Study 2
p= .012
p= .028

26. Days of Alcohol Use per Week
Study 1
Study 2
p= .015
p= .02

27. Alcohol outcomes in combined sample (N=161 of 428 alc dep)
Any drinking:
OR= 0.40, p= .0007
Any heavy drinking
OR= 0.33, p= .001
Frequency of drinking
B= -1.08, p= .009
Frequency of heavy drinking
B= -1.09, p= .003
MI-PC= 0, MI-IOP= 1

28. Patients Not Engaged at 2 Weeks: Rates of Any Heavy Drinking in Each Follow-up Month

29. Patients Not Engaged at 2 Weeks: Frequency of Heavy Drinking Days in Each Follow-up Month

30. Alcohol Use in Patients Disengaged between weeks 3 and 7
Main Effects Analyses
Alcohol Use in Patients Disengaged between weeks 3 and 7

31. Disengaged in weeks 3-7 in combined sample (N=73)
Any alcohol use
OR= 0.54, p= .16
Any heavy alcohol use
OR= 0.67, p= .36
Frequency of use
B= -0.84, p= .23
Frequency of heavy use
B=-1.03, p= .10
MI-PC= 0, MI-IOP= 1

32. Alcohol Use in Patients Disengaged at both 2 and 8 weeks
Main Effects Analyses
Alcohol Use in Patients Disengaged at both
2 and 8 weeks

33. Disengaged at weeks 2 and 8 in combined sample (N=86)
Any alcohol use
OR= 1.12, p= .79
Any heavy alcohol use
OR= 1.43, p= .45
Frequency of use
B= -0.34, p= .58
Frequency of heavy use
B= 0.02, p= .97
MI-PC= 1, no further outreach=0

34. Main Effects Analyses
Cocaine Use Outcomes

35. Cocaine use (N= 409) PTs disengaged at w2 (N=159):
NS (P values .13 to .86)
PTs disengaged in w3-7 (N=69):
NS (p values .16 to .74)
(results in direction of IOP better than PC)
PTs disengaged w2 and w8 (N=84):
NS (p values .14 to .42)
(results in direction of NFO better than PC)

36. Moderators
Study, site, and dependence status (current vs. past) did not interact with treatment condition in most analyses
Exception:
In patients re-randomized at 8 weeks, those with past alcohol dependence benefited from MI-PC, whereas those with current dependence did not.
Same finding obtained with re-randomized cocaine dependent patients

37. Conclusions
Providing substance dependent patients who fail to engage in IOP a choice of other treatment options does not improve alcohol or cocaine use outcomes
In fact, outreach without a choice of other treatments leads to better alcohol use outcomes in those who do not engage in IOP initially

38. Conclusions
No advantage to providing outreach and a choice of interventions to patients who engage initially but then drop out
Providing further outreach with a choice of interventions to those not engaged at 2 and 8 weeks did not improve SUD outcomes compared to no further outreach
Possible exception: Patients with past rather than current dependence at intake

39. Encouraging results
It is possible to successfully implement a SMART project in SUD patients
Use of telephone MI made it possible to at reach most patients after 1st and 2nd randomization, even though they were not engaged in treatment at that point.
Significant treatment effects obtained (although in the opposite direction of what was predicted!)

40. Study limitations Did not consider impact of TX choice at intake
Alternative treatments were not provided by IOP staff, and were delivered at a different location
We did not offer some possible TX combi-nations (e.g., IOP+meds, meds+CBT, etc.)
No TAU control at first randomization, no MI-IOP condition at second randomization

41. Challenges in SMART trials for Substance Dependence
PTs who are doing badly are hard to reach and are often unwilling to participate further in treatment of any sort
Identifying non-responders early in the process and delivering modified intervention in a timely fashion
Mechanisms of action in behavioral treatment options may not be sufficiently different that PT doing poorly in one treatment will respond to another option
Having power to include all relevant comparison conditions at each randomization
Still using group average results to develop decision rules for individuals

42. Exciting new developments
Using mobile health communication technology to:
Assess clients much more frequently, without becoming a burden to them
Make use of passive, as well as active, assessments (e.g., activity level, location)
Deliver automated interventions immedi-ately when non-response is detected
Conduct SMART studies with many separate randomizations

43. Funding Support for this study provided by NIH grants:
P60 DA05186 (O’Brien, PI)
P01 AA (McKay, PI)
K24 DA (McKay, PI)
RC1 AA (Lynch, PI)

44. Collaborators Penn Other Institutions Dave Oslin Kevin Lynch
Tom Ten Have
Debbie Van Horn
Michelle Drapkin
Other Institutions
Susan Murphy, Inbal Nahum-Shani, Danny Almirall, University of Michigan
Linda Collins, Penn State

45. Acknowledgments Our Research Team Oubah Abdalla John Cacciola
Rachel Chandler
Dominic DePhilippis
Michelle Drapkin
Ayesha Ferguson
Ellen Fritch
Jessica Goodman
Angela Hackman
Dan Herd
Laurie Hurson
Ray Incmikoski
Laura Harmon
Megan Long
Jen Miles
Jessica Olli
Zakkiyya Posey
Alex Secora
Tyrone Thomas
Debbie Van Horn
Sarah Weiss
Tara Zimmerman