2. University of Michigan, Biostatistics
SMART Design
Kelley M Kidwell, PhD
University of Michigan, Biostatistics
Tools to Apply Novel Methods for the Design and Analysis of Clinical Trials

3. Dichotomy in how we PRACTICE vs. study treatment
I want to motivate my talk by discussing how there is a bit of a dichotomy in the way that we practice and study many treatments, especially for acute and chronic diseases. In these settings, treatment doesn’t stop, but is an ongoing process based on patient response and behaviors. For example, here is a recommendation for treating depression. We see here, it begins with initiating medication but then individuals are assessed and based on their response, the dose may be continued, adjusted, or the treatment changed all together.

4. Dichotomy in how we PRACTICE vs. study treatment
Similarly, here is a guideline to treating bladder Cancer where treatment depends on baseline risk and again, subsequent trt depends on risk

5. Dynamic Treatment Regimens (DTRs)
a.k.a. adaptive intervention, adaptive treatment strategy, stepped care, treatment policies
Sequence of individually tailored decision rules that specify whether, how and/or when to alter the intensity, type, dose or delivery of treatment at critical decision points in the course of care
Guide/Formula for treatment
Goal: operationalize sequential decision making with the aim of improving clinical practice
This is a fancy term for how we actually practice much of medicine or treatment, especially for chronic diseases or acute illnesses that may wax and wane

6. Dynamic Treatment Regimens (DTRs)
Prostate Cancer:
First receive combination chemotherapy paclitaxel + estramustine + etoposide (TEC).
If successful (a decrease of 40% or more in PSA from diagnosis, with no evidence of disease progression at any site) at 8 weeks, then continue TEC;
otherwise switch to cyclophosphamide + vincristine + dexamethasone (CVD).
Alcohol Abuse
First take naltrexone and receive in person medical management
After 2 weeks, but before 8 weeks, if the individual has 2 or more heavy drinking days then add cognitive behavioral therapy
After 2 weeks, but before 8 weeks, if the individual has less than 2 heavy drinking days, replace in person medical management with telephone disease management
Med Mgmt: MM is a face-to-face, basic, minimal clinical support for the use of effective pharmacotherapy and reduction in drinking
Assessed weekly for drinking behavior

7. Dynamic Treatment Regimens (DTRs)
Sequences of treatments are relevant:
Waxing and waning of disease/disorder
No widely effective treatment
Treatments may be costly or burdensome
Adherence problems
Within and between person heterogeneity

8. Dichotomy in how we practice vs. STUDY treatment
But even though much of clinical practice involves these tailored treatment sequences, we often run clinical trials that looks at just one piece of the treatment. Like here a standard parallel, 2 arm trial for depression assessing CBT vs. psychodynamic therapy
The efficacy of CBT and Psychodynamic therapy in the outpatient treatment of major depression: a RCT, American Journal of Psychiatry 2013

9. Some Consequences
Compare treatments A vs. B for first-line treatment
Response Rates
A: 60%
B: 50%
Treatment A wins
Test efficacy of second-line treatment C for non-responders
A followed by C: 10%
B followed by C: 40%
Treatment B followed by C wins
Overall sequence
A,C: 64% (60%+40%*10%)
B,C: 70% (50%+50%*40%) A B C
The way we generally study medicine in this stage specific manner: first finding the best front-line treatment and then assessing the best second line treatment in those who fail the best front-line therapy may actually be missing some of the best sequences. A toy example is shown here where the best initial treatment followed by the best follow up treatment in non-responders is actually not the best overall strategy, but we wouldn’t be able to assess that if we never went on to test C following the runner up treatment B.

10. Some Consequences
Thus, not using the best trial designs may be a piece of the puzzle to why we’re putting so much money and time into drugs that don’t come to fruition

11. Sequential Multiple Assignment Randomized Trial
A type of multi-stage randomized design
Trial participants are randomized to a set of treatment options at critical decision points over the course of treatment
All individuals participate in all stages of the trial
Subsequent randomization is based on information leading up to that point (tailor treatment)
DTRs embedded in design
Goal: develop effective DTRs
A type of trial design that can identify the best tailored sequences of trts or DTRs is called a SMART design. This is NOT an adaptive trial where parameters change for future participants based on previous participants, but rather a multistage design where the same patients are followed throughout, but they may be re-randomized based on their intermediate outcomes. The goal of a SMART is to identify or develop effective dynamic treatment regimens so that these DTRs are embedded within the design.

12. SMART Example C R Responders D A Non-responders E R Entry F R C RB
Non-responders E
For example, this is a generic 2-stage SMART design where participants are randomized between A and B initially and then based on their intermediate response status, they are re-randomized into some treatment: C or D for responders and E or F for non-responders. 1 participant follows one pathway through the SMART design, but the SMART embeds multiple DTRs. R F

13. SMART Example: DTRs C R Responders D A Non-responders E R Entry F R CB
Non-responders E
Here is one DTR within this SMART where first you receive A and if you respond, you receive C and if you don’t respond then you receive E> R F

14. SMART Example: DTRs C R Responders D A Non-responders E R Entry F R CB
Non-responders E
Here’s another DTR. There are actually 8 embedded DTRs in this design. R F

15. SMART Example C Responders A Non-responders E R Entry F R C RespondersB
Non-responders E
SMART designs can look different so that only one group of participants (non-responders) are re-randomized R F

16. SMART Example C Responders A Non-responders E R Entry F R C RespondersB
Non-responders E
And the designs do not have to be balanced. Further, there can be any number of treatments at each stage and more than 2 stages

17. SMARTS in the Field Oncology Schizophrenia Drug abuse Depression ADHD
Insomnia
Alcoholism
Bipolar
Obesity
Conduct problems
OCD
Smoking cessation
Autism
Suicide prevention
There have been a decent number of SMARTs designed and implemented in the field. These are some areas where SMARTs have been run and the Penn State Methodology website details a number of these designs.

18. SMART Benefits Investigates Questions of
What treatment ?
Which order?
When/for whom to change treatment or dose?
When/for who to combine therapies?
How/when to measure early response to tailor subsequent treatment?
Takes advantage of intra- and inter-patient heterogeneity
So, SMARTs can answer questions like What treatment, but also address issues of which order should the treatment be in and when and for should it change? SMARTs take advantage of both intra-and inter-patient heterogeneity to try to identify the best treatment guidelines.

19. SMART Benefits Delayed Effects – treatment synergies or antagonisms
Prescriptive Effects – initial treatment may elicit symptoms to better match individual to subsequent treatment
Sample Selection Effects – individuals who enroll in, remain in or are adherent in a SMART may be different from those in other designs
SMARTs are particularly useful trial designs when there may be delayed, prescriptive or sample selection effects. The delayed effects are that which I showed in the toy example where the best initial treatment may not lead to the best overall strategy when followed by another treatment. There may be treatment synergies or antagonisms that a SMART can detect . Further SMARTs allow for the collection of data that may better match subsequent treatment and participants may be more attracted to a SMART design knowing they will receive follow-up treatment if their initial treatment is not successful so that SMART participants may be more generalizable than those in standard trial designs.
Delayed effect: Treatment may not appear best initially but may lead to
longest survival when followed by maintenance treatment
Treatment may produce highest number of responders initially,
but may produce subsequent side effects reducing maintenance
options, maintenance effectiveness, or adherence to
maintenance treatment

20. SMART: Scientific Aims
Compare first-stage treatments
Compare second-stage treatment options or strategies (for non-responders)
Compare or identify the best DTR
Develop a more deeply tailored or personalized DTR
SMARTs can identify the best DTRs, but the primary aim do not necessarily have to be about the DTRs. The primary aim, the one that sample size is based on may be based on comparing first-stage treatments.

21. SMART Example C R Responders D A Non-responders E R Entry F R C RB
Non-responders E
For example, the primary aim may be : does the best treatment strategy begin with A or B? Comparing those in yellow vs. Blue. This would require the same number of patients as a standard 2 arm trial, but you’d be able to explore DTRs, estimate the outcome for the embedded DTRs and potentially choose the best or set of best to go forward with in another, confirmatory trial. R F

22. SMART: Scientific Aims
Compare first-stage treatments
Compare second-stage treatment options or strategies (for non-responders)
Compare or identify the best DTR
Develop a more deeply tailored or personalized DTR
SMARTs can also address questions about the best second-stage treatment options for responders or non-responders (or both). The SMART may be powered on this aim or it may be secondary.

23. SMART Example C R Responders D A Non-responders E R Entry F R C RB
Non-responders E
For example, we may be interested in if the best treatment for non-responders is E or F. We can pool those who received E and compare those who received F and adjust for their initial treatment in analysis. If this was the main question of interest, the sample size could also be calculated similar to a standard 2 group trial, but we’d have to upweight it by the percentage of expected non-responders to find the total sample size needed at the start of the trial. R F

24. SMART: Scientific Aims
Compare first-stage treatments
Compare second-stage treatment options or strategies (for non-responders)
Compare or identify the best DTR
Develop a more deeply tailored or personalized DTR
Because it is a SMART, one aim will undoubtedly be to compare DTRs or to identify the best DTRs.

25. SMART Example C R Responders D A Non-responders E R Entry F R C RB
Non-responders E
For example, we may be interested in comparing one DTR (A,C,E) to the DTR (B,D,F). Perhaps one of these is the simplest or least burdensome or costly DTR and the other is the most aggressive/burdensome/costly DTR. This sample size and resulting analysis is not a standard calculation and requires methods developed specifically for SMART designs. References for how to calculate this type of sample size and analyze such data are given at the end. R F

26. SMART: Scientific Aims
Compare first-stage treatments
Compare second-stage treatment options or strategies (for non-responders)
Compare or identify the best DTR
Develop a more deeply tailored or personalized DTR
Aims may go beyond identifyin the best embedded DTR and try to personalize the DTR even more.

27. SMART Example C R Responders D A Non-responders E R Entry F R
Within 1 month C R
Responders D B
Non-responders E
For example, by collecting baseline and time-varying information throughout the trial, we can assess which characteristics are associated with particular DTRs and develop more personalized guidelines. This is unlikely to be a primary goal of the SMART and so not powered on (in fact power analyses for this type of aim do not exist), This analysis requires specific methods for SMART design adapted from computer science, like q-learning or other recursive regression algorithms. R
Females, Age>50
adherent F

28. SMART Design Principles
Keep it simple, straightforward
Power for simple important primary hypothesis
Take appropriate steps to develop a more deeply personalized DTR
So, a SMART may look more complex, but the goal is to keep it simple and straightforward, to power for primary hypotheses and to take appropriate steps to develop more deeply personalized DTRs

29. SMART Design Principles
At each stage or critical decision point
Restrict treatment options only by ethical, feasible, or strong scientific considerations
If you restrict randomization use a low dimensional summary (easy to use in actual clinical practice)
Collect additional auxiliary time-varying measures
Additional auxiliary time-varying measures such as quality of life, toxicities, adherence, etc. that may be affected by the first treatment and allow for better assignment of second treatment.

30. Summary
Dynamic treatment regimens are guidelines for clinical practice
A SMART is a clinical trial design that can build better DTRs
SMARTs do not need to be complicated or require larger sample sizes

31. Resources: Websites https://methodology.psu.edu/ra/adap-inter/projects

32. Resources: Introductory Articles
Lei H, Nahum-Shani I, Lynch K, Oslin D, Murphy SA. A “SMART” design for building individualized treatment sequences. The Annual Review of Clinical Psychology, :21-48.
Almirall, D., Nahum-Shani, I., Sherwood, N.E., Murphy, S.A. Introduction to SMART designs for the development of adaptive interventions: with application to weight loss research. Translational behavioral Medicine, (3):
Chakraborty, B. and Murphy, S. A. Dynamic treatment regimes. Annual Review of Statistics and its Applications :
Nahum-Shani I, et al. Experimental design and primary data analysis methods for comparing adaptive interventions. Psychological Methods :
Kidwell, K.M. SMART designs in cancer research: past, present and future. Clinical Trials (4):
Lavori, P.W., Dawson, R. Introduction to Dynamic Treatment Strategies and sequential multiple assignment randomization. Clinical Trials (4):

33. Resources: Textbooks
Adaptive Treatment Strategies in Practice: Planning Trials and Analyzing Data for Personalized Medicine. Ed. Kosorok & Moodie ASA-SIAM.
Statistical Methods for Dynamic Treatment Regimes: Reinforcement Learning, Causal Inference, and Personalized Medicine. Chakraborty and Moodie Springer.