1. Piloting and Sizing Sequential Multiple Assignment Randomized Trials in Dynamic Treatment Regime Development 2012 Atlantic Causal Inference Conference May 25, 2012—Johns Hopkins Daniel Almirall & Susan A. Murphy TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAA

2. Outline Dynamic Treatment Regimes Sequential Multiple Assignment Randomized Trial (SMART) External Pilots –Tailoring Variables –Transition to Next Stage –Assessment Schedule –Sizing a Pilot SMART 2

3. 3 Dynamic treatment regimes are individually tailored sequences of treatments, with treatment type and dosage changing according to patient outcomes. Operationalizes clinical practice. k Stages for one individual Patient data available at j th stage Action at j th stage (usually a treatment)

4. Dynamic Treatment Regimes A dynamic treatment regime (DTR) is a sequence of decision rules, one per treatment stage. Each decision rule inputs one or more tailoring variables and outputs a treatment action. The tailoring variables are (summaries of) patient data available at each stage. 4

5. 5 Example of a Dynamic Treatment Regime (DTR) Adaptive Drug Court Program for drug abusing offenders. Goal is to minimize recidivism and drug use. Marlowe et al. (2008, 2009)

6. 6 Adaptive Drug Court Program

7. 7 Sequential, Multiple Assignment, Randomized Trial (SMART) At each stage subjects are randomized among alternative options. For k=2, data on each subject is of form: A j is a randomized treatment action with known randomization probability.

8. 8 Usually the treatment options for A 2 are restricted by the values of one or more summaries of (X 1, A 1, X 2 ) These summaries are embedded tailoring variables; they are embedded in the experimental design. The embedded tailoring variable(s) restrict the class of DTRs that can be investigated using data from the SMART.

9. Pelham ADHD Study Begin low dose Med 8 weeks Assess- Adequate response? Continue, reassess monthly; randomize if deteriorate Med ++ Random assignment: BMOD + Med No Yes Begin low-intensity BMOD 8 weeks Assess- Adequate response? Continue, reassess monthly; randomize if deteriorate BMOD + Med Random assignment: BMOD ++ Yes No Random assignment: 9

10. ADHD: Embedded Tailoring Variable Early response is determined by two teacher- rated instruments, ITB and IRS. Binary embedded tailoring variable R=0 if ITB 3; otherwise R=1. R is the embedded tailoring variable. 10

11. External Pilot Studies Goal is to examine feasibility of full-scale trial. –Can investigator execute the trial design? –Will participants tolerate treatment? –Do co-investigators buy-in to study protocol? –To manualize treatment(s) –To devise trial protocol quality control measures Goal is not to obtain preliminary evidence about efficacy of treatment/strategy. –Rather, in the design of the full-scale SMART, the min. detectable effect size comes from the science. 11

12. Embedded Tailoring Variable Don’t use an embedded tailoring variable unless the science demands it. If you have an embedded tailoring variable make it simple (e.g. binary measure of (non-) response) –Non-responders likely to fail if continue on current treatment OR responders unlikely to gain much benefit if they stay on current treatment. –Usually need to use analyses of existing data to justify the use of the tailoring variable 12

13. Jones’ Study for Drug-Addicted Pregnant Women rRBT 2 wks Response Continue on same Random assignment: Increase scope/intensity Nonresponse tRBT Random assignment: Decrease scope/intensity 2 wks Response Random assignment: Increase scope/intensity Continue on same Decrease scope/intensity Nonresponse 13

14. Missing Tailoring Variable How to manage missingness in the embedded tailoring variable for purposes of randomizing/assigning subsequent treatment? –VERY different from handling missing data in a statistical analysis. –Tailoring variable is part of the definition of the treatment and experimental design. 14

15. Missing Tailoring Variable Need to formulate a fixed, pre-specified rule to determine subsequent treatment if tailoring variable is missing. –Unexcused visit==non-response –Use a rule that depends on all observed data, including the data collected when the subject again shows up at a clinic visit. –Try out the rule in pilot. 15

16. Assessment Schedule How often should the tailoring variable be measured? Example: Alcoholism study with weekly assessments of days of heavy drinking. –Weekly assessments were insufficient and likely a pilot study would have detected this. 16

17. 17 Oslin’s ExTENd Study Nonresponse if HDD>4 8 wks Response TDM + Naltrexone CBI Random assignment: CBI +Naltrexone Nonresponse Nonresponse if HDD >1 Random assignment: Naltrexone 8 wks Response Random assignment: CBI +Naltrexone CBI TDM + Naltrexone Naltrexone Nonresponse

18. Outcome Assessment versus Tailoring Variable Assessment Keep these separate. –Tailoring variable assessment done at clinic visit by clinical staff or clinical lab or participant. Outcome assessment done at research visit by independent evaluator or independent lab or participant. Autism & Adolescent Depression Examples Try out in Pilot Study 18

19. Transition Between Stages Clinical staff disagree with when 2 nd stage treatment is introduced. Non-responding subject refuses 2 nd stage treatment. –This may be VERY important scientifically –Cocaine/Alcoholism Example Test in Pilot 19

20. Sample Size for a SMART Pilot Primary feasibility aim is to ensure investigative team has opportunity to implement protocol from start to finish with sufficient numbers –If investigator has good evidence to guess the response rate: Choose pilot sample size so that with probability q, at least m participants fall into the sub-groups (the “small cells”) –If little to no evidence concerning response rate, size the study to estimate the response rate with a given confidence interval width. 20

21. Pelham ADHD Study Begin low dose Med 8 weeks Assess- Adequate response? Continue, reassess monthly; randomize if deteriorate Med ++ Random assignment: BMOD + Med No Yes Begin low-intensity BMOD 8 weeks Assess- Adequate response? Continue, reassess monthly; randomize if deteriorate BMOD + Med Random assignment: BMOD ++ Yes No Random assignment: 21

22. Sample Size for a SMART Pilot There are 2 treatment actions in stage 1, k R treatments for responders, k NR treatments for non-responders. Investigator chooses q (say 80%) and m (say 3), and assumes overall non- response rate p NR (say 50%). Solve for N, the total sample size, where 22

23. Discussion SMART clinical trial designs are of growing interest in the clinical sciences. Because these designs are very new, they require a great deal of leadership on the part of the statistical community. The payoff for the statistician is –Inform clinical science in a novel manner –Unusual and novel trial data for methodological development 23

24. 24 This seminar can be found at: http://www.stat.lsa.umich.edu/~samurphy/ seminars/ACIC_2012.ppt Reference: Almirall D, Compton SN, Gunlicks-Stoessel M, Duan N, Murphy SA. Designing a Pilot SMART for Developing an Adaptive Treatment Strategy. To appear in Statistics in Medicine