1. Kathleen Carroll & Brian Kiluk Division of Substance Abuse Yale University School of Medicine Supported by NIDA Supplement to R01 DA15969 and P50 DA09241, U10 DA015831, R01 DA019078, & R01 DA 10679

2. Why do we need a sound and valid indicator?  Facilitation of comparisons across projects, meta-analyses  Set and monitor performance standards  Benchmarking  Clearly convey magnitude of treatment effects to stakeholders  Facilitate comparisons across common standard  Lack of incentive to improve performance and outcome (retention not appropriate standard)

3. Overview  Desirable characteristics of indicators  Strengths and weaknesses of common approaches  Overview of our project

4. “Traditional” indicators of clinical significance almost always translate to complete abstinence  Return to normative levels  Reliable change indices  Return to healthy functioning? (e.g.,equivalent of ‘no heavy drinking days’ for stimulant users)

5. What are we looking for in an indicator?  Easy to calculate, interpret  Psychometrically sound, reliable, replicable  Low susceptibility to missing data  Verifiable (biologic indicator, other)  Independence from baseline measures  Sensitive to treatment effects  Low(er) cost  Predicts long-term cocaine outcomes  Related to indicators of good long term functioning  Acceptable to field  Easily interpreted by clinicians, policy makers, payors

6. What is ‘success’ in treating stimulant users?  Durable periods of abstinence  Employment, productivity  Lack of criminal activity  Reduced use of expensive, avoidable health care resources 11% at end of treatment, 21% at end of 1 year follow up

7. Why not complete abstinence?  Insensitive to change  Difficult standard for most individuals (14% of our sample of 434)  Chronically relapsing disorder  Change is dynamic  Starting and remaining abstinent may imply questionable need for treatment  Our data: Weak relationship with cocaine use and functioning outcomes at one year

9. Retention Pros  Easy to calculate  Available for all participants  Indicator of treatment acceptability  Indicator of differential attrition/data availability across conditions Cons  May be more meaningful in some contexts than others  Participants leave treatment for different reasons  Is retention with continued use meaningful?  Is compliance with ineffective treatment meaningful?  Not related to long-term outcome in our sample

10. Percent negative urines Pros  Widely used and accepted  Less susceptible to demand characteristics, misrepresentation  Quantifiable, ability to detect new episodes  Very accurate, if appropriate schedule of collection and minimal missing data  Timing is critical (overlap, missing data) Cons  Recent use only (3-5)  High cost for frequent or quantitative  Sensitive to missing data, esp. with differential attrition  Depends on assumptions (missing, denominator)  Stimulants or all drugs?  Can’t back-fill  Problems with assuming missing=positive*

11. Calculating percent urine samples Example: 1 negative urine, 2 sessions, then dropout of 12 week trial. Based on submitted: 100% Based on possible: 50% Based on expected/ 1x 8% Based on expected/ 3x 3% Percent cocaine positive 0%

12. Longest consecutive x-free urine specimens Pros  Strong evidence of meaningful abstinence  Less susceptible to demand characteristics, misrepresentation  Quantifiable, ability to detect new episodes  Very accurate, if appropriate schedule of collection and minimal missing data  Timing is critical (overlap, missing data) Cons  High cost for frequent or quantitative  Very sensitive to missing data, esp. with differential attrition  Depends on assumptions (missing, denominator)  Stimulants or all drugs?  Can’t back-fill

13. Percent days abstinent, self report Pros  Widely used  Potentially available for all participants and all days if TLFB used with high data completion; highly flexible  True intention to treat possible  Can be reliable if methods to enhance reliability used (at a cost)  Our discrepancy rate=8-12% Cons  With high/differential dropout, what’s the denominator? Days in treatment versus days expected?  Not easy to correct with urine data if discrepancies high

14. Maximum days of abstinence, overall or in final x weeks Pros  Linked linked to longer-term cocaine use  Potentially verifiable if urines collected at appropriate intervals  Provides ‘grace period’  Easily dichotomized (eg 3 plus weeks) Cons  High complexity with missing data, especially dropouts  High complexity if discrepant urine data  Participants last 2 weeks or last 2 weeks of trial?  End of treatment or sometime within treatment?

15. Reduction in use: Frequency and or quantity Pros  Alternative to abstinence; more achievable target?  Highly compatible with random regression models  Sensitive to treatments that may take time for effects to emerge  Provides ‘grace period’  Easily dichotomized Cons  Complexity obtaining accurate estimates of frequency/quantity of use prior to baseline  When is reduction measured (last weeks? Entire course?  Costs of repeated quantitative urines, sensitivity to missing data

16. Issues in defining ‘reduction’  Patterns vary widely (binge versus low use)  Reliable estimation of quantity complex (illicit, no standard units, adulterants common, potency varies, hard to standardize ‘hits’ ‘joints’ ‘dime bags’)  Difficulty of estimating dollar value (commerce, shared use, sex for drugs)

17. Which indicator of treatment response? Loss of power with dichotomous, but also easily interpretable, calculable for all, relevant to clinical significance  Candidates *Complete abstinence *3 or more weeks of abstinence *End of treatment abstinence *Reduction of x percent “Good functioning”

18. Indicator Ease of computation VerifiabilityVulnerability to missing data Relative costOperationalization for these analyses 1 Days retained in treatment protocolC EasyYes-Low Days from randomization to endpt 2 Percentage of urine specimens testing positiveC Easy for complete dataYes, by definitionAssumes independence of urine specimens (denominator), assumes numerator is unbiased by collection schedule or missing data. HighNumber of cocaine- negative urine specimens collected / all specimens collected 3 Maximum consecutive days abstinent CEasy for complete dataYes, provided appropriate schedule of data/urine collection Likely to result in casewise missingness or reduced sample size Moderate, due to biological verification and derivation from TLFB Longest continuous cluster of self-reported abstinence within treatment 4 Percent days of abstinence from cocaineC Depends on treatment duration, level of missing data, and intermittent missingness Yes, provided appropriate schedule of data/urine collection Likely to result in casewise missingness or reduced sample size Moderate, due to biological verification and derivation from TLFB Number of self-reported days of abstinence from cocaine / days in treatment (retention) 5 Maximum days of continuous abstinence during last two weeks of treatmentC Complex for intermittent and monotone, dropouts Yes, provided appropriate schedule of data/urine collection LowModerate, due to biological verification and derivation from TLFB For those retained 14+ days, longest cluster of abstinence in final 2 weeks; otherwise 0 6 Completely abstinent last two weeks of treatmentD EasyYes, provided appropriate schedule of data/urine collection LowModerate, due to biological verification and derivation from TLFB For those retained 14+ days, 0 days of use in last 14 days, otherwise 0 7 3 or more weeks of continuous abstinenceD EasyYes, provided appropriate schedule of data/urine collection LowModerate, due to biological verification and derivation from TLFB “Yes” if participant retained 21+ days, max days abstinent > 20. Otherwise No 8 2 or more weeks of continuous abstinenceD EasyYes, provided appropriate schedule of data/urine collection LowModerate, due to biological verification and derivation from TLFB “Yes” if participant retained 14+ days, max days abstinent > 13. Otherwise No Note. C=continuous, D=Dichotomous, TLFB=Timeline Followback method

19. Indicator Ease of computationVerifiabilityVulnerability to missing data Relative costOperationalization for these analyses 9 1 or more weeks of continuous abstinenceD EasyYes, provided appropriate schedule of data/urine collection LowModerate, due to biological verification and derivation from TLFB “Yes” if participant retained 7+ days, max days abstinent > 6. Otherwise No 10 Completely abstinent from cocaine during treatment D EasySameLowModerate, due to biological verification and derivation from TLFB 0 days of use and 0 positive urines 11 Completed treatment and abstinent in last weekD EasyYesLow Completion of treatment, 0 days of use in final week 12 Percent reduction in frequency of use (28 days prior/days last 4 weeks)C Complex, baseline definition can be arbitrary No, relies on accurate baseline/pretreatment assessment ModerateLowPercent days of use in final 28 days of treatment/ percent days of use in 28 days prior to baseline 13 50% reduction in frequency of useD Complex, baseline definition can be arbitrary Relies on access to accurate baseline/pretreatment level of use ModerateLow% reduction is 50% or higher 14 75% reduction in frequency of useD Complex, baseline definition can be arbitrary SameModerateLow% reduction is 75% or higher 15 Report no drug use, legal, employment, or psychological problems last 28 days of treatmentD EasyPartialLow Completes treatment, 0 days of problems in drug, legal, employment and psych ASI in past 28 days Note. C=continuous, D=Dichotomous, TLFB=Timeline Followback method

20. Indicator Ease of computation Biological verification Vulnerability to missing data Sensitivity to treatment effects Relationship with post tx outcomes Independent from baseline indicators Relationship to measures of general function 21-30 days of abstinence XRelies on appropriate schedule Low Completed treatment and abstinent in last 2 weeks XSame Low 50 % reduction Complex, baseline definition can be arbitrary Relies on having accurate baseline/pretreat ment assessment of use Moderate % days abstinent Depends on treatment duration, complex for dropouts, and intermittent missingness X, provided appropriate schedule of data/urine collection Moderate Max days consecutive abstinence Complex for intermittent and monotone missingness, dropouts X, provided appropriate schedule of data/urine collection Likely to result in casewise missingness or reduced sample size Percent neg ative urine specimens Easy except when missing data yes

21. So far…  Carroll, K.M., Kiluk, B.D., et al. (2014). Towards empirical identification of a reliable and clinically meaningful indicator of treatment outcome for illicit drug use. Drug and Alcohol Dependence, 137, 3-19.  Kiluk, B.D., et al. (2014). What happens in treatment doesn’t stay in treatment: Cocaine abstinence during treatment is associated with fewer problems at follow-up. J Consulting and Clinical Psychology, 82:619-27.  DeVito, E.E., et al. (2014). Gender differences in clinical outcomes for cocaine dependence: Randomized clinical trials of behavioral therapy and disulfiram. Drug and Alcohol Dependence, 145: 156-167.  Decker, S.E., et al. (2014). Assessment concordance and predictive validity of self-report and biological assay of cocaine use in treatment trials. The American Journal on the Addictions, 23, 466-74.  Kiluk, B.D., et al. (in press). Prompted to treatment by the criminal justice system: Relationships with treatment retention and outcome among cocaine users.