Aug 1, Conducting Cost-Effectiveness Analyses of Behavioral Interventions Todd H. Wagner, Ph.D. Mary K. Goldstein, M.D. Mary K. Goldstein, M.D.

Aug 1, Acknowledgements Partial funding was through a grant from the National Cancer Institute (2PO1 CA A1). Partial funding was through a grant from the National Cancer Institute (2PO1 CA A1). Dr. Goldstein ’ s contribution was informed in part by work conducted with a grant from the National Institute on Aging (R01 AG15110). Dr. Goldstein ’ s contribution was informed in part by work conducted with a grant from the National Institute on Aging (R01 AG15110). David Meltzer, Jodi Prochaska, Lisa Faulkner, and Stanford University seminar participants provided helpful comments. David Meltzer, Jodi Prochaska, Lisa Faulkner, and Stanford University seminar participants provided helpful comments.

Aug 1, Outline of the Talk Background on cost-effectiveness analysis (CEA) Background on cost-effectiveness analysis (CEA) A bias in CEAs for behavioral interventions A bias in CEAs for behavioral interventions How to fix the problem How to fix the problem Example Example Study design consideration Study design consideration

Aug 1, Economic Evaluations Most trials don’t include economic analysis Most trials don’t include economic analysis Economic analysis in only 0.2% of 50,000 trials conducted between Economic analysis in only 0.2% of 50,000 trials conducted between –Adams et al. Medical Care 30(3): Economic analysis in randomized clinical trials is increasing Economic analysis in randomized clinical trials is increasing

Aug 1, Dollars and Sense Economic studies inform decisions Economic studies inform decisions –formulary –adoption of new technology –scope of benefits –strategies for management of care –organization of health care

Aug 1, Cost Effectiveness Ratio Where C 1 is the average cost of the intervention group C 0 is the average cost of the control group E 1 is the average effectiveness of the intervention group E 0 is the average effectiveness of the control group

Aug 1, Standardization USPTF guidelines (Gold et al., 1996; summarized in three JAMA articles) USPTF guidelines (Gold et al., 1996; summarized in three JAMA articles) Drummond et al. (1997) Drummond et al. (1997) Created, along with journal editors, standards for both Created, along with journal editors, standards for both –Methods –Reporting

Aug 1, Whose Costs? Guidelines recommend a societal perspective for costs Guidelines recommend a societal perspective for costs Include Include –Provider –Payer –Patient

Aug 1, Effectiveness Guidelines recommend QALYs Guidelines recommend QALYs Values both quality and quantity of life Values both quality and quantity of life Each year of life is multiplied by a weighting factor (utility) Each year of life is multiplied by a weighting factor (utility) Utilities measure the preference of different health states on a 0-1 scale Utilities measure the preference of different health states on a 0-1 scale

Aug 1, Behavioral Interventions CEA framework holds, but caveats … CEA framework holds, but caveats … –Behavior change is a “ slow ” process –Treat many to prevent a few –Use of intermediate outcomes (proxies)

Aug 1, Intermediate Outcomes Outcome is clinically relevant and predicts mortality or morbidity Outcome is clinically relevant and predicts mortality or morbidity –Receipt of a mammogram –Substance use abstinence –Change in dietary fiber QALYs would require huge and/or very long studies QALYs would require huge and/or very long studies

Aug 1, CEA with an Intermediate Outcome Sufficient for publication Sufficient for publication Hard to interpret ICER Hard to interpret ICER –Can’t easily compare two CEAs with different intermediate outcomes –Can’t compare CEA to other CEA from another clinical area Sometimes only feasible approach Sometimes only feasible approach

Aug 1, CEA with QALYs Measure QALYs or Measure QALYs or Translate intermediate outcome to QALYs Translate intermediate outcome to QALYs –Either build a model de novo or use an existing model –Requires a lot of resources Most useful, but most challenging Most useful, but most challenging

Aug 1, CEA and Behavior Change CEA analysts treat behavior change as a dichotomous outcome CEA analysts treat behavior change as a dichotomous outcome Partial behavior change is not the same as no behavior change Partial behavior change is not the same as no behavior change “ Getting the person to recognize that they have a problem is half the battle. ” “ Getting the person to recognize that they have a problem is half the battle. ”

Aug 1, What ’ s Missing? Partial behavior change is missing from current models Partial behavior change is missing from current models –People who progressed in their process of changing their behavior but did not successfully change their behavior at the end of the study (Stages of Change)

Aug 1, Partial Behavior Change All behavioral interventions yield some partial behavior change All behavioral interventions yield some partial behavior change Amount depends on Amount depends on –the duration of the study –baseline stage of change

Aug 1, Partial Behavior Change Should we and can we value partial behavior change in a CEA?

Aug 1, Should We? NO Intentions are not the same as observed behavior change Intentions are not the same as observed behavior change Stage of change is flawed Stage of change is flawed Too difficult Too difficultYES Behavior change takes time Behavior change takes time Unobservable differences exist Unobservable differences exist Future benefits are important Future benefits are important Otherwise favors med/surg tx Otherwise favors med/surg tx

Aug 1, Can We? Behaviors are like “ value chains ” Behaviors are like “ value chains ” –A series of linked processes –Interventions may be designed to improve a link (stage-matched design)  Matching chemotherapy protocol to cancer stage –Interventions may have differential effects on different links Stages of Change (TTM) Model by Prochaska and DiClimente Stages of Change (TTM) Model by Prochaska and DiClimente

Aug 1, Stages of Change People progress through successive stages until they change their behavior People progress through successive stages until they change their behavior –Precontemplation: no self-recognition of a problematic behavior. –Contemplation: self-recognition of a problem without action. –Preparation: Planning to change behavior soon. –Action: In the process of change. –Maintenance: adherence to the new behavior over time. Art of behavioral interventions is to achieve action, and hopefully maintenance Art of behavioral interventions is to achieve action, and hopefully maintenance

Aug 1, Stages of Change and CEA Stages of change is critical for interpreting CEA results Stages of change is critical for interpreting CEA results –Stage of change may be associated with receptivity or motivation –Intervention effects can vary by stage of change –Incremental cost-effectiveness ratio can vary by stage of change

Aug 1, Integrating Stages of Change in a CEA

Aug 1, What’s Needed? Data on stages of change Data on stages of change Probability of moving from partial to successful behavior change Probability of moving from partial to successful behavior change –Note: these probabilities are not observed in the intervention group

Aug 1, Two Step Process Step 1 Step 1 –Calculate ICERs by baseline stage of change –This alone can provide much more useful and interpretable information for stage-matched interventions Step 2 Step 2 –Estimate probability of people moving into successive stages of change –For example, % moved from precontemplation to contemplation and preparation

Aug 1, Source of the Probabilities Probabilities from the literature or Probabilities from the literature or The control group (if possible) The control group (if possible) –Possible when control group gets usual care –If 10% of precontemplators in the control group changed behavior, this is the probability for the model

Aug 1, Hypothetical Example

Aug 1, Randomized Controlled Trial 2700 participants enrolled in 3 arms 2700 participants enrolled in 3 arms –Post card reminder (n=900) –Reminder phone call (n=900) –Personal motivational phone call (n=900) Sample from managed care plan Sample from managed care plan Managed care organization wants to know which reminder to use Managed care organization wants to know which reminder to use

Aug 1, Cost and Effectiveness

Aug 1, Report to MCO MCO should choose between postcard or reminder call MCO should choose between postcard or reminder call

Aug 1, Transition Probabilities

Aug 1, Including Partial Behavior Change Note motivational call is now most effective overall Note motivational call is now most effective overall

Aug 1, ICERs No single strategy is always preferred. No single strategy is always preferred. Motivational call is dominated by reminder call for contemplators and those in action Motivational call is dominated by reminder call for contemplators and those in action

Aug 1, Origin of These Ideas

Aug 1, Patient Reminders Mailed reminder vs no reminder for mammography Mailed reminder vs no reminder for mammography A meta analysis of 16 studies A meta analysis of 16 studies –US studies Pooled OR=1.48 (participants come from provider files) –Aust / NZ studies Pooled OR=5.57 (participants from voter lists) Wagner TH. The effectiveness of mailed patient reminders on mammography screening: a meta-analysis. Am J Prev Med. 1998;14(1):64-70.

Aug 1, CEA and Stage of Change Fishman P. et al. Cost-effectiveness of strategies to enhance mammography use. Eff Clin Pract Fishman P. et al. Cost-effectiveness of strategies to enhance mammography use. Eff Clin Pract Compared three alternative methods for increasing mammography screening Compared three alternative methods for increasing mammography screening –Reminder postcard –Reminder call –Motivational call

Aug 1, Fishman’s (2000) results ICER varied by prior mammography status (i.e., maintenance) ICER varied by prior mammography status (i.e., maintenance)

Aug 1, Feeling Lucky? The implications of a CEA should not vary by who enters the trial The implications of a CEA should not vary by who enters the trial Two randomized trials with same intervention Two randomized trials with same intervention –RCT 1: all participants are in preparation –RCT 2: all participants are in precontemplation

Aug 1, Clinical Trial Design If the effect of the intervention might vary by stage of change If the effect of the intervention might vary by stage of change –Qx: Enroll people from all stages? –Enroll sufficient numbers in each stage –Protect randomization

Aug 1, Conclusions Subgroup analysis by stage of change may be critical for interpreting the CEA Subgroup analysis by stage of change may be critical for interpreting the CEA Partial behavior change is important and can affect interpretation Partial behavior change is important and can affect interpretation These methods are appropriate for stage- matched studies These methods are appropriate for stage- matched studies Need to consider study design implications Need to consider study design implications