1. Aug 1, 2007 1 Conducting Cost-Effectiveness Analyses of Behavioral Interventions Todd H. Wagner, Ph.D. Mary K. Goldstein, M.D. Mary K. Goldstein, M.D.

2. Aug 1, 20072 Acknowledgements Partial funding was through a grant from the National Cancer Institute (2PO1 CA 55112- 05A1). Partial funding was through a grant from the National Cancer Institute (2PO1 CA 55112- 05A1). Dr. Goldstein ’ s contribution was informed in part by work conducted with a grant from the National Institute on Aging (R01 AG15110). Dr. Goldstein ’ s contribution was informed in part by work conducted with a grant from the National Institute on Aging (R01 AG15110). David Meltzer, Jodi Prochaska, Lisa Faulkner, and Stanford University seminar participants provided helpful comments. David Meltzer, Jodi Prochaska, Lisa Faulkner, and Stanford University seminar participants provided helpful comments.

3. Aug 1, 20073 Outline of the Talk Background on cost-effectiveness analysis (CEA) Background on cost-effectiveness analysis (CEA) A bias in CEAs for behavioral interventions A bias in CEAs for behavioral interventions How to fix the problem How to fix the problem Example Example Study design consideration Study design consideration

4. Aug 1, 20074 Economic Evaluations Most trials don’t include economic analysis Most trials don’t include economic analysis Economic analysis in only 0.2% of 50,000 trials conducted between 1966-1988 Economic analysis in only 0.2% of 50,000 trials conducted between 1966-1988 –Adams et al. Medical Care 30(3):231-43 Economic analysis in randomized clinical trials is increasing Economic analysis in randomized clinical trials is increasing

5. Aug 1, 20075 Dollars and Sense Economic studies inform decisions Economic studies inform decisions –formulary –adoption of new technology –scope of benefits –strategies for management of care –organization of health care

6. Aug 1, 20076 Cost Effectiveness Ratio Where C 1 is the average cost of the intervention group C 0 is the average cost of the control group E 1 is the average effectiveness of the intervention group E 0 is the average effectiveness of the control group

7. Aug 1, 20077 Standardization USPTF guidelines (Gold et al., 1996; summarized in three JAMA articles) USPTF guidelines (Gold et al., 1996; summarized in three JAMA articles) Drummond et al. (1997) Drummond et al. (1997) Created, along with journal editors, standards for both Created, along with journal editors, standards for both –Methods –Reporting

8. Aug 1, 20078 Whose Costs? Guidelines recommend a societal perspective for costs Guidelines recommend a societal perspective for costs Include Include –Provider –Payer –Patient

9. Aug 1, 20079 Effectiveness Guidelines recommend QALYs Guidelines recommend QALYs Values both quality and quantity of life Values both quality and quantity of life Each year of life is multiplied by a weighting factor (utility) Each year of life is multiplied by a weighting factor (utility) Utilities measure the preference of different health states on a 0-1 scale Utilities measure the preference of different health states on a 0-1 scale

10. Aug 1, 200710 Behavioral Interventions CEA framework holds, but caveats … CEA framework holds, but caveats … –Behavior change is a “ slow ” process –Treat many to prevent a few –Use of intermediate outcomes (proxies)

11. Aug 1, 200711 Intermediate Outcomes Outcome is clinically relevant and predicts mortality or morbidity Outcome is clinically relevant and predicts mortality or morbidity –Receipt of a mammogram –Substance use abstinence –Change in dietary fiber QALYs would require huge and/or very long studies QALYs would require huge and/or very long studies

12. Aug 1, 200712 CEA with an Intermediate Outcome Sufficient for publication Sufficient for publication Hard to interpret ICER Hard to interpret ICER –Can’t easily compare two CEAs with different intermediate outcomes –Can’t compare CEA to other CEA from another clinical area Sometimes only feasible approach Sometimes only feasible approach

13. Aug 1, 200713 CEA with QALYs Measure QALYs or Measure QALYs or Translate intermediate outcome to QALYs Translate intermediate outcome to QALYs –Either build a model de novo or use an existing model –Requires a lot of resources Most useful, but most challenging Most useful, but most challenging

14. Aug 1, 200714 CEA and Behavior Change CEA analysts treat behavior change as a dichotomous outcome CEA analysts treat behavior change as a dichotomous outcome Partial behavior change is not the same as no behavior change Partial behavior change is not the same as no behavior change “ Getting the person to recognize that they have a problem is half the battle. ” “ Getting the person to recognize that they have a problem is half the battle. ”

15. Aug 1, 200715 What ’ s Missing? Partial behavior change is missing from current models Partial behavior change is missing from current models –People who progressed in their process of changing their behavior but did not successfully change their behavior at the end of the study (Stages of Change)

16. Aug 1, 200716 Partial Behavior Change All behavioral interventions yield some partial behavior change All behavioral interventions yield some partial behavior change Amount depends on Amount depends on –the duration of the study –baseline stage of change

17. Aug 1, 200717 Partial Behavior Change Should we and can we value partial behavior change in a CEA?

18. Aug 1, 200718 Should We? NO Intentions are not the same as observed behavior change Intentions are not the same as observed behavior change Stage of change is flawed Stage of change is flawed Too difficult Too difficultYES Behavior change takes time Behavior change takes time Unobservable differences exist Unobservable differences exist Future benefits are important Future benefits are important Otherwise favors med/surg tx Otherwise favors med/surg tx

19. Aug 1, 200719 Can We? Behaviors are like “ value chains ” Behaviors are like “ value chains ” –A series of linked processes –Interventions may be designed to improve a link (stage-matched design)  Matching chemotherapy protocol to cancer stage –Interventions may have differential effects on different links Stages of Change (TTM) Model by Prochaska and DiClimente Stages of Change (TTM) Model by Prochaska and DiClimente

20. Aug 1, 200720 Stages of Change People progress through successive stages until they change their behavior People progress through successive stages until they change their behavior –Precontemplation: no self-recognition of a problematic behavior. –Contemplation: self-recognition of a problem without action. –Preparation: Planning to change behavior soon. –Action: In the process of change. –Maintenance: adherence to the new behavior over time. Art of behavioral interventions is to achieve action, and hopefully maintenance Art of behavioral interventions is to achieve action, and hopefully maintenance

21. Aug 1, 200721 Stages of Change and CEA Stages of change is critical for interpreting CEA results Stages of change is critical for interpreting CEA results –Stage of change may be associated with receptivity or motivation –Intervention effects can vary by stage of change –Incremental cost-effectiveness ratio can vary by stage of change

22. Aug 1, 2007 22 Integrating Stages of Change in a CEA

23. Aug 1, 200723 What’s Needed? Data on stages of change Data on stages of change Probability of moving from partial to successful behavior change Probability of moving from partial to successful behavior change –Note: these probabilities are not observed in the intervention group

24. Aug 1, 200724 Two Step Process Step 1 Step 1 –Calculate ICERs by baseline stage of change –This alone can provide much more useful and interpretable information for stage-matched interventions Step 2 Step 2 –Estimate probability of people moving into successive stages of change –For example, % moved from precontemplation to contemplation and preparation

25. Aug 1, 200725 Source of the Probabilities Probabilities from the literature or Probabilities from the literature or The control group (if possible) The control group (if possible) –Possible when control group gets usual care –If 10% of precontemplators in the control group changed behavior, this is the probability for the model

26. Aug 1, 2007 26 Hypothetical Example

27. Aug 1, 200727 Randomized Controlled Trial 2700 participants enrolled in 3 arms 2700 participants enrolled in 3 arms –Post card reminder (n=900) –Reminder phone call (n=900) –Personal motivational phone call (n=900) Sample from managed care plan Sample from managed care plan Managed care organization wants to know which reminder to use Managed care organization wants to know which reminder to use

28. Aug 1, 200728 Cost and Effectiveness

29. Aug 1, 200729 Report to MCO MCO should choose between postcard or reminder call MCO should choose between postcard or reminder call

30. Aug 1, 200730 Transition Probabilities

31. Aug 1, 200731 Including Partial Behavior Change Note motivational call is now most effective overall Note motivational call is now most effective overall

32. Aug 1, 200732 ICERs No single strategy is always preferred. No single strategy is always preferred. Motivational call is dominated by reminder call for contemplators and those in action Motivational call is dominated by reminder call for contemplators and those in action

33. Aug 1, 2007 33 Origin of These Ideas

34. Aug 1, 200734 Patient Reminders Mailed reminder vs no reminder for mammography Mailed reminder vs no reminder for mammography A meta analysis of 16 studies A meta analysis of 16 studies –US studies Pooled OR=1.48 (participants come from provider files) –Aust / NZ studies Pooled OR=5.57 (participants from voter lists) Wagner TH. The effectiveness of mailed patient reminders on mammography screening: a meta-analysis. Am J Prev Med. 1998;14(1):64-70.

35. Aug 1, 200735 CEA and Stage of Change Fishman P. et al. Cost-effectiveness of strategies to enhance mammography use. Eff Clin Pract. 2000 Fishman P. et al. Cost-effectiveness of strategies to enhance mammography use. Eff Clin Pract. 2000 Compared three alternative methods for increasing mammography screening Compared three alternative methods for increasing mammography screening –Reminder postcard –Reminder call –Motivational call

36. Aug 1, 200736 Fishman’s (2000) results ICER varied by prior mammography status (i.e., maintenance) ICER varied by prior mammography status (i.e., maintenance)

37. Aug 1, 200737 Feeling Lucky? The implications of a CEA should not vary by who enters the trial The implications of a CEA should not vary by who enters the trial Two randomized trials with same intervention Two randomized trials with same intervention –RCT 1: all participants are in preparation –RCT 2: all participants are in precontemplation

38. Aug 1, 200738 Clinical Trial Design If the effect of the intervention might vary by stage of change If the effect of the intervention might vary by stage of change –Qx: Enroll people from all stages? –Enroll sufficient numbers in each stage –Protect randomization

39. Aug 1, 200739 Conclusions Subgroup analysis by stage of change may be critical for interpreting the CEA Subgroup analysis by stage of change may be critical for interpreting the CEA Partial behavior change is important and can affect interpretation Partial behavior change is important and can affect interpretation These methods are appropriate for stage- matched studies These methods are appropriate for stage- matched studies Need to consider study design implications Need to consider study design implications