Cost-Effectiveness Analysis: New methods for old limitations? R Scott Braithwaite, MD, MSc, FACP R Scott Braithwaite, MD, MSc, FACP Yale University School of Medicine Yale University School of Medicine Connecticut VA Healthcare System Connecticut VA Healthcare System

CEA Limitations Difficulty interpreting “number” (ICER) Difficulty interpreting “number” (ICER) Is $59,000/life-year good or bad? Is $59,000/life-year good or bad? A policy tool in search of a U.S. policy “lever” A policy tool in search of a U.S. policy “lever” Other countries use it (e.g. Canada, U.K., Australia, Netherlands) but U.S. payers don’t Other countries use it (e.g. Canada, U.K., Australia, Netherlands) but U.S. payers don’t Does not incorporate quality of evidence Does not incorporate quality of evidence An analysis based on a high-quality study may appear to have same certitude as analysis based on expert opinion An analysis based on a high-quality study may appear to have same certitude as analysis based on expert opinion

New CEA Methods Decision rules for interpreting results Decision rules for interpreting results Linking results to policy “levers” in U.S Linking results to policy “levers” in U.S Incorporating quality of evidence Incorporating quality of evidence

Question: How can we use CEA to inform statin formulary decisions ? Case year-old female diabetic, prior MI, pre-treatment LDL 137 Case year-old female diabetic, prior MI, pre-treatment LDL 137 Case year-old male, no cardiac risk factors or history of cardiovascular disease, pre- treatment LDL 167 Case year-old male, no cardiac risk factors or history of cardiovascular disease, pre- treatment LDL 167 Note: Treatment of both patients endorsed by current NCEP guidelines Note: Treatment of both patients endorsed by current NCEP guidelines

Research to address limitations of cost- effectiveness analysis Decision rules for interpreting results Decision rules for interpreting results Linking results to policy “levers” in U.S Linking results to policy “levers” in U.S Incorporating quality of evidence Incorporating quality of evidence

Introduction CEA results estimate value CEA results estimate value Value allows maximizing health given budget Value allows maximizing health given budget What is high-value for one society at one time may be low-value for another society What is high-value for one society at one time may be low-value for another society So how to interpret CEA in the U.S. currently? So how to interpret CEA in the U.S. currently? CEA results often presented together with simple acceptability thresholds (e.g. “$50,000 per QALY”) CEA results often presented together with simple acceptability thresholds (e.g. “$50,000 per QALY”) Individual thresholds have little validity Individual thresholds have little validity Ranges (e.g. $50,000 per QALY to $100,000 per QALY) may be more valid and feasible Ranges (e.g. $50,000 per QALY to $100,000 per QALY) may be more valid and feasible

Objective To inform decision rules for CEA interpretation based on health care purchasing choices in US To inform decision rules for CEA interpretation based on health care purchasing choices in US Two distinct but complementary analyses to estimate upper- and lower-bounds of range Two distinct but complementary analyses to estimate upper- and lower-bounds of range

Methods Analysis #1: Estimate cost-effectiveness of “modern” health care Analysis #1: Estimate cost-effectiveness of “modern” health care Individuals prefer cost/benefit of modern health care to cost/benefit of pre-modern care Individuals prefer cost/benefit of modern health care to cost/benefit of pre-modern care If willing to pay for modern care, then should be willing to pay for services that have same or better cost-effectiveness as modern care If willing to pay for modern care, then should be willing to pay for services that have same or better cost-effectiveness as modern care May inform lower bound for decision rule May inform lower bound for decision rule

Methods Analysis #2 : Estimate cost-effectiveness of unsubsidized health insurance Analysis #2 : Estimate cost-effectiveness of unsubsidized health insurance Individuals prefer cost/benefit of no insurance to cost/benefit of unsubsidized insurance Individuals prefer cost/benefit of no insurance to cost/benefit of unsubsidized insurance Free rider effect: Pay 1/10 costs, get 2/3 benefit Free rider effect: Pay 1/10 costs, get 2/3 benefit If not willing to pay for unsubsidized insurance, then should be not willing to pay for services that have same or worse cost-effectiveness If not willing to pay for unsubsidized insurance, then should be not willing to pay for services that have same or worse cost-effectiveness May inform higher bound for decision rule May inform higher bound for decision rule

<100% 100%-199% 200%-299% 300%-399% ≥400% $75K % Willing to Pay for Health Insurance No Employer SubsidyEmployer Subsidy % Poverty Line Income (Family of 4)

Cost-effectiveness of modern health care Incremental benefits of modern care Incremental benefits of modern care 53% of observed mortality decrease 53% of observed mortality decrease 4.7 life-years 4.7 life-years Incremental lifetime costs of modern care Incremental lifetime costs of modern care $452,000 $452,000 Incremental cost-effectiveness of modern care Incremental cost-effectiveness of modern care $96,000 per life-year $96,000 per life-year Approximately $100,000 per QALY Approximately $100,000 per QALY Braithwaite RS et al, Med Care 2008; 46:

Cost-effectiveness of unsubsidized insurance Incremental benefits of buying (for 1 year) Incremental benefits of buying (for 1 year) Mortality reduced by 18% Mortality reduced by 18% LE increased by years LE increased by years Incremental costs of buying (for 1 year) Incremental costs of buying (for 1 year) $4100 $4100 Incremental cost-effectiveness of buying unsubsidized insurance Incremental cost-effectiveness of buying unsubsidized insurance $204,000 per life-year $204,000 per life-year Approximately $300,000 per QALY Approximately $300,000 per QALY Braithwaite RS et al, Med Care 2008; 46:

Interpreting cost-effectiveness results Cost-effectiveness of modern health care Cost-effectiveness of modern health care Inform lower (less inclusive) bound for rule Inform lower (less inclusive) bound for rule ≈ $100,000/QALY ≈ $100,000/QALY Cost-effectiveness of unsubsidized insurance Cost-effectiveness of unsubsidized insurance Inform higher (more inclusive) bound for rule Inform higher (more inclusive) bound for rule ≈ $300,000/QALY ≈ $300,000/QALY $50,000/QALY unlikely to be valid $50,000/QALY unlikely to be valid Acceptability range ($100,000/QALY to $300,000/QALY) likely more valid and feasible Acceptability range ($100,000/QALY to $300,000/QALY) likely more valid and feasible Braithwaite RS et al, Med Care 2008; 46:

Question: How can we use CEA to inform statin formulary decisions ? Case year-old female diabetic, prior MI, pre-treatment LDL 137 Case year-old female diabetic, prior MI, pre-treatment LDL 137 Incremental cost-effectiveness <$10,000/QALY Incremental cost-effectiveness <$10,000/QALY Favorable Favorable Case year-old male, no cardiac risk factors or history of cardiovascular disease, pre- treatment LDL 167 Case year-old male, no cardiac risk factors or history of cardiovascular disease, pre- treatment LDL 167 Incremental cost-effectiveness $420,000/QALY Incremental cost-effectiveness $420,000/QALY Unfavorable Unfavorable

Research to address limitations of cost- effectiveness analysis Decision rules for interpreting results Decision rules for interpreting results Linking results to policy “levers” in U.S Linking results to policy “levers” in U.S Incorporating quality of evidence Incorporating quality of evidence

Background Cost-sharing becoming a standard “volume knob” to control utilization in U.S. Cost-sharing becoming a standard “volume knob” to control utilization in U.S. Cost-sharing has Cost-sharing has Great potential to control costs Great potential to control costs Great potential to cause harm Great potential to cause harm Increasing calls to link cost-sharing to value (e.g. value-based insurance design) Increasing calls to link cost-sharing to value (e.g. value-based insurance design) No cost-sharing for high-value services No cost-sharing for high-value services Same or increased cost-sharing for low-value services Same or increased cost-sharing for low-value services

Background Possible way to link cost-effectiveness results to cost-sharing Possible way to link cost-effectiveness results to cost-sharing > $300K/QALY: Increase cost-sharing > $300K/QALY: Increase cost-sharing $100K-300K/QALY: No Δ cost-sharing $100K-300K/QALY: No Δ cost-sharing < $100K/QALY: Waive cost-sharing < $100K/QALY: Waive cost-sharing Cost-saving: Share cost-savings? Cost-saving: Share cost-savings? Braithwaite RS et al, Ann Intern Med. 2007; 146: Braithwaite RS et al, Ann Intern Med. 2007; 146:

Purpose To estimate the impact of value-linked cost- sharing if it were applied systematically across US health system To estimate the impact of value-linked cost- sharing if it were applied systematically across US health system

Methods Cost-sharing0%18%(Current)30%100% Relative utilization From RAND, we can estimate the impact of cost-sharing amount on health service demand Results confirmed by >100 observational studies

Age N Health expenditure New year = new “pull” from cost-effectiveness distribution InsuredUninsuredInsured, value-based cost-sharing High cost- sharing lowers health expenditures Prevailing cost-sharing maintains health expenditures Value increases, lowers, or maintains health expenditures Age N+1 Mortality

Current cost-shar Current insurance Current cost-shar Expand insurance Value-based cost-shar Current insurance Value-based cost-shar Expand insurance 20% 30% 40% 50%Copayment if low- value No cost-shar, Expand insurance Life Expectancy Gain (Years)

Current cost-shar Current insurance Current cost-shar Expand insurance Value-based cost-shar Current insurance Value-based cost-shar Expand insurance 20% 30% 40% 50%Copayment if low- value No cost-shar, Expand insurance Annual per-capita cost (2003 $)

Conclusions Value-linked cost-sharing may increase life expectancy from health care while reducing costs Value-linked cost-sharing may increase life expectancy from health care while reducing costs Costs may be lowered sufficiently to offset incremental expenditures from expanding health insurance coverage Costs may be lowered sufficiently to offset incremental expenditures from expanding health insurance coverage

Question: How can we use CEA to inform statin formulary decisions ? Case year-old female diabetic, prior MI, pre- treatment LDL 137 Case year-old female diabetic, prior MI, pre- treatment LDL 137 Incremental cost-effectiveness <$10,000/QALY Incremental cost-effectiveness <$10,000/QALY Favorable Favorable No cost-sharing No cost-sharing Case year-old male, no cardiac risk factors or history of cardiovascular disease, pre-treatment LDL 167 Case year-old male, no cardiac risk factors or history of cardiovascular disease, pre-treatment LDL 167 Incremental cost-effectiveness $420,000/QALY Incremental cost-effectiveness $420,000/QALY Unfavorable Unfavorable Higher cost-sharing (30%-35%) Higher cost-sharing (30%-35%)

Research to address limitations of cost- effectiveness analysis Decision rules for interpreting results Decision rules for interpreting results Linking results to policy “levers” in U.S Linking results to policy “levers” in U.S Incorporating quality of evidence Incorporating quality of evidence

Problem Clinicians and policy-makers often wonder “what goes into the model” Clinicians and policy-makers often wonder “what goes into the model” Current methods for cost-effectiveness analysis take into account uncertainty from random variation but not from low-quality evidence Current methods for cost-effectiveness analysis take into account uncertainty from random variation but not from low-quality evidence

Objective Can we augment standard cost-effectiveness analysis methods to develop a sensitivity analysis based on quality of evidence? Can we augment standard cost-effectiveness analysis methods to develop a sensitivity analysis based on quality of evidence?

Methods Basic concept of our approach Basic concept of our approach When potential information sources have insufficient quality of evidence, don’t use them When potential information sources have insufficient quality of evidence, don’t use them Instead, assume that little is known by using uninformative distributions over wide range Instead, assume that little is known by using uninformative distributions over wide range Don’t obscure questionable data under a “false veneer of mathematical certitude” Don’t obscure questionable data under a “false veneer of mathematical certitude” Warning! If you set evidence standards very high, not much of the available evidence may qualify. Warning! If you set evidence standards very high, not much of the available evidence may qualify.

Methods Assess quality of evidence using USPSTF guidelines Assess quality of evidence using USPSTF guidelines Study design Study design Design differs from controlled experiment Design differs from controlled experiment Internal validity Internal validity Results represent truth in study population Results represent truth in study population External validity External validity Results represent truth in target population Results represent truth in target population Our approach can be used with any evidence- evaluation hierarchy Our approach can be used with any evidence- evaluation hierarchy We chose USPSTF guidelines because of ubiquity not because of rigor We chose USPSTF guidelines because of ubiquity not because of rigor

Methods Set minimum standard in each evidence domain Set minimum standard in each evidence domain These can be “dialed” up or down at will These can be “dialed” up or down at will Evaluate each possible source of evidence Evaluate each possible source of evidence If source meets evidence criterion, use its 95% confidence interval in the analysis If source meets evidence criterion, use its 95% confidence interval in the analysis If evidence does not meet criterion, do not use it in the analysis If evidence does not meet criterion, do not use it in the analysis Instead use uninformative (“wide”) distribution Instead use uninformative (“wide”) distribution

Test Case: Directly observed therapy for HIV antiretrovirals Base Case: No evidence criteria Base Case: No evidence criteria All 17 data sources eligible for parameter estimation All 17 data sources eligible for parameter estimation Study Design set to highest standard (“1”) Study Design set to highest standard (“1”) 13 out of 17 sources were eligible 13 out of 17 sources were eligible Internal Validity set to highest standard (“good”) Internal Validity set to highest standard (“good”) 9 out of 17 sources were eligible 9 out of 17 sources were eligible External Validity set to highest standard (“high”) External Validity set to highest standard (“high”) 5 out of 17 sources were eligible 5 out of 17 sources were eligible All three criteria set to highest standards All three criteria set to highest standards 3 out of 17 sources were eligible 3 out of 17 sources were eligible

Results: All Evidence Braithwaite RS et al, Ann Intern Med 2007; 146:

Results: Internal Validity “Good” Braithwaite RS et al, Ann Intern Med 2007; 146:

Results: Study Design “1” Braithwaite RS et al, Ann Intern Med 2007; 146:

Results: External Validity “High” Braithwaite RS et al, Ann Intern Med 2007; 146:

Results: All 3 Criteria Braithwaite RS et al, Ann Intern Med 2007; 146:

Conclusions Quality of evidence may have profound impact on the precision and estimates of CEAs Quality of evidence may have profound impact on the precision and estimates of CEAs Stricter evidence criteria may produce more uncertain results because there are fewer studies to base assumptions on Stricter evidence criteria may produce more uncertain results because there are fewer studies to base assumptions on Approach shows when evidence is not good enough for decision making Approach shows when evidence is not good enough for decision making Need higher-quality information on HIV DOT Need higher-quality information on HIV DOT

Question: How can we use CEA to inform statin formulary decisions ? Case year-old female diabetic, prior MI, pre- treatment LDL 137 Case year-old female diabetic, prior MI, pre- treatment LDL 137 Incremental cost-effectiveness <$10,000/QALY Incremental cost-effectiveness <$10,000/QALY Value: Favorable Value: Favorable Likely <$100,000/QALY if strict evidence std? Yes Likely <$100,000/QALY if strict evidence std? Yes Decision: No cost-sharing Decision: No cost-sharing

Question: How can we use CEA to inform statin formulary decisions ? Case year-old male, no cardiac risk factors or history of cardiovascular disease, pre-treatment LDL 167 Case year-old male, no cardiac risk factors or history of cardiovascular disease, pre-treatment LDL 167 Incremental cost-effectiveness $420,000/QALY Incremental cost-effectiveness $420,000/QALY Value: Unfavorable Value: Unfavorable Likely ≥$300,000/QALY if strict evidence std? Yes Likely ≥$300,000/QALY if strict evidence std? Yes Decision: Higher cost-sharing (30%-35%) Decision: Higher cost-sharing (30%-35%)

Cost-Effectiveness Analysis Limitations (and possible solutions) Difficulty interpreting end-result Difficulty interpreting end-result Give policy makers decision rules demarcating low from intermediate from high cost-effectiveness Give policy makers decision rules demarcating low from intermediate from high cost-effectiveness A policy tool in search of a U.S. policy “lever” A policy tool in search of a U.S. policy “lever” Link CEA results to level of cost-sharing Link CEA results to level of cost-sharing Does not incorporate quality of evidence Does not incorporate quality of evidence Let policymakers and payers specify minimum threshold of evidence for decision making Let policymakers and payers specify minimum threshold of evidence for decision making

Questions ?????? Special thanks to Special thanks to Mentors Mentors Amy C Justice, MD, PhD Amy C Justice, MD, PhD Mark S Roberts, MD, MPP Mark S Roberts, MD, MPP Funders Funders NIAAA NIAAA RWJ RWJ Co-authors, including Co-authors, including David O. Meltzer, MD, PhD David O. Meltzer, MD, PhD Joseph King, MD Joseph King, MD Alison B. Rosen, MD, ScD Alison B. Rosen, MD, ScD John Concato, MD, MSc John Concato, MD, MSc