Value of Information Calculations to Inform and Prioritize Clinical Research Investments David Meltzer MD, PhD The University of Chicago

Overview Cost-effectiveness analysis has long been used to assess the value of medical treatments and the information that comes from diagnostic tests –Cost-effectiveness measured in Cost/QALY –Net health benefits = gain in QALYs – opportunity cost of spending in QALYs –Net monetary benefit = $ value of improved health - costs Newer value of information techniques have extended these tools to assess the value of medical research

Research as Value of Information: Analogy to Diagnostic Testing Test Don’t Test S H S H Max{pU(T|S)+(1-p)U(T|H), pU(N|S)+(1-p)U(N|H)} U(T|S) U(N|H) pU(T|S)+(1-p)U(N|H)

Value of Information Approach to Value of Research Without information –Make best compromise choice not knowing true state of the world (e.g. don’t know if intervention is good, bad) With probability p:get V(Compromise|G) With probability 1-p:get V(Compromise|B) With information –Make best decision knowing true state With probability p:get V(Best choice|G) With probability 1-p:get V(Best choice|B) Value of information = E(outcome) with information - E(outcome) w/o information = {p*V(Best choice|G) + (1-p)*V(Best choice|B)} - {p*V(Compromise|G) + (1-p)*V(Compromise|B)} = Value of Research

Practical Applications of Value of Information VOI requires modeling population value of information where VOI based on decision models –IVOI modeled with decision model –UK (NICE): Alzheimer’s Disease Tx, wisdom teeth removal Minimal modeling approaches to VOI –IVOI comes (nearly) directly from clinical trial –US (NIH): CATIE Trial of atypical antipsychotics Bound with more limited data (burden of illness)

“Bayesian Value of information analysis: An application to a policy model of Alzheimer's disease.”

Uncertainty in Incremental Net Benefits

Cost-Effectiveness Acceptability Curve

Value of Research by Value of Health

Value of Research by Time Horizon

Contributors to Value of Research

Practical Applications of Value of Information VOI requires modeling population value of information where VOI based on decision models –IVOI modeled with decision model –UK (NICE): Alzheimer’s Disease Tx, wisdom teeth removal Minimal modeling approaches to VOI –IVOI comes (nearly) directly from clinical trial –US (NIH): CATIE Trial of atypical antipsychotics Bound with more limited data (burden of illness)

Expected Value of Research on the Comparative Cost-Effectiveness of Antipsychotics Drugs of Antipsychotics Drugs David Meltzer MD PhD Department of Medicine, Department of Economics, Harris School of Public Policy, & University of Chicago CERT Chicago IL (Joint work with Anirban Basu PhD, University of Chicago & Dr. Herbert Y. Meltzer, Vanderbilt University)

The Clinical Antipsychotic Trials in Intervention Effectivness (CATIE) $42.6 million, NIMH-funded randomized trial of Atypical Antipsychotic Drugs (A-APDs) and a Neuroleptic (Perhphenazine) in patients with established schizophrenia Major findings o Discontinuation rates similar with A-APDs and Perphenazine o Perphenazine cost-effective first-line treatment Impact o Frequently discussed in coverage decisions o Some have argued results should be considered definitive

The Clinical Antipsychotic Trials in Intervention Effectivness (CATIE) Limitations o Continuation was major endpoint o Limited precision in estimates of effectiveness, costs o Small differences in effectiveness/costs across many persons could be of great value Important to know value of potential future research o Help prioritize individual research opportunities o Facilitate rational investment decisions

CATIE Cost-Effectiveness Results Monthly Costs Mean (sd) ($) QALY Mean (sd) ICER ($/QALY) Perphenazine817 ( 728)0.722 (0.0064) - Olanzapine1619 (1442)0.723 (0.0063)9,624,000 Risperidone1635 (1457)0.706 (0.0066)Dominated Quetiapine1680 (1497)0.721 (0.0065)Dominated ( Ref: Rosenheck et al, 2006; Private Communications with Dr. Rosenheck) Only statistically significant difference: QALY Perphenazine > QALY Risperidone (p-val QALY Risperidone (p-val < 0.001)

Aims Primary Aim : To determine the expected value of more precise determination of effects of AAPDs and Perphenazine on costs and QALYs. Secondary Aim : To determine the optimal sample size for a future trial of the effects of AAPDs and Perphenazine on costs and QALYs

Methods Used for Value of Research Expected value calculated based on the welfare of the prevalent cohort over their lifetimes and the welfare of next 20 incident cohorts over their lifetimes 3% discounting was used

Simulated Distribution of Mean QALYS (Based on uncertainty around CATIE results)

Simulated Distribution of Mean Costs (Based on uncertainty around CATIE results)

Realizations of Value of Research Over Time Total Value to Prevalent Cohort: $207 billion Total Value to Each Incident Cohort: $6.6 billion Total Value to Prevalent & Next 20 Incident Cohorts: $342 billion

Value of Research and Acceptability Profile

Optimal Sample Size for A Future Trial Traditional (Deterministic) Power Calculations (at $50K/QALY) Largest effect size in NMB between an atypical & perphenazine based on CATIE results: $15,680 (sd=$315,000) vs. $26,296 (sd=$140,000) Sample size required for alpha = 0.05 & power = 0.80: 8,300 for each arm Power associated with n of CATIE = 400/arm & alpha = 0.05: 10%

Net Expected Value of Sample Information (at $50K, $100K and $150K/QALY) Optimal sample size for each arm = 22,500

CATIE VOI Conclusions The value to more precisely establishing the cost-effectiveness of typical/atypical antipsychotics is enormous. The results of CATIE should not be viewed as definitive. Further studies of the comparative cost-effectiveness of typical/atypical antipsychotics with adequate sample size to answer such questions have high expected value. Optimal sample sizes may be exceptionally large, raising interesting questions as to how clinical trials on such a scale might be executed. Large scale social experiments may provide an interesting model for such studies.

Conclusions Value of information (VOI) analysis can be used to develop prospective estimates of the value of research VOI can be used to prioritize areas of clinical research or choose among study designs Decision models or minimal modeling approaches –Additional minimal model applications in progress –Erlotinib (Tarceva) (+gemcitabine) in advanced pancreatic CA –Azithromycin vs. Augmentin in acute sinusitis Incorporating VOI into research prioritization is a work in progress –Use by funding agencies? –Use by investigators?

Acknowledgments Collaborators: Anirban Basu Ph.D., Jeanette Chung Ph.D., Ties Hoomans Ph.D., Herbert Meltzer M.D. Funding: AHRQ BCBS Evidence-Based Practice Center, AHRQ Hospital Medicine and Economics Center for Education and Research in Therapeutics, Best Practice Inc, National Institute of Aging, National Institute of Mental Health