“New methods in generating evidence for everyone: Can we improve evidence synthesis approaches?” Network Meta-Analyses and Economic Evaluations Petros Pechlivanoglou PhD Child Health Evaluative Sciences The Hospital For Sick Children, Toronto Institute for Health Policy Management and Evaluation University of Toronto, Toronto CADTH 2016

Disclosure No funding received for this work Talk relies in part on the work of Cooper et al (2015) and Dias et al (2013). Acknowledgements MSc students: Troy Francis Yasmin Saeed Dias,S, et al. "Evidence synthesis for decision making 3 heterogeneity—subgroups, meta-regression, bias, and bias adjustment." Medical Decision Making 33.5 (2013): Cooper et al RFP Topic: Use of Network Meta – analysis to Inform Clinical Parameters in Economic Evaluations; white paper, CADTH (2015)

Aims Usefulness of relying NMA in EE Main concepts in embedding NMAs in EE Preliminary results of literature review Some recommendations on using NMA evidence in EE

Introducing NMA in EE Evidence-based decision making: evaluation of both cost and health impact of all relevant treatments. Ideally, optimal decisions are based on multi-arms RCTs, including all relevant treatments. BUT High costs and regulatory issues preclude such RCTs. New health technologies compared to standard care or placebo. NMAs allow the economic evaluation of all treatment options using comparative effectiveness estimates NMA development pushed by EE !!

Uncertainty, NMA and EE NMAs help us to more properly characterize uncertainty in treatment comparisons The importance of accurate uncertainty estimate: – Non-linearity of decision models implies inaccurate uncertainty estimates result to bias E(g(x)) ≠ g(E(x)) – Accurate characterization of uncertainty is important for decision making – Value of Information (VOI) increasingly used as a method to guide investments and research – VOI relies on accurate estimates of uncertainty.

HTA Process Clinical Evidence – Identify relevant evidence (Systematic review) – Synthesize Evidence (Meta – Analysis / NMA / narrative) – Interpret / Appraise Economic Evidence – Identify relevant EE evidence – Conceptualize model – Identify input – Execute model – Interpretation / Appraise ELSI

HTA Process Clinical Evidence – Identify relevant evidence (Systematic review) – Synthesize Evidence (Meta – Analysis / NMA / narrative) – Interpret / Appraise Economic Evidence – Identify relevant EE evidence – Conceptualize model – Identify input – Execute model – Interpretation / Appraise ELSI

Conceptualizing an NMA-EE model Comparators Study Design Outcomes Model Structure

Conceptualizing an NMA-EE model Comparators – Wider sets than those clinically meaningful to take advantage of indirect comparisons – Lumping of Comparators Differences in dose/ treatment durations / administration Study Design RCTs for treatment effect BUT Open-label extensions for long-term follow up (!) RCT/ surveys on HRQoL (!) (i) : new methods in development

Conceptualizing an NMA-EE model Outcomes – Discrepancy between what is clinically and what economically relevant in evidence synthesis – Sparsity / large presence of zeros – Surrogate endpoints – Correlated outcomes

Conceptualizing an NMA-EE model Model structure BAYESIAN One step NMA – EE (NMA and EE estimated together) Two Step NMA – EE (NMA posterior embedded in EE) FREQUENTIST Two Step NMA – EE (assuming Multivariate distribution) Two Step NMA – EE (Ignoring Correlations) Separate MAs – EE

Conceptualizing an NMA-EE model The Baseline treatment NMAs: comparing treatments relative effectiveness. EE: estimates of absolute risk ( the transition probabilities) Estimate of baseline risk required: Standard of care Alternatively Placebo/No treatment (not always appropriate) The most tested treatment in the network

Conceptualizing an NMA-EE model The Baseline treatment NMAs focus on comparing treatments with respect to their relative effectiveness. Decision models require estimates of absolute risk ( the transition probabilities) To make use of NMA results in EE an estimate of baseline risk is required.

Lit. review of NMA in EE What is “current practice” in NMA – EE integration? Literature review for applications of NMA-informed EE Searched through MEDLINE, EMBASE, Pubmed, NHS EED Non systematic component: Google scholar, pearl growing search. Outcomes / Design / Comparators / Model structure

Preliminary results: Studies Identified107 (39 analyzed so far) NMAEE No. of treatments No. of outcomes in3.3 No. of studies in NMA26.6 Inconsistency check21.6% Software 50% WinBUGS 10% OpenBUGS 3% EXCEL 8 EXCEL 2 TreeAge 2 Str8 Bayesian framework70% Baseline40% from NMA Control over model67% Type of model60% MTC only 18% ITC/MTC 22% TC 51%State transition 17% decision tree 1 study AUC, DES Country64% UK Evidence synthesis in QoL13%

Incorporating uncertainty around NMA parameters “SE to match 95% confidence intervals provided by ITC” “the 95% confidence limits from the MTC were used to estimate the variability” “ PSA values sampled from the WinBUGS CODA output“ “…were defined directly from values recorded in the 10,000 iterations of the NMA”

Conclusion NMAs can provide information that is compatible with the needs of EEs Care needs to be taken on choosing the appropriate NMA method as potential for large variations exists Preliminary results of review show mixed quality on methods used to incorporate NMAs in EE Standardizing practice (guidelines update /software development) and developing know-how needed