Comparative Drug Effects and Network Meta-Analyses Maylis COSTE, Chrissie FLETCHER EFSPI Statistical Leaders Meeting, June 8th 2011 EFSPI Statistical Leaders Meeting, June 8th 2011 1

Introduction Discussion on NMA Network Meta-Analysis: context and example Overview of documents and initiatives on Network Meta-Analysis Discussion on NMA Context of CER Involvement of pharmaceutical statisticians A multidisciplinary collaboration Need for reference documents and Guidance EFSPI Statistical Leaders Meeting, June 8th 2011 EFSPI Statistical Leaders Meeting, June 8th 2011 2

Context of Network Meta-Analysis Aim: To evaluate relative efficacy of several drugs or interventions for decision-making Method: To establish a network of evidence To perform an integrated and unified analysis To estimate pairwise treatment comparisons and/or classify competitors, incorporating direct and indirect evaluations To investigate validity and assumptions (heterogeneity, consistency,…) EFSPI Statistical Leaders Meeting, June 8th 2011 EFSPI Statistical Leaders Meeting, June 8th 2011 3

Meta-Analyses and types of comparisons Direct comparisons  Meta-analysis No direct comparison but a common comparator  Indirect comparison Direct comparison(s) and indirect comparison(s)  Mixed comparison (including Meta-analysis) Direct comparison(s) and indirect comparison(s) with a (several) common comparator(s)  Network Meta-Analysis Treatment X versus Placebo An other active treatment A therapeutic class Some active treatments (considered sequentially) Several (all) active treatments (considered simultaneously) EFSPI Statistical Leaders Meeting, June 8th 2011 EFSPI Statistical Leaders Meeting, June 8th 2011 4

An example of Network Meta-Analysis Comparative efficacy and acceptability of 12 new-generation antidepressants: a multiple-treatments meta-analysis A. Cipriani and al., Lancet 2009 EFSPI Statistical Leaders Meeting, June 8th 2011 EFSPI Statistical Leaders Meeting, June 8th 2011 5

An example of Network Meta-Analysis (cont.) Figure 3: Efficacy (response rate) and acceptability (drop-out rate) of the 12 antidepressants Odds ratios – 95% Credibility Intervals Random effect model within a Bayesian framework EFSPI Statistical Leaders Meeting, June 8th 2011 EFSPI Statistical Leaders Meeting, June 8th 2011 6

An example of Network Meta-Analysis (cont.) Figure 4: Ranking for efficacy (solid red line) and acceptability (dotted blue line) EFSPI Statistical Leaders Meeting, June 8th 2011 EFSPI Statistical Leaders Meeting, June 8th 2011 7

An example of Network Meta-Analysis (cont.) Investigation of inconsistency (Dias, 2010) Distributions of 2 selected treatment difference Posterior densities of the mean log-odds ratio (LOR) calculated using the full MTC model (black), and direct and indirect evidence only (red and blue respectively) MTC : LOR = -0.181 [-0.359 ; -0.001] Direct : LOR = -0.394 [-0.640 ; -0.144] Indirect : LOR = 0.037 [-0.209 ; 0.277] Inconsistency estimate = -0.429 [-0.775 ; -0.085] Test for Inconsistency: P = 0.008 < 5% EFSPI Statistical Leaders Meeting, June 8th 2011 EFSPI Statistical Leaders Meeting, June 8th 2011 8

Perspectives on Comparative Drug Effect Relative efficacy of drugs: an emerging issue between regulatory agencies and third-party payers HG Eichler & coll Nature Reviews - Drug Discovery Vol 9, Apr. 2010, pp 277-291 “All stakeholders, including academic groups, will need to agree on and apply common statistical research standards for non interventional approaches to Relative Efficacy Assessment: that is, Indirect Comparisons, Network Meta-Analyses and Observational Studies. This may encompass some form of preregistration of study protocols before funding and start of the study.” EFSPI Statistical Leaders Meeting, June 8th 2011 EFSPI Statistical Leaders Meeting, June 8th 2011 9

EFSPI Statistical Leaders Meeting, June 8th 2011 ISPOR initiative Internal Society of Pharmacoeconomics and Outcome Research (ISPOR) Part 1: Interpreting Indirect Treatment Comparisons & Network Meta-Analysis for Health Care Decision-making Report of the ISPOR Task Force on Good Research Practices JP Jarsen (MAPI Values) and US&EU collaborators Multiple treatment comparison & evidence networks Synthesis of evidence Analysis Critically reviewing and interpreting an indirect treatment comparison or network meta-analysis Decision making in the absence of direct / indirect treatment comparisons of RCTs EFSPI Statistical Leaders Meeting, June 8th 2011

EFSPI Statistical Leaders Meeting, June 8th 2011 ISPOR initiative (cont.) Part 2: Conducting Indirect Treatment Comparisons & Network Meta-Analysis Studies Report of the ISPOR Task Force on Indirect treatment Comparisons D. Hoaglin & US and EU collaborators Good Research Practices Guidance on more-technical aspects of conducting NMA Models (fixed and random effects) Frequentist versus Bayesian framework Model validation Example Bayesian Network  Two research practices papers to be published in Value in Health EFSPI Statistical Leaders Meeting, June 8th 2011

EFSPI Statistical Leaders Meeting, June 8th 2011 EUnetHTA Initiative Work Package 5 – Relative Effectiveness Assessment (REA) of Pharmaceuticals EUnetHTA : European network for Health Technology Assessment (network of government appointed organisations, a large number of regional agencies and non-for-profit organisations producers or contributors to HTA in 29 European countries) Lead Partners: the Netherlands, France Project: to scientifically summarize the available methodology on REA and come to common methodology that will closely relate to what is already happening in daily practice Two assessments of RE: Rapid assessment for a new technology at the time of introduction to the market and in comparison to standard care Full assessment of (all) available technology(ies)  Presentation of Rapid and Full model (12/2012) EFSPI Statistical Leaders Meeting, June 8th 2011

EFSPI Statistical Leaders Meeting, June 8th 2011 DIA initiative DIA Comparative Effectiveness Research Scientific Working Group (CER-SWG) MD Rotelli, FDA & EU members(C. Fletcher, J. Roehmel, M.A. Paget) Non competitive collaboration among staff from regulatory agencies, pharmaceutical and biotech companies and academia to share ideas and advance on the science of CER Scientific forum for the discussion and development of innovative methods and software for the design, analysis, and interpretation of CER Education and promotion of the dissemination of methods and best practices in CER, setting expectations for high quality standards, and sharing experiences on case studies Engagement in dialogue and advocacy efforts with industry leaders, the scientific community and regulators to develop a world-wide consensus position on when and how to consider the use of CER  Scientific White Paper, DIA public conferences, submissions to DIJ,… EFSPI Statistical Leaders Meeting, June 8th 2011

EFSPI/PSI HTA SIG Initiative Network Meta-Analysis for Health Technology Assessment Seminar at PSI Conference (May 2011) B. Jones and collaborators (C. Fletcher) Concepts used in NMA Description of work of Evidence Synthesis sub-team Forthcoming paper: methods, example and statistical programs Publication to be submitted in Pharmaceutical Statistics Journal “Network Meta-analysis: what every pharmaceutical statistician should know?” EFSPI Statistical Leaders Meeting, June 8th 2011

Basler Biometric Section Initiative BBS Seminar (May 2011) Comparative Quantitative Assessments Benefit-Risk & Effectiveness Industry Perspective on Comparative Effectiveness Research (CER) and the impact of Health Technology Assessment (HTA) in Europe (C. Fletcher) EFSPI Statistical Leaders Meeting, June 8th 2011

EFSPI Statistical Leaders Meeting, June 8th 2011 Discussions on NMA Relevance for CER Involvement of pharmaceutical statisticians A multidisciplinary collaboration Need for reference documents or guidance from Working groups or Regulatory bodies  Position of Statistical Leaders EFSPI Statistical Leaders Meeting, June 8th 2011

EFSPI Statistical Leaders Meeting, June 8th 2011 Discussion (1) Relevance of Network Meta-Analysis for CER Do we consider that statistical methods on NMA are sufficiently developed and published to generate relevant and reliable information from a CER perspective? Are there technical aspects on NMA that need further research (investigation of assumptions, inconsistency, meta-regression,...)? Should other considerations on NMA application be considered (structure and properties of network, multiple endpoints, heterogeneity with small numbers of studies by treatments, placebo arm…)? Could Individual Data Bases improve notably the conclusions drawn from NMA? Feasibility? How could we support this statistical (academic) research on NMA? EFSPI Statistical Leaders Meeting, June 8th 2011

EFSPI Statistical Leaders Meeting, June 8th 2011 Discussion (2) Involvement of pharmaceutical statisticians in Network Meta-Analyses Are pharmaceutical statisticians sufficiently aware and experienced on NMA? How to develop the methodology and scientific skills among the non-academic community of statisticians to be qualified to conduct or interpret NMA? What should be the topics a qualification on NMA should cover: protocol, analysis (statistical methods, validation and software) and interpretation? How could we support informing and training the statistical (non academic) community on NMA? EFSPI Statistical Leaders Meeting, June 8th 2011

EFSPI Statistical Leaders Meeting, June 8th 2011 Discussion (3) Network Meta-Analysis : a cross-functional collaboration Who is the initiator of a NMA (HTA, “Big Pharma”, Cochrane Collaboration,…)? For HTA initiative, who would participate in protocol, analysis and communication of results (pharmaceutical companie(s), regulatory bodies, specialized CROs, academics…)? What would be its place in drug development (initial submission, B/R reevaluation,…)? For pharmaceutical initiative, which departments should be involved : statistics, clinics, regulatory affairs, scientific documentation…? Should a NMA be periodically updated to take into account new evidence? How could we support the involvement of pharmaceutical statisticians in NMA? EFSPI Statistical Leaders Meeting, June 8th 2011

Back-up Specifications of Network Meta-Analysis Criteria for trials selection pathology patients (with or without restriction to indication,…) trial design (RCT, cross-over, placebo-controlled,…) competitors (registered or in development, whatever the dosage, individually or according to therapeutic class, placebo arm, with or without background therapy,…) evaluation criteria (“common” criteria on efficacy, safety, drop-out, other…) Data sources publications EPAR, SBA, EudraCT, Data Bases EFSPI Statistical Leaders Meeting, June 8th 2011 EFSPI Statistical Leaders Meeting, June 8th 2011 20

EFSPI Statistical Leaders Meeting, June 8th 2011 Discussion (4) Need for reference documents or guidelines from Working groups or Regulatory bodies on Network Meta-Analyses Are there other groups involved in this area? Who would be the best partner? Do we need (further) guidance on NMA? from HTA? Pharmaco- economics? CHMP? What are the need in terms of good practice of NMA of non academic statisticians (PSI initiative)? of other contributors? How could we participate in Working Groups to disseminate information and participate in Guidelines on NMA? EFSPI Statistical Leaders Meeting, June 8th 2011