Incorporating data from single-arm studies into network meta-analyses Steve Kanters PhD(c), Kristian Thorlund PhD, Edward Mills PhD, Jeroen Jansen PhD, Nick Bansback PhD April 14th, 2015

Network meta-analyses (NMA) An expansion of traditional pairwise meta-analyses that consider multiple treatments at a time NMA combine direct and indirect comparisons to make the most of the available evidence The utility of NMA is in providing comparative efficacy for all therapeutics of a given medical condition Presently, NMAs are generally restricted to RCT evidence Alternative sources of evidence include comparative observational studies and single-arm studies

Potential limitations to RCT evidence A large phase 3 RCT is at the top of the hierarchy of evidence In some situations it may be viewed as being lower on a ‘hierarchy of relevance’ than other designs Timeliness: A large phase 3 RCT can take years to complete the relevance of its findings may be reduced by the time of reporting E.g. in oncology, if findings of several uncontrolled trials and observational studies may have already shown promising results Ethics: RCTs will often be needed to confirm treatment effects, but not always ethical. Underpowered: For safety endpoints, observational studies can be much more relevant because RCTs are likely to be too short for safety outcomes

When are other sources of evidence needed? For some interventions only single arm trials or observational evidence is available. i.e., to connect the network. e.g., rare diseases. RCTs tend to be powered for efficacy and in turn are often underpowered for safety. Observational studies can often be larger and longer and hence better inform safety. Observational studies may shed light on efficacy and safety within sub-populations. RCTs dominated by Caucasian participants may not speak to Asian or Black populations. In time-to analyses, single arm phase IV trials may help supplement time-to information for both efficacy and safety.

Purpose Methods have already been suggested for combining comparative observational studies to RCTs However uptake has been slow The purpose of today’s talk is to discuss how to integrate single-arm evidence into NMA We provide motivational examples , but perhaps the most convincing is the integration of non-comparative phase IV trials to safety analyses

Standard NMA models

Standard model definitions θj are the likelihood parameters transformed by the appropriate link function E.g. logit(pj), yj, log(rj) μj are the study effects: the part of the observed outcome attributable to prognostic factors δj is the comparative treatment effect that we seek to solve for

Adjusted indirect comparison 8

Adding Studies of other epidemiological design to an RCT NMA

Single-arm studies (& comparative observational studies) Uncontrolled studies: Impossible to disentangle study effects from treatment effects. Only observed outcomes. However, it can be useful to add these kinds of studies to synthesis of RCT evidence…. …as long as we acknowledge their limitations with the analyses methods!

Combining RCT and to other designs How can we incorporate single-arm evidence to NMA? Use the single-arm evidence to create informative priors Create a virtual comparison based on patient characteristics

Informative priors > 65% of NMAs conducted today are conducted using Bayesian hierarchical models The majority of the remaining 35% are restricted to adjusted indirect comparisons using the Bucher method These tend to start with non-informative priors for the model parameters. Specifically: dAB ~ N(0, 0.0001) μj ~ N(0, 0.0001) If single-arm evidence exists for both treatments A and B, we can use this evidence to create informative priors on dAB For example, if dealing with a dichotomous outcome with linear model for mean difference dAB ~ N(yB,endo – yA,endo, precendoω), where ω is a correction weight Note that it does not make sense to construct informative priors on μ as this is study specific rather than treatment specific

Indirect comparison (NMA) incorporating single arm trial 1 A B C D A B C D 2 Interested in relative treatment effect of D versus A, B and C. Only single arm trial for D Prediction of comparator arm given patient characteristics in single arm trial for D. Creation of ‘virtual’ CD trial. 3 A B C D Incorporation of ‘virtual’ CD trial in network

Relative advantages of each method Both approaches allow for the integration of single-arm evidence to NMA Informative priors offer a more convenient way to weight the evidence Direct inclusion into the NMA requires a more contrived reduction of the effective sample size Direct inclusion lends itself better to all additional manipulations of the NMA, such as meta-regression adjustments

Applications

Single-arm evidence as prior Example 1 - Meningitis Cryptococcal meningitis is a leading cause of HIV-associated death and is the most common cause of meningitis in sub-Saharan Africa Multiple guidelines recommend use of Amphotericin B (AmB) in combination either 5-flucytosine (5FC), where available, or fluconazole (Azole) Despite high level of recommendations: No RCTs have shown mortality benefit for addition of Azoles to AmB Single, recent RCT has shown mortality benefit for addition of 5FC to AmB

( ) Observational studies Randomized Controlled Trials Campbell JI, Kanters S, Bennett JE, Thorlund K, et al. Comparative effectiveness of induction therapy for human immunodeficiency virus-associated cryptococcal meningitis: A network meta-analysis. Open Forum Infect Dis. 2015;2(1) ( ) Single-arm Studies

Example 1 – Meningitis Methods applied Pooled single arm results for each intervention Used single-arm based comparative effects as ‘informative priors’ in the Bayesian NMA model Estimated expected comparative pairwise efficacy by taking the difference between single arm results. Penalized precision of single arm comparative estimates by 4

Rationale for Penalization Amphotericin + Azole Amphotericin + 5FC Conceptual Indirect Comparison Conceptual control

Example 1 – Meningitis Results Heterogeneity in the model was reduced By 26% Model fit was improved DIC 144 vs. 234 Effect estimates were more precise Two comparisons became “statistically significant”

Example 2 – Hepatitis C Sofosbuvir is a recently licensed direct acting antiviral (DAA) for hepatitis C Single arm trials makes up much of the evidence for the two Sofosbuvir regimens Non-RCT evidence is required to connect the network, particularly when restricted to non-cirrhotic patients We analyzed the network by Directly including single-arm evidence by using virtual comparisons Integrating the single arm data through informative priors With informative priors with decreased precision (factor of 4) Excluding the single arm-evidence

Application to Hepatitis C Non-cirrhotic patients only Full network Randomized Controlled Trials Single-arm Studies ( ) 22

Odds ratios for sustained virological response P2bR = pegylated interferon alpha-2b; P2aR = pegylated interferon alpha-2a BOC = boceprevir; TEL = telaprevir; SIM = simeprevir; SOF = sofosbuvir; LDV = ledipasvir; (SDT) = standard duration therapy; (RGT) = response guided therapy

Take home messages

Take home messages Typically evidence synthesis of only RCT evidence has good internal validity. In some cases adding single-arm evidence can be very informative, especially when there are a limited number of RCTs. We have to be aware of limitations of observational single-arm evidence Analyses should be done using multiple methods, including those restricted to RCTs (if possible) It comes back to validity vs. precision

Thank You