Journal Club FLUshing Out the Evidence for Neuraminidase Inhibitors in the Treatment of InFLUenza Manish Khullar BSc.(Pharm) Sukhjinder Sidhu BSc.(Pharm) Interior Health Pharmacy Residents Infectious Disease Rotation May 1, 2014

2014

The Controversy

Influenza Signs/symptoms Complications Fever Sore throat Rhinitis Nonproductive cough Myalgias Malaise Complications Secondary infections (bacterial pneumonia, otitis media) Hospitalizations Death

How does Oseltamivir Work? influenza virus spreads by airborne droplets and binds to a cell in the upper respiratory tract and gets taken up into the cell by endocytosis viral RNA is released and is used to produce new viral components To be released from the cell, neuraminidase cleaves receptor and the replicated virus is released for continued viral replication. neuraminidase inhibitor prevents this cleaving step, halting viral replication. This is why neuraminidase inhibitors are more effective when taken within 48 hrs of symptom onset

Our PICO Design Meta-analysis Population Hospitalized patients with confirmed or suspected exposure to influenza Intervention Oseltamivir 75 mg PO bid x 5 days Comparator Placebo Outcome Reducing mortality Reducing morbidity (i.e. pneumonia) Reducing length of hospital stay Minimizing adverse events

Cochrane PICO D Meta-analysis P Hospitalized patients with confirmed or suspected exposure to natural occurring influenza I Oseltamivir 20 RCTs 11 treatment (adults) 4 treatment (children) 5 prophylaxis F/U 6-42 days Zanamivir 26 RCTs 14 treatment (adults) 2 treatment (children) 10 prophylaxis F/U 5-35 days C Placebo O Many outcomes for prophylaxis, treatment, on-treatment and off-treatment Not mortality They really didn’t have a focused clinical question so right off the bat its hard to figure out what it is they’re asking and what it is they are wanting to know  Tx vs Px, adults vs children; and 2 different drugs with a whole whack load of outcomes

Title Title does not state it’s a meta-analysis or systematic review Authors are Cochrane Collaboration Group Can imply it’s at least a systematic review

Introduction Rationale Influenza antivirals are commonly used and stockpiled Previous reviews have risk of reporting bias On list of WHO essential drugs Objectives potential benefits and harms of NIs for influenza in all age groups… all clinical study reports of published and unpublished R, PC trials and regulatory comments What benefits? What harms? All age groups – very broad category; severe vs. mild; outpatients vs. inpatients

Methods Eligibility Criteria Studies NIs RCTs for prophylaxis, post-exposure prophylaxis and treatment of influenza Published and unpublished trials Manufacturer-funded and non-manufacturer funded clinical trials No specific length of follow-up considered Exclusion criteria not identified Described what Primary and Secondary outcomes they wanted to answer with the meta-analysis, however they did not use these outcomes as eligibility study Should they have included observational data? people most at risk of severe outcomes (older people, comorbidities etc) are not included in RCTs; this study looks at previous healthy people and doesn’t include observational studies that have shown protective benefits against severe outcomes But observational studies are weaker than RCT data….but addresses key points in at risk and seriously ill/hospitalized patients *cant compare apples and oranges so its good that they didn’t do that in this case but is this information important? - it is important and it makes sense they keep it like with like but in terms of generalizability? NO because we use this in very sick patients so not ethical and standard of care is if you’re very sick and its during flu season and suspect influenza you will be put on Tamiflu THOUGHTS?

Methods Eligibility Criteria Participants Previously health children and adults Exposed to naturally occurring influenza with or without symptoms Excluded people with illnesses with more significant effects on the immune system (i.e. malignancy or HIV infection) All age groups – very broad category; severe vs. mild; outpatients vs. inpatients

Methods Information Sources Electronic databases CENTRAL, MEDLINE, MEDLINE (Ovid), EMBASE, PubMed (NOT MEDLINE), DARE, NHSEED, HEED January 2010 – July 2013 Clinical study reports Extensive searches conducted Regulatory information searches Trial registries; manufacturer submissions to regulators; drug product info sheet; previous published reviews; Health Technoology Assessment documents; public and manufacturer reigsters Regulatory – FDA, EMEA, Roche, Japanese regulator (PMDA, SBA) They looked at multiple sources and lots of grey literature and were actually quite proud of themselves that they did this. This is important to keep in mind, especially later on when we get into their sensitivity analyses

Methods Search Strategy Just one example; another provided was from EMBASE.com Unaware of limits used therefore would be hard to reproduce

Methods Study Selection 2 authors reviewed title & abstracts 4 authors independently read all data definitely include; definitely exclude; need more information 2 more authors reviewed for inclusion in Stage 1

Methods Study Selection Stage 1  assessing the reliability and completeness of trial data authors discussed face-to-face each trial with comments and other information from regulatory sources decision to whether move trial to stage 2 via consensus Stage 2 satisfied following criteria: completeness – CONSORT-specified methods & specified results internal consistency external consistency

Methods Data Collection Process Utilized a modified CONSORT statement-based extraction template 2 authors each searched oseltamivir and zanamivir trials Disagreements were resolved amongst each other in oseltamivir group and by a 3rd author in the zanamivir group For clinical study reports, complete list of trials were sent to manufacturers asking them to check accuracy and completeness of their list

Methods Risk of Bias in Individual Studies Used Cochrane “Risk of bias” tool Review author judged risk of bias Bias assessed at outcome and study level Studies included had high risk of bias Some outcomes from studies were poorly documented/collected They were more conservative in their approach to Bias’s – if unknown, classified study as high risk of bias They didn’t use any JADAD scales and overall this review seemed to look at more quantity than quality

Methods Synthesis of Results Chi2: test for heterogeneity I2: level of statistical heterogeneity Threshold for significance unknown Tau2: estimate of between-study variance Combining data using random-effects approach Chi – tells you if heterogeneity is present or not (p < 0.1); assesses whether observed differences are compatible with chance alone I2 – tells you magnitude of heterogeneity; represents the percentage of total variation across studies due to heterogeneity 0-25% - not important 25-50% - be careful > 50% - heterogeneity Tau2 - estimated standard deviation of underlying effects across studies; > 1 = presence of substantial statistical heterogeneity WHAT SHOULD THEIR THRESHOLD BE?

Methods Risk of Bias Across Studies Included unpublished trials -They assessed that although they are RCTs they question the methodology so why not just analyze the only high quality studies? -They did assess this but didn’t do anything about it because they included these studies still which makes this interesting

Methods Additional analyses Subgroup analysis to investigate high estimates of heterogeneity Meta-regression to investigate pneumonia heterogeneity Sensitivity analysis Fixed-effect method of Mantel and Haenszel to supplement primary analyses using random-effects method Peto’s method used when sparse data and borderline sensitivity Sensitivity -Another type of analysis that would be interesting to see and be of use is one involving study design quality (ie. high vs low quality) so see how the results would differ but unfortunately they didn’t do this. 2) Subgroup Analysis of severely ill (would be good to look at severely ill patients because in clinical practice the main people who get tamiflu are those with severe illness) WHAT ANALYSES WOULD BE BENEFICIAL?

Results Study Selection 208 studies identified form various sources 123 studies excluded 19 studies awaiting classification 66 studies for which clinical study reports requested 53 studies met eligibility 5 trials excluded due to incompleteness 2 trials excluded as didn’t fit inclusion criteria 46 trials included 20 oseltamivir 26 zanamivir 123 trials excluded from entering stage 1 due to PK studies, active comparator, compared higher vs. lower dose schedules, on-going trials; 19 awaiting assessment

Results Risk of Bias within Studies Presented selection, attrition, reporting, performance and detection bias To address issue of reporting bias, they ignored published trial reports if clinical study reports and regulatory information were available Random sequence generation missing in many trials Blinding may have been affected in many trials Placebos may have contained active substances Others – data on participants were not reported in oseltamivir trials; data on effects of rescue/relief meds were incomplete in clinical study reports of oseltamivir and not reported separately in zanamivir trials High risk of bias in studies included – affecting validity of included studies; however they were conservative with labeling “high risk of bias” – pg 17 - Placebo pills – zanamivir – lactose powder, which can cause bronchospasm; oseltamivir – dehydrochloric acid/Ca-PO4 dehydrate, which can cause GI symptoms

Results No analysis conducted on mortality outcome Discussed deaths in the oseltamivir and zanamivir arms, but no statistical analyses completed THOUGHTS?

Time to First Alleviation of Symptoms in Adult Treatment (ITT Population) Oseltamivir vs. Placebo Orient to forest plot Outcome Result is reported as a mean difference Using random effects model to pool results from 11 trials vs. fixed effects model – will discuss this in relation to the results on the next slide If diamond falls left of the line = outcome in favor of oseltamivir; if it falls right of the line = outcome in favor of placebo Given evaluating mean difference, line of no difference is 0 (i.e. if confidence interval crosses/touches line of no difference = not statistically significant) Lets take a closer look at the result

Time to First Alleviation of Symptoms in Adult Treatment (ITT Population) Oseltamivir vs. Placebo In adult treatment, oseltamivir reduced the time to first alleviation of symptoms by 16.8 hrs = representing a 10% reduction from 7 to 6.3 days MAKE SURE YOU SAY: AT THE END OF THE DAY DO WE CARE about this clinically? Discuss heterogeneity analysis and use of random effects model 3. Random Effects model -more conservative test that makes it more difficult to make overall estimate precise -problem with this model, is it assumes distribution of effects across different studies and follows a normal distribution which may not be the case and may make it difficult to show overall statistical effect *when see middle of the road heterogeneity (ie 30-50%) then switching to random effects model will show you as it could make the diamond wider and move it to the right to make it more conservative. Clinically significant reduction? When to use random vs. fixed effects?

Complication: Pneumonia Oseltamivir vs. Placebo Orient to forest plot Outcome Result is reported as a risk ratio Using random effects model to pool results vs. fixed effects model Given evaluating risk ratio, line of no difference is now 1 2 separate analyses completed – 1. Trials with non-specific diagnosis of pneumonia 2. Trials with specific diagnosis of pneumonia Will focus on those trials with a specific diagnosis of pneumonia

Complication: Pneumonia Oseltamivir vs. Placebo NNT = 100 Oseltamivir reduced the risk of pneumonia by 31% vs. placebo in those with a diagnosis of pneumonia (5 trials), however this is not a statistical difference given diamond crosses line of non-significance Once again looking at the heterogeneity tests …. not appropriate to use random effects model For interest sake, they combined the trials with a reliable diagnosis of pneumonia and those without and found a RRR of 45% with a NNT of 100 favoring oseltamivir This is a perfect case of outcome level bias because they noticed that diagnosis of pneumonia was weak in a number of trials yet they included them in the analysis. So why did they include it?

Nausea in Adult Treatment (On-Treatment) Oseltamivir vs. Placebo Orient to forest plot Outcome Result is reported as a risk ratio Using random effects model to pool results Line of no difference is 1 11 trials included

Nausea in Adult Treatment (On-Treatment) Oseltamivir vs. Placebo NNH = 28 Oseltamivir was associated with more nausea vs. placebo (NNH 28), which is statistically significant given diamond doesn’t cross line of non-significance Use of random effects in this analysis appropriate given I2 -more conservative test that makes it more difficult to make overall estimate precise -problem with this model, is it assumes distribution of effects across different studies and follows a normal distribution which may not be the case and may make it difficult to show overall statistical effect

Vomiting in Adult Treatment (On-Treatment) Oseltamivir vs. Placebo Orient to forest plot Outcome Result is reported as a risk ratio Using random effects model to pool results Line of no difference is 1 11 trials included

Vomiting in Adult Treatment (On-Treatment) Oseltamivir vs. Placebo NNH = 22 Oseltamivir was associated with more vomiting vs. placebo (NNH 22), which is statistically significant given diamond doesn’t cross line of non-significance Use of random effects in this analysis was not appropriate given I2

Results Risk of Bias Across Studies No funnel plot or Egger’s test done to rule out publication bias Made every effort to minimize publication bias by including non-published trials No discussion on bias across studies, only bias within studies Outcome bias – attrition bias from incomplete outcome data; detection bias (blinding of outcome assessment)  unable to ID all data for all outcomes in all oseltamivir trials and 8 of zanamivir trials; Study bias – selection bias from random sequence generation; reporting bias; other bias (active placebos); performance bias (blinding of participants) This brings us back again, if these trials had a lot of bias, why include them or why not do a sensitivity analysis afterwards looking at the higher vs lower quality of studies. ASK EVERYONE: SHOULD WE BE WORRIED ABOUT PUBLICATION BIAS IN THIS STUDY? SHOULD WE BE WORRIED ABOUT PUBLICATION BIAS?

Results Additional Analysis Subgroup analysis Time to first alleviation of symptoms in adults by infection status Pneumonia (diagnosis vs. non-diagnosis) Time to first alleviation.. done only in zanamivir trials – therefore we did not discuss Pneumonia subgroup analysis was presented in outcome data Other subgroup analyses that would have been beneficial = severity of illness, hospitalized vs. non-hospitalized for example

Discussion Summary of Evidence NIs have small, non-specific effects on reducing time to alleviation of influenza-like illness symptoms in adults Treatment trials… do not settle the question of whether complications of influenza, such as pneumonia are reduced Use of oseltamivir increases the risk of adverse events such as nausea, vomiting…

Discussion Conclusions … appears to be no evidence for patients, clinicians or policy-makers to use these drugs (NIs) to prevent serious outcomes Implications for future research More effective preventative measures Early identification of complications Page 41 – discuss in context to Kaiser 2003 and Hernan and Lipsitch

Limitations Did not formally assess mortality outcome Effect of complications was based on unclear and potentially unreliable definitions Included many trials with high risk of bias Affects validity of the results Authors do not summarize results in the context of observational study results that are driving standard of care in influenza treatment A generalized conclusion is made without taking severity of illness into account Based upon these trials and high risk of bias – it is difficult to conclude effects on a broad range of populations; would have been beneficial to do a sensitivity analysis to conclude results of the higher quality RCTs

Our Conclusions Population studied was broad Not generalizable to specific groups Doesn’t address the question of whether to give NIs in high risk patients More reviews needed to draw definite conclusions about mortality and reductions in complications More reviews needed in severe influenza (i.e. critical care patients) Standard of care is to give Tamiflu to very sick patients… how will you ever get ethical approval to put sick patients on placebo  observational studies and M-A, may open up RCT potential or a subgroup analysis of the more severe patients

Your Conclusions? Cochrane IDSA