Describing rare and serious harms of interventions B C Reeves 1, A Herxheimer 2, G A Wells 1, G Gyte 3 1. Non-Randomised Studies Methods Group; 2. Adverse Effects Methods Group; 3. Pregnancy and Childbirth Collaborative Review Group Introduction Systematic reviews need to consider all effects of an intervention, i.e. harms as well as benefits. Failing to do so means that a review presents a partial summary of the evidence about the effects of an intervention (even if the evidence about benefit is not biased), which may mislead health care professionals & users). Evidence about rare and serious harms rarely comes from randomised controlled trials (RCTs). Frequencies of serious harms (SAEs) are usually estimated from databases, longitudinal case series, case reports or custom-designed cohort studies. Evidence about SAEs from RCTs may not be applicable because RCTs often exclude people most at risk of serious harms from a new intervention. In non-randomised studies (NRS), data quality may be poor and ascertainment of SAEs uncertain. Intervention effects estimated from NRS are susceptible to confounding. Objectives To describe (a) relevant information when reporting rare, serious adverse effects (SAEs) of interventions and (b) factors that influence requirements. Methods We considered of examples of SAEs associated with specific interventions (see Tables 1 & 2). Examples were chosen to illustrate different combinations of factors hypothesised to influence the information requirements of users when weighing up beneficial and harmful effects of an intervention: Margin of benefit over next best treatment “Valuation” of estimated beneficial and harmful effects Availability of alternative intervention (with lower risk of SAE) Background risk of SAE (rare,>1% & <5% vs very rare, ≤1%) IndicationInterventionComparatorPopulationIntended benefitImplicated SAE 1. Planning where to give birth Home birthHospital birthPregnant women, <35 years, uncomplicated pregnancy & no known risk factors for IPPM Less morbidity from obstetric intervention Intra-partum related perinatal mortality (IPPM) 2. Hypercholesteraemia CerivastatinAlternative drug to reduce low density lipoprotein level Women or men with hyper- cholesteraemia & no contra- indications to statin therapy Reduction in low density lipoprotein level Rhabdomyolysis 3. Neovascular age- related macular degeneration (nAMD) RanibizumabPegaptanibElderly women or menHalt progression of choroidal neovasc- ularisation & visual loss Arterial thrombo- embolic event Table 1: Examples of interventions and implicated SAEs IndicationSAE frequency with best practice RCT or NRS? SAE frequency with intervention RCT or NRS? Rare/ very rare? Alternative intervention available? Effectiveness of alternative intervention Benefit highly valued? 1. Planning where to give birth ≈ /1,000 births NRSNot knownNRSVery rareMidwifery-led unit Not knownYes 2. Hypercholesteraemia 5 /100,000 pyrs RCT≈250 /100,000 pyrs NRSVery rareYesSimilarNo 3. Neovascular AMD≈19-78 /1,000 pyrs RCT & NRS ≈23 /1,000 pyrs RCTRareYesLess effectiveYes Table 2: Estimated SAE frequencies with best practice and with intervention Observations If an SAE is very rare, the difference in SAE freq between intervention & “best practice” ≈ SAE freq in people with the intervention, and the SAE freq in people with the intervention will be rare even if the intervention ‘obviously’ causes the SAE (e.g. relative effect >10) Highly valued benefits, when no alternative intervention is available, may override aversion to a very rare SAE even if the intervention ‘obviously’ causes the SAE (Example 1) If alternative interventions have similar benefits, an intervention that obviously causes an SAE is unacceptable (Example 2) If an SAE is relatively common (≥1% and 1,000) and a causal link is difficult to establish from non-randomised studies (Example 3) Describing the risk of an SAE among people with an intervention is a prognostic research question; factors influence the risk of an SAE can be investigated to allow estimates of SAE frequency to be customised for individual patients. Conclusions: We propose that reviewers should distinguish evidence for a causal link between an intervention and an SAE, and the risk of the SAE among people having the intervention. For SAEs that are very rare with “best practice”, the risk of an SAE among people having an intervention is highly relevant and may be sufficient for decision-making. This is a simple descriptive statistic, easily understood and avoids debate about susceptibility to bias of data from NRS. Estimates of SAE freq can be qualified by describing the populations from which they were obtained, information about the characteristics of individuals that influence the SAE freq, and study limitations that compromise the validity of the estimates. Weighing up benefits and harms is particularly difficult when SAEs freq with best practice ‘rare’ (≥1% & <5%), cf. very rare (<1%). Economic factors (e.g. cost-effectiveness, cost impact) have not been considered. The SAE frequency with intervention for example 3 is very imprecise but expected to be higher than for best practice because of evidence for a similar drug.