1 Cell-Free Hemoglobin-Based Blood Substitutes and Risk of Myocardial Infarction and Death Natason et al., JAMA, Prepublished online April 28, 2008 at Journal Club Yulia Lin, Transfusion Medicine Resident University of Toronto May 16, 2008

2 How to critically appraise a meta- analysis? Guyatt G, Rennie D (ed). User’s guide to the medical literature: a manual for evidence-based clinical practice. Chicago, IL: AMA Press Chapter 1E: Summarizing the evidence.

3 Definitions Overview Summary of the medical literature that attempts to address a focused clinical question Systematic review Using methods designed to reduce the likelihood of bias Meta-analysis Review that uses quantitative methods to summarize the results

4 Steps to a Meta-analysis Question Population, Intervention, Control, Outcome Literature search Identify all relevant studies Inclusion and exclusion criteria Select and critically appraise Data abstraction Collect relevant information Analysis Guyatt G, Rennie D. User’s Guide to the Medical Literature (2002) ch 1E

5 How to critically appraise a meta- analysis? 1. Are the results of the study valid? 2. What are the results? 3. Will the results help me in caring for my patients?

6 Are the results of the study valid? Did the overview address a focused clinical question? Was the search for relevant studies detailed and exhaustive? Were the primary studies of high methodologic quality? Were assessments of studies reproducible? Question Literature search Selection & appraisal Data abstraction

7 What are the results? Were the results similar from study to study? What are the overall results of the review? How precise were the results? Analysis

8 Will the results help me in caring for my patients? Are the results generalizable and can they be applied to my patients? Were all clinically important outcomes considered? Are the benefits worth the harms and cost? External validity

9 Cell-Free Hemoglobin-Based Blood Substitutes and Risk of Myocardial Infarction and Death Natason et al., JAMA, Prepublished online April 28, 2008 at

10 Hemoglobin-based blood substitutes (HBBS) Ideally Universally available Eliminate need for refrigeration Long shelf-life Decreased risk of iatrogenic infection But the challenge = free hemoglobin Scavenges nitric oxide --> vascular thrombosis

11 Possible solutions? Chemically alter Hb to create larger, more stable HBBS molecule Cross-link Polymerize Pegylate

12 Copyright restrictions may apply.

13 Question The association between HBBSs and the risk of myocardial infarction and death Additional purpose Examine regulatory process that permitted repeated trials with theses agents despite safety concerns

14 Literature Search Search Strategy PubMed, EMBASE, Cochrane Human RCT in English 1980 to March 25, 2008 “blood substitutes” and “hemoglobin” Internet FDA advisory committee meeting Press releases from companies

15 Study Criteria Inclusion RCT Outcome variable: Death or MI Exclusion Did not involve a HBBS All patients were healthy volunteers or younger than 19 years Results included in a subsequent report

16 Copyright restrictions may apply.

17 Methods Standard data collection form Outcomes: mortality and MI Descriptive data Intention-to-treat analysis used when reported Missing data (1 treatment, 4 control) analyzed both as survivors and nonsurvivors Independent review by 2 authors with a 3 rd author resolving discrepancies

18 Statistics “We assessed the homogeneity using Breslow-Day test and an associated I 2 statistic” To pool or not to pool? Tests of heterogeneity determine whether the degree of variability in treatment effects between studies is greater than that expected by chance I 2 measures the extent of variation 0% indicates no observed heterogeneity Heterogeneity increases as I 2 increases

19 To pool or not to pool? Forest plots Pool if all point estimates are on the same side of the line or if confidence intervals overlap

20 Statistics “Risks estimated using Cochran-Mantel- Haenszel test with 95% CI” How to pool the results? Results typically weighted based on study size (occasionally can be weighted based on quality of study)

21 Statistics “A fixed effects model was required because of the null values for the estimates of between-study variance” Models for pooling data in meta-analysis Fixed effects model Assumes there is a single true value underlying all study results such that if all studies were infinitely large, they would yield identical estimates of the effect Error comes only from within study variation

22 Models for pooling in meta-analysis Random effects model Assumes that studies are a random sample of a population of studies where each study estimates a different underlying true effect and the distribution of these effects is assumed normal around a mean value Error comes from both within study variation and between study heterogeneity Random effects model usually has a wider confidence interval If little heterogeneity, compared to within- study variance, approximates fixed effects model

23 Results 16 trials of 5 HBBSs met inclusion criteria Type: 4 double-blind, 7 single-blind, 4 open- label, 1 uninformative Patients: 5 trauma, 10 surgical, 1 stroke Outcomes: 12 used death, 10 used MI Median time from completion of trial to publication 4 years (1 to 6 years) Trials described in Table 2

24 Copyright restrictions may apply. NNH 62

25 Copyright restrictions may apply. NNH 50

26 Copyright restrictions may apply.

27 Subgroup analyses Results consistent regardless of Patient population Control: non-blood vs. blood product Product removed Type of product Low vs. High tetramer content Low vs. High P50 content Unpublished vs. published

28 Copyright restrictions may apply. Cumulative Mortality and MI By 2000, 11 studies completed Mortality RR 1.27; 95% CI MI RR 2.77; 95% CI

29 The regulatory process Sponsors are required by law to report their results to the FDA after studies are completed (whether or not published) Data are not made public unless product is approved or an advisory committee is convened Article outlines pitfalls of this system including that 5 trials are ongoing and 1 is being planned

30 Are the results of the study valid? Did the overview address a focused clinical question? PIO  Intervention and outcome Population was varied and one might expect that this might lead to variations in the absolute risk of mortality/MI but not necessarily the relative risk Control varied but no difference in sub-group analysis Was the search for relevant studies detailed and exhaustive? Yes, included unpublished studies and attempts to contact the companies for more complete data Assumed? Hand searching the reference lists of the articles or consulting experts in the area No change when unpublished studies excluded

31 Are the results of the study valid? Were the primary studies of high methodologic quality? Restricted to RCT but all RCTs included Did not comment on validity of primary studies Were assessments of studies reproducible? Yes 2 investigators reviewed the studies with a 3 rd author resolving any discrepancies

32 What are the results? Were the results similar from study to study? Yes, both by I 2 statistic and forest plot What are the overall results of the review and how precise were the results? HBBS associated with increased death/MI Death: RR 1.30; 95% CI, MI: RR 2.71; 95% CI,

33 Will the results help me in caring for my patients? Are the results generalizable and can they be applied to my patients? Yes – covers a variety of conditions and settings Were all clinically important outcomes considered? The study considered the adverse effects of therapy It did not report on the treatment effect though the endpoints in Table 2 listed avoidance of allogeneic transfusion and in one case, disability score Are the benefits worth the harms and cost? Weighing the risk of increased mortality and MI against the risks of avoidance of allogeneic transfusion No

34 Conclusion Agree with authors’ conclusions “In the analysis of available data from clinical trials, HBBSs was associated with a significantly increased risk of death and MI” Questions?