Mike Fischer, M.D., M.S. Slides from Niteesh Choudhy, M.D., Ph.D. Evaluating the Medical Evidence ​A TOOLKIT FOR THE INTERPRETING THE EFFECTIVENESS OF INTERVENTIONS

Take home points Evidence-based medicine has revolutionized the way health care is delivered 1 There is “evidence” to support whatever you believe! 2 Academic detailers are ambassadors of the evidence and need to know how to embrace its strengths and limitations 3

Contributors to cardiovascular death rates Evidence matters Evidence-based medicine aims to apply the best evidence gained from the scientific method to clinical decision making Gained prominence in the early 1990’s De-emphasizes intuition, unsystematic clinical experience and pathophysiologic rationale Application of evidence in patient care has resulted in substantial reductions in morbidity and mortality Contributors to cardiovascular death rates SOURCES: Guyatt et al. JAMA 1992;268:2420-2425; Ford et al. NEJM 2007; 356:2388-98

The volume of “evidence” is overwhelming NOT ALL EVIDENCE IS OF EQUAL QUALTY In 1992, internists needed to read an estimated 17 articles every day of the year in order to “keep up” with the literature The volume of published articles since then has increased exponentially Made more difficult because not all evidence is of equal quality (i.e. difficult to identify those studies that are particularly important) Creates a virtually impossible problem for practicing physicians SOURCES: Davidoff et al BMJ 1995; 310: 1085; http://www.nlm.nih.gov/bsd/medline_lang_distr.html

A hypothetical example A new cholesterol lowering pill, nolipid, has been synthesized and developed into tablet form for oral consumption In a prospective study, nolipid: significantly reduced LDL cholesterol levels by 50% (p<0.0001) was well tolerated had no adverse effects WOULD YOU RECOMMEND THE USE OF NOLIPID FOR PATIENTS WITH ELEVATED CHOLESTEROL?

The questions we should be asking: Did they choose the right comparator? Did they choose the right outcome? Absolute or relative changes? Overall or subgroup results?

Choosing the right comparator SPARCL 4731 patients who had stroke or TIA one to six months before study entry and NO CAD Randomized to atorvastatin 80 mg daily or placebo Significant reduction in primary end-point (fatal or non-fatal stroke) Placebo: 13.1% Atorvastatin 11.2% SOURCE: SPARCL investigators. NEJM 2006; 355: 549-59

Choosing the right comparator SPARCL BUT… Many patients in SPARCL would already be on a statin according to current treatment guidelines SPARCL should have compared high and lower intensity statin therapy More generally, to get FDA approval, drugs generally only need to demonstrate superiority over placebo but in reality, clinicians and decision makers want information about comparative efficacy/safety Current NCEP/ATPIII cholesterol treatment guidelines Risk Category LDL Goal (mg/dL) LDL Level at Which to Consider Drug Therapy (mg/dL) CHD or CHD Risk Equivalents (10-year risk >20%) <100 130 (100–129: drug optional) SOURCE: SPARCL investigators. NEJM 2006; 355: 549-59

Evaluating the right outcome EZETIMIBE AND THE ENHANCE TRIAL 720 patients with familial hypercholesterolemia Randomized to simvastatin + ezetimibe or simvastatin alone Substantial reductions in LDL from combination therapy A widely used “surrogate” outcome in cardiovascular trials However, there was no change in atherosclerosis (carotid-artery intima-media thickness) Thankfully, this was the trial’s “primary” outcome although many other trials that preceded it only evaluated LDL SOURCE: Kastelin et al. NEJM 2008; 1431-43

Surrogate end-points Use of surrogate end-points may lead to rapid and appropriate dissemination of new treatments (e.g. HIV) However, may also lead to excess morbidity/mortality (e.g. inotropes may improve hemodynamics but some may cause excess mortality) The majority of clinical trials focus on these outcomes A surrogate end-point is “a laboratory measurement or physical sign used as a substitute for a clinically meaningful end-point that measures directly how a patient feels, functions, or survives”

Looking at the “right” results THE SUB-GROUPS OF CHARISMA Enrolled 15,603 patients with established cardiovascular disease or multiple risk factors Randomized to clopidogrel 75 mg or placebo added to aspirin 75-162 mg daily (median follow-up duration 28 months) The published conclusion: HR: 0.93 (95% CI 0.83 to 1.05)

Overall v. subgroup results CHARISMA Numerous pre-specified sub-group analysis some of which reached (borderline) statistical significance When analyzing multiple subgroups, some will reach statistical significance by chance alone While there are statistical methods to deal with this, we should ideally focus on the overall trial results (or the results of a limited set of prespecified subgroups) RR 0.88 (95%CI: 0.77-0.998)

Absolute v. relative risks JUPITER TRIAL NEJM 2008; 359: 2195-207

Absolute v relative reductions JUPITER TRIAL NEJM 2008; 359: 2195-207

Overall v. subgroup results? An EBM toolkit Something is often better than nothing We often care about how well a new treatment compares with the existing standard of practice (not placebo) Right comparator? Surrogate end-points are easier to measure and are often sufficient for a new drug to be approved If available, we should really focus on “hard” outcomes Right end-point? Often possible to find subgroups that derive less or more benefit from a treatment Should focus on the overall trial results (entire cohort, primary outcome) Overall v. subgroup results? Absolute and relative effects can lead to very different assessments of the benefit/safety of a treatment Should use both when weighing the significance of a therapy Right effect measure?

Take home points Evidence-based medicine has revolutionized the way health care is delivered 1 There is “evidence” to support whatever you believe! 2 Academic detailers are ambassadors of the evidence and need to know how to embrace its strengths and limitations 3