Evaluating the Medical Evidence ​ A TOOLKIT FOR THE INTERPRETING THE EFFECTIVENESS OF INTERVENTIONS Niteesh Choudhy, M.D., Ph.D.

Take home points Evidence-based medicine has revolutionized the way health care is delivered 1 There is “evidence” to support whatever you believe! 2 Academic detailers are ambassadors of the evidence and need to know how to embrace its strengths and limitations 3

Evidence matters  Evidence-based medicine aims to apply the best evidence gained from the scientific method to clinical decision making  Gained prominence in the early 1990’s  De-emphasizes intuition, unsystematic clinical experience and pathophysiologic rationale  Application of evidence in patient care has resulted in substantial reductions in morbidity and mortality SOURCES: Guyatt et al. JAMA 1992;268: ; Ford et al. NEJM 2007; 356: Contributors to cardiovascular death rates

The volume of “evidence” is overwhelming NOT ALL EVIDENCE IS OF EQUAL QUALTY  In 1992, internists needed to read an estimated 17 articles every day of the year in order to “keep up” with the literature  The volume of published articles since then has increased exponentially  Made more difficult because not all evidence is of equal quality (i.e. difficult to identify those studies that are particularly important)  Creates a virtually impossible problem for practicing physicians SOURCES: Davidoff et al BMJ 1995; 310: 1085;

A hypothetical example  A new cholesterol lowering pill, nolipid, has been synthesized and developed into tablet form for oral consumption  In a prospective study, nolipid:  significantly reduced LDL cholesterol levels by 50% (p<0.0001)  was well tolerated  had no adverse effects WOULD YOU RECOMMEND THE USE OF NOLIPID FOR PATIENTS WITH ELEVATED CHOLESTEROL?

The questions we should be asking: Did they choose the right comparator? Did they choose the right outcome? Absolute or relative changes? Overall or subgroup results?

Choosing the right comparator SPARCL  4731 patients who had stroke or TIA one to six months before study entry and NO CAD  Randomized to atorvastatin 80 mg daily or placebo  Significant reduction in primary end-point (fatal or non-fatal stroke)  Placebo: 13.1%  Atorvastatin 11.2% SOURCE: SPARCL investigators. NEJM 2006; 355:

Choosing the right comparator SPARCL BUT…  Many patients in SPARCL would already be on a statin according to current treatment guidelines  SPARCL should have compared high and lower intensity statin therapy  More generally, to get FDA approval, drugs generally only need to demonstrate superiority over placebo but in reality, clinicians and decision makers want information about comparative efficacy/safety SOURCE: SPARCL investigators. NEJM 2006; 355: Current NCEP/ATPIII cholesterol treatment guidelines Risk Category LDL Goal (mg/dL) LDL Level at Which to Consider Drug Therapy (mg/dL) CHD or CHD Risk Equivalents (10-year risk >20%) <100  130 (100–129: drug optional)

Evaluating the right outcome EZETIMIBE AND THE ENHANCE TRIAL  720 patients with familial hypercholesterolemia  Randomized to simvastatin + ezetimibe or simvastatin alone  Substantial reductions in LDL from combination therapy  A widely used “surrogate” outcome in cardiovascular trials  However, there was no change in atherosclerosis (carotid-artery intima-media thickness)  Thankfully, this was the trial’s “primary” outcome although many other trials that preceded it only evaluated LDL SOURCE: Kastelin et al. NEJM 2008;

Surrogate end-points  Use of surrogate end-points may lead to rapid and appropriate dissemination of new treatments (e.g. HIV)  However, may also lead to excess morbidity/mortality (e.g. inotropes may improve hemodynamics but some may cause excess mortality)  The majority of clinical trials focus on these outcomes A surrogate end-point is “a laboratory measurement or physical sign used as a substitute for a clinically meaningful end-point that measures directly how a patient feels, functions, or survives”

Looking at the “right” results THE SUB-GROUPS OF CHARISMA  Enrolled 15,603 patients with established cardiovascular disease or multiple risk factors  Randomized to clopidogrel 75 mg or placebo added to aspirin mg daily (median follow-up duration 28 months)  The published conclusion: HR: 0.93 (95% CI 0.83 to 1.05)

Overall v. subgroup results CHARISMA  Numerous pre-specified sub- group analysis some of which reached (borderline) statistical significance  When analyzing multiple subgroups, some will reach statistical significance by chance alone  While there are statistical methods to deal with this, we should ideally focus on the overall trial results (or the results of a limited set of prespecified subgroups) RR 0.88 (95%CI: )

Absolute v. relative risks JUPITER TRIAL NEJM 2008; 359:

Absolute v relative reductions JUPITER TRIAL NEJM 2008; 359:

An EBM toolkit Something is often better than nothing We often care about how well a new treatment compares with the existing standard of practice (not placebo) Right comparator? Surrogate end-points are easier to measure and are often sufficient for a new drug to be approved If available, we should really focus on “hard” outcomes Right end- point? Often possible to find subgroups that derive less or more benefit from a treatment Should focus on the overall trial results (entire cohort, primary outcome) Overall v. subgroup results? Absolute and relative effects can lead to very different assessments of the benefit/safety of a treatment Should use both when weighing the significance of a therapy Right effect measure?

Take home points Evidence-based medicine has revolutionized the way health care is delivered 1 There is “evidence” to support whatever you believe! 2 Academic detailers are ambassadors of the evidence and need to know how to embrace its strengths and limitations 3