JAMA Facial Plastic Surgery: Evidence-Based Medicine and Level of Evidence Primer John S. Rhee, MD, MPH Wayne F. Larrabee, Jr, MD

1 What Is Evidence-Based Medicine? Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

What Is Evidence-Based Medicine? “The conscientious, explicit and judicious use of the current best evidence in making decisions about the care of individual patients.” – Sackett et al1 Integration of 3 critical elements Best research evidence Clinical expertise Patient values 1Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ. 1996;312(7023):71-72. Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

What Is Evidence-Based Medicine? Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

What Evidence-Based Medicine Is Not EBM is not “old hat” “Everyone is doing it already” EBM is not impossible to practice EBM is not “cookbook” medicine EBM is not cost-cutting medicine EBM is not restricted to randomized controlled trials (RCTs) and meta-analyses Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Case Example Dr Nash has been in practice for 10 years. He sees a 60-year-old woman in his office who would be a good candidate for a face-lift. Patient asks about “new medicine X” that may decrease postsurgical bruising. He is unsure if he should recommend the use of this medication. Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

The MEDLINE Search MeSH, Google? – rhytidectomy, “new medicine X,” aging face, ??? Limiting the search Randomized clinical trial (RCT)? Meta-analyses? Cochrane review? Level of evidence Best level is 4? Best level is 2? Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Level of Evidence Table 1 Properly powered and conducted RCT, systematic review or meta-analysis of RCTs 2 Well-designed controlled trial without randomization; prospective comparative cohort trial 3 Case-control studies; retrospective cohort study 4 Case series with or without intervention; cross-sectional study 5 Opinion of respected authorities; case reports Adapted from Oxford Centre for Evidence Based Medicine. http://www.cebm.net/index.aspx?o=1001. Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Level of evidence plays a role in just 1 circle (current best evidence) in the EBM picture Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

How to Critique an Original Study: Going Beyond Level of Evidence Are the results valid? Patient dropout rate Length of follow-up Patient cohort makeup What are the results? How large was the treatment effect? How precise was the estimate of the effect? How was the effect measured? Are the results applicable to my patients? Study patients similar to my patients? Outcome measures clinically relevant? Are my surgical skills and techniques similar to those of the study surgeons? Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Incorporate All 3 Elements of EBM Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Case Example (Reprise) Dr Nash integrates the 3 elements of evidence-based medicine to decide what to do for this patient Literature review and best evidence level Best available evidence is 2, no RCTs Predominantly level 4 evidence supporting efficacy of this medication Clinical expertise Comfortable with own surgical procedure and past outcomes without “new medication X” Surgical cohort in study was predominantly male with different surgical technique, so results may not be applicable for his patient Patient values Patient is aware of surgeon experience with procedure and made aware of current level of evidence and accepts recommendations of surgeon not to use “new medication X” Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

2 Facing Levels of Evidence The JAMA Facial Plastic Surgery Initiative Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Levels of Evidence Designation Only applicable to clinical, therapeutic (disease treatment) studies Examples include any treatment or interventions in which outcomes are provided Nonratable articles include (but are not limited to) narrative reviews, editorials, videos, and studies that are basic science, non–human-based, diagnostic, and cadaver/anatomy and survey/questionnaire based Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Building Blocks of Evidence-Based Practice OCEBM & USPSTF Individual Studies Original Articles Cochrane & QUORUM Systematic Reviews Meta-analyses GRADE & USPSTF Clinical Practice Guidelines Abbreviations: OCEBM, Oxford Centre for Evidence Based Medicine; USPSTF, US Preventive Services Task Force; QUORUM, Quality of Reporting Meta-analyses; GRADE, Grades of Recommendation Assessment, Development, and Evaluation. Evidence-Based Medicine and Level of Evidence Primer Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 • September 20, 2018 • Slide 15 15

Level of Evidence Table 1 Properly powered and conducted RCT, systematic review or meta-analysis of RCTs 2 Well-designed controlled trial without randomization; prospective comparative cohort trial 3 Case-control studies; retrospective cohort study 4 Case series with or without intervention; cross-sectional study 5 Opinion of respected authorities; case reports Adapted from Oxford Centre for Evidence Based Medicine. http://www.cebm.net/index.aspx?o=1001. Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Study Designs Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Descriptive Studies Observational, nonanalytic Describe patterns of “distribution of disease” Person, place, time Quick, easy, inexpensive Help in “formulation of hypotheses” Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Case Report – Level of Evidence: 5 Observational, descriptive Detailed report of an unusual observed characteristic of a single patient Advantages Inexpensive May lead to formulation of hypothesis Disadvantages Based on experience of 1 patient Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Case Series – Level of Evidence: 4 Observational, descriptive Characteristics observed in group of patients Advantages Inexpensive Useful in formulation of hypothesis Disadvantages No control group Bias related to subject or characteristic selection Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Cross-Sectional Studies – Level of Evidence: 4 Observational, descriptive Data on individual patients at 1 point in time Exposure and disease measured simultaneously Prevalence of disease and exposure can be calculated Useful for formulation of hypothesis about presence of association between exposure and disease Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Cross-Sectional Studies – Level of Evidence: 4 Advantages Inexpensive Best design for studying the status quo of disease or condition Disadvantages Provide only a “snapshot in time” “The chicken or the egg” dilemma Difficult to choose control or comparison group Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Study Designs Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Analytic Studies: Observational or Experimental Provide information about determinants of disease by testing hypotheses Observational analytic studies “Investigator simply observes” Experimental analytic studies “Investigator allocates exposure” Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Case-Control Studies – Level of Evidence: 3 Observational, analytic Cases matched with controls Cases = those with disease/outcome of interest Control = comparative group without disease Past “exposure” is determined Proportions with the “exposure” within each group are compared Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Case-Control Studies – Level of Evidence: 3 Advantages Quick and least expensive of analytic designs Efficient for rare diseases with long latent periods Disadvantages Control group selection difficult Disease incidence rates and relative risks cannot be calculated Temporal relationship between exposure and disease is sometimes difficult to establish Prone to selection and recall bias Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Prospective Cohort Studies – Level of Evidence: 2 Observational, analytic Participants are free of disease/outcome of interest at onset of study Participants selected on exposure/risk factor under study Exposed group (treatment group) vs unexposed group Participants followed over period of time to assess occurrence of disease or outcome Formulation of incidence rates and relative risk Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Prospective Cohort Studies – Level of Evidence: 2 Advantages Establishment of temporal relationship between exposure (treatment) and disease Assessment of multiple effects of a single exposure (treatment) Calculation of incidence rates and relative risk Selection bias can be controlled or minimized Disadvantages Very time-consuming and expensive Inefficient for rare diseases Dropout rate can affect validity of results An association but not necessarily causation between exposure (treatment) and disease can be made Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Study Designs Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Therapeutic Experimental Studies Randomized or nonrandomized Analytic studies Commonly called clinical trials Exposure/intervention is controlled or allocated by investigator Experimental group vs control group Participants screened for eligibility Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Nonrandomized Clinical Trials – Level of Evidence: 2 Comparison group is historical Historical controls Can lead to bias in analysis Weaker study design due to bias in patient assignment Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Randomized Clinical Trials – Level of Evidence: 1 “Gold standard” in study designs Randomized assignment to either experimental or control group Removes potential selection bias Potential for bias is prevented by blinding the participant to the intervention Double-blinded = when both investigator and participant are blinded to the intervention Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Randomized Clinical Trials – Level of Evidence: 1 Advantages Provide strongest evidence for determining causation Least number of problems or biases Best design to assess efficacy of a treatment or procedure Disadvantages Very expensive and time-consuming Inefficient if effect of treatment is small Criteria for measuring outcome may be controversial Ethical issues Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Meta-analysis Studies – Level of Evidence: 1 Results from published studies combined to provide overall conclusion Ideally comprised of RCTs only Smaller sample–sized studies that do not achieve statistical significance are combined Studies need to be similar with respect to disease and exposure/intervention Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Meta-analysis Studies – Level of Evidence: 1 Advantages Provide a reliable estimate of most likely effect of an exposure/intervention Useful for planning future trial with adequate power Disadvantages Carry risk of several biases Meta-analysis of observational studies especially suspect Too much bias and too many confounding variables Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Systematic Reviews – Level of Evidence: Variable A thorough and systematic review of the literature targeted by a specific research question Methods and results should be reproducible and transparent to all May assign levels of evidence to existing studies Identify knowledge gaps and point to needed areas of investigation Dependent on level of evidence of primary literature Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

Meta-analysis ≠ Systematic Review Evidence-Based Medicine and Level of Evidence Primer September 20, 2018 •

JAMA Facial Plastic Surgery: Evidence-Based Medicine and Level of Evidence Primer John S. Rhee, MD, MPH Wayne F. Larrabee, Jr, MD For more information and tools on evidence-based medicine, see http://www.JAMAevidence.com.