Chapter 7 The Hierarchy of Evidence

Chapter Overview The purpose of this chapter is to explore the concept of “evidence” as it applies to the advancement of health care practice and patient care. The growing focus on evidence-based medicine has been fueled by the advances in information technology. Clinical experience, combined with appraisal of the best available literature, should drive patient care decisions and the advancement of health care. Medical decisions become evidence based.

Evidence Across Patient Care Patient care includes: screening examinations, diagnostic procedures, providing a prognosis, developing a treatment plan, and implementing prevention efforts. Research efforts in each of these areas can inform and improve clinical practice. Keep in mind that differences in research methods and data analysis across patient care can result in differences in how the hierarchy of evidence is described.

Levels of Evidence Not all evidence is created equally. Differences in the strength of evidence come from two sources: sampling and research methods. Sampling: Larger samples are more likely to estimate true population values and result in narrower confidence intervals than small samples. Research methods: There are two models of a research method hierarchy. A hierarchy of strength of evidence for treatment decisions The Oxford Center for Evidence-Based Medicine hierarchy

A Hierarchy of Strength of Evidence for Treatment Decisions Within patient, randomized treatment order trials Systematic review of clinical trials with random assignment to treatment Clinical trial with random assignment to treatment Systematic review of non-random assignment of treatment trials Single trial with non-random assignment of treatment Laboratory studies related to physiological and biomechanical mechanisms underlying disease, injury, or treatment Opinion developed through informal clinical observations

The Oxford Center for Evidence-Based Medicine Hierarchy Level of Evidence Randomized controlled trials 1. a. Systematic review of RCTs b. Individual RCTs c. All-or-none studies Cohort studies 2. b. Individual cohort studies c. Outcomes research Case-control studies 3. a. Systematic reviews of case-control studies b. Individual case-control studies Case series / Case study 4. a. Includes poorly designed cohort and case-control studies Anecdotal evidence 5. Animal research Bench research Unpublished clinical observations (Adapted with permission from http://www.cebm.net/index.aspx?o=1025. Oxford Center for Evidence-Based Medicine--Levels of Evidence (March 2009). Accessed May 10, 2010.)

Within Patient, Randomized Treatment Order Trials Within patient, randomized treatment order trials are referred to by Guyett et al. as N-of-1 randomized trials. Within patient, randomized treatment order trials provide the strongest evidence. They are studies in which patients receive all interventions under consideration in random order over the duration of the investigation. They provide strong evidence and minimize the number of patients needed as there are no comparison groups. They minimize the influence of differences between patients in terms of responsiveness to intervention.

Limitations of Within Patient, Randomized Treatment Order Trials Trials require that the condition being treated is relatively stable and that the effect of an intervention does not influence the response to the other interventions being investigated. Trials are not common because few conditions treated by therapists and athletic trainers are stable across long periods of time and because the natural history and interventions directed at change are not reversible.

Systematic Reviews of Randomized Trials Systematic reviews: Research efforts in which data are acquired from existing literature through a planned and thorough search process. Data acquired through systematic review may undergo statistical analysis through a process called meta- analysis. Because data from multiple studies are combined, statistical power is increased providing for narrower confidence limits and greater confidence in the conclusions drawn from the analysis.

Randomized-Controlled Clinical Trials (RCTs) These studies prospectively assign patients to treatment groups and measure specific outcomes in order to assess the benefit of a particular intervention. These trials are prospective, in contrast to case series or case studies, in which patients are identified after a course of care for analysis. Prospective designs allow for greater control over factors that confound the identification of cause– effect relationships.

Cohort Studies Cohort studies may be prospective or retrospective in design. They involve the study of groups based upon exposure or intervention, and they assess for differences in outcomes. The principle difference between an RCT and a prospective cohort is that in the RTC, individuals are randomly assigned to interventions, whereas in a cohort study, groups are investigated. Random assignment in retrospective cohort studies is not possible because researchers identity groups based on exposure or intervention at some point in the past and then follow the groups forward. Prospective cohort designs can maximize efficiency, reduce research costs, and yield important information.

Outcomes Studies and Case-Controlled Studies Population-based “outcomes research” is similar to cohort studies. These studies seek to understand the end results of particular health care practices and interventions. Case-Controlled Studies These studies are similar to retrospective cohort studies or imbedded within a prospective cohort study. In these studies, comparisons are made between groups of subjects based on an outcome rather than an exposure or intervention.

Case Series and Case Studies Case series and case studies provide detailed descriptions of a series or a single case. By design, these reports provide no statistical comparison but rather describe the course of care and the outcome of one or more cases. Because only a single or a few cases are reported on, it is not possible to make inferences with regard to cause and effect or to generalize the outcome of the cases with confidence.

Bottom of the Evidence Hierarchies Guyatt et al. label one type “physiological studies.” Oxford Center for Evidence-Based Medicine identifies these studies as bench research and animal research. The other type is identified as unsystematic clinical observations, anecdotal evidence, or unpublished clinical observations. Conclusions drawn from bench (that is, basic) science and animal research cannot be generalized directly to human patients.

Hierarchy of Evidence for Diagnosis Oxford Centre for Evidence-Based Medicine identified five levels of evidence of diagnostic studies paralleling those of treatment and prevention studies. (Note: Diagram on Chapter 7, slide 6). Level 1: Level 1a evidence consists of systematic reviews of high- quality individual trials with homogeneous findings and clinical decision rules (CDRs). Level 1b studies consist of cohort studies and CDRs validated in a single setting. Level 1c studies reveal “absolute” SPIN or SNOUT. SpPIN refers to tests with near-perfect specificity (high specificity rules in). SnNOUT refers to tests with near-perfect sensitivity (high sensitivity rules out).

Hierarchy of Evidence for Diagnosis (continued) Level 2: Level 2a evidence consists of systematic reviews of 2b evidence. Level 2b evidence consists of exploratory (small sample) reports with good reference standards and CDRs not validated prospectively at one or more centers. Level 3: Level 3a evidence consists of systematic reviews of 3b reports. Level 3b reports involve the enrollment of non-consecutive patients or fail to consistently apply reference standards.

Hierarchy of Evidence for Diagnosis (continued) Level 4: Level 4 evidence comes from case-control studies or diagnostic studies without independent review of the reference standard. Level 5: Level 5 evidence stems from expert opinion and bench and animal science.

Hierarchy of Evidence for Prognosis Similar to hierarchy of diagnostic studies. Strong similarities in Level 1 evidence, with all or no case series replacing absolute SpPIN or SnNOUT in level 1c. Level 2 differences consist of “outcomes research” at level 2c, which does not exist for diagnostic studies. No Level 3 evidence exists for studies of prognosis. Level 4 consists of case series studies. Level 5 evidence is generally consistent across all categories of research.

Relevance of Evidence to Problem Ask the following about the papers you read: Are the results accurate, correct, and unbiased? Do the results apply to the patient I am treating? Will the results help me? Consider the whole of the patient being treated when applying the results of students. Problems occur when: The evidence is less strong or conflicting. The patient differs from those described in the research.

Chapter Summary and Key Points Confidence gained from experience and continued study of the literature helps the clinician cope with the stresses of providing optimal care and eases the anxiety of the patients we treat. N-of-1 trials and “Level 1” evidence should be weighted more heavily in clinical decisions than sources lower on the hierarchy. Hierarchies of evidence relate the strength of evidence in making decisions about patient care, not the sophistication of the research methods and data. Differences in the strength of evidence come from two sources: sampling and research methods. Poor methods increase the likelihood of biased data, threatening internal as well as external validity. Clinicians must carefully consider how strongly the available research will influence their decisions. All evidence is worthy of consideration for the patients seeking our care.