Appraising Randomized Clinical Trials and Systematic Reviews October 12, 2012 Mary H. Palmer, PhD, RN, C, FAAN, AGSF University of North Carolina at Chapel Hill

Objectives Discuss systematic reviews in the context of the pyramid of evidence Describe the importance of the CONSORT statement for randomized controlled trials Appraise select randomized controlled trials Appraise select systematic literature reviews Describe the process of synthesis and evaluation

Pyramid of Evidence

Appraising Systematic Reviews  Systematic reviews are a compilation of similar studies that address a specific clinical question.  Systematic reviews are not the same as a literature review or narrative review  The process the was used to conduce the review should be explicit  Look for a detailed description of the databases accessed, the search strategies, and the search terms  Clinicians should be able to clearly see which studies were included and which were excluded and how selected studies were assessed

Systematic Reviews A specific research question is answered. Detailed inclusion and exclusion criteria are explicated. Elaborate and thorough search strategies. Standardized review protocols that use trained reviewers (rather than PIs) Standardized abstracting process for capturing details Pre-established quality criteria to rate the value of the individual studies.

Systematic Reviews How is bias eliminated? – Databases accessed – Search strategies used – Search terms used – Years searched – Language used – Unpublished studies included (publication bias)

Systematic Reviews Randomized controlled trials PRISMA Meta-analyses and observational and studies MOOSE

Meta-Analysis Meta-analysis is a systematic review, not all systematic reviews are meta-analysis. Quality of studies included is critical. Meta-analyses pool data from individual studies – Increased sample size – Pooled statistical results – Definitive conclusions possible

Meta-Analysis (cont’d)  Remember, the report should inform clinicians about how data were extracted from the individual studies  Summary  Meta-analysis: the statistical approach to synthesizing the results of two or more studies  A relatively new methodology that has become a hallmark of EBP As with all methodologies, applicability must be considered

CONSORT CONSORT Translations The CONSORT website will announce and host translated CONSORT documents that we are aware of. If you are interested in translating a CONSORT document into another language, please see the CONSORT translation policy.CONSORT translation policy CONSORT 2010 Translations Chinese* CONSORT Statement PDF CONSORT Explanation & Elaboration PDF Checklist PDF / MS Word Flow Diagram PDF / MS WordPDF MS WordPDFMS Word

Appraising Randomized Controlled Trials (RCTs)  These studies can accurately establish cause and effect and can inform the efficacy and effectiveness of interventions  Questions related to validity of RCTs  Were the subjects randomly assigned to the experimental and control groups?  Was random assignment concealed from the individuals who were first enrolling subjects into the study?  Were the subjects and providers kept blind to study group?

Appraising the Validity of RCTs  Were reasons given to explain why subjects did not complete the study?  Were the follow-up assessments long enough to fully study the effects of the intervention?  Were the subjects analyzed in the group to which they were randomly assigned?  Was the control group appropriate?  Were instruments valid and reliable?  Were the subjects in each of the groups similar on demographic and baseline clinical variables?

Evaluation and Synthesis of Quantitative Studies  Once studies have been selected, they should be melded together into a synthesis upon which to base practice and standards  Individual studies should be evaluated using an evaluation table - see Table 5-8 in your textbook  Synthesis is NOT reporting the findings of study after study; it is combining, contrasting, and interpreting a body of evidence to reach a conclusion

Question  Tell whether the following statement is true or false.  Meta-analysis results in evidence that is applicable to a larger patient population than individual studies.

Answer  False  Rationale: In meta-analyses, combining the results of several studies produces a larger sample size and thus greater power to accurately determine the magnitude of the effect. This does not, however, increase the size of the relevant patient population.

Homework Find a systematic review or a randomized controlled trial related to your PICOT question. Write a critical appraisal and send it and a pdf of your article to: Deadline: October 22, 2012