NURS3030H NURSING RESEARCH IN PRACTICE MODULE 7 ‘Systematic Reviews’’ ©Janet Rush, RN, PhD, 2010 Graphic: Cochrane Library Logo

Objectives: Meaning, rationale, and usefulness of a Systematic Review Comparison with the ‘literature review’ Basic rules – applies critical appraisal skills … quantitative and qualitative The meta-analysis & meta-synthesis Critical appraisal of a systematic review

Recall - “5 S” levels of evidence (Haynes article) Systems Decision Support Tools Summaries ++ Clinical & management options Synopses Review,1 study Syntheses Sys. Revs + meta synthesis/meta analysis Single studies Haynes, B. (2007) – full citation in the Course Overview – additional readings We are here

The systematic review Rationale: Our motivation = evidence informed practice May not want to make a change in clinical care based on one single study/publication Many studies may be published on a clinical issue … but may have flawed designs or other limitations May not be able to do your own study (may not have skill, resources, time or ) May not have the resources or skills to review all studies Can we take the ‘best’ studies and combine them into one study? The birth of the Systematic Review!

The systematic review A special, very comprehensive study with specific ‘rules’ (ergo, ‘systematic’) Pools small research reports into one large study Applies critical appraisal skills Strengthens clinical decision-making Can be with or without meta analysis (meta synthesis)

Elements/Steps in a Systematic Review 1 Beginning …. A focused research question All parts of the question are included (PI/ECO) or qualitative question Aids in the search and selection of articles Lit search & Study: must define the ‘evidence’ selection strategy Databases, texts, reference lists, conference proceedings, experts, unpublished studies …. What resources were used? Need selection criteria – just like a sample, have inclusion/exclusion criteria Taken from the “P” (population) in the study question, the ‘O’, time limits, language preferences, and study designs (usually RCTs) Extensive review to select studies 2 people to select studies – must agree (Kappa stat) Disagreement? Accept/reject by consensus

Elements/Steps in a Systematic Review 2 Validity of the selected studies: Criteria defined System used to grade studies, e.g., Design – best = RCT Losses – all subjects accounted for? Test statistics & results – similar? Other major flaws? Reviewers (> 2) Agreement needed

Elements/Steps in a Systematic Review 3 Results consistent? Table is used (see next slide) Need to ascertain consistency if doing a meta-analysis (pooled statistic on the combined results) Can we use all studies even if the results were varied, study to study? It is OK if the results were not entirely consistent Homogeneity of the results – results similar Hetergeneity of the results – results different Test for heterogeneity done … tests the extent of variability between studies’ results If significant heterogeneity – random effects model for the meta-analysis If homogeneity of results – fixed effects model for the meta-analysis

Smith, et al 2004 Ontario RCT I = 54 C = 22 Flu shot Vs No shot S/S Author, Date, Place Design N Strategy Outcome Result (test stat) Comments Smith, et al 2004 Ontario RCT I = 54 C = 22 Flu shot Vs No shot S/S Flu X6 1 year Flu in I group (p = .02) Various Ages 6 not accounted for Jones 2006 Sask. Cohort Analytic I = 200 C = 198 “ NS Hosp. Workers Non RCT

Meta analysis: Statistically combines the individual studies Meta analysis: Gives pooled Odds Ratios & 95% CIs for each dependent variable … can be extensive Studies n OR (95% CI) 33 122 12 75 242 total OR “1” or “0”

Critical Appraisal of the SR Was the SR based on a focused research question? Were the selection criteria (for studies/evidence) clear and comprehensive? Were all relevant sources used? Was there agreement on articles selected? Were the studies subjected to a quality assessment system? Was a test for homogeneity done?

Critical appraisal of SR (with a meta-analysis) What were the overall results? … the summary measure ( test stat, OR etc), How large was the treatment effect? Based on the 95% CI, how precise was the treatment effect Were all clinically relevant outcomes considered Methodological strengths, weaknesses Are the results applicable to my patients or my setting?

Application … Critical Appraisal you ask the questions – is this rigorous? A study group published a systematic review and metasynthesis on the effectiveness of the flu vaccine in children (4-12 year olds). Their search for studies included Medline and CINHAL Their criteria were RCTs of flu vaccine, 2000-2007, English, w/ a 1- year follow up for Signs/Symptoms of flu PI selected the 8 studies for the team 2 study team members did the CrAppr. & tabled the results Individual study results showed both effectiveness and no differences between groups Pooled results of % with flu symptoms showed OR 3.4, 95% CI, -0.23 - 6.7 Concluded flu vaccine ineffective

SR of qualitative studies SRs began with quantitative approach Mostly about the effectiveness of treatments, interventions, programs … but also can consider causation studies Pooling qualitative studies – relatively new Pooling qualitative results – “meta synthesis”

Summary: Module 7 The background/rationale of the SR Benefits of the SR Basic rules for the SR & meta-analysis Critical appraisal – applying the rules Understanding the meta-analysis – reading and interpreting the results Considering SRs of qualitative studies