1. NURS3030H NURSING RESEARCH IN PRACTICE MODULE 7 ‘Systematic Reviews’’
©Janet Rush, RN, PhD, 2010
Graphic: Cochrane Library Logo

2. Objectives: Meaning, rationale, and usefulness of a Systematic Review
Comparison with the ‘literature review’
Basic rules – applies critical appraisal skills … quantitative and qualitative
The meta-analysis & meta-synthesis
Critical appraisal of a systematic review

3. Recall - “5 S” levels of evidence (Haynes article)
Systems
Decision Support Tools
Summaries
++ Clinical & management options
Synopses
Review,1 study
Syntheses
Sys. Revs + meta synthesis/meta analysis
Single studies
Haynes, B. (2007) – full citation in the Course Overview – additional readings
We are here

4. The systematic review Rationale:
Our motivation = evidence informed practice
May not want to make a change in clinical care based on one single study/publication
Many studies may be published on a clinical issue … but may have flawed designs or other limitations
May not be able to do your own study (may not have skill, resources, time or )
May not have the resources or skills to review all studies
Can we take the ‘best’ studies and combine them into one study?
The birth of the
Systematic Review!

5. The systematic review
A special, very comprehensive study with specific ‘rules’ (ergo, ‘systematic’)
Pools small research reports into one large study
Applies critical appraisal skills
Strengthens clinical decision-making
Can be with or without meta analysis (meta synthesis)

6. Elements/Steps in a Systematic Review 1
Beginning …. A focused research question
All parts of the question are included
(PI/ECO) or qualitative question
Aids in the search and selection of articles
Lit search & Study:
must define the ‘evidence’ selection strategy
Databases, texts, reference lists, conference proceedings, experts, unpublished studies …. What resources were used?
Need selection criteria – just like a sample, have inclusion/exclusion criteria
Taken from the “P” (population) in the study question, the ‘O’, time limits, language preferences, and study designs (usually RCTs)
Extensive review to select studies
2 people to select studies – must agree (Kappa stat)
Disagreement? Accept/reject by consensus

7. Elements/Steps in a Systematic Review 2
Validity of the selected studies:
Criteria defined
System used to grade studies, e.g.,
Design – best = RCT
Losses – all subjects accounted for?
Test statistics & results – similar?
Other major flaws?
Reviewers (> 2)
Agreement needed

8. Elements/Steps in a Systematic Review 3
Results consistent? Table is used (see next slide)
Need to ascertain consistency if doing a meta-analysis (pooled statistic on the combined results)
Can we use all studies even if the results were varied, study to study?
It is OK if the results were not entirely consistent
Homogeneity of the results – results similar
Hetergeneity of the results – results different
Test for heterogeneity done … tests the extent of variability between studies’ results
If significant heterogeneity – random effects model for the meta-analysis
If homogeneity of results – fixed effects model for the meta-analysis

9. Smith, et al 2004 Ontario RCT I = 54 C = 22 Flu shot Vs No shot S/S
Author, Date, Place
Design N
Strategy
Outcome
Result
(test stat)
Comments
Smith, et al
2004
Ontario
RCT
I = 54
C = 22
Flu shot Vs
No shot
S/S
Flu
X6 1 year
Flu in I group
(p = .02)
Various
Ages
6 not accounted for
Jones
2006
Sask.
Cohort
Analytic
I = 200
C = 198 “ NS
Hosp.
Workers
Non RCT

10. Meta analysis: Statistically combines the individual studies
Meta analysis: Gives pooled Odds Ratios & 95% CIs for each dependent variable … can be extensive
Studies n OR (95% CI) 33
122 12 75
242
total OR
“1” or “0”

11. Critical Appraisal of the SR
Was the SR based on a focused research question?
Were the selection criteria (for studies/evidence) clear and comprehensive?
Were all relevant sources used?
Was there agreement on articles selected?
Were the studies subjected to a quality assessment system?
Was a test for homogeneity done?

12. Critical appraisal of SR (with a meta-analysis)
What were the overall results? … the summary measure ( test stat, OR etc),
How large was the treatment effect?
Based on the 95% CI, how precise was the treatment effect
Were all clinically relevant outcomes considered
Methodological strengths, weaknesses
Are the results applicable to my patients or my setting?

13. Application … Critical Appraisal you ask the questions – is this rigorous?
A study group published a systematic review and metasynthesis on the effectiveness of the flu vaccine in children (4-12 year olds).
Their search for studies included Medline and CINHAL
Their criteria were RCTs of flu vaccine, , English, w/ a 1- year follow up for Signs/Symptoms of flu
PI selected the 8 studies for the team
2 study team members did the CrAppr. & tabled the results
Individual study results showed both effectiveness and no differences between groups
Pooled results of % with flu symptoms showed OR 3.4, 95% CI,
Concluded flu vaccine ineffective

14. SR of qualitative studies
SRs began with quantitative approach
Mostly about the effectiveness of treatments, interventions, programs … but also can consider causation studies
Pooling qualitative studies – relatively new
Pooling qualitative results – “meta synthesis”

15. Summary: Module 7 The background/rationale of the SR
Benefits of the SR
Basic rules for the SR & meta-analysis
Critical appraisal – applying the rules
Understanding the meta-analysis – reading and interpreting the results
Considering SRs of qualitative studies