1. Dr. Maryam Tajvar Department of Health Management and Economics
SYSTEMATIC REVIEW
Dr. Maryam Tajvar
Department of Health Management and Economics

2. دانش، مهارت و تجربیات شما؟

4. Pooled effect

5. Traditional vs. Systematic Reviews

6. Traditional vs. Systematic Reviews
Open to biases
Methods not published
No protocol
Sample of research
Subjective review of study quality
An unreliable basis for policy and practice
SYSTEMATIC
Bias is minimised
Exhaustive efforts to identify ALL relevant research
Criteria based review of quality
Methods published
Protocol based
A firm basis for policy and practice

7. What is a systematic review?
“Systematic reviews use objective and transparent methods to identify, evaluate and summarise ALL relevant research findings. To avoid bias, the approach and methods, are set out in advance. When carried out well, systematic reviews provide the most reliable evidence about the effects of tests, treatments and other interventions used in health and social care.”
CRD (Centre for Reviews and Dissemination), 2017

8. Why do a systematic review?
To carry on from where others have already reached, or positon your project relative to previous work
To identify gaps in the literature
To avoid reinventing the wheel
To identify other people working in the same filed
To increase power and precision
To systematically examine primary study method quality for bias and heterogeneity
To inform guidelines, policy & decisions
To identify influential works in your area
To establish consistency of findings (opposing vs. confirming views and aspects)
To identify most productive methodology for your research
To identify sources of information or data that might be useful to you

9. How do a systematic review?

10. A systematic review proposal should include:
Background
Research question/hypotheses
Methods:
- Study inclusion/exclusion criteria
- Search strategies
- Methods of selection
- Assessment of study quality and validity
- Data extraction
- Data synthesis and interpretation
References

11. Example of a Systematic Review Proposal

12. B. Executing (Conducting a review)
4. Identification of literature 5. Selection of studies 6. Quality assessment 7. Data extraction 8. Data synthesis

13. 4. Identification of literature (Searching)
Search strategy :
Sources:
Searching electronic databases (e.g. Medline, Embase, ISI, Scopus, PubMed)
Searching Reference lists
Hand searching relevant specialized journals (e.g. ….)
Grey literature, conference proceedings, documents and reports (e.g. ….)
Contacting key informants in the field
Browsing in libraries
Search terms: mesh terms, keywords

14. Search Strategy:
Getting the most extensive coverage of primary research possible
This is not the same as finding lots of journal articles
Start with individual broad concepts
and combine them together
gradually narrowing your
search down
as you go

15. 5. Selection of studies Types of studies • Primary studies
• previous reviews
Exclusion and inclusion criteria(studies should meet ALL the inclusion criteria and NONE of the exclusion criteria)
• Participants
• Interventions
• Outcomes
• Design
Language restriction
• Translation facilities may limit papers
Study selection process
• By searching first titles, then abstracts, then full-texts
Reviewers
• More than one observer
• Strategy for resolving disagreements

16. 6. Quality assessment How do we know what is a good quality study?
• By methodological hierarchy
• By quality components within method
• By study conduct
• Using checklists and scales (for various types of studies)
Remove or keep poor quality studies? (Based on proposal)

17. 7. Data extraction Develop a data extraction form
Create a database with the information
Double entry the recorded information
Seek clarifications from authors where possible (e.g. missing data)
Data manipulations (if necessary)

18. 8. Data Synthesis Data Synthesis is….
• Tabulation of study characteristics
• Tabulation of study results
• Statistical combining of results (maybe)
• Identification of themes to help explain outcomes
Data Synthesis is Narrative or mathematical [Descriptive or meta-analysis]
Publication Bias (Next slides)

19. Publication Bias
The accessibility of research results may be related to;
Publications with statistically significant results
Year of publication
Type and language of publication
Multiple publications
Author identity
Institution
Country of origin
Journal
Coverage by databases and indexes

20. Selective reporting of studies
Statistically significant results more likely to: be published (publication bias) be published rapidly (time lag bias) be published in English (language bias) be published again (multiple publication bias) be cited by others (citation bias)

21. Selective reporting of outcomes
Hahn 2002: LREC-approved protocols compared with published reports
8 (53%) of 15 studies mentioned analysis plan
7 (88%) of 8 did not follow plan
Chan 2004: 102 trials, 122 publications, 3736 outcomes
Comparing publications to protocols, 62% had at least one primary outcome that was changed, introduced, or omitted.
von Elm 2006: 234 trials, 376 publications
32% of outcomes in publication not in protocol
41% of outcomes in protocol not in publication
reporting associated with P<0.05; OR: 4.1 (CI 1.8 to 9.7)

22. C. Communicating (Reporting and dissemination)
9. Writing up the review
10. Getting the evidence into practice

23. 9. Writing up the review
Background (aim, objective, questions, rational, …)
Methods
Results
Discussion
- Interpret Results:
• provide an estimate of the overall effectiveness (effect sixe) of an intervention
• review effectiveness in different studies, populations and settings
• investigate differences
- Consider:
• Limitations, including publication and other biases
• Strength of evidence
• Applicability
• Economic implications
• Implications for future research

24. 10. Getting the evidence into practice
Disseminate to Ministry of Health
Disseminate to public
Disseminate in academic conferences
Publish it

25. Guidelines on how to write reviews & meta-analyses:
QUORUM statement: For meta-analysis of RCTs
MOOSE guidelines: For meta-analysis of observational studies
CONSORT Statement–Transparent reporting of Trials
ASSERT-A Standard for the Scientific and Ethical Review of Trials
EQUATOR-Enhancing the Quality and Transparency Of health Research

26. Sources and references of systematic reviews
Cochrane Library (via
Cochrane Database of Systematic Reviews (Cochrane Reviews)
Database of Abstracts of Reviews of Effects (DARE)
Cochrane Central Register of Controlled Trials (CENTRAL)
Cochrane Database of Methodology Reviews
Health Technology Assessment Database
NHS Economic Evaluation Database (NHS EED)
Cochrane Methodology Register
NCCHTA Website (
Centre for Reviews and Dissemination website (
Search Strategy from McMaster Hedges Team (Montori et al, BMJ 2005)
Database of Reviews of Effectiveness (
Cochrane Collaboration (
Joanna Briggs Institute and collaboration
Review Manager (
The Cochrane Reviewers’ Handbook (
CRD’s guidance for Undertaking Systematic Reviews of Research on Effectiveness (
Randomised Controlled Trials (
School of Health and Related Research, Sheffield -
Especially for non-RCTs –Campbell Collaboration
Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version [updated March 2011]. The Cochrane Collaboration, Available from