1. Best Practice Systematic Review
Emily Kothe
@emilyandthelime

2. Should you do a Review?
Not necessarily…

3. Reasons to do a review
You have a research question that is best answered by synthesising existing evidence rather than running a new study
There is no recent review that answers that same question
You want to explore apparent inconsistencies in previous studies

4. Reasons not to do a review
They take a very long time
They are really hard
Not every topic needs a review

5. Reviews take a very long time
Category
Mean ± SD
Median
Range
Authors/team members
5±3 5
1–27
Time (in weeks; registered project start to publication date)
67.3±31.0
65.8
6–186
Quantitative analysis yield rate (%)
2.6±4.7
1.0
0.03–32.43
Qualitative analysis yield rate (%)
2.7±4.6
0.05–26.19
Borah R, Brown AW, Capers PL, et al Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry BMJ Open 2017;7:e  doi:  /bmjopen

6. Not every topic needs a review
Analysis from:

7. Bad reasons to do a review*
You want a publication early in your thesis
Your supervisor told you to do one
You want to get your head around the existing literature

8. What type of review should you do?
There are a lot more than you think…

9. Types of Reviews
Factor
Evidence Briefing
Mapping Review
Scoping Review
Rapid Evidence Assessment
Rapid Realist Review
Systematic Review
Meta-analysis
Question
Narrow
Broad
Timescale
2 weeks
4-16 weeks
24 weeks
8-24 weeks
36-52 weeks
52-78 weeks
Resources
Single reviewer, limited databases
Single reviewer, comprehensive databases
Single review, limited databases
Double reviewer, limited databases
Double reviewer, comprehensive databases
Adapted from: Booth, A. (2016) EVIDENT Guidance for Reviewing the Evidence: a compendium of methodological literature and websites DOI: /RG

10. Types of Reviews
The type of review you conduct will depend on the purpose of the review, your question, your resources, expertise, and type of data.
Don’t assume that it is better to do the “harder” type of review

11. What do you want to know?

12. Your review question
As with primary research, developing a good research question is the key to conducting a good review
What has been done?
What hasn’t been done?
What do we know?
What works?
How does it work?

13. PICO(S)
Participants
Intervention
Comparison
Outcome
Study design

14. How do you conduct your review?

15. Standards MECIR and MEC2IR Conduct Standards
PRISMA, MOOSE, MARS Reporting Standards

16. MEC2IR Standards Item No. Status Item Name Standard
Setting the research question(s) to inform the scope of the review 1
Mandatory
Formulating review questions
Ensure that the review question and particularly the outcomes of interest, address issues that are important to stakeholders such as consumers, practitioners, policy makers, and others. 2
Pre-defining objectives
Define in advance the objectives of the review, including participants, interventions, comparators, and outcomes. 3
Highly desirable
Considering potential adverse effects
Consider any important potential adverse effects of the intervention(s) and ensure that they are addressed. 4
Considering equity and specific populations
Consider in advance whether issues of equity and relevance of evidence to specific populations are important to the review, and plan for appropriate methods to address them if they are. Attention should be paid to the relevance of the review question to populations such as low socioeconomic groups, low or middle-income regions, women, children, people with disabilities, and older people.

17. Write and Register a Protocol
The protocol sets out in advance the methods to be used in the review with the aim of minimizing bias.
The background section of the protocol should communicate the key contextual and conceptual factors relevant to the review question and provide the justification for the review.
The protocol should specify:
The review question.
Study inclusion and exclusion criteria using the relevant PICOS elements.
The protocol should also specify the methods which will be used to:
Identify research evidence
Select studies for inclusion
Data extract included studies
Quality assess included studies
Synthesise results
Disseminate the review findings

18. PROSPERO
PROSPERO Protocols

19. Search the Literature
Talk to a Librarian to help develop/review your search terms
Include robust methods to locating unpublished and poorly indexed literature
Document your process

20. Press

21. Screening
Screening is one of the most time consuming parts of running a review, this is what distinguishes your review from a non-systematic “cherry picking” review of the literature.
Use processes that will keep your screening transparent and reduce error
Pilot everything

22. My Current favourite Screening Tool: Rayyan

23. Data Extraction Use a standard data extraction process for all studies
Duplicate extraction of key study data where possible (most important to duplicate results)
Record source location for all extracted data

24. Assessing Risk of bias
You should incorporate risk of bias assessments into interpretation of findings
There are widely used tools for randomised and quasi-experimental designs (Cochrane Risk of Bias Tool and ROBINS-I)
There is less consensus about appropriate tools for observational studies
Make domain assessments of bias rather than global scores

25. Synthesising results What does it all mean?
Don’t just count number of significant and non-significant studies – that is just a really bad meta-analysis!
Remember to consider risk of bias when synthesising results

26. Creating summary tables in Mail merge
Reduce error and improve formatting consistency