1. Introduction to systematic reviews
Gerlinde Pilkington & Geoff Bates
Public Health Institute, LJMU

2. Session objectives At the end of this session you will be able to:
Describe the purpose of a systematic review and the types of questions they can answer.
Understand when and why you would want to undertake a systematic review.
Identify the key features of a systematic review and understand what a good quality systematic review includes.

3. Instructions
You have 2 sets of statements linked to the basic steps in undertaking a review of the literature.
Working in groups match up the numbered statements for each step that you think apply when undertaking a ‘traditional’ literature review and those that apply when undertaking a systematic review.
Gerlinde to give instructions
There are confusing terms that are used to describe reviews, not all of them mean the same thing even though they are often described in similar ways. A common confusion is between traditional reviews, what you might have done as a chapter of your dissertation for example, and a systematic review – can you unpick that and figure out the differences between what many people consider a literature review, and a systematic review? Five minutes to sort and five minutes to discuss

4. Step
Traditional review
Systematic review
1. Defining a question
May or may not be clearly defined
Clearly defined and well-focused
2. Writing a protocol
Not usually required
Recommended/essential
3. Methodology
Does not follow explicit or rigorous methodology
Follows explicit and rigorous methodology
4. Searching
No pre-defined search criteria and not necessarily comprehensive
Comprehensive and explicit searching methods used and reported
5. Inclusion and exclusion criteria
Not essential and no selection of studies based on explicit pre-defined criteria
Essential, selection based on explicit pre-defined criteria
6. Screening titles and abstracts; selecting full-text
Generally carried out by one researcher
Systematic screening and selection
7. Quality assessment
Not necessarily
Yes
8. Data extraction
Yes but no explicit methods
Yes using explicit methods
9. Analysis and synthesis
No clear method of synthesis; typically narrative
Recognisable method, can involve meta-analysis, narrative or qualitative synthesis
10. Application
Any field
11. Replication
Not easy
Yes replicable

5. What is a systematic review?
A review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant research, and to collect and analyse data from the studies that are included in the review
Cochrane Handbook for Systematic Reviews of Interventions

6. So, what are systematic reviews good for?

7. From ‘Bad Pharma’ by Ben Goldacre
“Instead of just mooching through the research literature, consciously or unconsciously picking out papers here and there that support your pre- existing beliefs, you take a scientific, systematic approach to the very process of looking for scientific evidence, ensuring that your evidence is as complete and representative as possible of all the research that has ever been done”
From ‘Bad Pharma’ by Ben Goldacre
Quote from Ben Goldacre sums up the differences

8. Why do we need systematic reviews?
Huge increase in the publication of research articles and other sources of relevant information
Amount and complexity of available information has increased
People who use research information and people who make decisions about the delivery or organisation of services rely on systematic reviews
To help them organise and prioritise relevant research information
To keep up to date with advances in their field

9. Research question/policy concern
What evidence exists?
Results to inform policy
Results to inform future research
New primary research

10. Why should I engage with systematic reviews as a PhD student?
Let’s consider…
Why should I engage with systematic reviews as a PhD student?
Why would I do a systematic review as a PhD student?

12. The systematic review Aims to answer a specific research question
Valid research method
Informs primary research, research agendas
Transferable skills
No ethics 

13. Systematic review isn’t the easy choice though…

14. A scientific tool Reviewing research systematically involves
Identifying relevant research (finding)
Describing relevant research (describing)
Critically appraising research reports in a systematic manner (appraisal)
Bringing together the findings into a coherent statement (synthesis)
Reviewing research systematically involves
4 key processes:
Reiterate - systematic review is scientific tool, can be used for answering many questions and used in many ways

15. Example reviews

16. Conceptualisation of systematic reviews
‘Conventional’/ Aggregative
'What works?'
Diagnostic test
Prevalence
Interpretive/ Configurative
Meta-ethnography
Critical interpretative synthesis
Meta-narrative review
Systematic reviews can look at quantitative data, or qualitative data, or both
It can be useful to think of systematic reviewing methods as falling into two ‘camps’.
Conventional systematic review methods have developed as a specific methodology for searching for, appraising, and synthesising the findings of primary studies. They are mainly concerned with assembling and pooling data, and synthesising results from primary studies addressing a similar topic.
These methods have demonstrated considerable benefits in synthesising certain forms of evidence. Particularly those where the aim is to test theories, especially evidence about "what works". 
Interpretive - Qualitative synthesis methods serve as a means of interpreting and synthesising qualitative findings across individual studies
Purpose is to present new perspectives on topics through interpreting findings from different qualitative studies and to advance knowledge and theory

17. Types of review question
To present a concept
Descriptive
Explore preferences about what should happen
Normative
Investigate a relationship between two or more variables
Observational/relational
Investigate the effect of one or more independent variables on one or more outcome variables
Causal
Explore factors that cause a condition, event or process
Theoretical

18. Types of review question
What is the nature of the literature on the relationship between obesity and sedentary behaviour in young people aged 6-16?
Descriptive
What are the views and experiences of people who inject drugs in relation to hepatitis C testing and diagnosis?
Normative
Are adults with disabilities at an increased risk of violence compared to those without disabilities?
Observational/relational
What are the effects of fluoridation of drinking water supplies on the incidence of caries?
Causal
What are the links between crime, fear of crime, the environment and health and wellbeing?
Theoretical

19. Extra slide for explaining how the type of question/data found is reflected in analysis methods

20.  Option A better Option B better 
meta analysis can be used to take findings from several studies, aggregate them, and determine which option works best – we’re not relying on one set of findings

22. Mapping qualitative synthesis methods
Idealist
Realist
Meta-narrative
Critical interpretive synthesis
Meta-ethnography
Grounded theory
Thematic synthesis
JBI meta-aggregative approach
Framework synthesis
Ecological triangulation
Subjective idealism
Objective idealism
Critical realism
Scientific realism
There is no shared reality independent of multiple alternative human constructions
Qualitative synthesis methods serve as a means of interpreting and synthesising qualitative findings across individual studies.
Falling within the interpretive/configurative review camp, their purpose is to present new perspectives on topics through interpreting findings from different qualitative studies and to advance knowledge and theory.
The range of different methods for synthesising qualitative research has grown over recent years as shown in the Figure. And the figure also shows how these different approaches have arisen along the spectrum of how researchers’ view knowledge. Although presented as continuum, the methods above can be broadly grouped into idealist and realist categories.
There is a world of collectively shared understandings
Knowledge of reality is mediated by our perceptions and beliefs
It is possible for knowledge to approximate closely an external reality
Taken from a presentation by Karin Hannes. Based on Barnett-Page E, Thomas J. Methods for the synthesis of qualitative research: a critical review. BMC Medical Research Methodology 2009, 9: 59.

23. Essential steps of a systematic review
Review protocol
Comprehensive literature search
Study selection (2-stage process)
Quality assessment
Data extraction
Synthesis/analysis
Reiterate they’ve seen these steps before when we did the game earlier

25. Features in a systematic review
What makes an article a ‘systematic review’?
If we read an article reporting findings from a research study, we have certain expectations e.g. in an article based upon a survey:
Number of participants and their characteristics
How data was collected
Survey outcomes and statistical tests
The same principle applies to systematic review articles.

26. Common features in a review
Systematic reviews share many identifiable features and share common terminology
Just like with articles reporting other methodologies:
there is still great variation between articles (particularly between different types of SRs).
You will find the quality and reporting of published systematic review articles varies from very good to very poor.

27. How do we know this article is a systematic review?

28. A logical structure Articles are likely to have: Introduction Method
Results
Discussion
Conclusion
We can contrast that with literature reviews that do not follow the systematic methodology approach

29. Comprehensive methods section
Describes in detail how and where the authors have searched for evidence, and what they searched for

30. Comprehensive methods section
Describes what evidence was being searched for (inclusion criteria), and any evidence that was not searched for (exclusion criteria).

31. Comprehensive methods section
Explicitly demonstrates how they ended up with the articles that were included in the review (text and/ or diagram)

32. Comprehensive methods section
Describes what happened to articles that were included in the review, and how decisions were made.

33. Clear evidence of critical appraisal
Describes how study quality was assessed, and reports the findings of this process
Discusses the implications of this on the review findings

34. Detailed information on included studies
Describes key characteristics of articles included in the review in text and/or tables

35. Clear presentation of results
This will depend on the type of review, but will typically be a mixture of tables and text enabling clear comparison between studies on key outcomes.

36. Clear presentation of results

37. Clearly expressed limitations and conclusions
Recognises limitations through the methods applied (e.g. and/ or articles identified (e.g. small numbers of studies, low applicability, poor methodological quality of articles)
Highlights the implications of the review, and how it adds to the existing evidence
Identifies gaps in the evidence and recommendations for future research

38. Based upon a protocol
Systematic reviews should always be based upon a review protocol, developed before the review is started.
A good protocol should:
Describe the current evidence base in your topic area
Clearly identify the question that the review will address
Outline the methods that will be used to answer the question

39. Why do a protocol? Enables thorough planning of review methods
Helps to ensure the review question and approach are appropriate and realistic
Enables others to compare the protocol with the completed review – which reduces risk of selective reporting bias
Allows others to replicate the review

41. Are all systematic reviews useful?
Poor quality primary research = evidence that may not be reliable/ useful.
Poor quality systematic review = evidence that may not be reliable/ useful.
When we read a systematic review we need to critically evaluate the methods and reporting in the same way as we would do when we look at primary research.
Knowing what makes a ‘high quality’ review is also helpful when undertaking a review.

42. Tools to assess the quality of reviews
There are several tools designed to help you with this.
It is a similar process to the ‘quality assessment’ that takes place within a systematic review, but applied to a review article rather than a primary research article.
The tools ask questions about how the review was carried out, and what was reported.

43. AMSTAR tool

45. ≈ ≈

47. Using AMSTAR Have a go at using the AMSTAR tool.
Briefly look over the criteria and discuss in your groups
What do you think of the AMSTAR tool?
Is there anything that you don’t understand?
2. Apply the tool to the BMJ review article and discuss in your groups.
How do you find using the AMSTAR tool?
How well does the article perform? Do you agree with each other?

50. Resources Books: Doing a systematic review: a student's guide
Systematic reviews in the social sciences
An introduction to systematic reviews
Online:
Systematic reviews, CRD's guidance for undertaking reviews in healthcare
Learning from research: systematic reviews for informing policy decisions
We’ll send them slides – if they want further info look here

51. PROSPERO website: www.crd.york.ac.uk/prospero/#index.php
PRISMA (reporting systematic review guidelines):
PRISMA journal article:
PRISMA reporting guidelines for review protocols:
AMSTAR checklist:
AMSTAR journal article:
CASP checklist for appraising systematic reviews:

52. #&❤’d
This book is a good resource for PG students, but also supervisors and anyone new to reviewing. – I know it’s a plug for my colleagues