Dr. Maryam Tajvar Department of Health Management and Economics SYSTEMATIC REVIEW Dr. Maryam Tajvar Department of Health Management and Economics

دانش، مهارت و تجربیات شما؟

Pooled effect

Traditional vs. Systematic Reviews

Traditional vs. Systematic Reviews Open to biases Methods not published No protocol Sample of research Subjective review of study quality An unreliable basis for policy and practice SYSTEMATIC Bias is minimised Exhaustive efforts to identify ALL relevant research Criteria based review of quality Methods published Protocol based A firm basis for policy and practice

What is a systematic review? “Systematic reviews use objective and transparent methods to identify, evaluate and summarise ALL relevant research findings. To avoid bias, the approach and methods, are set out in advance. When carried out well, systematic reviews provide the most reliable evidence about the effects of tests, treatments and other interventions used in health and social care.” CRD (Centre for Reviews and Dissemination), 2017

Why do a systematic review? To carry on from where others have already reached, or positon your project relative to previous work To identify gaps in the literature To avoid reinventing the wheel To identify other people working in the same filed To increase power and precision To systematically examine primary study method quality for bias and heterogeneity To inform guidelines, policy & decisions To identify influential works in your area To establish consistency of findings (opposing vs. confirming views and aspects) To identify most productive methodology for your research To identify sources of information or data that might be useful to you

How do a systematic review?

A systematic review proposal should include: Background - Research question/hypotheses Methods: - Study inclusion/exclusion criteria - Search strategies - Methods of selection - Assessment of study quality and validity - Data extraction - Data synthesis and interpretation References

Example of a Systematic Review Proposal

B. Executing (Conducting a review) 4. Identification of literature 5. Selection of studies 6. Quality assessment 7. Data extraction 8. Data synthesis

4. Identification of literature (Searching) Search strategy : Sources: Searching electronic databases (e.g. Medline, Embase, ISI, Scopus, PubMed) Searching Reference lists Hand searching relevant specialized journals (e.g. ….) Grey literature, conference proceedings, documents and reports (e.g. ….) Contacting key informants in the field Browsing in libraries Search terms: mesh terms, keywords

Search Strategy: Getting the most extensive coverage of primary research possible This is not the same as finding lots of journal articles Start with individual broad concepts and combine them together gradually narrowing your search down as you go

5. Selection of studies Types of studies • Primary studies • previous reviews Exclusion and inclusion criteria(studies should meet ALL the inclusion criteria and NONE of the exclusion criteria) • Participants • Interventions • Outcomes • Design Language restriction • Translation facilities may limit papers Study selection process • By searching first titles, then abstracts, then full-texts Reviewers • More than one observer • Strategy for resolving disagreements

6. Quality assessment How do we know what is a good quality study? • By methodological hierarchy • By quality components within method • By study conduct • Using checklists and scales (for various types of studies) Remove or keep poor quality studies? (Based on proposal)

7. Data extraction Develop a data extraction form Create a database with the information Double entry the recorded information Seek clarifications from authors where possible (e.g. missing data) Data manipulations (if necessary)

8. Data Synthesis Data Synthesis is…. • Tabulation of study characteristics • Tabulation of study results • Statistical combining of results (maybe) • Identification of themes to help explain outcomes Data Synthesis is Narrative or mathematical [Descriptive or meta-analysis] Publication Bias (Next slides)

Publication Bias The accessibility of research results may be related to; Publications with statistically significant results Year of publication Type and language of publication Multiple publications Author identity Institution Country of origin Journal Coverage by databases and indexes

Selective reporting of studies Statistically significant results more likely to: be published (publication bias) be published rapidly (time lag bias) be published in English (language bias) be published again (multiple publication bias) be cited by others (citation bias)

Selective reporting of outcomes Hahn 2002: LREC-approved protocols compared with published reports 8 (53%) of 15 studies mentioned analysis plan 7 (88%) of 8 did not follow plan Chan 2004: 102 trials, 122 publications, 3736 outcomes Comparing publications to protocols, 62% had at least one primary outcome that was changed, introduced, or omitted. von Elm 2006: 234 trials, 376 publications 32% of outcomes in publication not in protocol 41% of outcomes in protocol not in publication reporting associated with P<0.05; OR: 4.1 (CI 1.8 to 9.7)

C. Communicating (Reporting and dissemination) 9. Writing up the review 10. Getting the evidence into practice

9. Writing up the review Background (aim, objective, questions, rational, …) Methods Results Discussion - Interpret Results: • provide an estimate of the overall effectiveness (effect sixe) of an intervention • review effectiveness in different studies, populations and settings • investigate differences - Consider: • Limitations, including publication and other biases • Strength of evidence • Applicability • Economic implications • Implications for future research

10. Getting the evidence into practice Disseminate to Ministry of Health Disseminate to public Disseminate in academic conferences Publish it

Guidelines on how to write reviews & meta-analyses: QUORUM statement: For meta-analysis of RCTs MOOSE guidelines: For meta-analysis of observational studies CONSORT Statement–Transparent reporting of Trials ASSERT-A Standard for the Scientific and Ethical Review of Trials EQUATOR-Enhancing the Quality and Transparency Of health Research

Sources and references of systematic reviews Cochrane Library (via www.nelh.nhs.uk) Cochrane Database of Systematic Reviews (Cochrane Reviews) Database of Abstracts of Reviews of Effects (DARE) Cochrane Central Register of Controlled Trials (CENTRAL) Cochrane Database of Methodology Reviews Health Technology Assessment Database NHS Economic Evaluation Database (NHS EED) Cochrane Methodology Register NCCHTA Website (www.ncchta.org) Centre for Reviews and Dissemination website (http://www.york.ac.uk/inst/crd/reviews.htm) Search Strategy from McMaster Hedges Team (Montori et al, BMJ 2005) Database of Reviews of Effectiveness (http://www.york.ac.uk/inst/crd/darehp.htm) Cochrane Collaboration (www.cochrane.org/reviews/index.htm) Joanna Briggs Institute and collaboration Review Manager (http://www.cc-ims.net/RevMan) The Cochrane Reviewers’ Handbook (http://www.cochrane.dk/cochrane/handbook/hbook.htm) CRD’s guidance for Undertaking Systematic Reviews of Research on Effectiveness (http://www.york.ac.uk/inst/crd/report4.htm) Randomised Controlled Trials (http://www.bmjpg.com/rct/contents.html) School of Health and Related Research, Sheffield -http://www.shef.ac.uk/scharr/sections/ir/links Especially for non-RCTs –Campbell Collaboration http://www.campbellcollaboration.org/ Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from http://handbook.cochrane.org.