Quality and reporting of literature search strategies in systematic reviews published by London School of Hygiene & Tropical Medicine affiliated authors: an assessment using PRISMA, AMSTAR and PRESS criteria Jane Falconer User Support Services Librarian

Question Yes No N/A Electronic search strategy is described for at least one database 59.8% 40.2% 0% Synonyms are used? 47.5% 48.4% 4.1% A controlled vocabulary is used? 30.3% 66.4% 3.3% Terms are truncated? 10.7% 84.4% 4.9% The search syntax is adapted to each database? 13.1% 68.9% 18.0%

Methodology Random Sample Analysis Literature Search Web of Science Core Content Organisation = LSHTM Title contains “systematic review” Year = 2015 or 2016 n = 44 58 item data extraction form with criteria from PRISMA AMSTAR PRESS

PRISMA reporting characteristics Names of databases unambiguously stated Supplier of databases unambiguously stated Start/end dates clearly stated to at least mm/yy accuracy Fully repeatable search for one database Fully repeatable search for all databases Language not limited, or rationale for limit provided 50%

AMSTAR appraisal criteria Were both keywords and thesaurus terms provided?

PRESS quality criteria – basic skills

PRESS quality criteria – subject headings Are the subject headings relevant? Are all relevant subject headings included, including previous terms? Are subject headings chosen at correct level? Not too broad/too narrow? Are subject headings exploded where necessary? Are both subject headings and terms in free text used for each concept? n = 33

45% 25% 7% Overall quality score No search strategy was published Showed serious flaws No search strategy was published Major problems leading to lack of confidence in study validity 25% Could not be assessed due to lack of information Minor problems unlikely to impact study validity 7% Good quality search, clearly reported No problems with reporting or search quality

Image in the public domain, via Wikimedia Commons

References Gómez-Sánchez, A. F., et al. (2016). Evaluating the information retrieval quality and methodological accuracy of systematic reviews and meta-analysis on congenital malformations (2004-2014). Paper presented at the 15th EAHIL Conference, Seville, Spain. http://www.bvsspa.es/eahil2016/wp-content/uploads/2016/05/G4.pdf CWTS (Centre for Science and Technology Studies) (2017) CWTS Leiden Ranking 2017. [website] . Leiden University, Netherlands. http://www.leidenranking.com/ranking/2017/list THE (2014) REF 2014 results: table of excellence. [website] https://www.timeshighereducation.com/news/ref-2014-results-table-of-excellence/2017590.article Page, M. J., et al. (2016). Epidemiology and Reporting Characteristics of Systematic Reviews of Biomedical Research: A Cross-Sectional Study. PLoS Med, 13(5), e1002028. doi:10.1371/journal.pmed.1002028 Sampson, M., & McGowan, J. (2006). Errors in search strategies were identified by type and frequency. J Clin Epidemiol, 59(10), 1057-1063. doi:10.1016/j.jclinepi.2006.01.007 Bullers, K., et al. (2018). It takes longer than you think: librarian time spent on systematic review tasks. J Med Libr Assoc, 106(2), 198-207. Page, M. J., & Moher, D. (2017). Evaluations of the uptake and impact of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement and extensions: a scoping review. Systematic Reviews, 6(1), 263. doi:10.1186/s13643-017-0663-8 Mead, T. L., & Richards, D. T. (1995). Librarian participation in meta-analysis projects. Bull Med Libr Assoc, 83(4), 461-464. Rethlefsen, M. L., et al. (2015). Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. J Clin Epidemiol, 68(6), 617-626. All photographs from Pexels (https://www.pexels.com) unless otherwise noted and distributed under a CC0 license.

Jane Falconer jane.falconer@lshtm.ac.uk @falkie71 ORCID: 0000-0002-7329-0577