1. Compliance of Veterinary Medicine Systematic Reviews with Literature Search Reporting Standards MLA 2013 One Health - Boston Lorraine Toews MLIS Health Sciences Library, University of Calgary Alberta, Canada 1

2. Introduction “Poorly reported studies distort the available pool of research evidence and compromise its reliability.” Simera I, Kirtley S, Altman DG. Maturitas 2012 May;72(1):84-7. Reporting “deficiencies make it difficult … to evaluate the reliability of the presented findings” Simera I, Moher D, Hirst A, Hoey J, Schulz KF, Altman DG. BMC Med. 2010 Apr 26;8:24 2

3. Introduction “Searching is critical to conducting systematic reviews. Searches for systematic reviews need to be constructed to maximize recall and deal with … potentially biasing factors. Errors made in the search process may results in a biased or otherwise incomplete evidence base for the review.” Sampson M, McGowan J. Errors in search strategies were identified by type and frequency. J Clin Epidemiol. 2006 Oct;59(10):1057-63 3

4. Purpose Purpose of my study … to examine ▪ Reporting of literature searches in systematic reviews in veterinary journals ▪ Compliance with PRISMA reporting standard ▪ Assess conduct of review 4

5. Methods ▪ MEDLINE journal title search ▪ 14 high impact veterinary journal titles ▪ Limited to … ▪ Titles “meta-analysis”, “systematic review” ▪ Publication type “meta-analysis” ▪ Published mid-2009 to December 2012 5

6. Methods: Journals Searched American Journal of Veterinary Research Journal of Veterinary Pharmacology & Therapeutics Australian Veterinary JournalNew Zealand Veterinary Journal Canadian Journal of Veterinary Research Preventive Veterinary Medicine Canadian Veterinary JournalVeterinary Journal JAVMAVeterinary Record Journal of Veterinary Diagnostic Investigation Veterinary Research Journal of Veterinary Internal Medicine Veterinary Surgery 6

7. Methods: Systematic Reviews 2009-2012 JournalsNumber of Systematic Reviews JAVMA4 Journal of Veterinary Diagnostic Investigation 1 Journal of Veterinary Internal Medicine3 Preventive Veterinary Medicine9 Veterinary Journal7 Veterinary Research1 TOTAL25 7

8. Methods: PRISMA Elements PRISMA reporting items examined … ▪ Information sources searched ▪ Databases ▪ Grey literature ▪ Full electronic search strategy ▪ Limits ▪ Publication date ▪ Language ▪ Article, appendix, supplementary data reviewed for PRISMA reporting elements 8

9. PRISMA: Information Sources Searched “ Describe all information sources … databases with dates of coverage … in the search …” 9

10. Databases Searched Why is this important? Systematic review literature searches - comprehensive Incomplete or biased search can result in … ▪ Biased evidence base ▪ Inaccurate/over estimate of intervention effect 10

11. Results: Databases Searched Database%of Reviews Searched CAB Abstracts 64% PubMed 56% Agricola 32% MEDLINE 32% Web of Science 20% BIOSIS 12% Google Scholar 12% IVIS |Scopus | AGRIS | Current Contents |Global Health| Global Health | Google | Library catalogues |LILACS | PubMedCentral 8% 11

12. Results: Database Coverage Databases Searched in ReviewsDatabase Coverage of Veterinary Journals * CAB Abstracts PubMedScopus AgricolaScience Citation Index Expanded MEDLINEWeb of Science BIOSIS EMBASE Google ScholarMEDLINE IVIS |Scopus | AGRIS | Current Contents |Global Health| Global Health | Google | Library catalogues |LILACS | PubMedCentral * Grindlay, Brennan, Dean 2012 JVME 39(4): 408 12

13. Results: Databases Searched Number of Databases Searched % of Reviews that Searched Databases 1 database 8% 2 databases 20% 3 databases 24% 4 databases 12% 5 databases 16% 6 databases 4% 10 databases 4% 13

14. Results: Grey Literature Searched Only 32% of reviews searched for grey literature Only 24% reported searching for conference proceedings 14

15. Grey Literature Searched Why is this important? Counteract publication bias ▪ Over half of trials in conference proceedings never published (Scherer et al. Cochrane Database Syst Rev 2007 2) ▪ Published trials showed greater treatment effect than unpublished trials (Hopewell et al. Cochrane Database Syst Rev 2007 2) ▪ Reviews based only on published literature risk exaggerating intervention effectiveness (Relevo et al. J Clin Epidemiol. 2011 Nov;64(11):1169) 15

16. PRISMA Search Strategy “Present full electronic search strategy for at least one database … … including any limits used … … such that it could be repeated” 16

17. Search Strategy Why is this important? Systematic review literature search aims to … ▪ Be comprehensive ▪ Minimize bias ▪ Be reproducible 17

18. Search Strategy Elements High sensitivity strategies ▪ Free-text & subject headings used ▪ Subject headings exploded ▪ Variant spelling, truncation Strategies customized for each database ▪ Subject headings vary between databases ▪ Indexing varies between databases Cochrane Handbook Pt 2 Ch 6 18

19. Results: Search Strategy 0% Line-by-line full search strategy 12% Used subject headings 0% Reported exploding subject headings 48%Limited by publication date 20%Limited by language 19

20. Conclusions ▪ Reporting often incomplete ▪ Not comply fully with PRISMA 12% did not report databases searched 64% did not report if grey literature searched 0% reported full electronic search strategy 20

21. Conclusions Gaps in reporting … unable to evaluate ▪ Completeness & bias of search ▪ Replicate search 21

22. Conclusions Reporting gaps suggest … ▪ Conduct of searches fell short of best practices ▪ Validity of reviews questionable … based on incomplete, biased search 22

23. Thank you for listening … Questions? 23