1. Meet the PRESS, and other secrets of systematic searchers CADTH SYMPOSIUM BREAKFAST SESSION TUESDAY, APRIL 12, 2016

2. Meet the PRESS, and other secrets of systematic searchers: an update on literature search essentials for finding the evidence Amanda Hodgson, CADTH Dr. Jessie McGowan, University of Ottawa Melissa Severn, CADTH Danielle Rabb, CADTH 1

3. Disclosure Funded by federal, provincial, and territorial ministries of health. Application fees for three programs: CADTH Common Drug Review (CDR) CADTH pan-Canadian Oncology Drug Review (pCODR) CADTH Scientific Advice The presenters declare that there are no conflicts of interest. 2

4. Agenda Introduction (Amanda Hodgson) PRESS Update (Dr. McGowan) Filters Update (Melissa Severn) Grey Literature Update (Danielle Rabb) 3

5. Literature search standards - resources https://www.cadth.ca/resources/finding-evidence 4

6. Literature search standards - resources Cochrane Handbook – Chapter 6: searching for studies (2011) http://handbook.cochrane.org/ Institute of Medicine. Finding what works for healthcare: standards for systematic reviews. Section 3. (2011) http://www.nationalacademies.org/hmd/Reports/2011/Finding- What-Works-in-Health-Care-Standards-for-Systematic- Reviews.aspx 5

7. Literature search standards - resources Summarized Research in Information Retrieval for HTA SuRE Info http://www.htai.org/vortal/?q=sure-info 6

8. Literature search standards - resources Listservs of interest: Canmedlib (CHLA) Expertsearch (MLA) Cochrane (Information Retrieval Methods group) HTAi (Information Resources Group) 7

9. Literature search standards - Groups of Interest Health Technology Assessment International Information Resources Group (HTAi - IRG) http://www.htai.org/index.php?id=555 Cochrane Information Retrieval Methods Group (IRMG) http://methods.cochrane.org/irmg/ Canadian Health Libraries Association (CHLA) http://www.chla-absc.ca/ 8

10. What is PRESS? Peer Review of Electronic Search Strategies The PRESS 2015 Guideline is an includes two documents: PRESS Guideline Explanation & Elaboration (E&E) PRESS Guideline Statement

11. Report is found at: https://www.cadth.ca/sites/default/files/pdfCP0015_PRESS_Upd ate_Report_2016.pdf Report is found at: https://www.cadth.ca/sites/default/files/pdfCP0015_PRESS_Upd ate_Report_2016.pdf

13. Why do we need PRESS? In preparing systematic review (SR) and health technology assessment (HTA) reports, researchers aim to provide comprehensive, unbiased, and high-quality distillations of relevant evidence. One step in assuring the integrity of the searching process is the peer review of the work of the librarian or information specialist who is seeking the evidence for the researchers.

14. Searching and PRESS PRESS focuses on the quality of the Boolean search and is only one aspect of a comprehensive search for SRs and HTAs. Other important aspects include the search plan, search validation, and search reporting.

15. Searching and PRESS The Boolean search is typically the core element in the search plan, identifies the majority of relevant evidence and is the most productive method of identifying relevant studies. Peer review of the search strategy is one quality assurance step and provides a subjective validation. Other validation approaches include reaching data saturation, testing the search of the main database against eligible studies and testing using a related article search and simple search protocol.

16. Research components Systematic Review The SR search focused on research and reports on electronic search strategies within a health science context. 6,608 records were identified; 39 Survey: a total of 174 surveys (117 were valid and 108 were complete). Consensus Forum The objective was to create recommendations related to the existing PRESS checklist items based on updated SR evidence and survey findings. The outcome was that, of the seven existing PRESS elements, six were retained and one (“Search strategy adaptations”) will no longer be a PRESS element.

17. PRESS 2015 Implementation Four components: 1.PRESS 2015 evidence-based checklist  2.PRESS 2015 recommendations for librarian practice 3.PRESS 2015 assessment form 4.PRESS 2015 implementation strategies

18. PRESS 2015 evidence-based checklist

20. Search Filters

21. What is a Search Filter? Pre-defined Pre-tested Pre-saved Also referred to as hedges, clinical queries, optimal search strategies or methodological search strategies

22. Why Use Search Filters? Reduce number of results Extract higher level evidence Extract a specific concept Remove off-topic results Save time and money

23. Typical Search Strategy INDICATION TERMS INTERVENTION TERMS SEARCH FILTER(S)

24. Types of Search Filters – Specific methodology RCTs, systematic reviews – Clinical queries diagnosis, prognosis, adverse events – Population specific humans, pediatrics InterTASC Search Filter Resource http://www.york.ac.uk/inst/crd/intertasc/ http://www.york.ac.uk/inst/crd/intertasc/ CADTH Filters: Strings Attached https://www.cadth.ca/resources/finding-evidence/strings-attached- cadths-database-search-filters https://www.cadth.ca/resources/finding-evidence/strings-attached- cadths-database-search-filters

25. Methodology: RCTs 1. Randomized Controlled Trial.pt. 2. Randomized Controlled Trials as Topic/ 3. Randomized Controlled Trial/ 4. Randomization/ 5. Double-Blind Method/ 6. Single-Blind Method/ 7. Placebos/ 8. (random* or sham or placebo*).ti,ab 9. ((singl* or doubl*) adj (blind* or dumm* or mask*)).ti,ab 10. ((tripl* or trebl*) adj (blind* or dumm* or mask*)).ti,ab 11. or/1-11

26. Clinical Query: Prognosis 1.Incidence/ 2.Mortality/ 3.Follow up studies/ 4.Prognos*.ti,ab 5.Predict*.ti,ab 6.Course*.ti,ab 7.or/1-6 25

27. Population Specific: Human Filter ((Animals[MESH] OR Animal Experimentation[MESH] OR "Models, Animal"[ MESH] OR Vertebrates[MESH]) NOT (Humans[MESH] OR Human experimentation[MESH])) 26

28. Challenges to Filter Development Author related Titles Abstracts Indexer related Subject headings Publication type tags Database related Translation Testing 27

29. Gold Standard Testing A gold standard is a set of relevant records against which a filter’s performance can be assessed Developed through a range of techniques Gold standard sets can range in size 28 Harbour J, et al. Reporting methodological search filter performance comparisons: a literature review. Health Info Libr J. 2014;31(3):176-194

30. Sensitivity/Recall & Precision Sensitivity (or Recall): proportion of relevant records retrieved by the filter Precision: proportion of relevant records in the total retrieved records 29 http://www.creighton.edu/fileadmin/user/HSL/docs/ref/Searching_-_Recall_Precision.pdf

31. Performance of Search Filter in Ovid MEDLINE - Example https://www.cadth.ca/sites/default/files/pdf/H0490_Search_Filters_for_Economic_Evaluations_mg_e.pdf Development and Testing of Search Filters to Identify Economic Evaluations in MEDLINE and EMBASE, Oct 2009, pg. 15

32. 31 NEW! https://www.cadth.ca/resources/finding-evidence/strings-attached-cadths-database-search-filters

33. Diving into the Grey: Searching for grey literature 32

34. What is grey literature? Grey Literature is a field in library and Information science that deals with the production, distribution, and access to multiple document types produced on all levels of government, academics, business, and organization in electronic and print formats not controlled by commercial publishing i.e. where publishing is not the primary activity of the producing body. Source: GreyNet.org. The Grey Literature Network Service http://www.greynet.org/ http://www.greynet.org/ 33

35. Trends in grey literature PISA Declaration on Policy Development for Grey Literature Resources http://www.greynet.org/greyforumseries/policydevelopment. html http://www.greynet.org/greyforumseries/policydevelopment. html Teaching grey literature Publishing of Big Data Metadata standards Access points and access repositories (pan-Canadian HTA database) 34

36. Grey Literature - Groups of Interest 35

37. What is grey literature? 36

38. Clinical Practice Guidelines 37

39. Clinical Practice Guidelines 38

40. Clinical Practice Guidelines 39

41. Regulatory approvals 40

42. Regulatory approvals 41

43. Clinical Trials 42

44. Clinical Trials 43

45. Advisories and Warnings 44

46. Advisories and Warnings 45

47. Advisories and Warnings 46

48. What is grey literature? Clinical Practice Guidelines Position Statements Regulatory approvals Safety information & adverse event reports Recalls & alerts Clinical trial data (CSRs, abstracts, posters) Supplemental information to published literature 47

49. Canadian Producers 48

50. International Producers 49

51. Benefits Value of grey literature for systematic reviews Unpublished trials Conference abstracts Guidelines Can reduce publication bias Current Usually free Unique content Information on non-mainstream topics 50

52. Challenges Large volume of material Inconsistent access points Time-consuming to find Not always available online Quality of evidence varies Bibliographic information may be missing Inconsistent language, no subject indexing 51

53. Before you begin… How much time do you have? What type of information is required? How are you incorporating your grey lit search? How well do you know the sites you are searching? Do you have a well defined PICO? 52

54. The CADTH Experience Reasons for searching grey literature: To see other HTAs To find the best evidence available Adhere to standards for systematic reviews – the exhaustive search for trials Help our researchers understand the topic area 53

55. The CADTH Experience The Practical Stuff Grey literature searching is built into overall search time Customized checklists (Grey Matters) Results are sent separately from database results (word document) We do not differentiate between commercial and non- commercial sources for our clients 54

56. Grey Matters 55 Grey Matters checklistGrey Matters checklist: CADTH tool for finding grey literature information

57. Grey Matters Categories: Databases National & international HTA organizations Clinical practice guidelines Drug class reviews Health economics Safety/advisories Clinical trial listing Regulatory approvals Drug formularies Statistics/prevalence Other 56

58. Grey Matters Light checklistchecklist: Top 14 Resources for Evidenced Based Medicine Searching Categories: HTA organizations Safety/advisory resources Clinical practice guidelines 57

59. Pan-Canadian HTA database 58

60. Pan-Canadian HTA database The HTA Database Canadian search interface lets you: Automatically limit search results to HTAs produced in Canada or internationally Search in English or French Limit your search by province Easily organize your search by: Topic (using medical subject headings) Title Author Agency (CADTH, NICE, INESSS, etc.)

61. Pan-Canadian HTA database The pan-Canadian Health Technology Collaborative invites other HTA producers in Canada to contribute to the database. For information about how to get involved, please contact CdnHTA@cadth.caCdnHTA@cadth.ca

62. 61 @CADTH_ACMTS linkedin.com/company/cadth slideshare.net/CADTH-ACMTS youtube.com/CADTHACMTS cadth.ca/photoblog requests@cadth.ca Connect With Us

63. 62 Subscribe Sign up at www.cadth.ca/subscribe to get updates sent directly to your inbox. CADTH E-Alerts Calls for stakeholder feedback and patient group input, plus other time-sensitive announcements. New at CADTH Monthly newsletter including a summary of new reports plus corporate news, announcements of upcoming events, and more. CADTH Symposium and Events Updates about our flagship annual Symposium, workshops, webinars and other events.

64. 63