Meet the PRESS, and other secrets of systematic searchers CADTH SYMPOSIUM BREAKFAST SESSION TUESDAY, APRIL 12, 2016

Meet the PRESS, and other secrets of systematic searchers: an update on literature search essentials for finding the evidence Amanda Hodgson, CADTH Dr. Jessie McGowan, University of Ottawa Melissa Severn, CADTH Danielle Rabb, CADTH 1

Disclosure Funded by federal, provincial, and territorial ministries of health. Application fees for three programs: CADTH Common Drug Review (CDR) CADTH pan-Canadian Oncology Drug Review (pCODR) CADTH Scientific Advice The presenters declare that there are no conflicts of interest. 2

Agenda Introduction (Amanda Hodgson) PRESS Update (Dr. McGowan) Filters Update (Melissa Severn) Grey Literature Update (Danielle Rabb) 3

Literature search standards - resources 4

Literature search standards - resources Cochrane Handbook – Chapter 6: searching for studies (2011) Institute of Medicine. Finding what works for healthcare: standards for systematic reviews. Section 3. (2011) What-Works-in-Health-Care-Standards-for-Systematic- Reviews.aspx 5

Literature search standards - resources Summarized Research in Information Retrieval for HTA SuRE Info 6

Literature search standards - resources Listservs of interest: Canmedlib (CHLA) Expertsearch (MLA) Cochrane (Information Retrieval Methods group) HTAi (Information Resources Group) 7

Literature search standards - Groups of Interest Health Technology Assessment International Information Resources Group (HTAi - IRG) Cochrane Information Retrieval Methods Group (IRMG) Canadian Health Libraries Association (CHLA) 8

What is PRESS? Peer Review of Electronic Search Strategies The PRESS 2015 Guideline is an includes two documents: PRESS Guideline Explanation & Elaboration (E&E) PRESS Guideline Statement

Report is found at: ate_Report_2016.pdf Report is found at: ate_Report_2016.pdf

Why do we need PRESS? In preparing systematic review (SR) and health technology assessment (HTA) reports, researchers aim to provide comprehensive, unbiased, and high-quality distillations of relevant evidence. One step in assuring the integrity of the searching process is the peer review of the work of the librarian or information specialist who is seeking the evidence for the researchers.

Searching and PRESS PRESS focuses on the quality of the Boolean search and is only one aspect of a comprehensive search for SRs and HTAs. Other important aspects include the search plan, search validation, and search reporting.

Searching and PRESS The Boolean search is typically the core element in the search plan, identifies the majority of relevant evidence and is the most productive method of identifying relevant studies. Peer review of the search strategy is one quality assurance step and provides a subjective validation. Other validation approaches include reaching data saturation, testing the search of the main database against eligible studies and testing using a related article search and simple search protocol.

Research components Systematic Review The SR search focused on research and reports on electronic search strategies within a health science context. 6,608 records were identified; 39 Survey: a total of 174 surveys (117 were valid and 108 were complete). Consensus Forum The objective was to create recommendations related to the existing PRESS checklist items based on updated SR evidence and survey findings. The outcome was that, of the seven existing PRESS elements, six were retained and one (“Search strategy adaptations”) will no longer be a PRESS element.

PRESS 2015 Implementation Four components: 1.PRESS 2015 evidence-based checklist  2.PRESS 2015 recommendations for librarian practice 3.PRESS 2015 assessment form 4.PRESS 2015 implementation strategies

PRESS 2015 evidence-based checklist

Search Filters

What is a Search Filter? Pre-defined Pre-tested Pre-saved Also referred to as hedges, clinical queries, optimal search strategies or methodological search strategies

Why Use Search Filters? Reduce number of results Extract higher level evidence Extract a specific concept Remove off-topic results Save time and money

Typical Search Strategy INDICATION TERMS INTERVENTION TERMS SEARCH FILTER(S)

Types of Search Filters – Specific methodology RCTs, systematic reviews – Clinical queries diagnosis, prognosis, adverse events – Population specific humans, pediatrics InterTASC Search Filter Resource CADTH Filters: Strings Attached cadths-database-search-filters cadths-database-search-filters

Methodology: RCTs 1. Randomized Controlled Trial.pt. 2. Randomized Controlled Trials as Topic/ 3. Randomized Controlled Trial/ 4. Randomization/ 5. Double-Blind Method/ 6. Single-Blind Method/ 7. Placebos/ 8. (random* or sham or placebo*).ti,ab 9. ((singl* or doubl*) adj (blind* or dumm* or mask*)).ti,ab 10. ((tripl* or trebl*) adj (blind* or dumm* or mask*)).ti,ab 11. or/1-11

Clinical Query: Prognosis 1.Incidence/ 2.Mortality/ 3.Follow up studies/ 4.Prognos*.ti,ab 5.Predict*.ti,ab 6.Course*.ti,ab 7.or/1-6 25

Population Specific: Human Filter ((Animals[MESH] OR Animal Experimentation[MESH] OR "Models, Animal"[ MESH] OR Vertebrates[MESH]) NOT (Humans[MESH] OR Human experimentation[MESH])) 26

Challenges to Filter Development Author related Titles Abstracts Indexer related Subject headings Publication type tags Database related Translation Testing 27

Gold Standard Testing A gold standard is a set of relevant records against which a filter’s performance can be assessed Developed through a range of techniques Gold standard sets can range in size 28 Harbour J, et al. Reporting methodological search filter performance comparisons: a literature review. Health Info Libr J. 2014;31(3):

Sensitivity/Recall & Precision Sensitivity (or Recall): proportion of relevant records retrieved by the filter Precision: proportion of relevant records in the total retrieved records 29

Performance of Search Filter in Ovid MEDLINE - Example Development and Testing of Search Filters to Identify Economic Evaluations in MEDLINE and EMBASE, Oct 2009, pg. 15

31 NEW!

Diving into the Grey: Searching for grey literature 32

What is grey literature? Grey Literature is a field in library and Information science that deals with the production, distribution, and access to multiple document types produced on all levels of government, academics, business, and organization in electronic and print formats not controlled by commercial publishing i.e. where publishing is not the primary activity of the producing body. Source: GreyNet.org. The Grey Literature Network Service

Trends in grey literature PISA Declaration on Policy Development for Grey Literature Resources html html Teaching grey literature Publishing of Big Data Metadata standards Access points and access repositories (pan-Canadian HTA database) 34

Grey Literature - Groups of Interest 35

What is grey literature? 36

Clinical Practice Guidelines 37

Clinical Practice Guidelines 38

Clinical Practice Guidelines 39

Regulatory approvals 40

Regulatory approvals 41

Clinical Trials 42

Clinical Trials 43

Advisories and Warnings 44

Advisories and Warnings 45

Advisories and Warnings 46

What is grey literature? Clinical Practice Guidelines Position Statements Regulatory approvals Safety information & adverse event reports Recalls & alerts Clinical trial data (CSRs, abstracts, posters) Supplemental information to published literature 47

Canadian Producers 48

International Producers 49

Benefits Value of grey literature for systematic reviews Unpublished trials Conference abstracts Guidelines Can reduce publication bias Current Usually free Unique content Information on non-mainstream topics 50

Challenges Large volume of material Inconsistent access points Time-consuming to find Not always available online Quality of evidence varies Bibliographic information may be missing Inconsistent language, no subject indexing 51

Before you begin… How much time do you have? What type of information is required? How are you incorporating your grey lit search? How well do you know the sites you are searching? Do you have a well defined PICO? 52

The CADTH Experience Reasons for searching grey literature: To see other HTAs To find the best evidence available Adhere to standards for systematic reviews – the exhaustive search for trials Help our researchers understand the topic area 53

The CADTH Experience The Practical Stuff Grey literature searching is built into overall search time Customized checklists (Grey Matters) Results are sent separately from database results (word document) We do not differentiate between commercial and non- commercial sources for our clients 54

Grey Matters 55 Grey Matters checklistGrey Matters checklist: CADTH tool for finding grey literature information

Grey Matters Categories: Databases National & international HTA organizations Clinical practice guidelines Drug class reviews Health economics Safety/advisories Clinical trial listing Regulatory approvals Drug formularies Statistics/prevalence Other 56

Grey Matters Light checklistchecklist: Top 14 Resources for Evidenced Based Medicine Searching Categories: HTA organizations Safety/advisory resources Clinical practice guidelines 57

Pan-Canadian HTA database 58

Pan-Canadian HTA database The HTA Database Canadian search interface lets you: Automatically limit search results to HTAs produced in Canada or internationally Search in English or French Limit your search by province Easily organize your search by: Topic (using medical subject headings) Title Author Agency (CADTH, NICE, INESSS, etc.)

Pan-Canadian HTA database The pan-Canadian Health Technology Collaborative invites other HTA producers in Canada to contribute to the database. For information about how to get involved, please contact

linkedin.com/company/cadth slideshare.net/CADTH-ACMTS youtube.com/CADTHACMTS cadth.ca/photoblog Connect With Us

62 Subscribe Sign up at to get updates sent directly to your inbox. CADTH E-Alerts Calls for stakeholder feedback and patient group input, plus other time-sensitive announcements. New at CADTH Monthly newsletter including a summary of new reports plus corporate news, announcements of upcoming events, and more. CADTH Symposium and Events Updates about our flagship annual Symposium, workshops, webinars and other events.

63