The Bahrain Branch of the UK Cochrane Centre In Collaboration with Reyada Training & Management Consultancy, Dubai-UAE Cochrane Collaboration and Systematic Review Workshop, February 2007, Dubai - UAE Dr. Zbys Fedorowicz, Dr. Dunia Al Hashimi, Dr. Ahmed Al Asfoor W10

Systematic Evidence Searching

Content What does it mean to search systematically? Foundations of good searching Search mechanics Sources of evidence Beyond databases Etc. …

Objectives Comprehensive review of relevant sources Comprehensive overview of search strategy (syntax and mechanics of vocabularies) To specifically be applied to the research topic, but also transferable to your future evidence retrieval needs

Growth of Medical Information million articles were being published each year in 20,000 journals a pile of paper 500 metres high if researchers tried to stay current by reading two articles per day, in one year they would fall 55 centuries behind! or, if you try to read everything of possible biomedical relevance, you would have to read 5,500 articles per day.

Systematic Searching

Thorough searching much research is never published not all research is published in journals not all research published in journals is indexed on major databases not all research indexed on databases can be easily retrieved

‘Bias’ publication bias –Positive results language bias –Positive results more likely to be published in English language publications geographical coverage bias of journals and databases

Why do we search widely? Savoie et al estimated that 29.2% of items in their review were uncovered by: –searching the web –handsearching –scanning reference lists –personal communication –searching specialised databases and web sites.

Publication bias Song et al. Publication and related biases. Methods to dealing with publication and related biases in systematic reviews: –literature searching –locating unpublished studies –assessment of the risk of publication and related biases –detecting publication bias

Why search widely…. Allen & Hanburys found that: –only 51% of the clinical trials relating to their respiratory products were published in journals indexed by MEDLINE, EMBASE or CINAHL (46% are in grey lit) Wallace et al. –11 of 65 trials in end stage renal disease reviews were found by searching beyond major databases.

Why search widely.... Long lead times before publication: –publication gaps after conference presentation –indexing publication lag before recorded in databases Cheng et al, 1998 –8.1% of a set of conference papers achieved publication within 12 months –29% within 2 years and 40% within 5 years

Delay and non publication Non-publication Cheng et al, 1998 –only 32% of abstracts presented at chronic fatigue conferences were subsequently published in full Petticrew et al, 1999 –50.6 % of oral paper presented at the Society for Social Medicine 1996 achieved publication

Key factors thorough searches searching beyond published articles citation searching Internet searching hand searching searching for ongoing research

How to identify sources of information ask your librarian, check shelves/catalogue directories of databases search published information guides directories of special collections

Publication types Journal articles - MEDLINE, EMBASE, Cancerlit, AMED etc Conference papers - Index to Conference Proceedings, Inside Conferences Dissertations - Dissertation Abstracts Grey literature (reports, working papers, government documents) - SIGLE, Health Management Information Consortium

Importance of the Search The quality of the systematic review is directly dependant upon the quality of the search. A systematic review requires a comprehensive search for all relevant published and unpublished evidence to consider for inclusion Must be transparent and reproducible Bias and Completeness

Searching for the Qualitative More challenging than quantitative (where validated filters abound) Qualitative methodology terms abound, and do so inconsistently Techniques do exist to identify qualitative studies Do you need to do so?

Searching systematically Aim to include all comparable data Methods and inclusion criteria should be explicit Aim to avoid or eliminate any bias Results/ conclusions should be reproducible (like an experiment)

5 steps to searching 1.Formulate a research question 2.Brainstorm for synonyms/related terms 3.Define the scope of the research question 4.Identify appropriate types of data and sources for that data 5.Search, evaluate results, revise from step 1 if necessary

Methods - databases selected Medline Embase ERIC  most relevant to medical education

Methods - strategies Three levels of strategy: 1.standard (most users; limited search syntax) 2.enhanced (some use of search syntax) 3.expert (full use of search syntax) syntax includes: free-text, controlled vocabulary, term explosions, phrase lists, subheadings, sub strings, filters, proximity operators, etc...

Methods - handsearching “...refers to the planned searching of a journal page by page (i.e. by hand), including editorials, letters, etc., to identify all relevant items.” time consuming and meticulous produces the “gold standard” by which search efficiency can be measured

Reasons for shockingly poor performance I.Incomplete coverage of journals II.No properly indexed database for the studies III.Existing controlled vocabularies are inadequate.

Defining & Refining the Question

Defining & Refining the Query  Key, as everything follows from it questions must be precisely defined –list inclusion and exclusion criteria most queries have a single question –composite or multiple questions are possible...

Defining and Refining the Query define and write down the issue down in the form of a question or hypothesis identify and expand the essential concepts set out the scope of the search query (e.g. inclusion and exclusion criteria).

Converting a question into a search - Create list of terms Gather terms which describe subject of interest Identify synonyms Identify abbreviations Identify related terms Identify any Transatlantic differences - e.g. pressure sores/decubitus ulcers Identify combination groups

Database Mechanics Controlled vocabularies Free text Search commands

Controlled Vocabulary or Free Text Controlled Vocabulary –(nearly always) produces higher quality results – the Ovid default “ Map Term to Subject Heading” –process by which concepts of a document are identified and represented by the thesaurus Free Text –easiest, but lower quality –sometimes necessary for specific pieces of information

Controlled Vocabularies/Indexing Standardised hierarchical lists of indexed terms forming a subject tree process by which concepts of a document are identified and represented by the thesaurus makes possible to extract all bibliographic and contextual information to answer queries concepts based on selectivity (concepts of interest to users) and exhaustivity (all useful concepts, whether explicit or implicit)

Focus or Explode? Focus: brings up only records of articles in which the Subject heading is a primary topic. This reduces the number of articles retrieved. Explode: to search for a Subject heading and include all the narrower Subject headings which branch down from it, select the Explode option - recommended.

Free Text Searching Can search specific fields or all fields must search all possible variants and synonyms to be comprehensive can find things that the indexing missed can be very useful in combination with Controlled Vocabulary searching

Retrieving the evidence

Databases: Medical

Medline World’s largest biomedical database produced in Bethesda, MD by the National Library of Medicine Index Medicus, Index of Dental Literature, International Nursing coverage in the areas of allied health, biological and physical sciences, humanities and information science as they relate to medicine and health care, communication disorders, population biology, and reproductive biology

Medline ˜ 11,000,000 records in 3,900+ journals the single most important database for most research in medicine training database for today

‘Other’ Medlines Time segments, look for: OvidMEDLINE(R) 1966 to September Week In-process OldMedline ( ) Medline Daily Update option

Embase Medline’s rival, Excerpta Medica by Elsevier Science biomedicine and pharmaceuticals, as well as nursing, dentistry, veterinary medicine, psychology, and alternative medicine 9,000,000+ records in 3,500 journals

Embase ~ % overlap with Medline much better pharmaceutical information better European coverage and (arguably) better international coverage

PsychINFO/ClinPSYC Global coverage over 1,400 journals are indexed, as well as books and grey literature PsychINFO also indexes dissertations ClinPSYC indexes clinical psychology citations, does not include grey lit, and updates only quarterly

CINAHL Cumulative Index to Nursing & Allied Health largest source for nursing and PAMs 70% overlap with MeSH, Medline’s controlled vocabulary but also the British Nursing Index...

Grey Literature & more!

Grey Lit Defined “That which is produced on all levels of government, academics, business and industry in print and electronic formats, but which is not controlled by commercial publishers.” (Fourth International Conference on Grey Literature, 1997)

Academic papers Census data Committee reports Conference papers Corporate documents Discussion papers Dissertations Government reports House journals Market surveys Newsletters Ongoing research Preprints Proceedings Research reports Standards Technical reports Theses Trade literature Translations Working papers

Grey Literature Not a comment on quality notoriously difficult to locate, retrieve and manage (diverse formats in a fluid environment) often for a limited or selected audience evaluation process decisive

Advantages Can be created comparatively quickly (process less structured and formal) and can reach level of practice much sooner can provide context for traditionally lit, insight from environments events occur in key for intra-professional communication (frequently in-process, policy oriented, informal and/or discipline specific) often very succinct and focused

Advantages Growth in volume: –British Library (1992) had 3,000,000 items –British Library (2000) had 17,000,000 items growing faster than conventional literature

Grey Searches Measured Grey Searching reveals items not found be other methods one comparison of grey & conventional searches across various medical topics found % of evidence was grey another found one third of RCTs found only by grey methods yet another revealed commercial literature has a 15% larger estimation of effect

Why search widely.... Long lead times before publication: –publication gaps after conference presentation –indexing publication lag before recorded in databases Cheng et al, 1998 –8.1% of a set of conference papers achieved publication within 12 months –29% within 2 years and 40% within 5 years

Delay and non publication Non-publication Cheng et al, 1998 –only 32% of abstracts presented at chronic fatigue conferences were subsequently published in full Petticrew et al, 1999 –50.6 % of oral paper presented at the Society for Social Medicine 1996 achieved publication

Internet Enormous effect on growth and diversification of grey literature web pages (personal and organisational), listservs, usenet, blogs, digital repositories, various other electronic files much quicker than paper

Print Problematic for collecting, indexing and storing (both for individuals and organisations) costs time involved

Grey Lit Anarchy No centralised authority to store or distribute (nor is one likely) the web has greatly improved the situation some efforts to standardise and centralise, but largely American or British

Search Engines

Consider: + Ease of use Increasing access to quality evidence Time involved - Quality of information needs to be evaluated Time involved

Google Tips Automatic AND Most important terms first “Phrase searching” +-~ Now supports truncation (?)

Other Methods & Sources

Hand Searching Not all journals are indexed by databases not all citations are indexed sufficiently to be retrieved indexing can be selective; conferences and supplements are frequently missing keys are: (a) identifying resources (b) identifying journals

National Library Catalogues British Library COPAC Library of Congress; National Library of Medicine National Library of Canada National Library of Australia National Library of the Netherlands (all online, free and in English)

Theses & Dissertations Dissertations Abstracts: the last two years of dissertations and previews; full database and text require a subscription Index to Theses: complete list of theses and abstracts accepted by British and Irish universities