1. Literature Searching & Information Retrieval for Systematic Reviews
Anne Donnelly
Academic Support Librarian
27th November 2017
Hello! I’m Anne Donnelly, the Academic Support Librarian for the School of Philosophy, Psychology & Language Sciences.
This is the first of my two contributions to today’s programme.
This first session will take the shape of a general introduction to literature searching for systematic reviews.

2. Session content
Getting started on your literature search – things to think about.
Strategies for finding the literature.
Searching the literature – key approaches.
Searching the literature – some tools & tips.
Additional resources & information.
While I have had no active involvement in undertaking a systematic review, I have over the years done much to support colleagues who have sought to ensure that they are approaching the literature searching aspect of their research in a systematic and methodical way.
In preparation for this session, I have also consulted a few ‘oracles’, including the Centre for Reviews and Dissemination at the University of York. This session will present a distillation of the best approaches to a systematic literature review and I will follow this up in the afternoon with some demonstrations of these approaches in action.
Thus, in this shorter session, I’ll briefly cover the principles, key characteristics and overall purpose of a systematic review, before moving on to the literature searching that is one of the key activities that are involved in undertaking one.

3. Finding the research literature: bear in mind
Not all research is reported or published in journals.
Not all research published in journals is indexed in major databases.
Research is not always described in the same way by authors or indexed consistently.
Many reviews do not appear in major databases and must be retrieved by e.g. searching the web, hand searching or scanning reference lists.

4. Credit: http://hlwiki.slais.ubc.ca/index.php/Scoping_reviews
First things first
Consider your research question.
You may wish to undertake a scoping review, i.e. a type of literature review that aims rapidly to ‘map’ the relevant research in a given field of interest, thereby seeing just what is out there and who else may have carried out similar research.
Scoping review
Systematic review
Study question
Often broad
Focused
Search strategy
Systematic & transparent
In/exclusion criteria
Flexible
Defined a priori
Study appraisal
Non/minor
Yes
Synthesis
Typically qualitative
Often quantitative
This may sound somewhat obvious but it really is essential.
A scoping review may help you to do just that. It will enable you to check out any recent studies in your research area. It should also help you to identify the terminology that describes your topic, thereby helping you establish appropriate search terms.
Credit:

5. Searching the literature: where?
Bibliographic databases, e.g. Medline, Web of science, PsycINFO, other specialist
Each has its strengths, with no single source indexing everything in the field covered.
Don’t worry about any overlap – that’s where your reference management system comes in (more of which later!)
Trials registers.
Grey literature - produced outwith traditional commercial or academic publishing and distribution channels.
Internet - you’ll need to be focused!
Hand-searching journals.
Reference lists.
Citation searching.
Depending on the nature of your research, and how all inclusive it needs to be, not all of the above may apply of course.

6. Searching the literature: points to ponder
Savoie et al (2000)
Estimated that 29.2% of items in their review were uncovered by:
searching the web
hand searching
scanning reference lists
personal communication
searching specialised databases and web sites
Cheng et al (1998)
Only 8.1% of a set of conference papers achieved publication within 12 months, 40% within 5 years.
Hopewell et al (2007)
Cochrane review trials with positive results are published sooner than other trials.
Indexing lag between publication and recording in databases.
So, although the principal bibliographic databases are the best sources for finding the information you seek, it’s good to remember that they do not index everything.

7. Getting started (1)
Frame your research question, e.g. What is the effectiveness of cognitive behaviour therapy in chronic fatigue syndrome?
This will help you clarify what you want to find out and help you achieve focus.
Identify the principal concepts
These will give you the keywords for your search.
Think about
Synonyms, related terms, homonyms, antonyms.
Current and historic variations in terminology & spelling.
Abbreviations & variations in word endings.
As already mentioned!
Think about how your key terms can be expanded to extend the reach of your search, as the same concepts may be described differently across the literature. Mind maps can be helpful here, but just use whatever works best for you!
3. Also, depending on how far back your literature search is going, it’s worth remembering that terminology changes over time and is not always applied consistently, either by authors in describing their work or by databases in indexing it. I’ll be going into this in a bit more detail this afternoon.

8. Getting started (2) 4. Decide on your inclusion & exclusion criteria
These will establish the boundaries of your search, which may relate to specific publication years, geography, age/sex of study subjects etc.
5. Plan the literature search
Identify the resources that will help you locate appropriate studies.
Decide on your search strategy, which may require adaptation for the difference resources used.
Decide how you will document the process & manage your results
This really needs to be done from Day One!
Just a few limits suggested here; you may think of others. Do be careful, however, not to overdo it and exclude more than you mean to.
More on resources to follow, but getting that search strategy right at the outset will pay dividends. Having said that, some adaptation will be required from one resource to another.
Cannot be over-emphasised!

9. Searching the literature: with PICO
P Population, Patient, Problem I Intervention (or exposure) C Comparison O Outcome e.g. What is the effectiveness of cognitive behaviour therapy in chronic fatigue syndrome? P Patients with chronic fatigue syndrome I Cognitive behavioural therapy (CBT) C Other therapies, e.g. nutritional supplements or placebo O e.g. Improved symptoms
PICO offers a useful model when framing a healthcare-related question and may or may not be appropriate for your particular research question.

10. Population Outcome Intervention Chronic fatigue syndrome
Improvement in symptoms
Cognitive behaviour therapy
Outcome
Your search will aim to retrieve the narrow ‘cross-over’ point of the various PICO elements of your research question.
Intervention

11. Boolean searching (AND, OR & NOT)
Results retrieved from your search …
Boolean operator
… must include all your search terms
Insert AND between your keywords to narrow your search.
… must include at least one of your search terms
Insert OR between your keywords to broaden your search.
… must not include a specified term
Insert NOT before the word that represents what you wish to exclude, to eliminate certain results, thereby narrowing your search.
The Boolean operators lie at the heart of literature searching.

12. Applying Boolean operators (1)
Examples
Retrieves
AND
children AND literacy
diet AND obesity
poverty AND health OR
1960s OR sixties OR ‘60s
paediatric OR pediatric
language OR dialect
NOT
pregnancy NOT childbirth
media NOT internet
insects NOT wasps
These multi-disciplinary Venn diagrams that I prepared earlier explain best of all how Boolean searching works!
You are likely to use AND and OR most of all.
NOT can be useful, but should be used with caution; you may exclude more than you wish to.

13. Applying Boolean operators (2)
What is the effectiveness of cognitive behaviour therapy in chronic fatigue syndrome??
Boolean operator
Examples
Retrieves
AND
chronic fatigue syndrome AND cognitive behaviour therapy OR
cognitive behaviour therapy OR cognitive behavior therapy OR CBT
NOT
cognitive behaviour therapy NOT group
The same approach, but applied to our example research question.

14. chronic fatigue syndrome cognitive behaviour therapy (UK)
Expanding the search
Keyword
Alternative
chronic fatigue syndrome
CFS ME
post viral fatigue syndrome
fibromyalgia
cognitive behaviour therapy (UK)
cognitive behavior therapy (US)
CBT
placebo
nutritional
supplements
effectiveness
treatment outcome
observational study
case control study
trial
Not everything is described the same way so, in addition to your own chosen terms, it’s useful to think of other commonly-used descriptors for the same thing.
Rather than use this is as a search term, though you can also do that, you could opt to identify the kinds of literature that contain evidence of something, e.g. an observational study.
Effectiveness: Some concepts can be difficult to capture adequately in a search. I found the Medline indexing term – treatment outcome – by trial & error! Rather than use this is as a search term, though you can also do that, you could opt to identify the kinds of literature that contain evidence of the effectiveness of something, e.g. an observational study.
Cognitive behaviour therapy: Some databases and search tools ‘translate’ UK’ to US spelling and vice versa but it’s worth remembering that some do not. (I didn’t and then wondered why I couldn’t find a particular paper!)
Chronic fatigue syndrome: Some of the suggestions here are by no means synonymous but, being related, could still yield useful papers.

15. Undertaking your search
Note
Trial & error may be required!
Some ‘translation’ may be needed when switching from one database to another.
Important to explore all possible keywords to address lack of consistency in the assignment of indexing terms.
Combining your terms with Boolean operators
e.g. (chronic fatigue syndrome OR CFS) AND (cognitive behaviour therapy OR CBT)
e.g. (CFS OR post viral fatigue syndrome) AND (placebo or supplement) AND randomized controlled trial
While there is a big overlap in indexing terms between one database and another, sometimes switching from one resource to another using the same search terms can, surprisingly, retrieve zero results!
Just a couple of ways of combining your terms. OR widens the scope of an individual search line, while AND brings focus to the overall search result.
I’ve got several more top tips that I’ll go over this afternoon during our hands on session in the Greenfield.

16. Widening your search: cited reference search
Why?
You have identified a good paper … Prins JB et al, Cognitive behaviour therapy for chronic fatigue syndrome: a multi centre randomised controlled trial. Lancet 2001, 357,
However, it’s a bit dated, and you want to find more recent research on the same topic.
How?
Ovid databases (PsychINFO, Medline, Embase): ‘Find citing articles’.
Web of Science: ‘Cited reference search’.
The Cited Reference Search works particularly well if you are struggling to find recent papers in a topic, but have at least one good one that is a bit dated.
We’ll do a search for more up-to-date papers this afternoon.

17. Using search filters What are they? Using search filters
Tried and tested strategies, often taking some time to design, that are intended for repeated used.
Ideal for systematic reviews where a high level of sensitivity is required.
Intended for repeated use, combined with specific search terms.
Using search filters
Not all are effective!
Check the scope and aims of a filter to assess its relevance and suitability for your research strategy.
Understand how they work, i.e. high sensitivity with high precision.
Use with caution and an awareness of their possible impact.
Necessary to ‘translate’ search terms for individual databases.
You will already have used filters, e.g. by date range or subject age group.
These are just more complex versions that aim to meet specific search criteria.

18. Example search filter
#1 randomized controlled trial [pt] #2 controlled clinical trial [pt] #3 randomized [tiab] #4 placebo [tiab] #5 drug therapy [sh] #6 randomly [tiab] #7 trial [tiab] #8 groups [tiab] #9 #1 OR #2 OR #3 OR #4 OR #5 OR #6 OR #7 OR #8 #10 animals [mh] NOT humans [mh] #11 #9 NOT #10 Glanville JM, Lefebvre C, Miles JN, Camosso-Stefinovic J. How to identify randomized controlled trials in MEDLINE: ten years on. J Med Libr Assoc Apr, 94(2):

19. ISSG Search Filters Resource
More on search filters
ISSG Search Filters Resource
Web directory of evaluated methodological search filters.
Bibliographic references to published filters.
Information about unpublished filters/filters in progress.
Independent, structured, critical appraisal for many of the methodological search filters.
Comparative data from independent testing of search filters.
Filters for systematic reviews and other studies.

20. Create a test set to test your strategy
What is a test set?
A set of bibliographic references that you would expect your search strategy to retrieve.
Why create a test set?
It will help you to identify a search strategy that will retrieve the results you want.
How to create a test set
Identify some key studies, either ones you are aware of or perhaps by checking studies that have been included in an earlier review.
Check that each is available in the database you are using and record the accession numbers, thereby creating your test set.
Testing your test set
Run your search strategy and check the results against your test set.
If not all records in your test set are retrieved, you will need to revised your search strategy!
Essentially a test set will ensure your search strategy is on course and you are using the most appropriate search terms.

21. Documenting the process – the theory
Why?
Transparency and reproducibility.
It’s good academic practice.
For the sake of your sanity – seriously!
What?
List databases and all other sources.
When?
As it happens, from Day One!

22. Documenting the process – in action!
Do it as it happens!
List the databases searched.
Note the dates of the last search for each database and the years of publication that were searched.
Document your search strategy.
Keep notes of your search results.
Set up alerts on the search tools to keep track of new research and articles published in your research area.
List grey literature sources.
List any journals and conference proceedings that were hand-searched.
List individuals and organisations contacted.
Make very good use of referencing software.
STARLITE [Sampling strategy, type of study, approaches, range of years, limits, inclusions and exclusions, terms used, electronic sources used.]
Not … “in a minute”, “later when I have a moment” ….
Keeps you on track and helpful later when you are writing things up.
Very important!
You may have to adapt from database to database and this is something someone reviewing your methodology will want to know.
Reference management software will help here.
An excellent function – available in most major databases – that will re-run your search as new records are added, and mail them to you.
These can be a useful source
Ditto.
Absolutely!

23. Bibliographic management tools
Take the misery out of reference management!
By helping you to …
Collect, organise, annotate & describe your references
Cite them in your work
Style your references for your dissertation, PhD thesis, journal paper or book chapter.
Some bibliographic management tools
EndNote X7 – provided on all UoE PCs and fully supported with training materials.
EndNote Web – free on the Web and fully supported with the training materials.
Some other free web-based systems: Mendeley, Zotero.
If you have not already started using one of these, it’s time to start thinking about it!
Lots of systems available, with in-house support available for many of them.
Web link at top of the slide to helpful tabulation of some of the available management software.
Image: flickr.com

24. Additional resources
Systematic Reviews: CRD’s guidance for undertaking reviews in health care
Gough, Oliver & Thomas. An introduction to systematic reviews. London: SAGE, 2017.
Q180.A1 Int 2017 – multiple locations

25. Further reading
Booth A. ‘Unpacking your literature search toolbox: on search styles and tactics’. Health Information and Libraries Journal 2008; 25(4):
Cheng, K., C. Preston, et al. (1998). "Time to publication as full reports of abstracts of randomized controlled trials in cystic fibrosis." Pediatric Pulmonology 26(2):
Cochrane Collaboration. Cochrane handbook for systematic reviews of interventions.
Centre for Reviews and Dissemination. Undertaking systematic reviews of research on effectiveness: CRD's guidance for those carrying out or commissioning reviews.
Egger, M., P. Juni, et al. (2003). "How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study." Health Technology Assessment 7(1): 1-76.
Hopewell S, McDonald S, Clarke MJ, Egger M. Grey literature in meta-analyses of randomized trials of health care interventions. Cochrane Database of Systematic Reviews 2007, Issue 2. Art. No.: MR DOI: / MR pub3.
Khan K S, Kunz R, Kleijnen J, Antes G Systematic Reviews to support Evidence Based Medicine London: Hodder Arnold 2011
Savoie, I., D. Helmer, et al. (2003). Beyond Medline: reducing bias through extended systematic review search. International Journal of Technology Assessment in Health Care 19(1):

26. Help Do please just get in touch for further information & guidance!
…. and don’t panic!