Literature searching peer review in practice: enhancing the skills of searchers Rachel Playforth & Igor Brbre Brighton and Sussex NHS Library & Knowledge Service HLG Conference 15 June 2018

Peer review (noun) evaluation of scientific, academic, or professional work by others working in the same field

Background 3 libraries located at 3 sites Brighton and Sussex NHS Library and Knowledge Service 3 libraries located at 3 sites 29 members of staff, 15 searchers Regular peer review of recently completed searches This project Demonstrate how searching peer review works in practice Assess the impact Areas for improvement

Literature review Pragmatic literature review April 2018. 16 papers selected. Results in three main categories: about the PRESS instrument reports on usage of search peer review checklists guidelines encouraging peer review of searching No reports of formal searching peer review practice in libraries were found.

Why we do it Continuous service improvement Professional development Knowledge management and exchange ‘to detect errors, to improve quality, and to reduce not only the risk of missing relevant studies but also the risk of identifying unnecessarily large numbers of irrelevant records’ - HTAi Vortal, 2017

PRESS Peer Review of Electronic Search Strategies 2006 - study reveals errors in systematic review search strategies 2007 - CADTH funded study to assess the elements important for peer review of search strategies 2008 - the first report including PRESS checklist published 2012, 2013 - research shows searching peer review improves quality 2014 - CADTH funds an update of the 2007 study 2016 - revised guideline statement and checklist published

PRESS and our adaptation Our form pre-2015 Our current form PRESS 2015 Search question Translation into concepts (PICO, followed up request?) [searcher supplied question and PICO] Translation Sources chosen Sources (strategy adapted?) [searcher supplied; single database only] Search strategy Search operators Boolean and proximity operators Subject headings Natural language Text word searching Spelling & syntax Spelling, syntax and line numbers Search filters Limits and filters Limits Search fields utilized [in Text word searching] Attachments and cover sheet Presentation (search notes and summary)   Time spent CPD needs identified General comments Overall evaluation

How we do it Managed through Searching and Teaching Peer Group Random pairings, every 4 months Review meeting in pairs Structured discussion using PRESS-based checklist Completed forms anonymised and summarised by chair of S&T Group Summary shared and discussed in the S&T group meeting Issues and CPD requirements taken forward and actioned

What we think of it Survey sent to 15 current and 7 former searchers Response rate n=11 (50%) Anonymous Asked about aspects of searching and opinions on peer review process All but 1 thought peer review had made them a better searcher

What we think of it Areas people found most difficult The peer review meeting itself The summaries discussed in our group meetings Suggestions Do it less frequently Review (some) searches before (and after) completion Adapt the form for non-database searches Make feedback more actionable, develop training

What we will do next Revise our peer review form Discuss peer reviewing (some) searches before completion

Collaboration options PRESS Forum http://pressforum.pbworks.com KnowledgeShare

What else do we do Since 2017 - peer review of teaching Journal club

Selected bibliography Sampson M, McGowan J. Errors in search strategies were identified by type and frequency. J Clin Epidemiol 2006;59(10):1057-63. [accessed 25 April 2018]. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0895-4356(06)00043-6 Sampson M, McGowan J, Cogo E, Grimshaw J, Moher D, Lefebvre C. An evidence-based practice guideline for the peer review of electronic search strategies. J Clin Epidemiol 2009 Sep;62(9):944-52 [accessed 25 April 2018]. Available from: https://www.jclinepi.com/article/S0895-4356(08)00320- X/fulltext McGowan, J., Sampson, M., & Lefebvre, C. (2010). An Evidence Based Checklist for the Peer Review of Electronic Search Strategies (PRESS EBC). Evidence Based Library and Information Practice, 5(1), 149-154 [accessed 25 Apr 2018]. Available from: https://journals.library.ualberta.ca/eblip/index.php/EBLIP/article/view/7402 Spry C, Mierzwinski-Urban M, Rabb D. Peer review of literature search strategies: does it make a difference? Presented at Canadian Health Libraries Association (CHLA) Conference; 22-25 May 2013; Saskatoon, Saskatchewan: Canada [accessed 25 April 2018]. Available from: https://www.cadth.ca/media/is/CSpry_Poster.pdf McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol. 2016 Jul;75:40-6 [accessed 25 April 2018]. Available from: https://www.jclinepi.com/article/S0895-4356(16)00058-5/fulltext

Contacts Rachel Playforth Rachel.playforth@bsuh.nhs.uk @archelina Igor Brbre Igor.brbre@bsuh.nhs.uk @iroberger @BrightonSx_LKS