1. linda.snell@mcgill.ca “I sometimes get an article to review that is outside my area of expertise” “Why was I asked to review this paper when it is clearly not appropriate for this journal?” “I recently received a nasty review: what do editors do with the highly ‘negative’ review?” “I am likely to do further reviews for a journal when I hear about the fate of the paper – what the other reviewers thought and whether it was accepted.”

2. Getting the most out of your reviewers Linda Snell MD MHPE FRCPC FACP Centre for Medical Education & Department of Medicine, McGill University, Montreal, Canada Veterinary Journal Editor’s Meeting Washington DC July 2007

3. linda.snell@mcgill.ca Characteristics of journal reviewers & reviews? A great reviewer / reviews …  timely  Constructive  Specific  Concise  Thorough  Doesn’t edit  Objective  Non nihilist A terrible reviewer / reviews …  yes >> no  Dismissive  Nasty  Doesn’t edit  Nit picker  Edits  Uncommunicative  No explanations  Delay then reject

4. linda.snell@mcgill.ca Learning objectives By the end of this presentation you will be able to:  Discuss the importance of reviewers in the peer review  List methods of reviewer recruitment & selection  Overview strategies that can assist in producing an excellent review  Outline how to recognize and reward reviewers …  And will have shared ideas for best practices

5. linda.snell@mcgill.ca Why peer review? AuthorReviewer Journal Advise editor Balanced Ethics Opportunity to learn Professional responsibility Improve quality of paper

6. linda.snell@mcgill.ca Why peer review? AuthorReviewer Journal QUALITY ‘Peer review is at the core of science and academic life.’ [ Bordage ]

7. linda.snell@mcgill.ca Reviewer recruitment & selection  Match reviewer / expertise with topic or article type  Characteristics of reviewer producing ‘good’ review [JAMA]  Younger  Stats / epi training  Expertise …  Complemented by judgment  Experienced  Interested  Motivated  Time to do it

8. linda.snell@mcgill.ca Do you have enough reviewers for your journal? Do you have enough high quality reviewers for your journal?

9. linda.snell@mcgill.ca Reviewer recruitment & selection

10. linda.snell@mcgill.ca Reviewer recruitment & selection  Journal  Manuscript citations / bibliographies  Literature search  Recently published authors  Role of editorial board / sub-editors  Network / personal contacts  Scientific organizations / professional meetings  Other …  Authors’ suggestions  Shared assignment with juniors  ‘Ads’ in journal..

11. linda.snell@mcgill.ca ‘The reviewer serves as advisor to the editor, peer assessor to the experienced researcher, and teacher to the less experienced author.’ [Caelleigh]

12. linda.snell@mcgill.ca Reviewers’ perspectives of process  Time spent in review 2-3 h  Process of review  Helpful to ask a colleague: stats, expertise, literature, validate  Problems with reviews  Reviewer: expertise, experience, expectations, being balanced  Manuscript: flawed, poorly written  Process: inaccessible references, time  Facilitators to the review process  Covering letter with goals; instructions; good review form; reviewer training; feedback on reviews; access to literature; sample copy of journal

13. linda.snell@mcgill.ca Guidance for reviewers  Covering letter with the manuscript  Statement of journal’s purpose  Role of the reviewer  Instructions for the review and forms  Deadlines  Reviewer etiquette  Format  Structured forms: provide a framework  Narrative: provides valuable feedback  Separate the review & recommendation re publication

14. linda.snell@mcgill.ca Reviewer training & mentoring  15-20% have it; 75-90% want it  Part of graduate education  Reading  Consulting editors  Consulting colleagues  Workshops or seminars at scientific meetings

15. linda.snell@mcgill.ca After the review  Inform reviewer of fate of manuscript  Share comments of other reviewers  Feedback to reviewer  Other perspectives  Benchmarking  Annual report to reviewer  Number and quality of reviews [Ann Int Med]

16. linda.snell@mcgill.ca Recognition and reward  Thank-you's:  In journal  Letter  Recognition of quality  ‘Distinguished reviewers’  Use of associated organizations – ‘merit points’  Academic advancement  …

17. linda.snell@mcgill.ca ‘The seasoned author anticipates peer review without particular relish but expects constructive criticism… Approached thoroughly, peer review represents collegial mentoring and contributes to the integrity of the scientific endeavor.’ [Pascoe, Vet.Surg. 2006]

18. linda.snell@mcgill.ca In summary …  Pick the right reviewers  Give them the right guidance  Recognize their contributions

19. linda.snell@mcgill.ca References Bordage G, Caelleigh A (eds). Review criteria for research manuscripts. Acad Medicine. 76(9):904-75, Sept 2001. Black N et al. What makes a good reviewer and a good review for a general medical journal? JAMA. 280(3): 231-4, July 15 1998. Snell L, Spencer J. Reviewer’s perceptions of the peer review process. Med Education. 39:90-97, 2005 Van Rooyen S et al. Effects of open peer review on quality of reviews. BMJ 318:23-7, 2 Jan 1999.