1. February 2016 Peer review: A global view A white paper from Taylor & Francis Research 2 Reader Conference Will Frass Senior Research Executive authorservices.taylorandfrancis.com/peer-review-in-2015

3. Assessment of the make-up of House of Lords: Peer Review

4. February 2016 Peer review: A global view A white paper from Taylor & Francis Research 2 Reader Conference authorservices.taylorandfrancis.com/peer-review-in-2015 Will Frass Senior Research Executive 1 Methodology 2 Ideal world & real world objectives 5 Different models of peer review 3 Ethical concerns 4 Timing discrepancies

5. 43,000 Science Technology Medicine 43,000 Humanities & Social Sciences 1 Methodology 2 Ideal world & real world objectives Peer review in 2015 | A global view A white paper from Taylor & Francis 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review

6. 43,000 Science Technology Medicine 43,000 Humanities & Social Sciences 1 Methodology 2 Ideal world & real world objectives Peer review in 2015 | A global view A white paper from Taylor & Francis 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review

7. 2,398 STM responses 4,750 HSS responses 1 Methodology 2 Ideal world & real world objectives Peer review in 2015 | A global view A white paper from Taylor & Francis 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review

8. Confidence Interval: 1.95% @ Confidence Level: 95% Confidence Interval: 1.34% @ Confidence Level: 95% 1 Methodology 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review 2,398 STM responses 4,750 HSS responses Peer review in 2015 | A global view A white paper from Taylor & Francis

9. 2% confidence interval at 95% confidence level for the population of all 2013 published authors 95% Result -2%+2% 1 Methodology 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review Peer review in 2015 | A global view A white paper from Taylor & Francis

10. 2,398 STM responses 4,750 HSS responses All survey respondents: STM Authors Reviewers Editors HSS Authors Reviewers Editors 1 Methodology 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review Peer review in 2015 | A global view A white paper from Taylor & Francis

11. 1 Methodology 2 Ideal world & real world objectives Qualitative research 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review Peer review in 2015 | A global view A white paper from Taylor & Francis Numbers:6 focus groups 46 participants Locations:United Kingdom South Africa China Participants:Editors, authors and reviewers Minimum of 2 articles peer reviewed (with any other publisher) Disciplines:Science, Technology & Medicine Social Sciences & Humanities

12. In an ideal world… …to what extent do you agree or disagree the following objectives should be the purpose of peer review? 1 Methodology 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review Peer review in 2015 | A global view A white paper from Taylor & Francis

13. In an ideal world… …to what extent do you agree or disagree the following objectives should be the purpose of peer review? Ideal World – rating out of 10 HSS 1 Methodology 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review

14. In an ideal world… …to what extent do you agree or disagree the following objectives should be the purpose of peer review? Ideal World – rating out of 10 HSS 1 Methodology 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review

15. In the real world… …to what extent do you agree or disagree that peer review is currently achieving the following objectives effectively? Ideal World – rating out of 10 5 6 7 8 9 10 Real World – rating out of 10 HSS 1 Methodology 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review

16. In the real world… …to what extent do you agree or disagree that peer review is currently achieving the following objectives effectively? Ideal World – rating out of 10 5 6 7 8 9 10 Real World – rating out of 10 HSS 1 Methodology 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review

17. In the real world… …to what extent do you agree or disagree that peer review is currently achieving the following objectives effectively? Ideal World – rating out of 10 5 6 7 8 9 10 Real World – rating out of 10 HSS Relevant to Aims & Scope 1 Methodology 2 Ideal world & real world objectives Ethical concerns 34 Timing discrepancies 5 Different models of peer review

18. HSS STM 2 Ideal world & real world objectives 3 Ethical concerns Timing discrepancies 45 Different models of peer review 1 Methodology

19. HSS STM 1 2 38 5 4 9 7 10 6 Detect plagiarism 1 2 38 5 4 9 7 10 6 Ideal world mean scoreReal world mean score 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review 1 Methodology

20. HSS STM 1 2 38 5 4 9 7 10 6 Improve quality of published article 1 2 38 5 4 9 7 10 6 Ideal world mean scoreReal world mean score 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review 1 Methodology

21. HSS STM 1 2 38 5 4 9 7 10 6 Provide polite feedback 1 2 38 5 4 9 7 10 6 Ideal world mean scoreReal world mean score 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review 1 Methodology

22. HSS STM 1 2 38 5 4 9 7 10 6 Correct spelling, punctuation & grammar 1 2 38 5 4 9 7 10 6 Ideal world mean scoreReal world mean score 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review 1 Methodology

23. HSS Most important objective… Expectation exceeds reality… Expectation matches reality… Reality exceeds expectation… PolitenessDetect Fraud Correcting spelling, punctuation & grammar Relevant to the Aims & Scope Improve quality of published paper STM 1 Methodology 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review

24. HSS Most important objective… Expectation exceeds reality… Expectation matches reality… Reality exceeds expectation… PolitenessDetect Fraud Correcting spelling, punctuation & grammar Relevant to the Aims & Scope Improve quality of published paper STM 1 Methodology 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review “The worst reviews are short, snitty, patronising and not remotely useful. The best are critically engaged, add something and improve the quality.” Editor, Linguistics, UK “The worst reviews are short, snitty, patronising and not remotely useful. The best are critically engaged, add something and improve the quality.” Editor, Linguistics, UK “Editors should be more pre-emptive in detecting plagiarism & other types of fraud.” Researcher, Medical Research, UK “Editors should be more pre-emptive in detecting plagiarism & other types of fraud.” Researcher, Medical Research, UK

25. How common are the following situations in peer review? Regional bias Seniority bias Gender bias 1 Methodology 2 Ideal world & real world objectives Peer review in 2015 | A global view A white paper from Taylor & Francis 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review

26. How common are the following situations in peer review? HSS STM 12345678910 123456789 Gender bias Regional bias Seniority bias 1 Methodology 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review

27. How common are the following situations in peer review? HSS STM 12345678910 123456789 Gender bias Regional bias Seniority bias 1 Methodology 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review

28. How common are the following situations in peer review? HSS STM 12345678910 123456789 Reviewers delay assessment Reviewers take ideas Reviewers use false identities 1 Methodology 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies Different models of peer review 5

29. 12345678910 Higher frequency of occurrences reported 1 Methodology 2 Ideal world & real world objectives Peer review in 2015 | A global view A white paper from Taylor & Francis Lower frequency of occurrences reported 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review

30. As an editor, what do you consider a realistic amount of time to expect reviewers to deliver their initial report? HSS 1 Methodology 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review

31. As an editor, what do you consider a realistic amount of time to expect reviewers to deliver their initial report? 6% HSS 54%96% 1 Methodology 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review

32. HSS As a reviewer, how long was the duration between your acceptance to review and the delivery of your initial report? 96% 1 Methodology 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review

33. HSS As a reviewer, how long was the duration between your acceptance to review and the delivery of your initial report? 96% 7%23%63%93% 1 Methodology 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review

34. HSS As an author, how long did you wait after submission before you received the peer reviewer’s initial comments? 1 Methodology 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review

35. HSS As an author, how long did you wait after submission before you received the peer reviewer’s initial comments? 96% 93% 1%5%14%44% 1 Methodology 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review

36. HSS 96% 93% Editors 2 months is realistic Reviewers 2 months to deliver report Authors 2 months to receive report 1 Methodology 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review

37. HSS 96% 93% 44% Editors 2 months is realistic Reviewers 2 months to deliver report Authors 2 months to receive report 1 Methodology 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review

38. STM HSS 94% 95% 60% Editors 2 months is realistic Reviewers 2 months to deliver report Authors 2 months to receive report 96% 93% 44% Editors 2 months is realistic Reviewers 2 months to deliver report Authors 2 months to receive report 1 Methodology 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review

39. 1 Methodology 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review As an author, I am usually kept well informed about the progress of my article through the peer review process STM HSS 10 – strongly agree1 – strongly disagree

40. 1 Methodology 2 Ideal world & real world objectives 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review As an author, I would like to see average peer review times, from submission to decision to publish, displayed on a journal's website STM HSS 10 – strongly agree1 – strongly disagree

41. 1 Methodology 2 Ideal world & real world objectives STM HSS AuthorsReviewersEditors Very comfortable 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review Uncomfortable

42. 1 Methodology 2 Ideal world & real world objectives STM HSS Double blind: Neither the author’s nor the reviewers’ names are known to each other 8.6 8.9 AuthorsReviewersEditors 8.1 8.48.0 Uncomfortable Very comfortable 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review

43. 1 Methodology 2 Ideal world & real world objectives STM HSS Single blind: Only the author’s name is known to the reviewer, but the reviewers’ names are not known to the author 4.1 4.0 AuthorsReviewersEditors 5.4 5.96.5 Uncomfortable Very comfortable 4.4 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review

44. 1 Methodology 2 Ideal world & real world objectives STM HSS Open: Both the authors’ and reviewers’ names are known to each other 5.3 5.1 AuthorsReviewersEditors 6.0 5.8 & 5.9 Very comfortable 5.7 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review Uncomfortable

45. 1 Methodology 2 Ideal world & real world objectives STM HSS Open and published: Both the authors and reviewers’ names are known to each other and the reviewers’ signed reports are published 4.7 AuthorsReviewersEditors 5.7 5.4 Uncomfortable Very comfortable 5.3 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review

46. 1 Methodology 2 Ideal world & real world objectives STM HSS Post-publication: Online readers comment on, or rate the paper following publication 4.4 AuthorsReviewersEditors 5.7 5.1 Uncomfortable Very comfortable 4.7 4.9 3 Ethical concerns 4 Timing discrepancies 5 Different models of peer review

47. 1 Methodology 2 Ideal world & real world objectives On double blind “You have to be quite secure about your career to un-blind yourself. I don’t want to offend a future employer or someone sitting on an interview panel” Researcher, Environmental Sciences, UK On open and published “I think this is the most transparent way…it may put some pressure on the reviewer, but it also gives him/her credit…” Reviewer, Humanities, Lebanon On post-publication “This method is limited to those who can actually read the articles (are subscribed) online unless the articles are open access..” Reviewer, Agriculture & Food Science, Zimbabwe 4 Timing discrepancies 5 Different models of peer review 3 Ethical concerns

48. AuthorsReviewersEditors authorservices.taylorandfrancis.com/ peer-review-in-2015 White Paper | Key Survey Data | References Still to come in 2016… @tandfauthorserv#tfPeerReview Available online now… Twitter… Motivations & Training| Geographic Breakdowns