1. Reviewing Papers: What Reviewers Look For Session 19 C507 Scientific Writing

2. Are Referees really Necessary?  Some look at reviewers (referees) solely as biased adversaries whose objectives are solely to reject, delay or scoop all papers submitted to them  Some feel that science would do quite well without reviewers at all, that we could publish all readable papers without delay and let the community at large sort the bad from good

3. Are Referees really Necessary?  For those that argue against the review process and for unrestricted access, they fail to distinguish between the formal and informal systems of communication in science

4. Price Says  “The scientist is more frequently than not passionate, biased, illogical, resistant to proof and to change, and beset by other similar human failings. It is, today, better to let this show and be understood than to pretend that it is not there.”

5. An Overlooked Function  One function of the peer review system that is often overlooked is its indirect influence on the initial preparation of a paper  Established scientists write their papers with a critical sense that anticipates peer reviewer questions

6. Does the System Work?  The answer to this question seems to be a resounding “Yes.”  All editors, and most authors will affirm that there is hardly a paper published that has not been improved, often substantially, by the reviewers  These same editors will also attest that examples of intentional delay, biased reporting or unethical behavior is rare

7. The Different Processes of Peer Review  Evaluations of papers for publication  Assessment of grant proposals  Recommendations for positions or tenure

8. Two Kinds of Consensus  There are times when referees will disagree about the suitability of a paper for publication, and so will the journal editor  Not infrequently, a paper rejected by one journal is accepted by another  These disagreements do not indicate a breakdown of the referee system; rather, they arise from differing views of two kinds of consensus

9. Consensus #1  First, there is the consensus that develops within an editorial board on what constitutes an acceptable paper for that journal

10. Consensus #2  The second consensus that enters into the evaluation of a scientific paper is the one that develops within a field of research  In some areas of science the fundamental algorithms and paradigms are well established, tested, verified and accepted

11. Possible Solutions  You could publish the paper and include the reviewer’s comments as a comment or addendum to the paper  Or you could consider open review- here, you seek comments from several referees to be published concurrently with the paper

12. Does Reviewer Anonymity Matter?  This remains controversial  But, by and large, reviewers really do try their best to offer useful comments to authors.  Reviewers need to report with candor and honesty; keep them anonymous is often believed to help do just that

13. Blind Review  We also remove the author’s name from a paper sent out for review  This is supposed to reduce bias  But this really does not work too well, just in terms of keeping identity confidential; we have too small a population of researchers in our profession

14. Reviewers  In general, journals send papers to at least 2 reviewers, but may at times use more  Reviewers are selected because: They have expertise of use to the journal They have expertise of use to the journal They support the goals and mission of the journal They support the goals and mission of the journal

15. General Questions Reviewers Need to Answer  Does this paper report a specific, identifiable advance in knowledge?  Has the work reported in this paper been published before?  Are the conclusions justified, soundly based and logically consistent?

16. General Questions Reviewers Need to Answer  Are the procedures and methods sufficiently clear that the work could be repeated by anyone knowledgeable in the field?  Are the references to previous work pertinent and complete?

17. General Questions Reviewers Need to Answer  Is the paper as concise as it could be, consistent with clarity?  Referees are not required to comment on writing style. However, they are invited to suggest changes that would remove ambiguity or clarify meaning

18. General Questions Reviewers Need to Answer  Are the figures and tables relevant and properly prepared?  Are the title and abstract truly descriptive of the contents?

19. Ethics for Reviewers  We remind reviewers that the paper under consideration is a confidential document that should not be discusses or shown to others without the express permission of the editor  As the reviewer’s detailed report may be relayed to the author as a guide for revising their paper, we request that you avoid harsh or abrasive statements (comments specifically to the editor should be sent as a separate written letter)

20. Ethics for Reviewers  Your anonymity as a reviewer will be preserved and you are asked not to identify yourself to the authors without the permission of the editor  The paper must be returned within a specified period of time

21. The JMPT  Let us say that a new paper is submitted to the JMPT  We acknowledge receipt of the paper on the day it is received, and a letter that “accepts the paper for consideration to publish” is sent to the principal author

22. The JMPT  Then, we ready two copies to go out for review  The paper is blinded  Two reviewers, with expertise in the subject are and for whom we believe to have no bias, are selected  Appropriate forms are sent with the paper

23. The JMPT  Reviewers then provide their critique  They are asked to return their review within 2 weeks. 10% might actually do so  The editor uses those reviews as a guide to his or her decision  Most frequently, the paper is returned for revision

24. The JMPT  On the review form, the reviewer can check off a set of good points, bad points, and then offer a comment on publishability and/or recommendations for improvement

25. The JMPT  Good Points Original Original Timely Timely Interesting to read Interesting to read Practical Practical Information important to document Information important to document Well suited to journal’s objectives Well suited to journal’s objectives Good research Good research Good review of literature Good review of literature Well written/edited Well written/edited Other good points Other good points

26.  Bad Points Too preliminary Too preliminary Duplicates other published research- no new insight Duplicates other published research- no new insight Not well suited to journal’s objectives Not well suited to journal’s objectives Faulty research design Faulty research design Poorly supported by references Poorly supported by references Highly speculative Highly speculative Plagiarized material Plagiarized material Not well written/edited Not well written/edited Other bad points Other bad points