1. Reviewing for Scientific Journals A Primer

2. Why Participate in Peer Review?
“Playing a part in the academic process and improving papers are the most important motivations for reviewers.”
Quoted from: Peer Review in A Global View. A White Paper from Taylor & Francis.

3. Evaluated by APS Publications Department (Quality Control)
Author Submits MS
Evaluated by APS Publications Department (Quality Control)
Evaluated by Editor-In-Chief
Assigned to Associate Editor
Invites 3-10 potential reviewers
Assigned to 2-3 Reviewers (3-10 invited, first 2-3 get assignment)
Critiques Received
Associate Editor Makes Decision and
Rates the Reviewers’ Performance
1 day**
1 day**
Consultation with AEs if Triage is considered (~10%)*
2 days**
This slide depicts the flow of manuscript processing once an author submits a manuscript through the electronic system.
Triage may increase with the elimination of submission fees as of Jan
14 days**
1-2 days**
* Note, not all journals triage papers. ** Times are approximate and are goals.

4. (major or minor). R1 sent back to original reviewers
Rejected
Revisions Needed
(major or minor). R1 sent back to original reviewers
Accepted

5. Other Possible Associate Editor Decisions
Rejection with option to resubmit as a new manuscript
Rejection with referral to Physiological Reports
Rejection without the option to resubmit

6. Who gets invited to review?
Recognized experts in the field
Junior reviewers under the supervision of a mentor
Editorial board members (provide ~30% of reviews)
Potential reviewers recommended by the authors (up to 4 max recommended per manuscript but AE usually chooses only 1)
Reviewers are asked to decline if:
there is a real or perceived conflict of interest
they cannot review in a timely manner (2 weeks)
the content is out of their area of expertise
It is an honor and quite prestigious for you to be asked to review a scientific manuscript. This endeavor is used as documentation of your international reputation in a given field. It will be considered by promotion and tenure committees as well as new position interviews.

7. How to be selected to review:
Become prominent in your field(s) of research
Publish in APS journals and present at APS conferences
Ask current reviewers for APS journals to recommend you when they decline their invitation to review
Tell Editorial Board members and other experts of your willingness to review (NETWORK!)
Share your interest to review with the Editor
It is important that you and your work are known if you wish to be considered as a manuscript reviewer. Obtain experience early in your training by asking your mentor to help review papers. Be sure he/she lets the editor know that you helped.
The better your reviews are the more you will be asked to evaluate papers. At some point you can request that the editor of a given journal consider you as a reviewer. Eventually, you may be considered for membership on the editorial board.
Do a good job when you review
- Be on time, thorough, polite, constructive and unbiased
- Maintain a high reviewer rating in the APS system
(reviewers are evaluated by the Associate Editor)

8. Primary Questions to Ask
Approaching a Paper-
Primary Questions to Ask
Is the paper within the scope of the journal? (generally assessed by EIC at submission)
Does the paper address an interesting, important, and novel question?
Are there surprising/novel findings that could lead the field in new directions or are the findings incremental?
Are new/novel methodologies (broadly inclusive) used?
While there is no one way to review a paper here are some recommendations for the novice reviewer that will help evaluate the most important components of a scientific manuscript. Negative findings may be important but one needs to be sure they are not related to a type II statistical error.

9. Primary Questions to Ask
Approaching a Paper-
Primary Questions to Ask
Is the manuscript readable?
Is the scientific approach appropriate for answering this question?
Are there obvious omissions?
Are the conclusions supported by the data?
Are the statistics appropriate?
If the paper is rejected, it is particularly important to address these questions.
Negative findings may be important and should not be dismissed out of hand.
While there is no one way to review a paper here are some recommendations for the novice reviewer that will help evaluate the most important components of a scientific manuscript. Negative findings may be important but one needs to be sure they are not related to a type II statistical error.

10. Initial Evaluation (suggested tactics)
Read the abstract - frames the larger view
Skip to the figures, figure legends, and tables. You can quickly get a sense of what sort of study this is – i.e. the level of detail, mechanistic or descriptive, quality of the data being presented, etc.
Read conclusions at end of discussion
First, get a sense of the overall importance and methodology of the study. You can use the abstract and figures to help with this without reading all the fine detail in the manuscript.

11. In Depth Evaluation Read the entire paper Pay attention to the details
Moving through the results and figures/tables, ask:
Is the question as stated in the Introduction actually being addressed by the methods and protocols being used?
Do the data support the conclusion?
Is the quality of the data sufficient to enable conclusions to be drawn from it?
Is the experimental design appropriate?
Are non-relevant data presented?
Do tables and figures duplicate the findings – which is the better format?
Consider the possibility of type II (false negative) errors from underpowered studies.
When doing an in-depth evaluation there are several questions you should keep in mind as you go through each section of the manuscript.

12. Be sure your overall evaluation is consistent with your review
Synthesizing Comments for the Editors
Remember – the Editor probably did not read it as carefully as you
Editors need the major questions answered – is this novel, exciting, correct, incremental, boring, useful or not?
Provide a sound, logical, and data-driven basis for your recommendations
Do not just copy and paste your comments to the Editors to the authors: EDITORS HATE THAT
Do not tell the Editors one thing, and then provide comments to the authors which send a different message
Be sure your overall evaluation is consistent with your review
There are several issues to keep in mind when writing your reviews. First, you should synthesize comments about the manuscript and study that are specific to the Editor/Associate Editor. These comments should focus on the importance and novelty of the study. Does it add substantially to our understanding of a process or are the data just incremental? You can also indicate any issues you may have with the writing and of course, any ethical issues that you are concerned about. It is important that you don’t just copy and paste the same comments that you make to the author. Finally, be sure that your overall evaluation of the manuscript is consistent with your review.

13. What makes a “Good” or “Bad” Review?
(for the authors)
Comments to authors should be different from those provided to the Editors. Here the level of detail should be enhanced.

14. Things That Make a Bad Review…..
Extreme brevity…even good papers need a review stating why it is good!
Rude, arrogant, or personal comments (would you say that to someone’s face?)
Scientific errors (or misquoted literature) by the reviewer (it happens!)
Mention of “acceptance” or “rejection” in the review (not the reviewer’s decision)
Sloppy writing with speling erors and not good grammer (please proof your reviews)
Recommendations based on opinion rather than the facts
It is important to remember that your review should be written in a direct, constructive and scholarly way. Sentences should be phrased in such a way as not to degrade or belittle the authors. Specific issues concerning interpretation of data or discussion points should include pertinent references when necessary. You are not obligated to re-analyze or re-draw the data but can make suggestions that the authors do so. Finally, remember that your review needs to be written as well as you would want the manuscript to be written.

15. So What Makes a Good Review?
Good reviews are very helpful to the authors and ultimately result in better science.

16. Things That Make a Good Review:
Critique the Science
List the major strengths and weaknesses of the science
Recommend changes to improve the science of the study and use references to support suggestions
Recommendation for additional experiments includes (and suggests):
No additional studies (accept or minor revision)
Minor additional studies (major revision)
Major additional studies (reject)
In a good review you will comment on strengths and weaknesses. It is okay to be complementary! You can comment on the experimental design, the area and its importance, the novelty, etc. If necessary, and with justification you can recommend that additional studies need to be done if there are gaping holes in data that are critical. You should divide your comments in Major issues and Minor ones.
If more work is needed to answer the question the authors posed you can propose that. If more studies would answer different (next-step) questions, generally do not propose.

17. Things That Make a Good Review: Critique the Presentation:
Briefly critique the writing style
Suggest changes to improve the presentation of the data
Too much? Too little? Fig/table overlap?
Propose clearer formats, if necessary.
Suggest changes to clarify, expand, or reduce sections
introduction, methods, results, discussion, conclusions, figures, tables, references
Major reorganization of the presentation or reanalysis of the data (implies major revision)
Other issues that you may comment on to the Authors can relate to the presentation of the data and the writing. Comment on the rationale and hypothesis (es). Do the data justify the conclusions? Does the Discussion provide a synthesis of new ideas?

18. Additional Evaluation Points (What reviewer’s submit to editors)
Each journal has a slightly different evaluation and reviewer form. Here are examples from AJP of the kind of information editors ask of reviewers in order to best evaluate the acceptability of a manuscript.

19. Additional Evaluation Points (What reviewer’s submit to editors)
Each journal has a slightly different evaluation and reviewer form. Here are examples from AJP of the kind of information editors ask of reviewers in order to best evaluate the acceptability of a manuscript.

20. Additional Evaluation Points (What reviewer’s submit to editors)

21. Reviewing Supplementary Data
Are the supplementary data appropriate as a supplement or should they be moved to the paper? Refer to the author guidelines pertaining to supplemental material
Data supplements are common place in many journals, even those published on-line only. However data supplements must be appropriate. As a reviewer you are responsible for also reviewing the data supplements whether they be expanded methods, videos, long data sets or software code. You should evaluate whether any of the supplemental material would be better suited as part of the main manuscript. For AJP journals where there are no page limitations authors are encouraged to provide all relevant figures and tables in the body of the manuscript. It is up to the Editor to determine if a given set of supplemental data is necessary.

22. The Reviewer as a Consultant:
Goal = improve the paper at hand,
not make it into a different study
Point out missing controls or studies needed to interpret data
Evaluate clarity of presentation style and order
If you suggest further studies:
Are they really necessary before conclusions can be made?
Do they fit within the scope of the work?
Can they be done within the 90 day revision window?
Please remember that the goal of the review process is to make science better. The published record of a set of experiments should be believable and should have been conducted with the rigor that will allow the data and conclusions to be upheld given the current level of technology. In this way as a reviewer you are contributing to the scientific record for many years to come.
A good reviewer is a consultant to the authors.
Goal = improve the paper at hand, not make it into a different study.
Point out missing controls or studies needed to interpret data.
Evaluate clarity of presentation style and order.
If you suggest further studies, are they really necessary before conclusions can be made?
Do the suggested additional studies fit within the scope of the work or would they lead to a major expansion?
Can the studies be done within the 90 day revision window? (i.e. new animal studies using a 6 month treatment protocol are not realistic to request).

23. Summary of a good review:
Indicate the major strengths and weaknesses of the study
Include references to support your comments
Suggest changes to improve the science of the study
Comment on the grammar or syntax, if certain phrases should be revised
Comment on the statistics. Are they appropriate?
Provide helpful comments to the Editor, telling what you really think and why!
Here are some take-home messages concerning manuscript reviewing.
Indicate the major strengths and weaknesses of the study for the authors (INCLUDING NOVELTY BUT INDEPENDENT OF THE WRITING STYLE).
Include references to support your comments (WHEN APPROPRIATE)
Suggest changes to improve the science of the study
Comment on the grammar or syntax, if certain phrases should be revised
Comment on the statistics. Are they appropriate?
Provide helpful comments to the Editor IN COMPLETE THOUGHTS, telling what you really think and why!

24. Reviewing a Revised Manuscript:
Were the authors responsive to your suggestions?
Are the revisions acceptable?
Did the authors explain why a suggestion was not acted upon?
If you asked for something you felt was important, make sure it was addressed.
Be careful not to review the other reviewers’ comments.
Don’t argue with the other reviewers in your comments to the authors.
When a manuscript is revised, the authors provide a version of the manuscript where the major changes are usually designated by red text. This may vary depending on journal guidelines. In addition a separate document is provided where the authors respond to the concerns of each reviewer on a point by point basis. As a reviewer you should evaluate how responsive the authors have been to your concerns. Did they respond in a measured, scientific way? Did they carry out additional analyses or experiments as recommended. If they did not, do they present a convincing argument why not?

25. Reviewing a Revised Manuscript (continued):
Four important things to remember:
Your opinions are just that. Allow the authors their approach if it is reasonable and the results significant.
The Associate Editor has to balance your comments with those of the other reviewers. Make it clear in your comments to the Editor why you recommended acceptance, revision, referral, or rejection.
Don’t be upset if the Associate Editor makes a decision that does not agree with your recommendations.
Don’t delay your review of the revised manuscript.
Finally, it is important that you understand that there are likely to be differences of opinions between you and the authors and the other reviewers for that matter. The Editor must make a decision based on the abundance of evidence in one or another direction. It is your responsibility as a reviewer to make you argument as strong as possible. Your opinion may be a minority opinion and the final decision may not be to your liking, but this is the role of the Editor. Revised manuscripts should be given a high priority of review and should be easier to evaluate than the original submission. If you are new at reviewing make sure you document this activity and discuss with your Chair at your annual evaluation.

26. Final thoughts:
Contributing to the peer review process is an important
part of a successful career.
Experience gained as a peer reviewer will help you in writing and revising your own manuscripts.
Good reviewers are hard to find, Editors appreciate carefully considered and concise reviews.
Recognize that the Editors grade your performance as a reviewer. If you are repeatedly asked to review for an APS journal, you must be doing a good job.
Reviewing is an obligation and can be a major career builder.

27. References
Peer Review in A Global View. A White Paper from Taylor & Francis.
Revising Your Manuscript: Responding to Critiques. S Duckles
Manuscript peer review: a helpful checklist for students and novice referees. DR Seals and H Tanaka
Peer Review and Publication in APS Journals. DJ Benos and M Reich
Statistics, authors, and reviewers: the heart of the matter. D Curran-Everett and DJ Benos
Demystifying the publishing process: a primer for early career investigators. IH Zucker
Here are some references where you can find more detailed information that may be helpful in the review process.

28. References
Reproducibility in Science: Improving the Standard for Basic and Preclinical Research. CG Begley and JPA Ioannidis
Beyond Bar and Line Graphs: Time for a New Data Presentation Paradigm. TL Weissgerberg, NM Milic, SJ Winham, VD Garovic.
Civil, sensible, and constructive peer review in APS Journals. H Raff and D Brown
How to review a paper. Dale J. Benos, Kevin L. Kirk, and John E. Hall.
The ups and downs of peer review. Dale J. Benos, Edlira Bashari, Jose M. Chaves, Amit Gaggar, Niren Kapoor,Martin LaFrance, Robert Mans, David Mayhew, Sara McGowan, Abigail Polter,Yawar Qadri, Shanta Sarfare, Kevin Schultz, Ryan Splittgerber, Jason Stephenson,Cristy Tower, R. Grace Walton, and Alexander Zotov.
Guidelines for reporting statistics in journals published by the
American Physiological Society Adv Physiol Educ 28: 85–87, 2004; /advan
Here are some references where you can find more detailed information that may be helpful in the review process.