Reviewing for Scientific Journals A Primer

Why Participate in Peer Review? “Playing a part in the academic process and improving papers are the most important motivations for reviewers.” Quoted from: Peer Review in 2015. A Global View. A White Paper from Taylor & Francis. http://www.tandf.co.uk/libsite/

Evaluated by APS Publications Department (Quality Control) Author Submits MS Evaluated by APS Publications Department (Quality Control) Evaluated by Editor-In-Chief Assigned to Associate Editor Invites 3-10 potential reviewers Assigned to 2-3 Reviewers (3-10 invited, first 2-3 get assignment) Critiques Received Associate Editor Makes Decision and Rates the Reviewers’ Performance 1 day** 1 day** Consultation with AEs if Triage is considered (~10%)* 2 days** This slide depicts the flow of manuscript processing once an author submits a manuscript through the electronic system. Triage may increase with the elimination of submission fees as of Jan 1 2016 14 days** 1-2 days** * Note, not all journals triage papers. ** Times are approximate and are goals.

(major or minor). R1 sent back to original reviewers Rejected Revisions Needed (major or minor). R1 sent back to original reviewers Accepted

Other Possible Associate Editor Decisions Rejection with option to resubmit as a new manuscript Rejection with referral to Physiological Reports Rejection without the option to resubmit

Who gets invited to review? Recognized experts in the field Junior reviewers under the supervision of a mentor Editorial board members (provide ~30% of reviews) Potential reviewers recommended by the authors (up to 4 max recommended per manuscript but AE usually chooses only 1) Reviewers are asked to decline if: there is a real or perceived conflict of interest they cannot review in a timely manner (2 weeks) the content is out of their area of expertise It is an honor and quite prestigious for you to be asked to review a scientific manuscript. This endeavor is used as documentation of your international reputation in a given field. It will be considered by promotion and tenure committees as well as new position interviews.

How to be selected to review: Become prominent in your field(s) of research Publish in APS journals and present at APS conferences Ask current reviewers for APS journals to recommend you when they decline their invitation to review Tell Editorial Board members and other experts of your willingness to review (NETWORK!) Share your interest to review with the Editor It is important that you and your work are known if you wish to be considered as a manuscript reviewer. Obtain experience early in your training by asking your mentor to help review papers. Be sure he/she lets the editor know that you helped. The better your reviews are the more you will be asked to evaluate papers. At some point you can request that the editor of a given journal consider you as a reviewer. Eventually, you may be considered for membership on the editorial board. Do a good job when you review - Be on time, thorough, polite, constructive and unbiased - Maintain a high reviewer rating in the APS system (reviewers are evaluated by the Associate Editor)

Primary Questions to Ask Approaching a Paper- Primary Questions to Ask Is the paper within the scope of the journal? (generally assessed by EIC at submission) Does the paper address an interesting, important, and novel question? Are there surprising/novel findings that could lead the field in new directions or are the findings incremental? Are new/novel methodologies (broadly inclusive) used? While there is no one way to review a paper here are some recommendations for the novice reviewer that will help evaluate the most important components of a scientific manuscript. Negative findings may be important but one needs to be sure they are not related to a type II statistical error.

Primary Questions to Ask Approaching a Paper- Primary Questions to Ask Is the manuscript readable? Is the scientific approach appropriate for answering this question? Are there obvious omissions? Are the conclusions supported by the data? Are the statistics appropriate? If the paper is rejected, it is particularly important to address these questions. Negative findings may be important and should not be dismissed out of hand. While there is no one way to review a paper here are some recommendations for the novice reviewer that will help evaluate the most important components of a scientific manuscript. Negative findings may be important but one needs to be sure they are not related to a type II statistical error.

Initial Evaluation (suggested tactics) Read the abstract - frames the larger view Skip to the figures, figure legends, and tables. You can quickly get a sense of what sort of study this is – i.e. the level of detail, mechanistic or descriptive, quality of the data being presented, etc. Read conclusions at end of discussion First, get a sense of the overall importance and methodology of the study. You can use the abstract and figures to help with this without reading all the fine detail in the manuscript.

In Depth Evaluation Read the entire paper Pay attention to the details Moving through the results and figures/tables, ask: Is the question as stated in the Introduction actually being addressed by the methods and protocols being used? Do the data support the conclusion? Is the quality of the data sufficient to enable conclusions to be drawn from it? Is the experimental design appropriate? Are non-relevant data presented? Do tables and figures duplicate the findings – which is the better format? Consider the possibility of type II (false negative) errors from underpowered studies. When doing an in-depth evaluation there are several questions you should keep in mind as you go through each section of the manuscript.

Be sure your overall evaluation is consistent with your review Synthesizing Comments for the Editors Remember – the Editor probably did not read it as carefully as you Editors need the major questions answered – is this novel, exciting, correct, incremental, boring, useful or not? Provide a sound, logical, and data-driven basis for your recommendations Do not just copy and paste your comments to the Editors to the authors: EDITORS HATE THAT Do not tell the Editors one thing, and then provide comments to the authors which send a different message Be sure your overall evaluation is consistent with your review There are several issues to keep in mind when writing your reviews. First, you should synthesize comments about the manuscript and study that are specific to the Editor/Associate Editor. These comments should focus on the importance and novelty of the study. Does it add substantially to our understanding of a process or are the data just incremental? You can also indicate any issues you may have with the writing and of course, any ethical issues that you are concerned about. It is important that you don’t just copy and paste the same comments that you make to the author. Finally, be sure that your overall evaluation of the manuscript is consistent with your review.

What makes a “Good” or “Bad” Review? (for the authors) Comments to authors should be different from those provided to the Editors. Here the level of detail should be enhanced.

Things That Make a Bad Review….. Extreme brevity…even good papers need a review stating why it is good! Rude, arrogant, or personal comments (would you say that to someone’s face?) Scientific errors (or misquoted literature) by the reviewer (it happens!) Mention of “acceptance” or “rejection” in the review (not the reviewer’s decision) Sloppy writing with speling erors and not good grammer (please proof your reviews) Recommendations based on opinion rather than the facts It is important to remember that your review should be written in a direct, constructive and scholarly way. Sentences should be phrased in such a way as not to degrade or belittle the authors. Specific issues concerning interpretation of data or discussion points should include pertinent references when necessary. You are not obligated to re-analyze or re-draw the data but can make suggestions that the authors do so. Finally, remember that your review needs to be written as well as you would want the manuscript to be written.

So What Makes a Good Review? Good reviews are very helpful to the authors and ultimately result in better science.

Things That Make a Good Review: Critique the Science List the major strengths and weaknesses of the science Recommend changes to improve the science of the study and use references to support suggestions Recommendation for additional experiments includes (and suggests): No additional studies (accept or minor revision) Minor additional studies (major revision) Major additional studies (reject) In a good review you will comment on strengths and weaknesses. It is okay to be complementary! You can comment on the experimental design, the area and its importance, the novelty, etc. If necessary, and with justification you can recommend that additional studies need to be done if there are gaping holes in data that are critical. You should divide your comments in Major issues and Minor ones. If more work is needed to answer the question the authors posed you can propose that. If more studies would answer different (next-step) questions, generally do not propose.

Things That Make a Good Review: Critique the Presentation: Briefly critique the writing style Suggest changes to improve the presentation of the data Too much? Too little? Fig/table overlap? Propose clearer formats, if necessary. Suggest changes to clarify, expand, or reduce sections introduction, methods, results, discussion, conclusions, figures, tables, references Major reorganization of the presentation or reanalysis of the data (implies major revision) Other issues that you may comment on to the Authors can relate to the presentation of the data and the writing. Comment on the rationale and hypothesis (es). Do the data justify the conclusions? Does the Discussion provide a synthesis of new ideas?

Additional Evaluation Points (What reviewer’s submit to editors) Each journal has a slightly different evaluation and reviewer form. Here are examples from AJP of the kind of information editors ask of reviewers in order to best evaluate the acceptability of a manuscript.

Additional Evaluation Points (What reviewer’s submit to editors) Each journal has a slightly different evaluation and reviewer form. Here are examples from AJP of the kind of information editors ask of reviewers in order to best evaluate the acceptability of a manuscript.

Additional Evaluation Points (What reviewer’s submit to editors)

Reviewing Supplementary Data Are the supplementary data appropriate as a supplement or should they be moved to the paper? Refer to the author guidelines pertaining to supplemental material http://www.the-aps.org/mm/Publications/Info-For-Authors/Data-Supplements) Data supplements are common place in many journals, even those published on-line only. However data supplements must be appropriate. As a reviewer you are responsible for also reviewing the data supplements whether they be expanded methods, videos, long data sets or software code. You should evaluate whether any of the supplemental material would be better suited as part of the main manuscript. For AJP journals where there are no page limitations authors are encouraged to provide all relevant figures and tables in the body of the manuscript. It is up to the Editor to determine if a given set of supplemental data is necessary.

The Reviewer as a Consultant: Goal = improve the paper at hand, not make it into a different study Point out missing controls or studies needed to interpret data Evaluate clarity of presentation style and order If you suggest further studies: Are they really necessary before conclusions can be made? Do they fit within the scope of the work? Can they be done within the 90 day revision window? Please remember that the goal of the review process is to make science better. The published record of a set of experiments should be believable and should have been conducted with the rigor that will allow the data and conclusions to be upheld given the current level of technology. In this way as a reviewer you are contributing to the scientific record for many years to come. A good reviewer is a consultant to the authors. Goal = improve the paper at hand, not make it into a different study. Point out missing controls or studies needed to interpret data. Evaluate clarity of presentation style and order. If you suggest further studies, are they really necessary before conclusions can be made? Do the suggested additional studies fit within the scope of the work or would they lead to a major expansion? Can the studies be done within the 90 day revision window? (i.e. new animal studies using a 6 month treatment protocol are not realistic to request).

Summary of a good review: Indicate the major strengths and weaknesses of the study Include references to support your comments Suggest changes to improve the science of the study Comment on the grammar or syntax, if certain phrases should be revised Comment on the statistics. Are they appropriate? Provide helpful comments to the Editor, telling what you really think and why! Here are some take-home messages concerning manuscript reviewing. Indicate the major strengths and weaknesses of the study for the authors (INCLUDING NOVELTY BUT INDEPENDENT OF THE WRITING STYLE). Include references to support your comments (WHEN APPROPRIATE) Suggest changes to improve the science of the study Comment on the grammar or syntax, if certain phrases should be revised Comment on the statistics. Are they appropriate? Provide helpful comments to the Editor IN COMPLETE THOUGHTS, telling what you really think and why!

Reviewing a Revised Manuscript: Were the authors responsive to your suggestions? Are the revisions acceptable? Did the authors explain why a suggestion was not acted upon? If you asked for something you felt was important, make sure it was addressed. Be careful not to review the other reviewers’ comments. Don’t argue with the other reviewers in your comments to the authors. When a manuscript is revised, the authors provide a version of the manuscript where the major changes are usually designated by red text. This may vary depending on journal guidelines. In addition a separate document is provided where the authors respond to the concerns of each reviewer on a point by point basis. As a reviewer you should evaluate how responsive the authors have been to your concerns. Did they respond in a measured, scientific way? Did they carry out additional analyses or experiments as recommended. If they did not, do they present a convincing argument why not?

Reviewing a Revised Manuscript (continued): Four important things to remember: Your opinions are just that. Allow the authors their approach if it is reasonable and the results significant. The Associate Editor has to balance your comments with those of the other reviewers. Make it clear in your comments to the Editor why you recommended acceptance, revision, referral, or rejection. Don’t be upset if the Associate Editor makes a decision that does not agree with your recommendations. Don’t delay your review of the revised manuscript. Finally, it is important that you understand that there are likely to be differences of opinions between you and the authors and the other reviewers for that matter. The Editor must make a decision based on the abundance of evidence in one or another direction. It is your responsibility as a reviewer to make you argument as strong as possible. Your opinion may be a minority opinion and the final decision may not be to your liking, but this is the role of the Editor. Revised manuscripts should be given a high priority of review and should be easier to evaluate than the original submission. If you are new at reviewing make sure you document this activity and discuss with your Chair at your annual evaluation.

Final thoughts: Contributing to the peer review process is an important part of a successful career. Experience gained as a peer reviewer will help you in writing and revising your own manuscripts. Good reviewers are hard to find, Editors appreciate carefully considered and concise reviews. Recognize that the Editors grade your performance as a reviewer. If you are repeatedly asked to review for an APS journal, you must be doing a good job. Reviewing is an obligation and can be a major career builder.

References Peer Review in 2015. A Global View. A White Paper from Taylor & Francis. http://www.tandf.co.uk/libsite/ Revising Your Manuscript: Responding to Critiques. S Duckles http://www.the-aps.org/careers/careers1/mentor/workshop/01wrkshp.htm Manuscript peer review: a helpful checklist for students and novice referees. DR Seals and H Tanaka http://advan.physiology.org/cgi/reprint/23/1/S52 Peer Review and Publication in APS Journals. DJ Benos and M Reich http://www.the-aps.org/careers/careers1/GradProf/peerreview.htm Statistics, authors, and reviewers: the heart of the matter. D Curran-Everett and DJ Benos http://advan.physiology.org/content/33/1/80 Demystifying the publishing process: a primer for early career investigators. IH Zucker http://ajpheart.physiology.org/content/309/4/H529 Here are some references where you can find more detailed information that may be helpful in the review process.

References Reproducibility in Science: Improving the Standard for Basic and Preclinical Research. CG Begley and JPA Ioannidis http://circres.ahajournals.org/content/116/1/116.short Beyond Bar and Line Graphs: Time for a New Data Presentation Paradigm. TL Weissgerberg, NM Milic, SJ Winham, VD Garovic. http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002128 Civil, sensible, and constructive peer review in APS Journals. H Raff and D Brown http://ajpheart.physiology.org/content/305/3/H265.short How to review a paper. Dale J. Benos, Kevin L. Kirk, and John E. Hall. http://advan.physiology.org/content/27/2/47.long The ups and downs of peer review. Dale J. Benos, Edlira Bashari, Jose M. Chaves, Amit Gaggar, Niren Kapoor,Martin LaFrance, Robert Mans, David Mayhew, Sara McGowan, Abigail Polter,Yawar Qadri, Shanta Sarfare, Kevin Schultz, Ryan Splittgerber, Jason Stephenson,Cristy Tower, R. Grace Walton, and Alexander Zotov. http://advan.physiology.org/content/31/2/145.long Guidelines for reporting statistics in journals published by the American Physiological Society Adv Physiol Educ 28: 85–87, 2004;10.1152/advan.00019.2004. Here are some references where you can find more detailed information that may be helpful in the review process.