Cleveland State University ESC 720 Technical Communications How to Respond to Peer Reviews Dan Simon 1

Overview of the Process Peer reviewers make comments, criticisms, and suggestions about your paper You get a chance to revise the paper and respond to the peer reviews (often with a deadline) The same peer reviewers will see your revised paper and your response document Is it worth trying to resubmit? Do the reviewers have a negative view of the paper? Will they find additional problems in the resubmission? 2

How to irritate peer reviewers Argue with their comments Ignore their comments Make it hard for them to see how you addressed their comments in the revised paper How to please peer reviewers Thank them for their comments Go the extra mile in addressing their comments and revising your paper Make it easy and convenient for them to see how you addressed their comments in the revised paper 3

Case Study: AESCTE-D Jan. 6 – initial submission Jan. 20 – paper assigned to assoc. editor (AE) Jan. 29 – AE sends 8 reviewer invitations March 3 – 3 reviews received March 12 – AE decision “revise and resubmit” May 20 – revision 1 submitted May 22 – paper assigned to AE May 26 – AE decision “revise and resubmit” 4

May 26 “Revise and Resubmit” Letter I am not sending out your revised paper for review because you did not adequately respond to the reviewer comments. Your response letter needs to not only answer the reviewer comments, but also needs to explain where and how you modified your paper in response to each comment. If the reviewer has a question about your paper, then it is likely that many readers will have the same question, so you need to address the reviewer comments not only in your response letter, but also in the paper itself. If you feel that you can suitably address the reviewers' comments, I invite you to revise and resubmit your manuscript. 5

AESCTE-D Jan. 6 – initial submission … May 26 – AE decision “revise and resubmit” May 27 – revision 2 submitted May 30 – paper assigned to AE May 31 – AE sends 3 reviewer invitations July 7 – 3 reviews received July 14 – paper rejected July 15 – author asks for clarification 6

AESCTE-D R2 Reviews Reviewer #1: While the manuscript is potentially interesting, its technical contribution is still questionable, since the author has, in my view, not satisfactorily addressed the comments from the last round of reviews. … As I pointed out last time, Theorem 1 does not apply to changing set points. … This is symptomatic of a broader problem afflicting the paper, as other reviewers point out … The comments from other reviewers are also addressed in an unsatisfactory manner. … Reviewer #2: I could repeat some of what I said in the previous review … Reviewer #3: Accept. 7

Dear. Professor X, I take no pleasure in rejecting papers. After the first round of reviews, I thought that it would eventually be accepted. However, when you submitted the first revision, you did not respond adequately to the review comments. I sent the paper back to you, without review, asking for a better response. When you submitted the second revision, 2 of the 3 reviewers wrote that you still did not respond adequately. I encourage you in future submissions to be as careful as possible in explaining how you modified your paper based on review comments. Responding to review comments is not an exercise to see how few changes you can make while still satisfying the reviewers. Review comments give you an opportunity to improve the paper not only for the reviewers, but also for the readers of the paper, many of whom will have perspectives similar to those of the reviewers. 8

Case Study: INS-D April 9 – Initial review Sep. 27 – Revision 1 received Oct. 3 – Revision 1 Reviewed: “Unfortunately, I have to recommend rejection, mainly because the authors basically ignored the most important comment in my previous review.” 9

Case Study: SIGPRO-D – Review of Revised Paper: There are two ways to address reviewer comments: (a) Accept the comments as constructive criticism, and use the comments to improve the paper; (b) Defend the paper against the comments and leave the paper as originally written. The authors have chosen the second option. This is unfortunate because the paper seems to have potential, but I do not think that it is publishable unless the authors address the issues raised by the reviewers. Following are point-by- point discussions of my first review … 1.This comment has been addressed in the revised paper. However, the authors added a new figure called "Figure 2b" to the paper. But the paper already had a Figure 2a, 2b, and 2c. 2.This comment was not addressed. … 10

Case Study (continued): 3. This comment was not addressed. The purpose of the comment was not that the authors provide a hand-written derivation to the reviewer, but that the authors modify the paper to include the derivation for the readers. 4. This comment was addressed in the revised paper. 5. This comment was addressed half-way. 6.This comment was not addressed. The purpose of the comment was not that the authors provide an answer only for the reviewer, but that the authors modify the paper to answer the question for the readers. 7.… 11

Replying to Review Comments Add a statement of appreciation to the reviewers at the beginning of your reply document Go the extra mile in modifying your paper “Response to Reviewer” Document: Repeat each reviewer comment, along with your response and a short description of how and where you addressed it Revised Submission: Highlight the changes in red font 12

Replying to Review Comments: Example We would like to thank the reviewers for their comments and suggestions. Our responses to the reviewer comments are listed below. Changes in the revised paper are in red font. 1.In Section 2, please describe the mutation process. Reply: We added a description of mutation just prior to Algorithm 1, starting at the bottom of page 5. 2.It could be interesting if authors expand the experimental part by considering zero mutation rates. Reply: We do not test with a zero mutation rate because the theory in this paper requires that the mutation rate be positive (see Theorem 3). We have clarified this in the text a couple paragraphs before Table 1 (page 23). 3.… 13

Replying to Review Comments: Example 1.The article assumes that the reader knows about CMA-ES and PSO2011. Otherwise, they will need to read references [35-39] and [24], which makes reading this work tedious. Moreover, the proposed methodology does not present a major contribution … Response: We agree the paper should be more self- contained. Although the reader should have familiarity with EAs to get the most out of this paper, we added some background about CMA-ES and PSO2011 in Sections 2.1  2.4 to make the paper more self-contained. The main contribution of this paper is the introduction of biogeography-based hybridization strategies. This idea has not been published before now. We have revised the paper to emphasize the contribution in the abstract, introduction (3rd paragraph), and conclusion (1st paragraph). 14

Return politeness for rudeness: Review Comment: How do you expect me to evaluate the relevance of your findings without a statistical analysis? What you want to say: Anyone with half a brain can see the relevance of our data – we don’t need a lousy statistical analysis! (Moron.) What you should say: We thank the reviewer for this insightful comment. We have now performed a student’s t-test to determine the significance of differences between the experimental and control groups. Our analysis shows that the p-value is less than 0.05, indicating statistical significance. This information has been added to the Results section (pg. 12, paragraph 2, line 4). 15

A Few Lessons: “Revise and resubmit,” or “Accept subject to modifications,” is not the same as acceptance Do not take reviewers for granted – appreciate them Just because the reviewer is wrong does not mean that you are right Consider reviews (even rejections) as an opportunity to improve your paper 16

How to respond to peer reviews: Get mad, cry, and curse – then get over it Edit carefully, take your time, but be prompt Include the original comments and give point-by-point responses (Review 1, Comment 1, etc.) Pay careful attention to detail Disagree if you must, but do so carefully – choose your battles wisely Do not submit the same version to a different journal If reviewers contradict each other: Explain to both reviewers why you believe one of the suggestions would be more effective in improving the paper Be prepared to shorten your paper – extra material can be uploaded to a web page (yours or the journal’s) 17

18 Review Comment: The method the authors propose is clearly wrong. How NOT to respond: Yes, we know. We thought we could still get a paper out of it. Sorry for wasting your time. Correct response: The reviewer raises an important concern. However, as the focus of our work is exploratory and not performance-based, validation was not considered to be of critical importance to the contribution of this paper. This point is addressed in detail on page 19 of the revision.

19 Review Comment: The authors fail to reference the work of Smith et al., who solved the same problem 20 years ago. How NOT to respond: Oops. We didn’t think that the reviewers would know about that paper. Actually, Smith’s solution is much better than ours. Correct response: The reviewer raises an important concern. However, our work is based on different first principles (e.g., we use different variable names), and has a much nicer figures. This point is addressed in detail on page 13.

20 Review Comment: This paper is poorly written and scientifically unsound. I strongly recommend rejection. How NOT to respond: reviewer! I will track you down, and I will get sweet revenge when I review YOUR paper! Correct response: The reviewer raises an important concern. However, it appears that the reviewer did not fully grasp the scope and significance of this work, and may have misjudged the paper based on faulty assumptions. We address the reviewer’s concerns in detail on page 17.

What you want to say: You don’t have a clue what we were trying to say! What you should say: Thank you for your question. Some of the text was ambiguous, and we have modified our explanations on page 11 to be more clear. What you want to say: No one knows the answer to your question! What you should say: Your question is important, and we have added some discussion of it in the “future work” section of the conclusion. 21

What you want to say: The extra test that you suggest would take forever! What you should say: The suggested test is interesting and would provide important information about our algorithm. The suggested test is outside the scope of our study, but we added discussion about it to the “future work” section. What you want to say: We’re not claiming that we proved anything – we’re just presenting data to support a hypothesis! What you should say: We agree that our statements were too definitive; we have edited the text on p. 7 on to soften our claims. 22

What you want to say: Did you even read what we wrote?!? What you should say: We have altered the text on p. 19 to be more clear. What you want to say: You are SO picky about grammar and formatting! What you should say: We appreciate the reviewer’s attention to detail, and we have corrected the text as suggested. 23

Acknowledgments: comments-by-peer-reviewers comments-by-peer-reviewers content/uploads/downloads/2013/08/AJE-Responding-to- Reviewers.pdf content/uploads/downloads/2013/08/AJE-Responding-to- Reviewers.pdf Comments-from-Peer-Reviewers.aspx Comments-from-Peer-Reviewers.aspx 24