Refereeing “And diff’ring judgements serve but to declare, That truth lies somewhere, if we knew but where.” – William Cowper, Hope.

Slides:



Advertisements
Similar presentations
To achieve a Level 7 you need to…. To achieve a Level 6 you need to…
Advertisements

© Nick Feamster and Alex Gray Reading and Reviewing Papers Nick Feamster and Alex Gray College of Computing Georgia Institute of Technology.
Critical Reading Strategies: Overview of Research Process
How to review a paper for a journal Dr Stephanie Dancer Editor Journal of Hospital Infection.
Customer Success is Our Mission MILCOM 2008 Reviewer Guidelines Rev B 8 July 2008.
Dr Ronni Michelle Greenwood Autumn  Introduction  Method  Results  Discussion.
Cleveland State University ESC 720 Writing in Electrical and Computer Engineering Peer Review Dan Simon 1.
Publishers of original thinking. What kinds of academic writing are there? There are many kinds of writing that originates from academia. In my view there.
Submission Process. Overview Preparing for submission The submission process The review process.
GETTING PUBLISHED Chapter 18.
Summary-Response Essay
The Rosabeth Moss Kanter Award Module 2, Class 2 A Teaching Module Developed by the Curriculum Task Force of the Sloan Work and Family Research Network.
Reviewing Papers: What Reviewers Look For Session 19 C507 Scientific Writing.
CPSC 699. Summary Refereeing is the foundation of academic word: it promotes equity, diversity, openness, free exchange of ideas, and drives the progress.
Responsible Conduct of Research, Scholarship, and Creative Activities Peer Review Responsible Conduct of Research, Scholarship, and Creative Activities.
Reviewing the work of others Referee reports. Components of a referee report Summary of the paper Overall evaluation Comments about content Comments about.
Slide 1Xiao Qin, Auburn University How to Write Papers Dr. Xiao Qin Auburn University These slides are.
ALEC 604: Writing for Professional Publication
CMPUT Teaching and Research Methods1 CMPUT603 - Fall 2005 Topic2: Refereeing (After Alan J. Smith, “The Task of the Referee”, IEEE Computer, April,
Scholarship Skills Tim Sheard & Todd Leen 1 Lecture 19 Scholarship Skills Tim Sheard, PSU Todd Leen, OGI-OHSU All material © 1996, 1997, 1999, 2000 David.
Statistical presentation in international scientific publications 5. A statistical review (group work) Malcolm Campbell Lecturer in Statistics, School.
Radiography Peer Review - make your contribution Dr Pauline Reeves Associate Editor (Clinical Imaging)
ALEC 604: Writing for Professional Publication Week 10: Faculty/Peer Reviews.
The peer review process and the task of a referee
Peer Review for Addiction Journals Robert L. Balster Editor-in-Chief Drug and Alcohol Dependence.
FISH 521 Peer review. Peer review Mechanics Advantages Challenges Solutions.
Advanced Research Methodology
How to Write a Scientific Paper Hann-Chorng Kuo Department of Urology Buddhist Tzu Chi General Hospital.
How to Write a Literature Review
The Submission Process Jane Pritchard Learning and Teaching Advisor.
“Knowing Revisited” And that’s how we can move toward really knowing something: Richard Feynman on the Scientific Method.
11 Reasons Why Manuscripts are Rejected
READING A PAPER. Basic Parts of a Research Paper 1. Abstract 2. Introduction to Technology (background) 3. Tools & techniques/Methods used in current.
CRITICAL APPRAISAL OF SCIENTIFIC LITERATURE
Chris Luszczek Biol2050 week 3 Lecture September 23, 2013.
How your submission will be evaluated by European Urology reviewers: Reviewer template and Publication guidelines Jim Catto Associate Editor European Urology.
Research Methods and Techniques Lecture 1 Introduction & Paper Review 1 © 2004, J S Sventek, University of Glasgow.
 How to referee. Refereeing is excellent practice for  developing critical appraisal skills  understanding how good (and bad) papers are written 
Ginny Smith Managing Editor: Planning and Urban Studies Taylor & Francis Ltd.
1 How to review a paper by Fabio Crestani. 2 Disclaimer 4 There is no fixed mechanism for refereeing 4 There are simple rules that help transforming a.
Software Engineering Experimentation Rules for Reviewing Papers Jeff Offutt See my editorials 17(3) and 17(4) in STVR
I-Search Paper Purpose You will be writing a personal research paper, sometimes called an I-Search paper. You will pick a subject to which you have a personal.
Being an Effective Peer Reviewer Barbara Gastel, MD, MPH Texas A&M University
Morten Blomhøj and Paola Valero Our agenda: 1.The journal NOMAD’s mission, review policy and process 2.Two reviews of a paper 3.Frequent comments in reviews.
MedEdPORTAL Reviewer Tutorial Contact MedEdPORTAL
How to write a professional paper. 1. Developing a concept of the paper 2. Preparing an outline 3. Writing the first draft 4. Topping and tailing 5. Publishing.
Reviewing the Research of Others RIMC Research Capacity Enhancement Workshops Series : “Achieving Research Impact”
Reviewing Papers© Dr. Ayman Abdel-Hamid, CS5014, Fall CS5014 Research Methods in CS Dr. Ayman Abdel-Hamid Computer Science Department Virginia Tech.
Maximizing the Probability of Journal Article Acceptance By Ron C. Mittelhammer.
 An article review is written for an audience who is knowledgeable in the subject matter instead of a general audience  When writing an article review,
Responding to Reviewers. Rare to get an acceptance with no changes So two paths, rejection or revise and resubmit Rejection Revise and Resubmit.
The Task of the Referee Arnon Rungsawang Massive Information & Knowledge Engineering COmputer and Network SYstem Laboratory Department.
Guide for AWS Reviewers Lois A. Killewich, MD PhD AWS AJS Editorial Board.
Technical Writing: An Editor’s Perspective Michael K. Lindell Hazard Reduction & Recovery Center Texas A&M University.
Scientific Writing Scientific Papers – Original Research Articles “A scientific paper is a written and published report describing original research.
Scientific Peer Review Yixin Chen, Associate Professor Computer & Information Science University of Mississippi April 9, 2013.
How to get a paper published Derek Eamus Department of Environmental Sciences.
How to Really Review Papers CS 8803 AIC. Prvulovic: CORD 2 Paper Reviewing Algorithm Read the paper Think about it Take a look at related work Leave it.
Sept 17, 2007C.Watters 1 Reviewing Published Articles.
A gentle introduction to reviewing research papers Alistair Edwards.
Abstract  An abstract is a concise summary of a larger project (a thesis, research report, performance, service project, etc.) that concisely describes.
How to get your research published.
HR0160 DEVELOPING EMPLOYABILITY AND STUDY SKILLS Thinking Skills: Critical Thinking and Analysis.
Dr.V.Jaiganesh Professor
Starter- Debriefing List the Six parts to a debriefing process.
How to Really Review Papers
CSCD 506 Research Methods for Computer Science
Software Engineering Experimentation
Strategi Memperbaiki dan Menyiapkan Naskah (Manuscript) Hasil Review
Dr John Corbett USP-CAPES International Fellow
Presentation transcript:

Refereeing “And diff’ring judgements serve but to declare, That truth lies somewhere, if we knew but where.” – William Cowper, Hope

Refereeing: What’s in it for you? “Refereeing is … a good way to learn to write better papers; evaluating the work of others gives one insight into one’s own.” (Smith) “Refereeing is … a good way to learn to write better papers; evaluating the work of others gives one insight into one’s own.” (Smith) Guidelines for reviewing are, by implication, guidelines for writing. Guidelines for reviewing are, by implication, guidelines for writing.

What is a Publishable Paper? Sufficient contribution Sufficient contribution New and interesting results New and interesting results Insightful synthesis of existing results Insightful synthesis of existing results Useful survey or tutorial Useful survey or tutorial Small results OK—might spark new research Small results OK—might spark new research But not But not Papers that have major flaws or significant minor flaws Papers that have major flaws or significant minor flaws Papers that are mostly repetitions of other papers Papers that are mostly repetitions of other papers Papers that have good ideas badly expressed (encourage a rewrite) Papers that have good ideas badly expressed (encourage a rewrite)

The Referee Report Summarizes the point of the paper Summarizes the point of the paper Evaluates significance and quality Evaluates significance and quality Gives a recommendation Gives a recommendation Justifies the recommendation Justifies the recommendation

Make Reports Constructive Acceptance Acceptance Persuade editors that the paper should be accepted. Persuade editors that the paper should be accepted. Superficial comments provoke suspicion that the paper has not been read carefully. Superficial comments provoke suspicion that the paper has not been read carefully. Describe changes required to fix residual faults. Describe changes required to fix residual faults. Rejection Rejection Clearly explain why—detail faults. Clearly explain why—detail faults. Explain what the authors might do next. Explain what the authors might do next. Do further work Do further work Submit to a different venue Submit to a different venue Correct major presentation flaws to make the results clear enough to be judged Correct major presentation flaws to make the results clear enough to be judged Consider additional questions, whose answers would make the work worthwhile Consider additional questions, whose answers would make the work worthwhile

Issues in Evaluating a Paper (a checklist) What is the purpose? What is the purpose? (What’s the problem?) Is it appropriate for the venue? Is it appropriate for the venue? Does the paper recognize previous work? Does the paper recognize previous work? Is the goal significant? Is the goal significant? (Why’s the problem a problem?)

Issues in Evaluating a Paper (a checklist …) Does the paper achieve its goal? Does the paper achieve its goal? (What’s the solution?) Is the research method valid and are correct conclusions drawn? Is the research method valid and are correct conclusions drawn? (Why’s the solution a solution?) Is the presentation satisfactory? Is the presentation satisfactory?

Other Issues Timely response Timely response Confidentiality Confidentiality Single-blind reviews Single-blind reviews Misuse of information to which the reviewer is privy Misuse of information to which the reviewer is privy Delicate problem of asking the author to cite the referee’s own work (no easy solution) Delicate problem of asking the author to cite the referee’s own work (no easy solution) Authors’ reputation Authors’ reputation Shouldn’t matter Shouldn’t matter Double-blind reviews Double-blind reviews Conflict of interest Conflict of interest

Other Issues … Reviewers often referee work that is unfamiliar or outside the referee’s domain of expertise. (implications, especially for conferences?) Reviewers often referee work that is unfamiliar or outside the referee’s domain of expertise. (implications, especially for conferences?) Shouldn’t recommend acceptance unless sure of the quality (implications, especially for conferences?) Shouldn’t recommend acceptance unless sure of the quality (implications, especially for conferences?)

Nitpicking on the Writing? Help authors improve their work. Help authors improve their work. Correct typos and suggest other improvements. Correct typos and suggest other improvements. For excessive numbers of errors, give examples and recommend proofreading. For excessive numbers of errors, give examples and recommend proofreading. Give non-native writers help, but also a break. (example) Give non-native writers help, but also a break. (example)example Frequent errors indicate that the author has not been careful—calling even the results into question. Frequent errors indicate that the author has not been careful—calling even the results into question. Must (reluctantly) accept some poorly written work, but results are of no value if the paper is so poor that it cannot be read. Must (reluctantly) accept some poorly written work, but results are of no value if the paper is so poor that it cannot be read.

Receiving Reviews of Your Paper Make use of reviews: One way to get peer feedback on your work One way to get peer feedback on your work When the reviewer didn’t understand—ask why and fix the probable cause When the reviewer didn’t understand—ask why and fix the probable cause When the reviewer makes a mistake—ask why and make it better the next time When the reviewer makes a mistake—ask why and make it better the next time

Your Reputation is at Stake “The [reviewer] forms an opinion of the author.” “The [reviewer] forms an opinion of the author.” “If the quality … [reflects] badly on the author, [the paper] should not even be submitted for publication.” “If the quality … [reflects] badly on the author, [the paper] should not even be submitted for publication.”