How to write a review. Outline What is a review? Why should you review? How do you review a paper? What not to do? What are the dilemmas? Case study.

Slides:



Advertisements
Similar presentations
Critical Reading Strategies: Overview of Research Process
Advertisements

How to upgrade from MPhil to PhD Status
Ethics Prof. Toby Walsh NICTA and UNSW. Ethics  Why?  Why should you worry about ethics?  What?  What should you worry about?  How?  How do you.
S3 Useful Expressions.
Reviewing Rejection Top Ten ( The most common reasons I reject papers that I am asked to review ) James Davis UC Santa Cruz 2005.
Submission Process. Overview Preparing for submission The submission process The review process.
WRITING CRITIQUE GROUP GUIDELINES Writing responses to your group members’ work and receiving responses from others is the most important step in revising.
Reviewing Papers: What Reviewers Look For Session 19 C507 Scientific Writing.
CPSC 699. Summary Refereeing is the foundation of academic word: it promotes equity, diversity, openness, free exchange of ideas, and drives the progress.
SENG 531: Labs TA: Brad Cossette Office Hours: Monday, Wednesday.
Experiences from Editing a Journal: Case EJOR Jyrki Wallenius Helsinki School of Economics EJOR Editor Outgoing Editor till June 30, 2005 EJOR.
Looking at Texts from a Reader’s Point of View
ALEC 604: Writing for Professional Publication Week 11: Addressing Reviews/Revisions.
Reading the Literature
Technical Writing Function. The purpose of having guidelines is to make the document more readable. Standard guidelines govern – Format – page layout,
Manuscript Writing and the Peer-Review Process
Peer Review for Addiction Journals Robert L. Balster Editor-in-Chief Drug and Alcohol Dependence.
Structuring an essay. Structuring an Essay: Steps 1. Understand the task 2.Plan and prepare 3.Write the first draft 4.Review the first draft – and if.
CHAPTER 3: DEVELOPING LITERATURE REVIEW SKILLS
Tips for writing well and getting your work published Madhukar Pai, MD, PhD McGill University, Montreal Editorial board member: Lancet Infect Dis PLoS.
The Art of Scientific Writing. Goals of Scientific Writing  Making a clear presentation of a complex scientific problem/accomplishment  Addressing a.
Writing Analytically.
“Knowing Revisited” And that’s how we can move toward really knowing something: Richard Feynman on the Scientific Method.
OB : Building Effective Interviewing Skills Building Effective Interviewing Skills Structure Objectives Basic Design Content Areas Questions Interview.
Easy Chair Online Conference Submission, Tracking and Distribution Process: Getting Started + Information for Reviewers AMS World Marketing Congress /
Evidence Based Medicine Meta-analysis and systematic reviews Ross Lawrenson.
Highlights from Educational Research: Its Nature and Rules of Operation Charles and Mertler (2002)
An Introduction to Empirical Investigations. Aims of the School To provide an advanced treatment of some of the major models, theories and issues in your.
 How to referee. Refereeing is excellent practice for  developing critical appraisal skills  understanding how good (and bad) papers are written 
1 How to review a paper by Fabio Crestani. 2 Disclaimer 4 There is no fixed mechanism for refereeing 4 There are simple rules that help transforming a.
Software Engineering Experimentation Rules for Reviewing Papers Jeff Offutt See my editorials 17(3) and 17(4) in STVR
Grade 8: Nonfiction Reading Research-Based Argument Essay RI.8.1 Cite the evidence that most strongly supports an analysis of what the text says explicitly.
How to read a scientific paper
LEVEL 3 I can identify differences and similarities or changes in different scientific ideas. I can suggest solutions to problems and build models to.
Reviewing Papers© Dr. Ayman Abdel-Hamid, CS5014, Fall CS5014 Research Methods in CS Dr. Ayman Abdel-Hamid Computer Science Department Virginia Tech.
REVIEWING MANUSCRIPTS TIPS FOR REVIEWING MANUSCRIPTS IN PEER REVIEWED JOURNALS Bruce Lubotsky Levin, DrPH, MPH Associate Professor & Head Dept. of Community.
Survey Questions First: Part I A change was made after a few classes to not adhere strictly by the book because I felt it went off on too many tangents.
Open Archive Workshop, CERN th March 2001 Peer Review - the HEP View Mick Draper, CERN ETT Division
 An article review is written for an audience who is knowledgeable in the subject matter instead of a general audience  When writing an article review,
Thomas HeckeleiPublishing and Writing in Agricultural Economics 1 Observations on assignment 4 - Reviews General observations  Good effort! Some even.
Manuscript Review Prepared by Noni MacDonald MD FRCPc Editor-in-Chief Paediatrics and Child Health Former Editor-in -Chief CMAJ
Dealing with Reviews. Rejection hurts, but is it fatal?
Writing the Argumentative/Persuasive Essay. What is an Argumentative Essay? The purpose of an argumentative essay is to persuade the reader to accept—or.
Argument & Counter- Argument Adopted from Baetty Language Centre – Andalas University.
Guide for AWS Reviewers Lois A. Killewich, MD PhD AWS AJS Editorial Board.
Writing Exercise Try to write a short humor piece. It can be fictional or non-fictional. Essay by David Sedaris.
Consensus Validation A Tool for Teams GALA Leadership Symposium October 11, 2013 Presenter: Mindy Taylor.
DISCUSS WORKSHOPS AND PEER EDITING How to get the most out of your Peer Review.
1 To accept or not to accept: The dilemma of an SPC a case study of AAAI-2013 paper Ariel Felner Ben-Gurion University Israel.
Experimental Psychology PSY 433 Chapter 5 Research Reports.
Roadmap for Publication and Maximizing Your Chances for Getting Published Nathan Pickett PhD candidate, Dept. of Geography and Atmospheric Sciences, University.
How To Be A Constructive Reviewer Publish, Not Perish: How To Survive The Peer Review Process Experimental Biology 2010 Anaheim, CA Michael J. Ryan, Ph.D.
How to Really Review Papers CS 8803 AIC. Prvulovic: CORD 2 Paper Reviewing Algorithm Read the paper Think about it Take a look at related work Leave it.
Sept 17, 2007C.Watters 1 Reviewing Published Articles.
A gentle introduction to reviewing research papers Alistair Edwards.
Listening Skills Sue Falkingham Audiologist/Hearing Therapist/RHAD.
3 Simple Steps to Reading Scholarly Articles
Experimental Psychology
The peer review process
How to Get Your Paper Rejected
Peer Reviews Tips for the author.
How to Really Review Papers
Create PT: Complete the Task
Dr. Zhen Jiang Computer Science Department West Chester University
How to Get Your Paper Rejected
Rob Holte University of Alberta
The Process of Getting Published: Reviews and Rejection
Software Engineering Experimentation
How to write good. How to write good Background: Reading Read the papers. Then read them again. Then again. Write out the structure of the paper. If.
Rob Holte University of Alberta
Presentation transcript:

How to write a review

Outline What is a review? Why should you review? How do you review a paper? What not to do? What are the dilemmas? Case study

What is a review? Something that will ruin your day Alan Bundy Even if it is good! The stamp of scientific quality Feedback from your peers Future directions? Prof. Alan Bundy

What is a review not? Acceptance/rejection Editors/Program committees accept/reject You recommend! A place for bias, prejudice, personal animosity, … Though it often appears to be so

Why should you review? Youd much rather enjoy Paphos And so would I! But science would grind to a halt Not immediately, of course

Reasons to review Duty Fairness 2-3 reviews written/ paper written Promotion Education Good reviewers write good papers?

Bad reasons to review To settle old scores To advance your own theories/hinder rivals To get latest results Unpublished papers are strictly confidential

How do you review? Read the paper Read the review form Useful dimensions to look at Novelty, Clarity, Importance, Timeliness … Read the paper Wait a few days Read paper Write review Everyone has their own method

How do you review? Put yourself in authors shoes Think how you would like to read this review Offer constructive criticism Dont just tell them something is inadequate! Tell them how they might fix it

What not to do? Miss the deadline We all hate late reviews Display partiality, bias, animosity, … Destructively criticize Always work out what they would need to do to fix problems

Collect and share reviews Learn if others agree with your opinions Thicken your skin Clearly, the author fails to understand the work of Walsh in this area … Since they mention no related work, this paper cannot be original … This idea is too simple not to exist already … This work is good but I dont understand why Bundy hasnt done this already?

Ethical dilemmas You are working on the same problem Talk to Editor/Program Chair You already reviewed and rejected paper Look for changes

Ethical dilemmas This journal submission already appeared at a conference Does it extend previous appearance? An almost identical paper already appeared Unless it was at a workshop, inform Editor/Chair

Case study Stochastic Constraint Programming By Toby Walsh Be frank, the feedback is good! What do you think?

What did reviewer 1 think? Appears to like it Main criticisms: Relationship to influence diagrams Algorithm performance Phase transition too preliminary The paper reads well. … I have a number of remarks though. First, from a probabilistic reasoning viewpoint, I wonder about the relationship between this framework and influence diagrams (or decision diagrams). It appears to me that what you have defined here is very closely related …Second, from a constraint satisfaction viewpoint: you gave us no indication of how well the different algorithms you presented work in practice. … Third, I think the discussion on phase transition cannot be left at this level. It is not surprising that we have a phase transition here, but what is interesting is the nature of this transition… I think this topic is too serious to sum it up in a small Section … [it] deserves a dedicated and more thorough treatment. I would have preferred to see this space dedicated to experimental results on the performance of presented algorithms

What did reviewer 2 think? Appears to like it Very relevant Moderately significant/original Good readability/English Minimal comments Total of their written comments: It would be nice to include the exact syntax of one SCprogram, as accepted by your system (?), say, for the example of Section 3.

What did reviewer 3 think? Again appears to like it Very relevant, very original, moderately significant Main criticisms Relationship to influence diagrams Phase transition too preliminary Clearly, one could just add constraints to influence diagram representation and extend algorithms to exploit them (my preferred approach) but the approach here is still very valid and could motivate researchers in MDPs and influence diagrams to treat constraints as special creatures … so that their special … algorithms can be exploited. I think the experimental portion of the paper should have been to compare the performance of the algorithms with the performance of traditional MDPs or influence diagram algorithms applied to this class of problems and I speculate that gain can be shown. I dont find the phase transition experiments of much value at this stage. So, there may be a phase transition, so what? I recommend that the author will carefully analyze their model against standard influnce diagrams or factored MDPs and discuss the pros and cons.

What did the IJCAI PC think? Paper was rejected Along with 75% of the other submissions A less good paper (my and reviewers opinions) was accepted! Some compensation $150,000 to be precise IJCAI 2001 logo

Conclusions Reviewing can be rewarding Both to authors and to reviewers Be constructive Think how you would react to the review Take on board your reviews Reviewers hate most being ignored!