Reviewing the Research of Others RIMC Research Capacity Enhancement Workshops Series : “Achieving Research Impact”

Reviewing the research of others The Peer-Reviewing Process (refereeing) – Normally 1-3 reviewers – “Double-blind,” the reviewers and the authors do not know each other – The editor mediates between them – A good review; Summarises the manuscript to show that the reviewer understood it Identifies its strengths and weaknesses Avoid self promotion Advises on how to improve it – How to strengthen weaknesses – Which additional references to read – Publication: Accept unconditionally Conditionally accept subject to modification Reject but encourage revision and resubmission Reject outright

What editors want to see (i) Proper Structure Abstract Introduction Literature Review Theoretical Framework Research Design Methodology Results Analysis Discussion Conclusions References What to look for Capable of standing in for the paper Brief, contextual, problem oriented Comprehensive but succinct Simple, clear, justifiable from the literature Relevant to the theory, practical Straightforward and honest description Concise, complete Make full and logical use of the results Interpretative and justifiable Relate discussion to the problem/objective Complete

What editors want to see (ii) Persuasion Persuasion – many journals reject up to 90% of submissions Logical flow of ideas and arguments Logical flow of ideas and arguments Something new, interesting, counter-intuitive Something new, interesting, counter-intuitive A contribution.. A contribution.. In the context of prior work In the context of prior work References; References; – Relevant – Adequate, not excessive – To the journal you’re submitting to – Recently published (4 years)

Plagiarism Taking someone else’s work as your own Not providing appropriate citation information to indicate authorship correctly. Plagiarism is a serious offence An author who plagiarises is likely to find that; – his/her article is automatically rejected – irrespective of the quality of the work done – he/she is blacklisted from that journal (and other journals) in the future Plagiarism is very easy to detect From Davison, R.M. (2011) Tutorial on Publishing ICT4D Research, Presented at the IFIP WG9.4 Conference, Kathmandu, Nepal, May.Tutorial on Publishing ICT4D Research

Rejecting manuscripts Inappropriate objectives Inappropriate objectives – Unclear, drifting; too many; too ambitious Incomplete/overdone literature review Incomplete/overdone literature review Conclusions do not arise from analysis Conclusions do not arise from analysis No data, no research No data, no research Too much speculation Too much speculation Confusing correlation with causation Confusing correlation with causation Unsuitable length; should be 5-8k words Unsuitable length; should be 5-8k words No story No story Uninteresting, boring Uninteresting, boring Trivial, irrelevant, no problem, done before Trivial, irrelevant, no problem, done before Poorly constructed, weakly argued Poorly constructed, weakly argued Ethical concerns Ethical concerns

A pre-review rejection example 1.The title seems to ask if ebooks are a Disruptive Innovation. There’s actually a good deal of literature on Disruptive Innovations but the manuscript makes no further mention of this concept, so it is unable to answer its own question. 2.There is literature on the Technology Acceptance Model (TAM) which examines the use of ebooks. Not only is the TAM a suitable theoretical framework for this study – in examining the ‘acceptance of e-textbooks’ – it has also been used previously (elsewhere) for this purpose. However, the authors have failed to pick up on this. 3.There is no explanation or reasoning of how the questions in the survey questionnaire were derived. E.g., what constructs/variables where being examined through the use of the questionnaire. There should be some structure to relate the questions to some variables of interest and at least to hypothesise some relationships. The Manuscript: Disruptive Innovation? What Higher Education Students in Ghana Think About E-textbooks.

Typical review form Use this as checklist for manuscripts

10 tips for reviewing 1.Understand your role 1.Understand your role – To evaluate and advise, not critique 2.Do it on time 3.Read manuscripts carefully 4.Find something positive to say 5.Don’t be mean 6.Be brief 7.Don’t nitpick 8.Develop your reviewing style; – Summary, strengths, weaknesses, advise. 9.Make a recommendation 9.Make a recommendation 10.Review unto others as you would have them review unto you. When a young researcher becomes known as an excellent reviewer, he or she may be selected as consulting editor, then associate editor, and then perhaps the primary editor of a journal. Referees, and often editors, are not usually paid.

Issues with peer reviewing The open peer review The open peer review Post-publication reviews Post-publication reviews Timescales Timescales Bias, unaccountable, incomplete Bias, unaccountable, incomplete Not designed to detect fraud Not designed to detect fraud Not full access to the data Not full access to the data Susceptible to control by established elites Susceptible to control by established elites