1. Peer review – a view from the social sciences
Dr Karim Murji

2. Aims of this session
To provide an overview of reviewing process for academic journals in the social sciences
To look at editorial role and processes
To identify good peer reviewing practice
To highlight benefits of peer reviewing [double blinded]

3. A gold standard in publishing… or a ‘sacred cow’?
Different kinds of ‘peer review’
-pre-publication [improves quality of papers; up to 80% rejected]
post-publication
Have we always had peer review?

6. Why is it important?
Peer-review in academia as a mechanism for assessment – for more than journals only
- Journal articles – and metrics
- Grants (awards and outcomes)
- Books (proposals and full manuscripts)
- REF/impact

7. How peer review is organised
The example of Sociology
The Editors - Are the ‘ultimate’ reviewers/decision makers
Editorial boards
- Ensure editorial accountability
- Provide continuity across editorships
- Maintain quality control
Adjudicate in differences of opinion
Individual [non board] reviewers
- Spread the workload
- Provide specialist expertise
- Have awareness of other national contexts

8. Becoming a reviewer Submit a paper or book review
Contact the editors direct
Get yourself onto an editorial or advisory board
Keep a visible and searchable web presence
Add yourself to the database
Conferences and events

9. Why do it? Professional responsibility/part of scientific community
You become part of the process by which the academy self regulates (and by which academic schools and universities are measured and assessed)
Increases recognition that you have expertise
Personal career development
But also
Can be satisfying and thought-provoking
It improves level of intellectual engagement with all academic writing

10. Seeing an article through the review process
All papers are processed electronically. On submission
1) Administrator checks for length and anonymity
2) An editor then reviews the paper to
ascertain whether it is worth sending out for review
If no, immediate ‘desk reject’
If yes, at least two – sometimes three – referees assigned.
Reviewers allocated 33 days to review; average turnaround to decision is 60 days

11. Getting reviewers
Board members can not decline, but non–board people can – and some/a few do not respond
Reviewer guidelines
Delays in responding to authors almost always due to late reviews; or, rarely, need for adjudication
‘League table’ of workload/average turnaround for board members

12. The requirements of peer review
In Sociology, 4 components:
-A grid for ranking the paper along a number of criteria
- Confidential comments to the editors
- Comments to the author(s)
- Recommendation/decision

16. Reviewer questions Read the manuscript thoroughly and ask yourself:
- Is it right for the journal? Does it fit better somewhere else?
Does it make clear what its main contribution is to sociology? Is it justified in claiming this contribution?
Is this contribution of significance and/or interest?
Is any empirical component sufficiently robust and explicit in method (including ethical considerations)?
Does it support the conclusions?
Is it well-written?

17. Decision categories [Immediate] Accept. This is exceptional/rare
Minor revisions – are they really minor?
Major revisions – the most common response
Are they really major? Or is it really a reject? (almost any paper can be made better)
Are you criticising it for not being the paper you would have written?
Reject

18. Areas to comment on Overall – contribution, argument, persuasiveness
Structure
Context/literature review
Methodology and methods
Data and evidence
Discussion and conclusion
Keep things in proportion
Are the issues/problems fixable?

19. A good review… Length – not too long or too short
Constructive comments
Appreciates the author’s perspective
Being critical – in a helpful way
Rejecting – and not using major revision

20. Poor/bad reviews Too accepting/uncritical Too short
Focus on reviewer’s interests [‘hobby horse’] rather than the paper’s intentions
Dismissive and destructive comments
Give a decision in the review

21. Writing a review Comments to the editors Comments to the author(s)
- In confidence so can be candid here
- Examples of comments received: notification that this has just been reviewed elsewhere; very strong criticisms that might be bruising; warnings of reputational risk to the journal; personal knowledge of author; limits of your expertise; declaration of conflict of position.
Comments to the author(s)
- Must support the recommendation but not declare it
- As much detail as needed, but not obliged to write very long, detailed and justificatory comments (quality of judgement is more important than quantity)
- Do not feel you need to point out typos etc (although do comment on quality of presentation)

22. Editor decision making
Comments and recommendations can be very different in content and outcome
The editors decide on the outcome
Where reviews differ, editors decide how to reconcile
- Make their own judgment, informed by divergent reviews or based on quality of reviewer comments
- Send it out to a third reviewer for extra review or adjudication
Usually the default outcome is to take the more expert/informed/critical review.

23. Re-reviewing a paper
Read the author’s accompanying note, but don’t leave it at that….
Check to see if they have addressed your concerns adequately
Do not feel obliged to accept the paper simply because they have revised it or because the other reviewer likes it.
Do not raise NEW issues that were already present in the original version.
It is quite alright to ask for further major revisions

25. Impact Factor:1.617 | Ranking:Sociology 21 out of 142

26. Is the future open review?
The single-blind model (reviewers unknown to authors) is adopted by most biomedical journals.
Blinding also reviewers (double-blind model) is tricky and does not improve reports' quality.
Open peer review ensures transparency about study reports that may influence clinical practice.
Also the editorial process should be made public, avoiding an excessive focus on reviewers' role.
Vercellini et al, European Jnl of Internal Medicine, April 2016