1. Critiquing research reports
PRM
Lecture 3

2. "If it's in print, it must be true."
Or must it?

3. How well was the research done? and, How well has it been reported?
2 questions
How well was the research done? and, How well has it been reported?

4. Evaluating the research
Positive as well as negative
Was there a well defined and understandable purpose?
Were the data gathered properly and were they the best for the purpose?
Were the data suitably studied and analyzed?
Were the reported results based strictly on the outcome of the study or analysis of the data?
Does the investigator's interpretation of the results make sense?

5. Evaluating the Reporting of the Research: Beginnings
An author's prime duty is to communicate to the reader
Has the author said all that needs to be said?
Does it do what it says on the tin?
Setting the scene
Goals
Definitions

6. Evaluating the Reporting of the Research: Content
Is it clear what data were used and why?
How was the data collected, and why?
Is this analysis explained and appropriate?
Is the results section clearly delineated?
Display
Clarity of communication

7. Evaluating the Reporting of the Research: Endings
Does the author summarise the study?
Objectivity
Recommendations
Appendices (not often used in journal articles)
Presentation – order, written style, avoidance of discriminatory (esp. sexist) language

8. Ethics Appropriately reviewed? Harm, deceptions, informed consent
Safeguarding
Benefits vs. Costs

9. Tips
Be sure to comment on the study's strengths as well as its limitations. The critique should be a balanced consideration of the worth of the research. Each research report has some positive features. Be sure to find them and note them.
Give specific examples of the study's strengths and limitations. Avoid vague generalizations of praise and fault finding.
Try to justify your criticisms. Offer a rationale for how a different approach would have solved a problem that the researcher failed to address.
Be as objective as possible. Try to avoid being overly critical of a study because you are not particularly interested in a topic or because you have a world view that is inconsistent with the underlying paradigm.
Perhaps suggest realistic alternatives that the researcher (or future researchers) might have considered. Don't just identify problems--offer some recommended solutions, making sure that the recommendations are practical ones.
Evaluate all aspects of the study--its substantive, theoretical, methodological, ethical, interpretive, and presentational dimensions.
Don’t be rude about other people’s work!

10. Checklist