1. Educational Psychology
Critique Stu’s Tips
These are fairly general and relate to any empirical paper reporting a study
Newman University, 2016

2. Your Target Article this year is …
Pygmalion Revisited: New Evidence for Student Expectancy Effects in the Classroom by Jamieson et al (1987) - Journal of Educational Psychology, Vol. 79, No. 4,

3. Your Target Article 2 I have chosen this article for several reasons:
It is available at Newman – A good start!
It looks at a psychological phenomenon in classroom settings
It has been cited since publication!

4. Caveat & Heuristic
What follows is just a guideline of how I might construct a critique!
Some of the “mechanical” stuff you can look in my schema up can be illuminating
Heuristic Best Advice …Dig around the literature and you might just come across critique of the target article in terms of methods employed or interpretation of data and meaning.

5. Heuristic
The secret to a critique is to be informed about the concepts involved and what the literature suggests.
1 Read paper thoroughly and Jot Down the concepts
2 Look up the concepts .. Using Google Scholar … Pygmalion in the Classroom
3 Use Google Scholar to access papers that have cited the study by Jamieson – There are over 40 that you can discover
4 Read through those that are available – even if just in abstracts to broaden your knowledge get interesting info and see if any have critiqued the paper in any way. Easy to do.

6. Sections (Sub-headings) for critique
Introduction
Summary of article
Critique of article
Conclusions

7. Introduction
What is the general background to the paper? That is, the increasing (over a time period) mismatch between student performance in formal school tests and their actual understanding of some basic concepts.
A very brief foray into the general area. And state that you will be reviewing it!
Perhaps a 100 words

8. Summary
In a few hundred words (no more than 250 here) you need to tell the reader what the paper was about, the questions it asked, the methods used, what the research found and finally how the author interprets these findings in relation to:-
Tip
Use the authors’ own summary as a guide …!

9. Critique
First point. While a critique should go for the jugular, if that is appropriate, it should be balanced.
Here are some things/questions to think about …

10. Critique: Authors Who are? .. Clinicians … educators ….researchers …
Look them up.
Look up their Publication history … quality & quantity in relation to this area.
Authors’ backgrounds are considered by Peer Review … even when it is meant to be anonymous … you can ask for certain experts to be reviewers and people tend to know what each other are up to.

11. Critique: Journal Quality
Peer reviewed or Vanity …JEP is peer reviewed. It is published under the auspices of the APA
Ranking (Impact Rating). Look it up on Google Scholar. Be mindful to compare the journal with ones that are in the same area (e.g. the BJEP) and are directly comparable in terms of readership. Read what impact factor means.
Use the impact factor of the journal (there is not one per se for the article) for 1987 – when the paper was published
Related to field … academic or practitioner
In this case it is rarely read by teachers … so mostly researchers. Read the blurb the JEP writes about itself

12. Critique: Funding Who by?
Alcohol or drug companies fund research that looks as if it will help them … a form of lobbying. An article funded by Bell’s Whisky saying that drinking Scotch is good for you might be biased.
This paper appears to be unfunded … but as it is a type of review itself … albeit with analyses it did not need much by way of funding …….

13. Critique: Presentation of Paper
Does it look right in terms of conventions – not always a bad thing – utility of communication format. Is it in readable English. Important here as the authors are Dutch … Double Dutch … use Pidgin language!
Do you understand it?
Jargon-ridden … This pertain to the Intro and Discussion mostly

14. Critique: Underpinning theory of paper
The theoretical model presented in the target article
Pygmalion in the classroom … It is a known phenomenon but is quite complex

15. Critique: Hypotheses Are they sensible? Do they follow/drop out of
1 The underpinning theory
2 The introduction of the paper
Can they be falsified?
Good science.

16. Critique: Methods Employed
The Method - Participants
Reliability
Payment to complete study
Attrition rates
Evidence gathered in addition to self reports, etc
The Method – Design
Controls and checks used … for confounding variables, etc.
The Method – Procedure
Was it the same for all participants
Could you replicate the study
Ethics … considered explicitly

17. Critique: The Results
Does it use standard analytical procedures … qualitative or quantitative?
Are they clear and statistically significant … does the data appear tampered with .. interpolations, etc?
Could the data have been analysed more fully/more appropriately?

18. Critique: The Discussion
Does it address fully the questions raised in introduction?
Are the authors’ interpretations of meaning of data fair?
Are there other interpretations of the data?

19. Conclusion: Contribution of paper to science … practise
Has it be cited since? Google Scholar again
Has it been criticised/praised? Read around
Has it affected theory or clinical practise Read around
Has their recommendation led to further work Read around

20. Conclusion: Final commentary
BE BALANCED & DON’T BE AFRAID TO ADD MATERIAL NOT CONSIDERED IN MY LITTLE CHECKLIST!
Cats have 9 lives and can be skinned in lots of ways …

21. That’s All Folks!