Educational Psychology Critique Stu’s Tips These are fairly general and relate to any empirical paper reporting a study Newman University, 2016

Your Target Article this year is … Pygmalion Revisited: New Evidence for Student Expectancy Effects in the Classroom by Jamieson et al (1987) - Journal of Educational Psychology, Vol. 79, No. 4,461-466

Your Target Article 2 I have chosen this article for several reasons: It is available at Newman – A good start! It looks at a psychological phenomenon in classroom settings It has been cited since publication!

Caveat & Heuristic What follows is just a guideline of how I might construct a critique! Some of the “mechanical” stuff you can look in my schema up can be illuminating Heuristic Best Advice …Dig around the literature and you might just come across critique of the target article in terms of methods employed or interpretation of data and meaning.

Heuristic The secret to a critique is to be informed about the concepts involved and what the literature suggests. 1 Read paper thoroughly and Jot Down the concepts 2 Look up the concepts .. Using Google Scholar … Pygmalion in the Classroom 3 Use Google Scholar to access papers that have cited the study by Jamieson – There are over 40 that you can discover 4 Read through those that are available – even if just in abstracts to broaden your knowledge get interesting info and see if any have critiqued the paper in any way. Easy to do.

Sections (Sub-headings) for critique Introduction Summary of article Critique of article Conclusions

Introduction What is the general background to the paper? That is, the increasing (over a time period) mismatch between student performance in formal school tests and their actual understanding of some basic concepts. A very brief foray into the general area. And state that you will be reviewing it! Perhaps a 100 words

Summary In a few hundred words (no more than 250 here) you need to tell the reader what the paper was about, the questions it asked, the methods used, what the research found and finally how the author interprets these findings in relation to:- Tip Use the authors’ own summary as a guide …!

Critique First point. While a critique should go for the jugular, if that is appropriate, it should be balanced. Here are some things/questions to think about …

Critique: Authors Who are? .. Clinicians … educators ….researchers … Look them up. Look up their Publication history … quality & quantity in relation to this area. Authors’ backgrounds are considered by Peer Review … even when it is meant to be anonymous … you can ask for certain experts to be reviewers and people tend to know what each other are up to.

Critique: Journal Quality Peer reviewed or Vanity …JEP is peer reviewed. It is published under the auspices of the APA Ranking (Impact Rating). Look it up on Google Scholar. Be mindful to compare the journal with ones that are in the same area (e.g. the BJEP) and are directly comparable in terms of readership. Read what impact factor means. Use the impact factor of the journal (there is not one per se for the article) for 1987 – when the paper was published Related to field … academic or practitioner In this case it is rarely read by teachers … so mostly researchers. Read the blurb the JEP writes about itself

Critique: Funding Who by? Alcohol or drug companies fund research that looks as if it will help them … a form of lobbying. An article funded by Bell’s Whisky saying that drinking Scotch is good for you might be biased. This paper appears to be unfunded … but as it is a type of review itself … albeit with analyses it did not need much by way of funding …….

Critique: Presentation of Paper Does it look right in terms of conventions – not always a bad thing – utility of communication format. Is it in readable English. Important here as the authors are Dutch … Double Dutch … use Pidgin language! Do you understand it? Jargon-ridden … This pertain to the Intro and Discussion mostly

Critique: Underpinning theory of paper The theoretical model presented in the target article Pygmalion in the classroom … It is a known phenomenon but is quite complex

Critique: Hypotheses Are they sensible? Do they follow/drop out of 1 The underpinning theory 2 The introduction of the paper Can they be falsified? Good science.

Critique: Methods Employed The Method - Participants Reliability Payment to complete study Attrition rates Evidence gathered in addition to self reports, etc The Method – Design Controls and checks used … for confounding variables, etc. The Method – Procedure Was it the same for all participants Could you replicate the study Ethics … considered explicitly

Critique: The Results Does it use standard analytical procedures … qualitative or quantitative? Are they clear and statistically significant … does the data appear tampered with .. interpolations, etc? Could the data have been analysed more fully/more appropriately?

Critique: The Discussion Does it address fully the questions raised in introduction? Are the authors’ interpretations of meaning of data fair? Are there other interpretations of the data?

Conclusion: Contribution of paper to science … practise Has it be cited since? Google Scholar again Has it been criticised/praised? Read around Has it affected theory or clinical practise Read around Has their recommendation led to further work Read around

Conclusion: Final commentary BE BALANCED & DON’T BE AFRAID TO ADD MATERIAL NOT CONSIDERED IN MY LITTLE CHECKLIST! Cats have 9 lives and can be skinned in lots of ways …

That’s All Folks!