I NTERVENTIONS ARE N ECESSARY BUT N OT S UFFICIENT Peter Tymms

Outline A quick look backwards The need for interventions Two recent eye openers Alternative perspectives Conclusions – For teaching – For policy making

Historically Education was an evidence free zone Experience, opinion and good ideas ruled Two example examples – Kenneth Baker – Estelle Morris

Interview of Lord Baker with Giles Dinot on 18 February 2011 Q:One criticism was that the National Curriculum just “taught to test”; is there a problem with that? A:I went to a Church of England primary school during the war in Southport. I’ve still got the little report books, and there they are, marks out of 100. I was quite clever in those days. I was being tested every term. And I took it home and showed my mother and father.

Estelle Morris: Education Secretary When in office she did not base any action on research but took considerable notice of the Daily Mail

The need for interventions Consider – £500 million on the National Literacy Strategy with no detectable impact – Etc etc etc World wide recognition of the issue – Campbell Collaboration – USA What Works Clearing House Slavin’s Best Evidence – John Hattie – Tool kit – CEM, Durham

Meta-analysis Example: use of digital technology to increase literacy in middle schools. (2005)

Forest Plot

Points to note Overall: ES=0.49 Variation of impact of interventions Need to know more …. But we now have a best estimate

First eye opener Slavin et al 2014 Cooperative Learning in mathematics has been shown to be effective in the USA Tried a large RCT in England – And failed “most surprising” Tried again with a revitalised approach – And failed again

Second eye opening Lemons et al 2014 Five RCTs of the efficacy of KG Peer-Assisted Learning on reading program. Over 9 years involving 2,591 students. Early results very positive and widely reported Most recent results – Failed Team: “shocked and depressed”

Explanations Slavin et al – “Differences between traditional teaching practices in England and in North America” – “Teaching methods proven to be effective in one culture and system cannot be assumed to be effective in another.” Lemons et al – “The changed context”. The bar had been raised in KG. – “the change agent - a no-nonsense Chief Instructional Officer” – The controls were different

A Further Complication We live in a complex interactive world. Even deterministic worlds are not predictable – But patterns appear (strange attractors) – Error propagation is the cause and it is summed up in the Dynamic linear model.

The nature of teaching Great teaching involves – Planning – The ability to ditch Plan A and go to Plan B – Dealing with idiosyncratic children classes Integrating professional judgement with scientific evidence The bottom line – Academic progress – Positive non-cognitive progress

Some features of policy making Making big decisions. John Maynard Keynes – "When the facts change, I change my mind. What do you do, sir?" But U-turns are bad news – “The lady’s not for turning” – Are “Reforms as Experiments” conceivable? Time scales are long

Propositions Teachers: – must have up-to-date evidence – must interact and collect feedback continuously – must not be held to account for processes – progress is the bottom line Policy makers should: – accumulate & generate evidence – run pilot projects for proposed policy initiatives – spend large sums only when with evidence – must monitor – be held to account for consequences