1. What Works? Evidence-Based Practice

2. Evidence-based Policy and Practice
In recent decades, there has been an increase in emphasis on policy which reflects evidence of what is successful:
David Blunkett, then Secretary of State for Education, argued that social scientists should examine what works and why (DfEE 2000).
Policy-makers goal is: ‘to combine high quality research, feeding into evidence based policy, with clear dissemination of our findings to policy makers in the department and of our interests and priorities to the research community’.
DfES Research Strategy
In the US: ‘the Clinton administration initiated standards-based reforms and pursued a policy of improving the implementation of government programmes. The Bush administration has continued the reforms while also advocating “evidence-based” policy making, especially in education’
OECD 2002:13

3. Evidence-based policy
What exactly is evidence-based policy?
Policy-making informed by the evidence of research?
Policy-making evidenced by confirmatory research?
Research commissioned to find the evidence for policy decisions?
Gorard (2002) argues the danger of research being reduced to the role of ‘legitimating policy’, used ‘to justify already-prepared programmes of action’
In a democracy perhaps we should not be surprised when governments with espoused ideological positions that people have voted for then find research evidence to support that view
But perhaps we might expect more responsible, open-minded government?
A problem with ‘what works’ models: who is asking the question? Who for? Whose purposes does it serve?

4. The Phonics Example
Context: a government who were wedded to a view that phonics is the most important aspect of a pedagogy for reading
Rose Report (2006): who is the author? A man known to support phonics
Clackmannanshire Report: government accurately cite result that phonics has improved word recognition; do not cite result that reading comprehension did not improve.
Torgerson et al (2006) meta-analysis: used as supporting evidence for synthetic phonics; though the report says ‘no statistically significant difference in effectiveness was found between synthetic phonics instruction and analytic phonics instruction’.
Camilli et al (2003) used as evidence: they state ‘phonics, as one aspect of the reading process, should not be over-emphasised’.

5. What Works?
This emphasis on evidence-based policy and practice is often called the ‘what works?’ model;
It privileges certain kinds of research, especially validation of experimental studies and the ‘gold standard’ randomised controlled trial (RCT);
It seeks a single truth: a definitive answer to a particular question;
It is not interested in understanding how or why something works;
It is often called the medical model of research.

6. The Medical Model
Medical research, because of the obvious risks to life, is based on very rigorous research;
Drugs testing is done using randomised controlled trials: very robust: they used randomisation so participants don’t know whether they are receiving the drug or the placebo; large-scale; sophisticated statistics used; and replication required;
Yet mistakes are made (thalidomide), advice changes (take aspirin/don’t take aspirin)
Many drug trials are funded by large pharmaceutical companies: vested interests?

7. Issues with the medical model
In what ways is education like health?
being a student is not a disease
education is not a treatment or a medicine
teachers’ goals are not making students better
teaching a lesson is not the same as administering a drug
learning is not the same as receiving a therapy
Even in medicine, the medical model is questioned – it simplifies the relationship between drug and disease, but often other factors complicate efficacy such as other underlying medical conditions , participants’ lifestyles, and whether patients actually stick to the drug regime given

8. Causal relationships
The medical model is based on a belief in strong causal relationships: x happens to y because of z (eg heart attacks in those with high blood pressure are reduced because of taking aspirin).
Same belief appropriated in an education ‘what works’ model: ‘the most rigorous quantitative study design for evaluating whether or not an intervention based on a causal question is effective is the RCT’ (Torgerson et al 2004:20).
But teaching and learning are complex activities which resist causal explanations: crucially, it is not a two way relationship of patient and drug; it is a multiple relationship of learner, intervention, teacher, class/school context, social context…
Has there ever been any educational strategy which was so universally reliable that you could say ‘This always works’?

9. Causal relationships?
Finding: infants who die a ‘cot-death’ are more likely to be sleeping on second hand mattresses
Conclusion: to avoid cot-death, make sure your baby has a new mattress
OR children in poor families who cannot afford new mattresses are at risk?
Finding: students in small classes in the UK make no more progress (sometimes less) than students in larger classes
Conclusion: small classes have no influence on educational outcomes
Or SEN children are more likely to be in small classes so the research is flawed?
Which cause?
Mosquitoes cause thousands of children to die from malaria each year
Lack of mosquito nets causes thousands of children to die from malaria each year
Poverty causes thousands of children to die from malaria each year

10. Critical problem What does it mean to say ‘what works’?
Education never works the same everywhere and for everyone as it depends on interpretation of meaning in a dialogue ie it is a ‘complex adaptive system’ not a ‘closed system’.
The only thing that is possible is to show ‘what worked’ once under certain circumstances.
Research then does not offer factual relationships between inputs and outputs but it can offer resources to teachers to help them make decisions in unique situations –it does not offer answers but helps us understand the context of the problems better and the options available; it supports professional decision-making.

11. A Question of Values
Education is not ‘indoctrination’ or even ‘training’ , as it involves the ideal of an educated person.
The question of what education is for is not one that can be avoided.
Even when goals are fixed, the educational question is not only ‘what is the most efficient way’ but also ‘what is the most desirable way’ and what is ‘the most educational way?’
The process of becoming educated is as important as the achievement of being educated.
From a research perspective, seeking to inform and understand professional practice through research evidence is important, and methods such as RCTS and experimental designs have an important role to play. But on their own they are not enough to answer the important educational questions we face – these questions are complex and require more diverse research approaches.

12. You and Your Research
The ideas and issues raised in this Powerpoint have direct implications for your research and how you choose to conduct it:
What do you believe your research is for?
Where did your research question/focus come from?
What do you believe about the ‘truth’ of a study?
Who does your research matter to?
What do you want your research to achieve?
How might your research be used?