What works? How can we draw lessons for practice, from high quality research? Tim Oates Group Director Assessment Research & Development

Any questions?

The first issue Defining what the issue is Using secondary research – judgements of utility and quality Undertaking primary research – participating – the ‘triad’ concept Undertaking primary research – initiating – ‘high barriers to entry’

Policy borrowing and cherry picking – international studies Learning everything, learning nothing, learning something Policy learning, not policy borrowing

Control factors 1curriculum content (nc specifications, support materials, etc) 2assessment and qualifications 3national framework for qualifications 4inspection 5pedagogy 6professional development 7institutional development 8institutional forms and structures (eg size of schools, education phases) 9allied social measures (linking social care, health care and education) 10funding 11governance (autonomy versus direct control) 12accountability arrangements 13labour market/professional licensing 14allied market regulation (eg health and safety legislation; insurance regulation) Oates T 2010 Could do better: using international comparisons to refine the National Curriculum in England Cambridge Assessment

Defining the nature of the problem – the humble literature search Identifying causal mechanisms – understanding theory Effect size – measurement (role of assessment) Ethics of intervention – ‘do no harm’ Foregone opportunity The nature of observation, explanation and action in complex social systems

A responsible approach to transnational comparisons (note not identical to control factors) Character and structure of the National Curriculum (size, focus, underpinning theory, and design principles) Curriculum coherence – content, pedagogy, assessment, support materials, drivers and incentives Teacher quality and teaching quality Demarcations between national requirement and school autonomy – locus of control School ethos and balance between National Curriculum elements, taught elements and untaught elements Family culture and national social culture General attitude to innovation Investment in education

Policy borrowing and cherry picking – international studies Curriculum coherence Curriculum control Ceteris Paribus Finland – the role of ideas in making a system work

cherry-picked myth Singapore Small – correct - 5.1m 2011 (world bank) Homogenous society – incorrect - 25% ethnic minority Low social inequality – incorrect – Finland 26.9 UK 36.0 Singapore 42.5 (GINI co-efficient) High levels of purchased tutoring – incorrect, counterfactual Authoritarian State – incorrect, failure of western educational researchers to deploy political analysis Human development index UK 28 Singapore 26 (HDI World Bank)

cherry-picked myth ‘suicide rates in Asian countries show how they have got education wrong…’ Wasserman D, Qi Cheng Qi, Gui-Xin Jiang World Psychiatry (1) 39 Global suicide rates among young people aged Lithuania 23.9 per 100,000 rank 2 New Zealand 15.3 rank 6 Norway 10.9 rank 14 Austria 9.6 rank 20 Finland 9.5 rank 22 USA 8.0 rank 34 Australia 8.9 rank 28 Singapore 8.5 rank 29 Japan 6.4 rank 39 Hong Kong 5.2 rank 50 Netherlands 4.6 rank 56 Sweden 4.3 rank 57 UK 4.2 rank 60 Rates corroborated by examination of press reports and official statistics for Culturally-determined ‘cause of death’ reporting and press reporting

EPPE – ecertsE ecertsR The importance of measurement Triangulation

AfL Inside the Black Box – high utility and quality Medway and Oxford – overdetermination by accountability?

Class size Thousands of studies Harvey Goldstein STAR study Is class size really the issue? Russell Hobby

Learning styles – Coffield et al critique Emotional intelligence – Immordino-Yang critique Text forms – Oates critique Conceptual misunderstanding – inside the pupils’ head Pedagogic quality – is any different happening?

Overdetermination Outside factors – CASE and ‘I think about science’ Behaviour and motivation Professional conflicts

Setting and streaming versus not Many different models – Rachel Marx It’s not yes or no, but what you put in place to ameliorate inevitable problems – eg allocating best teachers, permeability, models of ability

The first issue Defining what the issue is Using secondary research – judgements of utility and quality Undertaking primary research – participating – the ‘triad’ concept Undertaking primary research – initiating – ‘high barriers to entry’

Defining the nature of the problem – the humble literature search Identifying causal mechanisms – understanding theory Effect size – measurement (role of assessment) Ethics of intervention – ‘do no harm’ Foregone opportunity Collaboration – support Cycle 1 Design – execution – dissemination/implementation/impact Cycle 2 – measurement, evaluation The nature of observation, explanation and action in complex social systems Context is critical – interaction of control factors, explanatory factors