Evidence-Based Policy at the Cabinet Office Government Social Research Unit Impact and Insight Seminar Overseas Development Institute 17 October 2005 Evidence-Based Policy at the Cabinet Office Philip Davies PhD Government Social Research Unit Cabinet Office London SW1A 2WH www.gsr.gov.uk

Making Public Policy More Evidence-Based Government Social Research Unit Impact and Insight Seminar Overseas Development Institute 17 October 2005 Making Public Policy More Evidence-Based Philip Davies PhD Government Social Research Unit Cabinet Office London SW1A 2WH www.gsr.gov.uk

Outline Why evidence-based policy? Factors other the evidence Different types of evidence Some Problems Some solutions

Why Make Public Policy More Evidenced-Based? Effectiveness - ensure we do more good than harm Efficiency - use scarce public resources to maximum effect Service Orientation - meet citizen’s needs/expectations Accountability - transparency of what is done and why Democracy - enhance the democratic process Trust - help ensure/restore trust in government and public services

What is Evidence-Based Policy? “Evidence-based policy helps people make well-informed decisions about policies, programmes and projects by putting the best available evidence from research at the heart of policy development and implementation” (Davies, P.T., 1999a)

Factors Influencing Policy Making Experience & Expertise Pragmatics & Contingencies Judgement Lobbyists & Pressure Groups Evidence Resources Values and Policy Context Habits & Tradition

Different Types of Evidence for Policy Experimental Quasi-Experimental Counterfactual Experimental Quasi-Experimental Qualitative Theories of Change Social Ethics Public Consultation Surveys Admin Data Comparative Qualitative Implementation Evidence Ethical Evidence Descriptive Analytical Evidence Cost-Benefit Cost-Effectiveness Cost-Utility Econometrics Impact Evidence Surveys Qualitative Economic and Econometric Evidence Attitudinal Evidence Statistical Modelling Multivariate Analysis

Evidence-Based Policy Opinion-Based Policy Evidence-Based Policy Increasing Pressure (Time) Adapted from: Muir Gray 1997

Problem 1 - Social Scientific Knowledge Not all research is of sufficient quality Unclear objectives Poor research design Methodological weaknesses Inadequate data reporting Selective use of data Unsupported conclusions Uncertainty of scientific knowledge Different status of different fields of knowledge

Evidence-Based Policy Opinion-Based Policy Evidence-Based Policy Increasing Pressure (Time) Adapted from: Muir Gray 1997

Problem 2 - Different Notions of Evidence Policy Makers’ Evidence Researchers’ Evidence Colloquial (Contextual) Anything that seems reasonable Policy relevant Timely Clear Message ‘Scientific’ (Context free) Proven empirically Theoretically driven As long as it takes Caveats and qualifications

Problem 2 – UK Policy Makers’ Types of Evidence Quantitative and statistical evidence Economic evidence Qualitative evidence Surveys and attitudinal evidence Consultation evidence Behavioural evidence Anecdotal evidence Hard and soft evidence International evidence Experimental and Quasi-Experimental Evidence Systematic review/meta-analytical evidence X X

Problem 3 - Getting Research Into the Evidence Chain

Problem 4 – Uncertain Policy Logic/Theory of Change Theories of Change - A Policy Logic Model ‘Scared Straight’ Programmes Programme Theory Programme Evidence

Problem 5 – Inaccessibility of Evidence For Policymakers Research Evidence Is Too: Long Verbose Detailed Dense Impenetrable Jargonesque Methodological Untimely Non-relevant/irrelevant

Some Solutions Integrate and plan research into policy strategically (CRAG) Establish incentives to use evidence (PSG initiative) Establish ownership of the evidence Clarify the policy/practice issues with users ex ante Establish users’ theory of change/logic model Establish answerable questions Establish the policy/practice timetable Knowledge translation (establish key messages clearly) Use appropriate formats for presentation (e.g. 1:3:25) Persistence and Opportunism (Matthew Taylor, 2005)

Contact phil.davies@cabinet-office.x.gsi.gov.uk Government Social Research Unit Cabinet Office Admiralty Arch The Mall London SW1A 2WH England Tel: +44 (0)20 7276 1862 Fax: +44 (0)20 7276 1450 http://www.policyhub.gov.uk