1. Prime Ministers Strategy Unit Rapid Evidence Assessments Philip Davies PhD Government Chief Social Researchers Office Prime Ministers Strategy Unit Cabinet Office London SW1A 2WH DfES Seminar on Rapid Reviews 17 November 2004

2. Prime Ministers Strategy Unit Systematic reviews of existing evidence Timed to meet the needs of policy makers/practitioners Strategically using the three arms of systematic searching Rapid Evidence Assessments - What Are They? Electronic sources Print sources Grey Literature Critical appraisal of studies identified Summary of findings, with caveats and qualifications Tentative findings Interim Evidence Assessments

3. Prime Ministers Strategy Unit Rapid Evidence Assessments - Searching

4. Prime Ministers Strategy Unit Rapid Evidence Assessments - Critical Appraisal

5. Prime Ministers Strategy Unit Rapid Evidence Assessments - Advantages Provides sounder evidence than selective literature reviews Better than opinion-based policy Provides a challenge function to received wisdoms Challenges and strengthens a policys theory of change Provides more precise estimates of likely outcomes/effects Provides valid and reliable evidence on implementation Transparent strengths and weaknesses of evidence-base

6. Prime Ministers Strategy Unit Rapid Evidence Assessments - Limitations Usually not comprehensive use of existing literature Introduces selection and publication biases Can misrepresent the totality of evidence Can lead to Type I and Type II errors Need to be continued to produce full-blown systematic reviews Hence Interim Evidence Assessments to be preferred

7. Prime Ministers Strategy Unit phil.davies@cabinet-office.x.gsi.gov.uk Government Chief Social Researchers Office Prime Ministers Strategy Unit Cabinet Office Admiralty Arch The Mall London SW1A 2WH England Tel: +44 (0)20 7276 1862 Fax: +44 (0)20 7276 1450 Contact www.policyhub.gov.uk