1. Linda Croxford Centre for Educational Sociology University of Edinburgh Governing by numbers - performance management in school education

2. About the research “Governing by Numbers” study of Quality Assurance & Evaluation (QAE) in England and Scotland funded by ESRC part of a wider study with Denmark, Finland and Sweden entitled “Fabricating Quality in European Education” funded by the European Science Foundation (ESF) This presentation focuses on England, where performance management has taken its most extreme form

3. Overview of presentation 1.Role of numbers in governing education in England 2.Massive growth in production and flow of quantitative data on pupils and schools 3.Privatisation of data production services 4.Effects on pupils, teachers and schools 5.The “data dream” versus issues of data quality

4. “Our data are used to monitor and drive up performance, increasing accountability at school, local and national levels, and providing a basis for our partners to make informed decisions.” Department for Children, Schools & Families (DCSF) Analysis and Evidence Strategy 2009-10: Delivering evidence-based policy (July 2009) The role of numbers in governing education

5. Accountability in England l National Curriculum l National Testing (standard assessment tests or SATs) l Publication of results at school level -> League Tables l Parental choice informed by League Tables l Inspection by Ofsted – linked to audit framework l Targets for school improvement l Appraisal of teachers’ performance

6.  Steered by inspection framework - between 1997- 2004 all schools inspected at least twice – “Naming and shaming”  “Intelligent Accountability” ( Since 2004) starts with formal self-evaluation by schools - producing data to constantly audit performance and set targets  Inspections - at short notice - start with evidence from self-evaluation School data production and analysis

7.  Pupil-level data - characteristics, attendance, behaviour, attainment - passed to government agencies  Raise Online - “Reporting and Analysis for Improvement through School Self-Evaluation” - created by Ofsted and DCFS – to collect and analyse data on every pupil  Data feed National Pupil Database Data flow

8. “where we corral all of this data and information and at a glance now across all local authorities in England - you can go downstairs and look at a big screen and you can look across all the key performance areas - and that’s actually across all the social care areas as well as education. So at that level we’re doing quite active performance management of the system and that’s quite a powerful tool” (Interview with a Senior Official – DCSF) The National Pupil Database

9.  Data production is big business – testing, database systems, statistical packages, computer systems  Belief that private enterprise provides sophisticated systems, efficiency, transparency, cost cutting  But some high-profile problems eg SATS 2008  Lack of continuity, inconsistencies and unwillingness to take responsibilities for problems Privatisation of data production processes

10. Other research shows  League Tables and parental choice are associated with increased polarisation of school intakes  Increased problems for schools serving areas of deprivation  School practices such as ability grouping-> internal segregation  “Manipulation of performance”  eg “Teaching to the Test”, disproportionate focus on pupils at borderline of targets eg 5+A-C @GCSE, Unforeseen effects

11. We find that the use of national test results for the purpose of school accountability has resulted in some schools emphasising the maximisation of test results at the expense of a more rounded education for their pupils. A variety of classroom practices aimed at improving test results has distorted the education of some children, which may leave them unprepared for higher education and employment. House of Commons Select Committee 2008 (1)

12. We find that ‘teaching to the test’ and narrowing of the taught curriculum are widespread phenomena in schools... Tests, however, can only test a limited range of the skills and activities which are properly part of a rounded education, so that a focus on improving test results compromises teachers’ creativity in the classroom and children’s access to a balanced curriculum. House of Commons Select Committee 2008 (2)

13. ‘Because we have all this Key Stage Data and because it is longitudinal, we are practically, without boasting, we are probably the leading administration in the world as far as value-added measures and schooling are concerned. (Interview with DCSF Senior analyst) “Data dream” versus data quality

14.  On-going problems with national tests  Upward trends in attainment – countered by claims of exams getting easier (eg A-levels)  High staked ->manipulation of test results to improve “performance”  Comparison of school performance undermined by inadequate measures of school context Issues that raise questions about data quality

15.  We have argued that data have become central to education policy  Government places a great deal of trust in numbers  But data have their limits, and don’t by themselves offer solutions to policy problems.  The effort to collect more and better data places considerable pressure on the education system, and is very costly  What can RSS do to put data back in its “proper” place? Finally

16. Further details of this ESF/ESRC project Fabricating Quality in European Education/Governing by Numbers at http://www.ces.ed.ac.uk/research/FabQ/index.htm