University of Oxford National data – local knowledge Using administrative data David McLennan & Kate Wilkinson Social Disadvantage Research Centre Department of Social Policy and Social Work University of Oxford

Administrative data: a targeting, monitoring and evaluation resource

University of Oxford National Strategy for Neighbourhood Renewal  Aims to ‘narrow the gap’ between the most deprived neighbourhoods and the rest of the country of a range of key outcomes  Five priority themes identified:  Lower Worklessness  Lower Crime  Better Health  Better Skills  Better Housing the Physical Environment

University of Oxford Measuring Outcomes  Establish baseline  Pre- and Post-intervention  NDC v LA v Region v England  Time series  What is success?  Sustainability

University of Oxford Evaluation data sources Census + near 100% coverage of entire population + results reliable at small area level - only every ten years - few suitable indicators of social deprivation - extremely expensive Surveys + clear research focus + many valuable indicators of social deprivation - sampling error - results often not reliable at small area level - very expensive Administrative Data + near 100% coverage of population of interest + constantly updated + results reliable at small area level + already collected for operational purpose - some indicators are proxies - dependent upon support of data providers - data protection

University of Oxford Worklessness and Low Income

University of Oxford Worklessness & Low Income  Worklessness  Worklessness: Unemployment + Work-Limiting Illness  Unemployment: numbers and proportions of people aged receiving Job Seekers Allowance  Work-Limiting Illness: numbers and proportions of people aged receiving Incapacity Benefit or Severe Disablement Allowance  ‘Exit Rates’ from unemployment, illness and overall worklessness  Low Income  Proportion of adults and dependent children (aged 0-59) living in households receiving means-tested out-of-work benefits (Income Support + income-based Job Seekers Allowance)

University of Oxford Work and Pensions Longitudinal Study  Database of all spells of benefit receipt (DWP) and all tax records (HMRC) from June 1999 onwards.  Spells linked together using individual person unique reference number.  Includes details of person’s age, gender, home postcode, number of children, age of youngest child, spell type, spell start and end etc.  Over 164 million records and growing…  DWP’s primary research tool and the source for their neighbourhood statistics data

University of Oxford Worklessness example  An NDC area sees its unemployment rate change as follows:  12% in 2001  10% in 2003  8% in 2005 while the local authority unemployment rate stays the same.  Success?  Change in unemployment rate could be due to: a) unemployed people in the NDC area moving into jobs b) unemployed people in the NDC becoming unable to work due to illness c) new people moving into the NDC area who are not unemployed

University of Oxford Health

University of Oxford Health Indicators  Standardised Mortality Ratio  A measure of the number of deaths in the NDC area compared to the expected level given the area’s age and gender structure  Standardised Illness Ratio  A measure of the prevalence of illness in the NDC area compared to the expected level given the area’s age and gender structure  Mental Illness Rate  proportion of adults under 60 suffering from mood or anxiety disorders in each area  Low Birth Weight  Percentage of single live births classed as low birth weight in a 5 year time period

University of Oxford Standardised Mortality Ratio

University of Oxford Crime

University of Oxford Crime Indicators  Violence Rate  Number of violent crimes per 1000 ‘at-risk’ population  Burglary Rate  Number of burglaries per 1000 ‘at-risk’ properties  Theft Rate  Number of thefts per 1000 ‘at-risk’ population  Criminal Damage Rate  Number of criminal damage crimes per 1000 ‘at-risk’ population  ‘Total Crime’ Rate  Number of violence, burglary, theft and criminal damage crimes per 1000 ‘at-risk’ population

University of Oxford Crime Rate Numerators  Individual level recorded crime from all 39 police forces  Crime type, date/time occurrence, date recorded, grid reference and/or postcode of occurrence  33 different crime types under the broad headings of:  Violence crime  Burglary  Theft  Criminal Damage

University of Oxford

½ a crime to non-NDC area ½ a crime to NDC area

University of Oxford Estimates of properties / population ‘at-risk’  At risk properties = Residential properties + Commercial/industrial properties  At risk population = Resident population (minus prison) + Workers + Students + Shoppers / recreational users + Passers by + etc…

University of Oxford Estimates of properties / population ‘at-risk’  At risk properties = Residential properties Total Dwellings (Census) + Commercial/industrial properties + OS Address Point  At risk population = Resident population (minus prison) Resident Population (estimates) + Workers - Prison Pop + Students + Workplace Pop (Census) + Shoppers / recreational users + Passers by + etc…

University of Oxford Estimates of properties / population ‘at-risk’

University of Oxford Education

University of Oxford Indicators & data sources INDICATORSDATA SOURCES  Pupil attainment at Key Stage 2 (age 11) - % achieving level 4 in English, maths, science  Pupil Level Annual Schools Census (PLASC) – DfES, collected annually, pupil information including home postcode  National Pupil Database (NPD) – DfES, collected annually, pupil test scores  Pupil attainment at Key Stage 3 (age 14), % achieving level 5 in English, maths, science  Pupil attainment at Key Stage 4 – GCSE (age 16), % achieving 5 or more A*-C grades  % pupils staying in full-time education post 16  Child Benefit – HMRC, annual snapshot, counts of children by age, area of residence and gender  % year olds accepted to higher education  Universities and Colleges Admissions Service (UCAS) and Higher Education Statistics Agency (HESA) – collected annually, includes age, outcome of application and postcode

University of Oxford Using the education data  A picture of educational performance and attainment from age to allow comparison: over time; with Government targets; with district, regional and national figures  Cohort tracking – track performance of 2002 KS2 cohort to 2005 KS3  Example – KS4 performance , NDC, LA and Region

University of Oxford Data limitations – some examples  Staying on rate and entry to higher education indicators are proxies not actual measurements  Pupil cohorts can have different characteristics across years  Education indicators may not be comparable with locally sourced statistics or national statistics from different providers:  GCSE indicator dependent on whether pupils are included who left school at 16 and were not entered for any exams  KS2 and KS3 also depend on whether or not pupils who are absent for the test are included in the denominator( for example)  DfES generally produces data at school level rather than area level – all NDC pupils do not go to the same schools…

University of Oxford Coping with data limitations  Look at outcomes across the age range – performance across the range of indicators often varies by NDC  Consider change across a long time-series – data is available from , looking at a longer time period is a better indicator of long-term trends  Use survey data to supplement administrative indicators  Remember that indicator definitions are consistent between areas and over time BUT other factors (i.e. pupil characteristics) can influence performance and these vary over time and between areas In Summary…  Performance across the range of education indicators varies within an NDC  Changes over time are important as well as relative performance – all areas start from a different baseline

University of Oxford NDC variation across education indicators

University of Oxford Housing

University of Oxford Indicators & data sources  Mean price of houses sold by type: flats, terraced, semi-detached, detached, Source: Land Registry  Number of houses sold by type: flats, terraced, semi-detached, detached, Source: Land Registry Uses & limitations  Comparing change in house prices over time and relative to district, region and England However….  No information about type of house i.e. no. of bedrooms so difficult to make accurate comparisons  No information about turnover i.e. rate of house sales – will be available in the future  From current data difficult to draw conclusions about area desirability

University of Oxford Population Estimates

University of Oxford Methodology & data sources  Population counts by age and gender within 5 year age groupings from  Population counts from various administrative data sources:  Child Benefit (0-14)  Patient Registration (0-90+)  Super Older Persons Database (65+)  Only Patient Registration data covers the entire age range and all data sets known to have weaknesses in particular areas and for particular age ranges  Methodology developed to test data reliability and make population estimates based on relative accuracy of each data source  ONS work on producing small area estimates used to supplement and improve our methodology

University of Oxford Uses & limitations  Population change can be an indicator of area desirability….. BUT  Estimates are estimates and may be inaccurate – they rely on the quality of the administrative data  Local authority mid-year estimates have been revised since 2001  Populations may change as a result of housing regeneration

University of Oxford Summary

University of Oxford Administrative data indicators  Useful tool for measuring performance at NDC level because:  Data is collected routinely for operational purposes so enables a consistent time-series to be built up  They enable measurement across a variety of themes so can be linked to specific interventions  They do not suffer from sampling error so reliable at small area level  They can be easily and consistently compared over time and between areas but there are limitations….

University of Oxford Administrative data limitations  There may be differences in definitions, data sources and time collection points between local and national data  National data available from different providers and may use different definitions  Sometimes indicators are proxies or estimates of events or outcomes  Care needed attributing indicator changes to programme interventions Suggestions or comments on indicator packages and data provision are welcome and are best directed through the NDC Reference Group.