Validation, editing and imputation of the Family Resources Survey John Snow (DWP) Family Resources Survey ESDS Government FRS User Meeting April 2008.

Slides:

Advertisements

Similar presentations

Families and Children Study (FACS) Aims, coverage and methodology Figen Deviren Department for Work and Pensions.

Advertisements

An Introduction to the UK Data Archive and the Economic and Social Data Service November 2007 Jack Kneeshaw, UKDA.

The English Longitudinal Study of Ageing (ELSA) Data & Documentation 2008 Jibby Medina NatCen.

An Introduction to the Families and Children Study (FACS)

Research into an alternative sampling frame for the FRS Antonia Simon, Development Team, DWP.

1 Developments to the Family Resources Survey Family Resources Survey (FRS) and the EU-Statistics on Income and Living Conditions (EU-SILC) Valerie Christian.

Measuring Child Poverty Using the FRS Laura Adelman Family Poverty and Work Division.

Nutrition and consumption data from the EFS By Clare Burgon Defra Food Statistics Team

Expenditure & Food Survey – datasets and user guides Steven Dunstan EFS Senior Research Officer.

Households Below Average Income 23 rd April 2007 Presentation at FRS User Group Meeting By Nick Herbert - DWP.

Outline of talk The ONS surveys Why should we weight?

Use of FRS data in Pension Trends David Miller April 2009.

The General Lifestyle Module/Survey (The General Household Survey) Steven Dunstan, Survey Manager, ONS Riaz Ali, Senior Research Officer, ONS Office for.

Looking forward to the 2006/07 HBAI publication: New analyses and improvements Peter Matejic (DWP) Households Below Average Income ESDS Government FRS.

THE BRITISH CRIME SURVEY: PRESENTATION TO THE BCS USER GROUP 11 December 2007 Progress with data archiving Last meeting we described our review of current.

ESDS Using working with surveys: v.10/07 1 Further Applications of Linking and matching Anthony Rafferty & Jo Wathan Economic and Social Data Service (Government.

IHS: Requirements for Secondary Analysts Jo Wathan ESDS Government University of Manchester.

Integrated Household Survey An overview and update Neil Bannister and Gareth Powell Social Survey Division ONS.

Using LFS Longitudinal & Household Datasets Marilyn Thomas & William Barnes Office for National Statistics.

Family Resources Survey Data Collection Methods Jo Maher (National Centre for Social Research) Tom Howe (Office for National Statistics)

Households Below Average Income 2008/09

Summary overview of the Overseas Trade Statistics (OTS) Published by: HM Revenue & Customs – Trade Statistics Cust. Services:

1 Department for Work and Pensions, UK Replacement Rates in Pensim2 Stewart Butler Pensim2 Team Model Development Unit Department for Work and Pensions.

Household Projections for England Yolanda Ruiz DCLG 16 th July 2012.

Scottish Executive Large Scale Survey Developments Dr Robert Williams Analytical Services Group Office of the Chief Statistician.

© Institute for Fiscal Studies, 2007 Government Financial Support for Children across the United Kingdom: how does Northern Ireland compare? Stuart Adam.

Family Resources Survey Data Linking Jo Cockerham.

Access routes to 2001 UK Census Microdata: Issues and Solutions Jo Wathan SARs support Unit, CCSR University of Manchester, UK

Dr Veronique Siegler Office for National Statistics, Measuring National Well-being

Census.ac.uk Census Area Statistics and Casweb David Rawnsley Census Dissemination Unit (CDU) Mimas University of Manchester.

Beyond 2011 – A new paradigm for population statistics? Pete Benton, Beyond 2011 Programme Director Office for National Statistics, UK.

Household projections for Scotland Hugh Mackenzie April 2014.

What’s new in the Child Poverty Unit – Research and Measurement Team Research and Measurement Team Child Poverty Unit.

 Access & Dissemination of Data from the SHS Lisa Taylor SHS Research Officer.

UK Data Warehouse Work 23 rd May 2012 Paul Tutton, Sarah Ravenhill.

Effects of Income Imputation on Traditional Poverty Estimates The views expressed here are the authors and do not represent the official positions.

MEASURING INCOME AND POVERTY AT A NATIONAL LEVEL Sian Rasdale Social Justice Analysis, Scottish Government.

Scotland’s Labour Market – Latest Developments Denise Patrick Lifelong Learning Analytical Services Division 25 th May 2010.

UNITE and the benefit cap: building a new data platform Daniel Brown & William Baxter - DWP.

Better Information for Regional Government Marie Cruddas, Minda Phillips & Pete Brodie, ONS. Presented by Martin Brand, ONS Methodology Directorate.

Response to UKSA Monitoring Review on Income and Earnings statistics Joint response from ONS, DWP, HMRC July

Using IPUMS.org Katie Genadek Minnesota Population Center University of Minnesota The IPUMS projects are funded by the National Science.

Record matching for census purposes in the Netherlands Eric Schulte Nordholt Senior researcher and project leader of the Census Statistics Netherlands.

Plans for Access to UK Microdata from 2011 Census Emma White Office for National Statistics 24 May 2012.

Understanding Wales: Opportunities for Secondary Data Analysis Annual Population Survey/Labour Force Survey Melanie Jones School of Business.

Combining survey and administrative data to create a new input data file for National Accounts processes Shaun McLaughlin Central Statistics Office, Ireland.

The application of selective editing to the ONS Monthly Business Survey Emma Hooper Office for National Statistics

Scottish Household Survey Jonathan Waite SHS Research Officer Communities Analytical Services.

Additional analysis of poverty in Scotland 2013/14 Communities Analytical Services July 2015.

1 Data Linkage for Educational Research Royal Statistical Society March 19th 2007 Andrew Jenkins and Rosalind Levačić Institute of Education, University.

ELSA ELSA datasets and documentation available from the archive or by special arrangement Kate Cox National Centre for Social.

Jump to first page (o ns) Modernising Statistical Systems to improve Quality The experiences of the Office for National Statistics (ONS) Presented by Emma.

An overview 9 th September 2009 Families with Children Projection Model: From ONS Projections to DWP Policy.

General Register Office for S C O T L A N D information about Scotland's people Comparison between NHSCR and Community health index sources of migration.

Developing the income data in the Scottish Household Survey Chris Martin, Associate Director, Ipsos MORI Income and Poverty Statistics Seminar 11 th June.

The 2011 Census: Estimating the Population Alexa Courtney.

2009 Survey of Disability, Ageing and Carers (SDAC) – emerging data Presentation to Carers NSW Biennial Conference 17 March 2011 Steve Gelsi Assistant.

Keeping Children Safe Summer School.... Pathways to information 15 th September 2011

Workshop on MDG, Bangkok, Jan.2009 MDG 3.2: Share of women in wage employment in the non-agricultural sector National and global data.

An Editing Procedure for Low Pay Data Salah Merad, Mike Hidiroglou and Fiona Crawford Office for National Statistics, UK Survey Methods Division.

Using official data-sources Tom Spencer, Social Justice Analysis.

Information Directorate DWP Small Area Data NERIP Event – Version 1.

Evaluating imputation of sex and age for substitutes in substitute households Michael Ryan 2008 UNECE Work Session on Statistical Data Editing.

Implementation of Quality indicators for administrative data

Deriving a reliable measure of household income – DWP

The Indices of Deprivation Poverty and deprivation: Statistics for Action – Places: What are the local/ devolved administration/ UK needs? 10 February.

Transforming household finance statistics in the UK

Micro-simulating child poverty in 2010 and 2020: an update

Metadata used throughout statistics production

Quick statistics - how to deal with quality?

Presentation transcript:

Validation, editing and imputation of the Family Resources Survey John Snow (DWP) Family Resources Survey ESDS Government FRS User Meeting April 2008

Overview of FRS Processing ONS/NatCen Validation Editing the data Imputation Grossing Data release Family Resources Survey Derived Variables

Validation - Hierarchical Data Structure Family Resources Survey

Validating the data Data Validation – Initial checks (Year start) Documentation: metadata / One True Version (OTV) Check all new variables Check changes to variables (eg labels) Additional changes (eg routing) Family Resources Survey

Validating the data Data Validation – monthly checks Record creation General validation Set Type checks Skipped key variables Skipped important variables Family Resources Survey

Editing the data 1 Benefit Editing Near Zero Amounts Near identical Single / Multiple Benefits Combined Benefits Family Resources Survey

Editing the data 2 Pre-imputation editing Outliers – now run by ONS – 6 monthly Credibility Checks – now run by ONS Weekly amounts (Period codes) Post-imputation & DVs editing Credibility Checks – run by DWP Family Resources Survey

Imputation What is Imputation? To estimate missing values on item non response Why / when do we impute? How to impute, and checks? Algorithms Hotdecking Bias Family Resources Survey

Derived Variables What are Derived Variables (DVs)? Customised variables (Note: some created by ONS (eg DVMARDF) Created on data users request Used to carry out consistent analysis Regularly reviewed Family Resources Survey

Grossing What is Grossing? A factor which is applied to the sample data to give estimates which represent overall population. Control totals are applied, for males and females by age-band, by region Other control totals are number of lone parents, tenure type and Council Tax band Family Resources Survey

Releasing the data Within DWP: FRS Main Dataset Across analytical directorate Outside DWP: FRS Anonymised Dataset - Other Government Departments - Researchers via the UK Data Archive Family Resources Survey

Terms & Conditions for Using the FRS The dataset will be used for secondary analysis in the form of tabulations and statistical modelling - Outputs will not be produced that are likely to identify an individual -Regional results should not to be published below Government Office Region (GOR) If in doubt please contact the FRS team at DWP Family Resources Survey

Annual FRS Publication The Annual FRS Publication latest ; released on 27 March 2007 available online Family Resources Survey

Outside Government Benefit Take-Up Households Below Average Income UK Data Archive New Policy Institute Government DWP Pensioner Incomes Policy Simulation HMT HMRC DTI Scottish Executive Disability and Carers WWEG, PCD… Pensions Commission Defra DfES Europe : LIS Institute of Fiscal Studies The FRS Community Northern Ireland, DSD Family Resources Survey Family Poverty & Work ONS

For further help… IFD IM1 Phone Maxine Willitts(Team Leader: General, Data Release, report) (020) Jeremy Barton (Harmonisation, Methodology, Analysis)(020) John Snow (Imputation, Derived Variables)(020) Robert Chung(Benefit Editing, Data Access)(020) Any questions? Family Resources Survey