Presentation to Primary Health Alliance 7 June 2019 Conducting wellbeing measurement Presentation to Primary Health Alliance 7 June 2019 NOT GOVERNMENT POLICY – UNCLASSIFIED

Who is the Social Investment Agency Investing in what works for better lives Demonstrating the value of using data and evidence, and technology to improve decision-making Developing infrastructure, tools and methods for any agency to use Partnering with agencies to trial how government and NGOs can work together to innovate and embed more effective ways of working Establishing a Data Protection and Use Policy for the social sector NOT GOVERNMENT POLICY – UNCLASSIFIED

Two publications on social housing Measuring the wellbeing impacts of public policy: social housing Fiscal, focused on ROI Population of applications Group A: housed Group B: not-housed Government spend over the following six years Individual wellbeing Focus of this presentation Social Housing Test Case SIA’s wellbeing domains based off OECD wellbeing framework Very similar wellbeing domains to TSY Our focus is individuals & households NOT GOVERNMENT POLICY – UNCLASSIFIED

The wellbeing impact of social housing was measured retrospectively NOT GOVERNMENT POLICY – UNCLASSIFIED

Linked administrative and survey data are required Longitudinal admin data for intervention Cross sectional survey data for wellbeing These are available in the IDI Two sources of information: * Cross sectional wellbeing * Longitudinal intervention NOT GOVERNMENT POLICY – UNCLASSIFIED

The IDI combines a range of data Integrated Data Infrastructure Produced by SNZ Administrative records Survey & Census responses NOT GOVERNMENT POLICY – UNCLASSIFIED

Social housing dataset Drawn from HNZ transaction records Captures time stamped information on applications, tenancies (placement in social housing), end of tenancies Includes information on the individuals in the application that can be linked to other IDI data Complete data exists from 2001 to 2015 We used data from April 2007 through to June 2015 NOT GOVERNMENT POLICY – UNCLASSIFIED

New Zealand General Social Survey Sample size of c8500 per wave (8462 to 8795) Achieved response rate of 78% to 83% There is a significant change in the questionnaire between 2012 and 2014 We use the 2008, 2010, 2012, and 2014 waves Link rate for the surveys to the IDI is between 77% and 82% NOT GOVERNMENT POLICY – UNCLASSIFIED

Results across a range of wellbeing domains can be observed IDI disclaimer The results in presentation are not official statistics, they have been created for research purposes from the Integrated Data Infrastructure (IDI) managed by Statistics New Zealand. The opinions, findings, recommendations and conclusions expressed in this presentation are those of the author(s) not Statistics NZ, or other government departments. Access to the anonymised data used in this study was provided by Statistics NZ in accordance with security and confidentiality provisions of the Statistics Act 1975. Only people authorised by the Statistics Act 1975 are allowed to see data about a particular person, household, business or organisation and the results in this excel table have been supressed to protect these groups from identification. Careful consideration has been given to the privacy, security and confidentiality issues associated with using administrative and survey data in the IDI. Further detail can be found in the Privacy impact assessment for the Integrated Data Infrastructure available from www.stats.govt.nz. The results are based in part on tax data supplied by Inland Revenue to Statistics NZ under the Tax Administration Act 1994. This tax data must be used only for statistical purposes, and no individual information may be published or disclosed in any other form, or provided to Inland Revenue for administrative or regulatory purposes. Any person who has had access to the unit record data has certified that they have been shown, have read, and have understood section 81 of the Tax Administration Act 1994, which relates to secrecy. Any discussion of data limitations or weaknesses is in the context of using the IDI for statistical purposes, and is not related to the data’s ability to support Inland Revenue’s core operational requirements. NOT GOVERNMENT POLICY – UNCLASSIFIED

Wellbeing before and after being housed NOT GOVERNMENT POLICY – UNCLASSIFIED

Trends over time Housing outcomes by tenure The static NZGSS picture is the opposite of the combined NZGSS + HNZ data picture Source: NZGSS 2008-2014 + HNZ social housing dataset NOT GOVERNMENT POLICY – UNCLASSIFIED

This technique is not without limitations NOT GOVERNMENT POLICY – UNCLASSIFIED

Most significantly: non-causal “before” and “after”, not “control” and “treatment” Ashenfelter’s dip – people are likely to be in a worse than normal situation when they apply for social housing, and outcomes are likely to improve to some degree even if there was no intervention Impact on wellbeing does not have a strong causal interpretation, and estimates of the effect of an intervention are likely to be too high Other limitations: sample size, short term effects, selection bias for housed NOT GOVERNMENT POLICY – UNCLASSIFIED

Implications and opportunities Linked data enables new research approaches The wellbeing outcomes of individuals can be observed across a range of different domains Evaluations are possible retrospectively Technique has limitations NOT GOVERNMENT POLICY – UNCLASSIFIED