1. Presentation to Primary Health Alliance 7 June 2019
Conducting wellbeing measurement
Presentation to Primary Health Alliance
7 June 2019
NOT GOVERNMENT POLICY – UNCLASSIFIED

2. Who is the Social Investment Agency
Investing in what works for better lives
Demonstrating the value of using data and evidence, and technology to improve decision-making
Developing infrastructure, tools and methods for any agency to use
Partnering with agencies to trial how government and NGOs can work together to innovate and embed more effective ways of working
Establishing a Data Protection and Use Policy for the social sector
NOT GOVERNMENT POLICY – UNCLASSIFIED

3. Two publications on social housing
Measuring the wellbeing impacts of public policy: social housing
Fiscal, focused on ROI
Population of applications
Group A: housed
Group B: not-housed
Government spend over the following six years
Individual wellbeing
Focus of this presentation
Social Housing
Test Case
SIA’s wellbeing domains based off OECD wellbeing framework
Very similar wellbeing domains to TSY
Our focus is individuals & households
NOT GOVERNMENT POLICY – UNCLASSIFIED

4. The wellbeing impact of social housing was measured retrospectively
NOT GOVERNMENT POLICY – UNCLASSIFIED

5. Linked administrative and survey data are required
Longitudinal admin data for intervention
Cross sectional survey data for wellbeing
These are available in the IDI
Two sources of information:
* Cross sectional wellbeing
* Longitudinal intervention
NOT GOVERNMENT POLICY – UNCLASSIFIED

6. The IDI combines a range of data
Integrated Data Infrastructure
Produced by SNZ
Administrative records
Survey & Census responses
NOT GOVERNMENT POLICY – UNCLASSIFIED

7. Social housing dataset
Drawn from HNZ transaction records
Captures time stamped information on applications, tenancies (placement in social housing), end of tenancies
Includes information on the individuals in the application that can be linked to other IDI data
Complete data exists from 2001 to 2015
We used data from April 2007 through to June 2015
NOT GOVERNMENT POLICY – UNCLASSIFIED

8. New Zealand General Social Survey
Sample size of c8500 per wave (8462 to 8795)
Achieved response rate of 78% to 83%
There is a significant change in the questionnaire between 2012 and
We use the 2008, 2010, 2012, and 2014 waves
Link rate for the surveys to the IDI is between 77% and 82%
NOT GOVERNMENT POLICY – UNCLASSIFIED

9. Results across a range of wellbeing domains can be observed
IDI disclaimer
The results in presentation are not official statistics, they have been created for research purposes from the Integrated Data Infrastructure (IDI) managed by Statistics New Zealand.
The opinions, findings, recommendations and conclusions expressed in this presentation are those of the author(s) not Statistics NZ, or other government departments.
Access to the anonymised data used in this study was provided by Statistics NZ in accordance with security and confidentiality provisions of the Statistics Act Only people authorised by the Statistics Act 1975 are allowed to see data about a particular person, household, business or organisation and the results in this excel table have been supressed to protect these groups from identification.
Careful consideration has been given to the privacy, security and confidentiality issues associated with using administrative and survey data in the IDI. Further detail can be found in the Privacy impact assessment for the Integrated Data Infrastructure available from
The results are based in part on tax data supplied by Inland Revenue to Statistics NZ under the Tax Administration Act This tax data must be used only for statistical purposes, and no individual information may be published or disclosed in any other form, or provided to Inland Revenue for administrative or regulatory purposes.
Any person who has had access to the unit record data has certified that they have been shown, have read, and have understood section 81 of the Tax Administration Act 1994, which relates to secrecy. Any discussion of data limitations or weaknesses is in the context of using the IDI for statistical purposes, and is not related to the data’s ability to support Inland Revenue’s core operational requirements.
NOT GOVERNMENT POLICY – UNCLASSIFIED

10. Wellbeing before and after being housed
NOT GOVERNMENT POLICY – UNCLASSIFIED

11. Trends over time Housing outcomes by tenure
The static NZGSS picture is the opposite of the combined NZGSS + HNZ data picture
Source: NZGSS HNZ social housing dataset
NOT GOVERNMENT POLICY – UNCLASSIFIED

12. This technique is not without limitations
NOT GOVERNMENT POLICY – UNCLASSIFIED

13. Most significantly: non-causal
“before” and “after”, not “control” and “treatment”
Ashenfelter’s dip – people are likely to be in a worse than normal situation when they apply for social housing, and outcomes are likely to improve to some degree even if there was no intervention
Impact on wellbeing does not have a strong causal interpretation, and estimates of the effect of an intervention are likely to be too high
Other limitations: sample size, short term effects, selection bias for housed
NOT GOVERNMENT POLICY – UNCLASSIFIED

14. Implications and opportunities
Linked data enables new research approaches
The wellbeing outcomes of individuals can be observed across a range of different domains
Evaluations are possible retrospectively
Technique has limitations
NOT GOVERNMENT POLICY – UNCLASSIFIED