2018 Census content development Disability Presentation 2a 2018 Census content development Disability

2018 Census content review The programme has undertaken an assessment and consultation on the content for the 2018 Census This has been followed by our testing programme which informed the final content recommendations A refresh of content was necessary given the previous ‘no change’ census models Need for the information collected in the census to remain relevant, while retaining the value of longitudinal data Opportunities: Refreshing the content allows us to address important unmet information needs Leveraging the benefits of an improved processing system, revised methodologies, administrative data and the online first collection model will allow us to produce higher quality outputs for customers Challenges: Managing expectations around the amount of new content we can include on the forms. Increasing the total amount of content will likely lower overall response rates and data quality The high profile of the census and the level of detail that it can provide means that many customers may want information on a topic that is better suited in another survey collection

The stages to determine 2018 content  2013 Census content   Public engagement and consultation    Assessment of formal submissions    Question development and cognitive testing 2015-2016 Engagement and consultation have played a major roles in ensuring that the census content remains relevant and reflect real world change This census cycle we undertook engagement and consultation using three methods Face to face seminars and workshops in five main centres - In addition to meetings with key agencies at the end of 2014, we ran face-to-face seminars and workshops in Auckland, Hamilton, Wellington, Christchurch, and Dunedin in May 2015. These seminars involved a small group of technical experts, and provided an introduction to the modernised 2018 Census and a general overview of census content. The workshops included a detailed run-through of the Preliminary View of 2018 Census Content for each topic.  Public online engagement ran over nine weeks from April 2015 using the Loomio discussion forum. This is the first time Stats NZ has conducted live, supported, public online engagement. It was a unique opportunity for the public to have their say about census content and for us to learn to engage with the public in this way.  Formal submissions - The formal submission period started two weeks after online engagement in May 2015 and closed at the end of June. Formal submissions started after online engagement to encourage people to discuss and develop their thinking with others on the discussion forum before making a formal submission. Where appropriate we encouraged people to represent the views of their organisation. For formal submissions, we encouraged people to fill in a form on the Stats NZ website, but also accepted email and paper submissions. This approach was in line with our digital-first goal for the 2018 Census   Continuing question development and public tests during 2016-17    Government Statistician makes final decision on 2018 Census content

Content determination Does the proposed change add value to New Zealand's society and economy? Is the census the most appropriate information source? Does the proposed change reflect an enduring information need? Will the proposed change produce quality information? Is there continuity with previous census data? Is data consistent with other data collections? Is there general acceptance of the proposed change? Would the proposal questions be easy for respondents to complete? The content team reviewed submissions, and went through a content determination process.  The topics included in the census must reflect legal requirements, current information needs, and our aims for the census. The census must also reflect real-world change and maintain relevance.  Regardless of whether census forms are competed online or on paper, it is still important to consider the overall amount of information being collected and overall respondent burden., The size of the paper forms and ability to fit the content onto these is an important factor.  The census content and all topics raised during engagement and consultation were then evaluated against a set of criteria. Scoring of submissions: <60 – Currently recommending exclusion 60-69 – More content development and evidence (QD/content team/external) 70-74 – Currently recommended for inclusion, but some minor issues may need to be resolved 75+ - Currently recommended for inclusion

Content to questionnaire testing Cognitive testing to assess whether new and changed content is suitable for a self-completed questionnaire A number of topics did not progress past cognitive testing We have thoroughly tested new and changed content including cognitive testing and mass completion testing with testing the questionnaire in two major tests in July 2016 and March 2017.  Cognitive testing to assess whether new and changed content is suitable for a self-completed questionnaire A number of topics did not progress past cognitive testing (stepfamilies, gender identity, volunteering, license to occupy) Public content tests: March 2016 Field focused test Limited number of variables Housing quality questions were tested on dwelling form July 2016 Content focus Wide scope of content tested: 7 page individual form Two sample groups: paper and internet 8,000 households (paper forms) External reference group – 23,000 people tested online forms April 2017 Content further refined Using intended paper size for 2018: 4 pages While earthquake rescoped the test to not include test outputs, more focus put onto output and detail of data quality

Testing programme March 2016 July 2016 April 2017 Experimental design 22,000 households Split our sample to test different approaches Self response – 32 percent before field visits Response rate – 59% 64% took part online March 2016 Two sample groups 8,000 households (paper forms) External reference group – 23,000 people tested online forms Content focus 20% response rate July 2016 Rescoped after the November earthquake 20,00 households in Whanganui 36% response rate 76% took part online Marketing campaign, communications & community engagement tested April 2017 A number of small independent tests conducted together with larger tests in 2016 and 2017. (Already been reflected on from the content perspective in the first session). Testing process is an ongoing one even though our field tests have been completed. April 2017 We received 15,000 individual forms, and 7,000 dwelling forms.  It took people approximately eight minutes to complete their individual form online. Overall, 69% of online responses were completed on a desktop computer, 17% were by mobile and 14% were on a tablet. Most people completed their census test forms on 4 April 2017, despite the risk of flooding in Whanganui that day.

Washington Group Short Set Initially a matrix format was tested on paper due to uncertainty around space constraints, for the 2017 Census Test the format was revised given the space available We have thoroughly tested new and changed content including cognitive testing and mass completion testing with testing the questionnaire in two major tests in July 2016 and March 2017.  Cognitive testing to assess whether new and changed content is suitable for a self-completed questionnaire A number of topics did not progress past cognitive testing (stepfamilies, gender identity, volunteering, license to occupy) Public content tests: March 2016 Field focused test Limited number of variables Housing quality questions were tested on dwelling form July 2016 Content focus Wide scope of content tested: 7 page individual form Two sample groups: paper and internet 8,000 households (paper forms) External reference group – 23,000 people tested online forms April 2017 Content further refined Using intended paper size for 2018: 4 pages While earthquake rescoped the test to not include test outputs, more focus put onto output and detail of data quality

Other new content Usual residence one year ago Housing quality – damp, mould, access to basic amenities Travel to education - usual instead of census day - educational institution address Revisions to other content eg. appliances used to heat dwelling, sector of landlord categories All of these topics had quite a lot of work / testing done prior to the decision to exclude. Sexual orientation Data quality concerns Sensitivity of questions Stats NZ exploring further (GSS) A third category for sex Low quality data Low prevalence Paper form option Gender identity Data quality Complexity of concept Response behavior was variable Second address/residence Complex living situations Burdensome and difficult for respondents Data quality issues Step-families Difficulties in understanding ‘step’ Complex Further testing and development work Licence to occupy Term not well understood Confusion and errors

Other topics explored but excluded Sexual orientation A third category for sex Gender identity Second address / residence Step-families Licence to occupy All of these topics had quite a lot of work / testing done prior to the decision to exclude. Sexual orientation Data quality concerns Sensitivity of questions Stats NZ exploring further (GSS) A third category for sex Low quality data Low prevalence Paper form option Gender identity Data quality Complexity of concept Response behavior was variable Second address/residence Complex living situations Burdensome and difficult for respondents Data quality issues Step-families Difficulties in understanding ‘step’ Complex Further testing and development work Licence to occupy Term not well understood Confusion and errors