Carry Forward Imputation for Unit Nonresponse after a Survey Redesign Kimberly Ault

Definitions Carry Forward Imputation: Common imputation approach for longitudinal data where the last reported value is carried forward to the next round of data collection. Adjustment factors can be used to predict changes between rounds. Example: School X reports having 500 full-time students in year 1 and does not participate in year 2. Using CF imputation, the imputed value for school X in year 2 would be 500. An adjustment factor based on reported data (type of school, number of programs, or school size) can be used to increase or decrease the carried forward values. Unit Nonresponse: Failure of an eligible sample member to respond (all items are missing). RTI’s values and our dedication to innovative, objective research and technical services make us an outstanding partner for clients around the world whose greatest challenges demand rigorous approaches and science-based solutions.

Challenges after a survey redesign Historical data at broad field level: Computer Science Redesign data at more refined level: Computer Science Artificial Intelligence Information Technology Information Science Data Modeling Computer Graphics Networking and Telecommunications Information Systems Security Simple carry-forward of historical data not appropriate. Supplemental data source (where unit-level matching is possible) used to adjust the historical information to refined level. RTI’s values and our dedication to innovative, objective research and technical services make us an outstanding partner for clients around the world whose greatest challenges demand rigorous approaches and science-based solutions.

Redesign: Broad to Detailed Level Computer Science, General Computer & Information Sciences, General Artificial Intelligence Information Technology Informatics Computer & Information Sciences, Other Data Modeling/Warehousing & Database Administration Computer Graphics Modeling, Virtual Environments & Simulation Computer Systems Networking & Telecommunications Computer & Info. Systems Security/Information Assurance Computer & Info. Sciences & Support Services, Other

Ratio Adjustment for Unit Nonresponse For unit nonrespondents, all data needs to be imputed. For those with prior data available, CF imputation is the preferred method and can be used with a modification to address the more-refined data collection methods. Unit-level data can be matched to an additional data source that contains the more refined program codes that allows the prior data to be adjusted. For example, the reported counts in one program (e.g. Computer Science) in the prior round need to split out among the more-refined codes for the current year as if they would have been reported. RTI’s values and our dedication to innovative, objective research and technical services make us an outstanding partner for clients around the world whose greatest challenges demand rigorous approaches and science-based solutions.

Ratio Adjustment for Unit Nonresponse 𝐶 𝑡 (𝑖) - total count at school t for program i (from matched data source) 𝑃 𝑡 - prior reported count at school t 𝑃 𝑡 (𝑖) - estimated count at school t for each program i (up to n programs): 𝑷 𝒕 (𝒊)=𝑷 𝒕 ∗ 𝑪 𝒕 (𝒊)/ 𝒊=𝟏 𝒏 𝑪 𝒕 (𝒊) RTI’s values and our dedication to innovative, objective research and technical services make us an outstanding partner for clients around the world whose greatest challenges demand rigorous approaches and science-based solutions.

Ratio Adjustment for Unit Nonresponse 𝑖 𝐶 𝑡 (𝑖) 𝑷 𝒕 𝑪 𝒕 (𝒊)/ 𝒊=𝟏 𝒏 𝑪 𝒕 (𝒊) Old Code New Code 1 94 64 0.241 CS CS1 15 2 162 0.415 CS2 27 3 30 0.077 CS3 5 4 55 0.141 9 16 0.041 6 33 0.085 n = 390 1.00

Carry Forward Adjustment from Broad to Specific Prior Round: Broad Field (CS) 𝑷 𝒕 = 64 Current Round: Refined Field CS1 𝑷 𝒕 = 15 Current Round: Refined Field CS2 𝑷 𝒕 = 27 Current Round: Refined Field CS3 𝑷 𝒕 = 22

Challenges and Future Investigation Matching between data sources Rounding problems for ratio adjustments Future Investigation: Investigate the quality of imputations for unit nonrespondents in next survey cycle by comparing imputed values with reported values. How well did we estimate the true values?

RTI International Kimberly Ault, PhD RTI International Research Statistician 919-541-7455 ault@rti.org Since our founding with a single employee in 1958, we’ve grown into one of the world’s leading research institutes by remaining focused on our mission to improve the human condition by turning knowledge into practice.