1 Reengineering the SIPP: The Evolution of a Phoenix Joint Statistical Meetings July 30, 2007 David Johnson

2 Reengineering the SIPP

3 The Unique Value of SIPP To provide a nationally representative sample for evaluating: –annual and sub-annual dynamics of income –movements into and out of government transfer programs –family and social context of individuals and households –interactions between these items

4 Purpose of SIPP Recommendation 2-1: The two primary goals for SIPP should be to provide improved information on the distribution of income and other economic resources for people and families and on eligibility for and participation in government assistance programs. Within these two goals, most attention should be paid to improving the information for people who are economically at risk. A third important but subordinate goal is for SIPP to have a capability to respond to current policy needs for data in topical areas that are related to the core subjects of SIPP. –The Future of the Survey of Income and Program Participation, NAS, 1993

5 Department of Agriculture –Model food stamp eligibility and measure food stamp participation Department of Health and Human Services –Measure the economic effect of disabling conditions on children and adults, and determine "triggers" that cause people to go on or to go off programs. Social Security Administration –Model SSI benefits, and the restructuring of Social Security such as age threshold changes. Congressional Budget Office and Congressional Research Service –Use micro-simulation to measure participation in major government programs Current Uses of SIPP

6

7 Reengineering Goals Cost reduction Improved accuracy Improved timeliness and accessibility Improved relevance  Use of annual data collection  Focused content selection  Lower attrition rates  Integrated administrative data  Improved documentation  Improved processing system  Ongoing content determination process; use of “hooks”  Use of ACS

8 Main Components of the Reengineering Process Improved Processing System and Collection Instrument Development of Event History Calendar Meetings with stakeholders, development of survey content, and use of reimbursable supplements Use of ACS for sample Use of administrative records data to supplement and evaluate survey data

9 Timeline of SIPP/DEWS/Re-engineered SIPP (The Order of the Phoenix) Stopped work on converting existing SIPP to BLAISE and reprogramming post-data collection processing to SAS December 2005 January 2006 Preparing for budget release, sessions with key federal stakeholders were held to discuss future SIPP plans Publicly launched the “Program of Income, Wealth, and Health Insurance Measurement” and possible end of SIPP data collection in Sept 2006 February 2006 March/May 2006 CEPR circulated “Save the SIPP” letter. Census presented progress to date, and suggested using linked CPS data and administrative records data June 2006 Brookings Workshop: Concerns expressed by stakeholders about possible data gap and usefulness of admin data. Stakeholder matrices sent to major users in order to identify critical items. It was determined that admin records would only assist in data quality evaluation.

10 Timeline of SIPP/DEWS/Re-engineered SIPP First open stakeholder meeting held at Census. Publicly stated the Bureau was refocusing effort to use admin data to supplement data collection (not replace). New survey plans presented – EHC and ‘hooks’ to allow opportunities for reimbursable supplements. Announced plans to continue SIPP with reduced sample from 10/06-5/07 August 2006 Dec 2006/ Jan 2007 March 2006 April/May 2006 Topical content meetings held With additional funding, announced plans to continue SIPP with reduced sample through Sept, 2007 Another open stakeholder meeting at Census where more details were presented on content, structure, and plans for the new survey. Census presented plans for at CNSTAT Panel meeting. June/July 2006 Senate and House Appropriations Comm propose additional funding to continue SIPP collection in 2008

11 SIPP ’04 Panel Collection 2/04-1/08 (reduced sample 10/06-1/08) Improved Panel Data Collection 2011/12 and on Reengineering Planning 2006 – 2011 SIPP ’08 Panel Full Sample w/ TM Same as ’04 Panel Dress Rehearsal Test/Evaluation 9/09 EHC Paper Test 3/08 Current SIPP Status (subject to funding)

12 Improving Processing System Convert CAPI instrument to BLAISE Document processing system –Convert from FORTRAN to SAS –Document edit and imputation procedures Develop processing system in conjunction with survey for reengineered SIPP

13 Event History Calendars An EHC interview is centered around a customized calendar that shows the reference period under investigation The calendar contains timelines for different domains, for example, residence history, household composition, work history, and other domains relevant to the topic of study Landmark events, such as holidays and birthdays can be used to aid the respondent’s memory

14 EHC Format Survey Instrument – –Annual administration, Jan-April –Follow movers –Limited feedback Calendar – –Improvement on other designs by integrating more closely with Blaise, the instrument programming language. Simultaneous development of processing and instrument

15 Additional Validation with Admin. Records Prepare SOW for Instrument & Training 2008 EHC Paper Test Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Contract with RTI to prepare paper EHC and Training Field Staff Training Field EHC Evaluation Test in MD & IL Reference period for paper EHC Evaluation Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Field EHC Evaluation Test Coding Analysis EHC and SIPP 2004 Receive Admin. Records MD & IL Administrative Records from MD & IL to validate SIPP - EHC Census development of procedures for administering the paper EHC Evaluation SIPP 2004 Waves

16 Event History Calendar Conference "The Use of Event History Calendar Methods in Panel Surveys" December 5-6, 2007 at the US Census Bureau Agenda –Examining the Impact of Event History Calendar Interviewing on Data Quality from Disadvantaged Respondents - Belli –EHC in PSID –EHC in NLSY97 –EHC in Netherlands –EHC in LA Family and Neighborhood Survey –EHC in British Panel survey –Seam Bias in SIPP

17 Stakeholders and Content Review Responding users indicated a broad need for most of SIPP core content. About 40 stakeholders completed matrix Select areas were added based on lost topical module content. Held five subject area meetings to discuss specific content –Health –General income/Government programs –Assets and wealth –Labor force –Demographics and other items

18 Use of ACS Data in Survey Design Sampling Frame –American Community Survey (ACS) interviewed cases –Existing SIPP Primary Sampling Units (PSUs) –Goal is to use 2-years worth of ACS sample Oversample based on similar criteria as SIPP Sample size similar to SIPP (budget permitting)

19 Use of Administrative Records National-level prototype –SIPP linked with administrative records data at the person level Creation of SIPP Synthetic Beta File matched with SSA data State-level prototype –Many social programs administered at the state- level –SIPP data linked to social program data from the states of Maryland and Illinois

20 CNSTAT Panel Goals of the panel are to evaluate: – the costs/benefits of various strategies for data linkage, – accessibility of relevant administrative records, – operational feasibility of linking administrative records and survey data, – quality and usefulness of linked data, and – strategies for providing public access to the linked data while protecting the confidentiality of individual respondents

21 URL: Source: U.S. Census Bureau, Demographics Survey Division, Survey of Income and Program Participation branch Census Bureau Links: Home · Search · Subjects A-Z · FAQs · Data Tools · Catalog · Census 2000 · Quality · Privacy Policy · Contact Us HomeSearchSubjects A-ZFAQsData ToolsCatalog Census 2000QualityPrivacy PolicyContact Us Assessing Users’ Needs

EHC Evaluation 2009 Re-engineered SIPP Dress Rehearsal Jan --- May --- Sep --- Jan --- May --- Sep --- Jan --- May --- Sep --- Jan --- May --- Sep --- Jan 2009 SIPP Re-Engineering – Instrument Dev. Processing and Evaluation SIPP 2008 Panel – Waves 1 – 9 (planned) Eval. Analysis Systems Tests/Preparation 2009 Re-engineered SIPP Dress Rehearsal Reference Period Field Activities

23 Matching SIPP responses with Social Security Administrative Data on Retirement Benefits NO Retirement Benefit on SSA file Retirement Benefit on SSA file NO Retirement Benefit reported in SIPP 84.8%1.3% Retirement Benefit reported in SIPP 2.9%10.9%

24 Matching SIPP responses with Social Security Administrative Data on Retirement Benefits NO Retirement Benefit on SSA file Retirement Benefit on SSA file NO Retirement Benefit reported in SIPP 84.8%1.3% Retirement Benefit reported in SIPP 2.9%10.9%

25 Matching SIPP responses with Social Security Administrative Data on Retirement Benefits NO Retirement Benefit on SSA file Retirement Benefit on SSA file NO Retirement Benefit reported in SIPP 84.8%1.3% Retirement Benefit reported in SIPP 2.9%10.9%