Validating Self-Reported Education: Results of a Pilot Study Jesse Rothstein Cecilia Rouse Ashley Miller

How many high school graduates are there? Current Population Survey Gives “status completion” rate: # of 25-year-old graduates / 25-year-old population Common Core of Data Gives “school-based completion rate”: # of graduates in year t / # of 9 th graders in t-3. CPS estimates much higher than CCD estimates  2000/2 cohorts: 84% CPS vs. 70% CCD

CPS vs. CCD-based graduation rates

Two easy issues HS diploma attainment rises with age in CPS  Delayed completion?  Adult education / GEDs?  Misreporting? “9 th grade bulge”  CCD estimates usually use 9 th grade enrollment as denominator  Many students are held back in 9 th grade  9 th grade enrollment consistently about 7-10% higher than 8 th grade enrollment through 1990s These reduce the problem, but don’t eliminate it.

CPS vs. CCD-based graduation rates

Not very sensitive to definitions CCD-based estimates all tend to yield similar answers. Mishel and Roy (2006): Get same answers with CPS, Census, NELS, NLSY.

Alternative graduation rates, existing data

Not very sensitive to definitions CCD-based estimates all tend to yield similar answers. Mishel and Roy (2006): Get same answers with CPS, Census, NELS, NLSY. Except!  Some evidence that GEDs can account for a lot of the discrepancy (Heckman and LaFontaine 2007)

Alternative graduation rates, existing data

Potential explanations 1. Attainment increase with age is poorly understood 2. Proxy responses in CPS may overstate graduation 3. Respondents may overstate own graduation 4. “Some college” may not have graduated HS 5. Regular diplomas vs. GEDs, certificates of attendance, adult ed., etc. 6. Public vs. private schools 7. Immigrants/emigrants since HS 8. Grade retention 9. Divergent definitions in CCD 10. CPS excludes armed forces, prisoners 11. Poor coverage in CPS

Potential explanations that we will address 1. Attainment increase with age is poorly understood 2. Proxy responses in CPS may overstate graduation 3. Respondents may overstate own graduation 4. “Some college” may not have graduated HS 5. Regular diplomas vs. GEDs, certificates of attendance, adult ed., etc. 6. Public vs. private schools 7. Immigrants/emigrants since HS 8. Grade retention 9. Divergent definitions in CCD 10. CPS excludes armed forces, prisoners 11. Poor coverage in CPS

Our Sample…. Sample frame = Households likely to have an year old member (complied by a commercial vendor). We purchased 1,000 (randomly selected) phone numbers. Our target was 50 “completed” inteviews.

Survey implementation consisted of three steps. Step 1: Mimic the CPS Conduct a “CPS-style” interview with a HH member who was at least 18 years old (the “household respondent”). Note: We ended up with 2 versions (“A” and “B”) where version “A” asked for a full HH roster while version “B” asked initially if an year old lived in the HH; only asked questions about one year old. Probe more on education questions, particularly regarding alternative types of high school degrees. Ask for the name and location of the last high school attended by the young adult, as well as year of high school graduation.

Step 2: Verify Proxy Report and Respondent Understanding of CPS-style Education Question Ask similar “CPS-style” education questions of the young adult him or herself (“initial report”). Probe further on education questions (“final report”). Also ask for the name and location of last high school attended as well as year of high school graduation.

Step 3: Verify Proxy and Self-reports with Administrative Data Send a letter to the high school requesting verification of reported high school graduation information. Follow-up with a telephone call for a subset of schools that did not respond within days.

Table 1: Sample sizes (Part I) TotalVersion AVersion B (1)(2)(3) N%N%N% Number of phones called Non-working/non-residential13213%9323%397% Unable to make contact22322%164%20735% Made contact64565%30474%34158% Made contact Language problem102%5 51% Unresolved after max calls7311%21%7121% Refusal41865%22875%19056% Completed HH interview14422%6923%7522%

Table 1: Sample sizes (Part II) TotalVersion AVersion B (1)(2)(3) N%N%N% Completed HH interview No eligible youth5236%2536%2736% 1+ eligible youth in HH9264%4464%4864% All eligible youth Youth was HH respondent109%410%68% Failed to contact4135%615%3545% Refused1815%923%912% Turned out ineligible33%25%11% Completed self-report4538%1846%2735%

Table 1: Sample sizes (Part III) N% Completed self or proxy reports117 Ineligible (outside age range/ambiguous age)54% Did not attend high school33% Has GED43% Still in high school1513% Not enough identifying information119% Sent letter to high school7968% All letters sent to high schools79 No response2734% Declined to provide information68% Provided information4658%

Conclusion We find a high level of agreement between proxy respondents, the young adults themselves and high school administrative records on whether individual has a high school diploma (or equivalent). We see less agreement between proxy respondents and young adults on college attendance. Questions about modes of high school completion should be asked of those with a college education as well as those with only a high school education.