1. Measurement Error in Relationship and Marital Status Questions Nancy Bates Senior Researcher for Survey Methodology U.S. Census Bureau COPAFS Quarterly Meeting December 2, 2011

2. Research problem: definitions  Societal and legal definition of “marriage” has changed  New terms (e.g., same-sex husbands and wives, partner, civil unions, domestic partnerships)  State-to-state recognition and no federal level recognition  According to 2008 ACS: ~150,000 same-sex spousal couples but…  Only ~32,000 same-sex legally married couples in U.S. in 2008 (Williams Institute) Why the discrepancy? 2

3. Classification “error”  Are cohabiting gays & lesbians selecting “husband/wife” (even if not married)?  Is primacy effect a factor (‘husband/wife’ is 1 st in list, ‘unmarried partner’ is 13th)?  False positives caused by heterosexual married couples mis-marking gender?  Some combination of all the above? 3

4. Research to date  Interagency Workgroup on Measuring Relationships in Federal Household Surveys (IWMRFHS)  Focus Groups (Bates et al., 2010) –Understand terms commonly used –Interpreted as legal or something else? –Shortcomings of current questions? –Come up with alternative relationship and marital status questions to cognitively test  Cognitive Interviews (DeMaio and Bates, 2011) –Test 2 alternatives of each question among target populations –Select one set for further quantitative testing

5. Methodology: focus groups  18 groups in 8 geographically varied sites (15 with gay couples; 3 straight)  Included legally married, domestic partners, and no legal status Summary Findings:  Interpreted Qs in context of a federal form - that means legal status  Legally married anywhere = “husband/wife”  Very few selected “husband/wife” or “now married’ if not legally married  Need categories to reflect new legal unions  “Functional equivalence” problem  Most same-sex couples able to make selection in Relationship Q.  Marital status had bigger problems:  No place to indicate committed relationship for many same-sex couples  Consider measuring cohabitation/relationship status and marital status separately

6. Methodology: Cognitive Interviews  40 interviews - included both gays and straights  Married, domestic partnerships (DP)/civil unions (CU), no legal status  Cleveland, Charlotte, Las Vegas, DC metropolitan area  Observed as respondents completed one of two versions  Retrospective cognitive interviews –How and why respondents answered as they did? –Meaning of terms and categories? –Sensitive questions? –Showed alternate versions at end – preference?

7. New Relationship Questions Version 1 How is this person related to Person 1? Mark (X) ONE box. [ ] Husband/wife/spouse[ ] Parent-in-law [ ] Unmarried partner[ ] Son-in-law or daughter-in-law [ ] Biological son or daughter[ ] Other relative [ ] Adopted son or daughter[ ] Roomer or boarder [ ] Stepson or stepdaughter[ ] Housemate or roommate [ ] Brother or sister[ ] Foster child [ ] Father or mother[ ] Other nonrelative [ ] Grandchild

8. New Relationship Questions Version 2 How is this person related to Person 1? Mark (X) ONE box. [ ] Opposite-sex husband/wife/spouse[ ] Grandchild [ ] Same-sex husband/wife/spouse[ ] Parent-in-law [ ] Opposite-sex unmarried partner[ ] Son-in-law or daughter-in-law [ ] Same-sex unmarried partner[ ] Other relative [ ] Biological son or daughter[ ] Roomer or boarder [ ] Adopted son or daughter[ ] Housemate or roommate [ ] Stepson or stepdaughter[ ] Foster child [ ] Brother or sister[ ] Other nonrelative [ ] Father or mother

9. New Marital Status Questions Version 1 What is this person’s current marital status? Mark (X) ONE box. [ ] Now married [ ] In a registered domestic partnership or civil union [ ] Widowed [ ] Divorced [ ] Separated [ ] Never married

10. New Marital Status Questions Version 2 1. What is this person’s current marital status? Mark (X) ONE box. [ ] Now married - Skip to #4 [ ] Widowed [ ] Divorced [ ] Separated [ ] Never married 2. Is this person currently living in a registered domestic partnership or civil union? [ ] Yes - skip to #4 [ ] No

11. Last in series: cohabitation question 3. Is this person currently living with a boyfriend/girlfriend or partner in this household? [ ] Yes [ ] No  Allows same-sex couples without legal recognition to indicate relationship status  Allows for identification of sub-families  Currently asked in Current Population Survey

12. Results: Relationship Question  Most survey reports aligned with “true” legal status  Category ordering effect of Version 2 caused misreporting  Version that delineates “same-sex” / “opposite-sex” had most mentions for being potentially sensitive (but was also most preferred) Recommendation: Further test delineated Version 2 –Acts as consistency check to reduce gender misreports among straight married couples –However, new category of “same-sex husband/wife/spouse” could inflate gay couple reports (reorder categories to minimize) –Also, “same-sex / opposite-sex” labels could have negative impact on unit response (need to empirically test)

13. Reactions to Relationship with Same-Sex and Opposite-Sex categories “This [version with opposite-sex and same-sex labels] actually might, would offend me a little bit, because I think it’s …I feel like we’re kind of just lowering the standards..[as in] OK, we’ve got to conform to everyone. We want to be politically correct” - married straight female “...it does show how we grow as a country, that’s what the Census is, showing how we deposit ourselves all over the country and how we define our families” - married gay man “[that version] tries too hard…it’s making a point. [that version means] we’re gonna put it out there…that we’re including everybody. But, I get that..it’s fine…we’re in America” - married straight female

14. Results: Marital Status Question  Most survey reports were consistent with “true” legal status  Inconsistent reports involved “domestic partner/civil union” in some way  Confusion /misunderstanding of “domestic partnership/civil union” (mostly among straights)  Overwhelming preference for embedded Version 1, but… Recommendation: Further test Version 2 (separate questions) –Misunderstanding of DP/CU terms is widespread –Embedding may inflate DP/CU selection among straight couples –Reduce above by having separate question –Reorder to ask cohabitation first followed by DP/CU question

15. Reactions to DP/CU concept “two people who have lived together for at least 7 years” -straight married male “I might instead put registered DP or CU..if I lived with somebody in a committed relationship [and] we weren’t married. Now I have another option” -straight married female “In California, you can do a domestic partnership, which is same-sex marriage”- straight married male

16. Cohabitation Item  When space allows, keep cohabitation question  Re-order to follow Marital Status and precede Domestic Partnership/Civil Union question

17. Recommended: Relationship Q. How is this person related to Person 1? Mark (X) ONE box. [ ] Opposite-sex husband/wife/spouse[ ] Grandchild [ ] Opposite-sex unmarried partner[ ] Parent-in-law [ ] Same-sex husband/wife/spouse[ ] Son-in-law or daughter-in-law [ ] Same-sex unmarried partner[ ] Other relative [ ] Biological son or daughter[ ] Roomer or boarder [ ] Adopted son or daughter[ ] Housemate or roommate [ ] Stepson or stepdaughter[ ] Foster child [ ] Brother or sister[ ] Other nonrelative [ ] Father or mother

18. Recommended: Marital Status Q. 1. What is this person’s current marital status? Mark (X) ONE box. [ ] Now married - Skip to #4 [ ] Widowed [ ] Divorced [ ] Separated [ ] Never married 2. Is this person currently living with a boyfriend/girlfriend or partner in this household? [ ] Yes [ ] No - skip to #4 3. Is this person currently living in a registered domestic partnership or civil union? [ ] Yes - skip to #4 [ ] No

19. Next Steps  Recommended questions currently under review of interagency group  Piggybacking on the next ACS forms-length test  Mail only  Small test panel (5,000 mailout or only ~2,000 returns)  Need to test in other modes  Reinterview component would be ideal

20. Questions? Contact: nancy.a.bates@census.gov