1. Identifying Gender Minorities in Population-based Proxy Surveys
Mark Noyes, MPH
Brian Robertson, PhD
AAPOR, May 2018

2. Background
Current Survey Practices
Experimental Design
Results & Comparison to Other Surveys
Conclusions and Next Steps

3. Background

4. Concepts and Definitions
Sex and Gender
Sex: “[T]he traits that distinguish between males and females. Sex refers especially to physical and biological traits…”1
Sex is a set of physical characteristics
While not entirely binary, sex is generally measured as male and female
Gender: “[T]he condition of being male, female, or neuter… implies the psychological, behavioral, social, and cultural aspects of being male or female”1
Does not necessarily conform to physical sex
Finding ways to disambiguate from sex in a survey is often a struggle
1. American Psychological Association. (2015). APA dictionary of psychology (2nd ed.). Washington, DC: Author.

5. They’re so cute!

6. Survey Measures Proxy Surveys
Surveys taken by one individual about another individual(s) for whom they have appropriate knowledge.
Gender identity can be fraught and not immediately obvious if a person doesn’t make it known- can proxies appropriately assess gender minorities?

7. Questions this raises
What are the best ways to identify members of gender minority populations in surveys?
We still encounter societal stigma related to these groups, and the perception that asking these questions could encourage non-response.
Is it possible to use questions related to gender identity in a proxy survey?
This would be required to get an accurate assessment of this population’s prevalence on a national level.

8. Current Survey Practice

9. Surveys which already ask about gender identity
Several federally sponsored surveys ask questions (Current Measures of Sexual Orientation and Gender Identity in Federal Surveys, 2016)
There are more, these are those that appeared in a federal survey published in 2016.
All surveys about respondents, varying methods of data collection
Ortman et al (2017) measuring SOGI, online study, non-representative sample
California Health Insurance Survey, (Jans, 2017)

10. Two primary methods of questioning
Single question structures
NIS, HCPS
Expanded answer categories to contain options for transgender and/or gender queer individuals
Two question structure
Other listed surveys
Two methods:
Asking about identifying as transgender directly (BRFSS, PATH)
Splitting sex and gender identity into separate concepts, questioned separately (NCVS)

11. Experimental Design

12. Survey background Oregon Health Insurance Survey
Fielded with a new methodology and questionnaire in 2017
9,000 households contacted via landline (33%) and cellphone (67%)
21,329 individuals represented in data set
Oversample of racial and ethnic minority groups
Questions on the topic of health insurance, health access and barriers to accessing care %
APPOR RR 3
13.5%
APPOR Coop 3
42.5%
APPOR Refusal Rate 3
20.7%

13. Two methods deployed – Treatment 1
A single question structure
Additional answer categories added for transgender and other gender minority groups
These categories were UNREAD and used only when individuals volunteered the information
Question Text
What gender is PERSON?

14. Treatment 2 Two question structure
First question assessing sex at birth
Second question asking about current gender identity
Additional explainer text written for interviewers to explain definitions, purpose, and importance of asking this as a separate item
Question Text
Q1: What sex was PERSON assigned at birth?
Q2: Now, does PERSON consider themselves...

15. Method of Categorization
In Treatment 1 individuals volunteering a gender minority category were placed into the category of an identified gender minority individual.
In Treatment 2, individuals identifying themselves as a member of a minority group in question 2, or who’s identified gender in Q2 and Q1 did not align, were placed into the category of identified gender minority.

16. Results

17. # GM Individuals Identified
Results by Treatment
Treatment Group
# GM Individuals Identified
Population
Rate
MDR Treatment 1 6
12,629
0.05%
MDR Treatment 2 91
8,700
1.05%

18. Comparison to other surveys
Our estimates show respondents are as capable and willing to identify gender minority individuals in a proxy survey as in an individual survey
Source
Identified
Population %
2014 BRFSS
752
166,907
0.45%
NATS
233
75,233
0.31%
2017 Ortman et al [Treatment 1 Self] 27
2,493
1.08%
2017 Ortman et al [Treatment 1 Proxy] 22
3,126
0.70%
2017 Ortman et al [Treatment 2 Self] 48
2,498
1.92%
2017 Ortman et al [Treatment 2 Proxy] 61
3,095
1.97%
2015 CHIS NR
19,334
0.2%
2017 MDR Treatment 1 6
12,629
0.05%
2017 MDR Treatment 2 91
8,700
1.05%

19. Non-response
Because of the method of data storage, specific non-response rates for each treatment could not be identified.
Overall, imputation was conducted for sex and/or gender on 0.7% of individuals in the data set.
This is less frequently than
Race (3.7%)
Ethnicity (1.4%)
Any health care expense
Income
This is in line with other survey findings.

20. Other relative or non-relatives
Family Relationships
Individuals were asked to identify not only other individuals in their household, but also what their relationship is to that individual
Proxies were able to identify gender minority individuals in all types of family relationship
Head of Household
Spouse or partner
Child, grandchild
Parent, grandparent
Sibling
Other relative or non-relatives
Male
4,552
2,467
2,973
113 74
231
Female
4,359
3,122
2,713
253 67
205
Transgender female-to-male 14 11 17 3
Transgender male-to-female 16 7 1 4
Genderqueer 9 2
Something else
Unsure 43 19

21. Conclusions and Next Steps

22. Effectiveness
Though the true population estimate remains unknown, proxy measurement of gender minority individuals is clearly possible.
Proxy family members were able and willing to identify gender minority household members.
They were able to do this in all types of family relationships.
Current estimates are likely lower than true, due to factors such as societal stigma.
Questions must be crafted to make the availability of gender minority options explicit, and should likely separate the constructs of sex and gender to obtain the best results.

23. Further Research
Attempt to assess differences between asking about current gender identity as male/female or as male/female/transgender
What are we actually measuring?
When asking a ambiguous sex/gender question, do people who do not identify with their physical sex answer about sex or about gender?
Find true population rates
Weighting: What do we do?