1. A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning Alessandra Gaia Institute for Social and Economic Research (ISER) University of Essex
Thank you very much for having me here today. I will talk about a comparison of self-reported sexual identity using direct and indirect questioning techniques.

2. Motivation 1/2
“sexual orientation”: multidimentional construct: attraction, behaviour and self-identification (Laumann et al., 1994)
included in the list of protected attributes in the UK Equalities Act (2010), to tackle stigma, harrassment, and discrimination
crucial to obtain high quality data…
…but challenging
Operational problems:
Multidimentional construct
Sexuality is a sensitive topic
“sexual orientation” is included in the list of protected attributes in the UK Equalities Act (2010), to tackle stigma, harrassment, and discrimination. Thus, obtaining good quality data on this topic is crucial. However, this is also challenging. Indeed, there are several operational problems to asking survey questions on sexual orientation. Firstly, according to early work from Laumann and his colleagues, sexual orientation is a multinational construct involving attraction, behaviour and self-identification. Secondly, sexuality is among the most sensitive topics in surveys.
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017

3. Motivation 2/2
Estimates of sexual minorities in the UK seem inconsistent
Wave 3 Year 2 of Understanding Society: the UK Household Longitudinal Study (UKHLS)
2012 Integrated Household Survey
Bisexuality (HIS) < bisexuality (UKHLS)
especially among young people
Difference in data collection protocols: modes (IHS: F2F with showcards & Tel, vs. UKHLS: CASI), and wording.
Thus, there are theoretical reasons why sexual orientation may be misreported in surveys, and/or may be subject to item non response. From an empirical point of view, there seem to be an inconsistency of estimates on sexual orientation in the UK. Wave 3 Year 2 of Understanding Society: the UK Household Longitudinal Study (UKHLS) overlaps with the 2012 round of the Integrated Household Survey; a comparison of figures suggests underreporting of bisexuality, especially among young people in the IHS. The two studies implement different data collection protocols: Firstly, they use different data collection modes (with the IHS adopting a and the UKHLS asking the survey question in the self completion section of the questionnaire) and they use slightly different wording.
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017

4. Research Idea
Compare the UKHLS protocol with the IHS protocol, using the Item Count Technique as a benchmark the Item Count Technique pros: no one will know the respondent’s answers cons: no individual level data will the respondent understand? the “Longitudinal Item Count Technique (LICT)”
My research idea is to compare these two data collection protocols using the Item Count Technique as a benchmark. I will describe the technique in detail in the next slides, for the moment let me just anticipate that this is a technique was over 25 years ago, it is based on the idea of eliciting more candid answers as it allows higher respondents’ privacy; (at least in its original formulation) it has the caveat that it doesn’t allow to derive individual level characteristics. I propose a modification of this technique, that allows the derivation of individual level estimates; and it is called the Longitudinal Item Count Technique (LICT). And I will explain the design in more detail in the next slides.
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017

5. Method: Item Count Technique (ICT)
Where:
P is the estimated prevalence of the sensitive behaviour
X is the number of items counted
Longitudinal ICT (LICT)
The item count technique
Assumption: time invariant behaviour between t1 and t2
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017

6. Method: Item Count Technique (ICT)
Group A
Group B
“I have at least once been sexually attracted to someone who …
• is the same sex as me
• has a disability
• is fit and muscular
• grew up with me in my local area
• is ten or more years older than me
How many statements are true for you?
The item count technique
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017

7. Method: Longitudinal Item Count Technique (LICT)
Wave 1
Wave 2
“I have at least once been sexually attracted to someone who …
• is the same sex as me
• has a disability
• is fit and muscular
• grew up with me in my local area
• is ten or more years older than me
How many statements are true for you?
The item count technique
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017

8. Longitudinal Item Count Technique
Combined Method
Wave 1
Wave 2
Group 1
List + sensitive item
List
Longitudinal Item Count Technique
Group 2
Item Count Technique
The item count technique
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017

9. Research questions
What is the estimated prevalence of the lesbian gay and bisexual population obtained with the HIS protocol and the UKHLS protocol? ...and how does it compares with the measure obtained with the Item count technique? ... And with the “Longitudinal Item Count Technique” perform?
Thus my research questions are
What is the estimated prevalence of the lesbian gay and bisexual population obtained with an indirect questioning method, such as the “Item Count” indirect questioning Technique (ICT)?
...and how does it compares with the measure obtained with CASI or CAPI with showcards protocols?
...how does the “Longitudinal Item Count Technique” perform? (not only for measuring sexual identity, but also homosexual attraction and homoerotic behaviour)
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017

10. Data Experiment from the innovation panel wave 8 and 9
Probability sample of the UK population
IP is part of UKHLS study (but separate sample)
for methodological innovation & substantive research
Original sample: 1,500 households at wave 1
Refreshment sample at wave 4 and 7 (~ 500 hh)
Experiment from the innovation panel wave 8 and 9
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017

11. 2x2 Design IP8 IP9 UKHLS Question List List + Sensitive item
HIS Question
List + Sensitive item
Used for every item: sexual attraction, behaviour and identity
Two list Item Count Technique (total of 6 question)
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017

12. Results, IP8: sexual minorities
Note: N=1464 (UKHLS), 767 (IHS).
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017

13. Results, IP9: sexual minorities
Note: N=1464 (UKHLS), 544 (IHS)
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017

14. Results Low item non response, refusal, and don’t know...
...but, evidence of both “ceiling effects” and “floor effects”
estimates of behaviour and attraction are considered unreliable due to “ceiling” and “floor” effects
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017

15. Results
No significant difference among the two protocols, in IP8, nor in IP9
Even analysing the results by subgroups, e.g. age
Low item non response, refusal, and don’t know...
...but, both “ceiling effects” and “floor effects”
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017

16. Conclusion
While the item count technique was generally accepted by respondents, the derived estimates are not always reliable and there is inconsistency across the ICT and LICT
No difference in the two protocols: UKHLS and IHS
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017

17. Thank you!
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017

18. Appendix
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017

19. Item count tecniques
The next set of questions will ask you to count the number of statements that are true for you. Please only count the number of statements. We are not interested in knowing which statements are relevant for you. I have at least once been sexually attracted to someone who … • is the same sex as me • has a disability • is fit and muscular • grew up with me in my local area • is ten or more years older than me How many statements are true for you? None of the statements are true One of the statements are true Two of the statements are true Three of the statements are true Four of the statements are true Five statements are true
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017

20. Longitudinal Item Count Technique
Results
Item Count Technique
Longitudinal Item Count Technique
Same sex Attraction
16.6%
30.8%
Same sex Behaviour
9.5%
 27.7%
Gay/ lesbian/ bisexual
3.6%
 N.A.
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017

21. Longitudinal Item Count Technique (ICT) – 2 list version
Group 1
Group 2
“I have at least once been sexually attracted to someone who …
• is the same sex as me
• has a disability
• is fit and muscular
• grew up with me in my local area
• is ten or more years older than me
How many statements are true for you?
• wears the latest trends and fashions
• has a tattoo or body piercing
• is of a different ethnicity to me
• is from a different class background to me
How many statements are true for you?”
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017

22. IHS Question via face to face interview (with a show card)
“Which of the options on this card best describes how you think of yourself? Please just read out the number next to the description.”
Note: The categories on the show card should appear as follows:
27 Heterosexual/Straight
21 Gay / Lesbian
24 Bisexual
29 Other
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017

23. Item count tecniques
Lists made sense together Mix of (expected to be) high prevalence and low prevalence items
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017

24. IHS Questions via telephone interview
HIS Questions via telephone interview
I will now read out a list of terms people sometimes use to describe how they think of themselves: “Heterosexual or Straight”, “Gay or Lesbian”, “Bisexual”, or “Other”.
As I read the list again please say 'yes' when you hear the option that best describes how you think of yourself.
“Heterosexual or Straight”, “Gay or Lesbian”, “Bisexual”, or “Other”.
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017

25. UKHLS Question via self-completion (CASI or Web)
“Which of the following options best describes how you think of yourself?
Heterosexual/Straight
Gay / Lesbian
Bisexual
Other
Prefer not to say”
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017

26. Design List A & B Attraction List C & D Behaviour List E & F Identity
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017

27. Method: Item Count Technique (ICT)
Sample
I have at least once been sexually attracted to someone who …
• is the same sex as me
• has a disability
• is fit and muscular
• grew up with me in my local area
• is ten or more years older than me
How many statements are true for you?
The item count technique
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017

28. Method: Item Count Technique (ICT)
I have at least once been sexually attracted to someone who …
• has a disability
• is fit and muscular
• grew up with me in my local area
• is ten or more years older than me
How many statements are true for you?
I have at least once been sexually attracted to someone who …
• is the same sex as me
• has a disability
• is fit and muscular
• grew up with me in my local area
• is ten or more years older than me
How many statements are true for you?
The item count technique
A Comparison of Self-reported Sexual Identity Using Direct and Indirect Questioning | July 2017