Sampling Issues for Telephone Surveys in Scotland Gerry Nicolaas Survey Methods Unit National Centre for Social Research 13 January

This session will focus on: Surveys of the general population Probability sampling

Potential sampling advantages of telephone interviewing: No need for geographical clustering. Inclusion of more remote areas. Cheaper recalls

Telephone methods are not widely used in the UK for social surveys among the general population

Obstacles to selecting a representative and unbiased probability sample of the general population: Households with no telephone Unlisted telephone numbers Mobile phones

(1) Households without telephone: Proportion of households with no phone in Scotland declined from 5% in 1998 to less than 0.5% in Households with no phone tend to belong to the most socially and economically deprived groups. The exclusion of households with no phone would not introduce notable bias in a general population survey. Source: Taylor, 2003

(2) Unlisted telephone numbers: In % of telephone owning households (fixed lines) in Scotland were ex-directory Households with unlisted phone numbers tend to be: - Smaller than average - Headed by someone in social classes IV or V Source: Beerten and Martin, 1999

(3) Mobile phones: Proportion of households in Scotland with at least one mobile phone increased from 13% in 1998 to 80% in % of households only have mobile phone Mobile phone owners without a fixed line tend to be younger, lower socio-economic group, lower income, unemployed. Mobiles tend to be not listed in phone directories. Mobile phone numbers cant be linked to geogr. area. Mobiles belong to individual rather than household. Source: Taylor, 2003; Ofcom 2003

Sampling from telephone directories: Select samples systematically or use random numbers to select page numbers and phone numbers. Non-coverage of - households with no telephone, - households with only mobile phones, and - households with unlisted telephone numbers. Coverage= about 65% of Scottish population. Under-representation of - most socially & economically deprived groups, - young mobile people, - people living in small households.

Plus digit sampling: Sample is selected from the telephone directory and a fixed number is added to the last digit. Similar procedure replaces the last one or more digits with a random number. Sample will include unlisted numbers, but: - proportion of unlisted numbers is lower than in the population; - profile of the achieved sample reflects that of listed telephones rather than that of all telephones; - probabilities of selection are unknown & vary because of unequal distribution of listed & unlisted numbers.

Random Digit Dialling (RDD): RDD uses comprehensive list of of valid area codes and prefixes and adds randomly generated suffixes. OFCOM database of blocks of 10,000 numbers; e.g xxxx All telephone households have a known non-zero chance of selection (fixed telephones and mobiles). Exclusion of households without phone will not introduce notable bias But: High proportion of non-working numbers and other ineligibles (e.g. business): about 80% Sources: Nicolaas and Lynn (2002)

RDD hit rates can be improved: Advanced telephony systems can screen out most non- working numbers; RDD samples can be matched against yellow pages to remove listed business numbers; Adopt sample designs that improve hit rate, - Mitofsky-Waksberg two-stage method - List-assisted methods

Should mobile phone numbers be included in RDD sample? Exclusion of mobile phone numbers will result in under- coverage of younger people, lower socio-economic group, lower income group, the unemployed. Inclusion of mobile phone numbers will result in further reduction in effective sample size due to post-weighting of households with more than one phone (fixed & mobile) Inclusion of mobile phone numbers will increase ineligibles for surveys covering areas smaller than UK (e.g. Scotland).

Dual frame sampling: E.g, a sample of listed telephone numbers supplemented with a RDD sample. Directory status of each RDD interview must be known Sampling administration is more complicated. Data to be combined using post-stratified dual-frame estimators. Unclear whether this approach is feasible and whether the gains are worthwhile.

Single frame, dual mode approach: Use a frame with good coverage (e.g. PAF) Telephone interviews for those sampled cases with a matched telephone number. Postal questionnaires or face-to-face interviews for those without a matched telephone number. But: - low matching rates; - costly if unmatched cases interviewed face-to-face; - relatively low response rates if postal questionnaires are sent to unmatched cases; - possibility of mode effects.

Conclusion: None of the sampling methods are perfect. Choice of sampling method depends on specific survey. On the whole, RDD appears to be superior method: - Complete coverage of all households with phones. - Non-coverage of households without phone can be ignored - Sampling frame is accurate (Ofcom database). - Precision of estimates is high (unclustered sample). - Relatively cheap and easy to select RDD sample. - Fieldwork efficiency can be improved. But the proportion of mobile only households needs to be monitored. Response rates?

Another sampling issue: Respondent selection Include all eligible household members or select one at random with post weighting for unequal selection probabilities. Random selection of one eligible household member tends to be most common approach for telephone surveys Gold standard is Kish Method but some claim this requires too much info up front and may reduce response. Most common method is Last/Next Birthday method. NatCen experiment showed no significant difference between the two methods in response rates nor sample compositions. Source: Tipping & Nicolaas (2001)

Suggested reading: Beerten & Martin (1999) Household ownership of telephones and other communication links: implications for telephone surveys. Surv. Methodol. Bull., 44: 1-7. Collins (1999) Sampling for UK telephone surveys. JRSS(A) 162: 1-4. Collins & Sykes (1987) The problems of non-coverage and unlisted telephone numbers in telephone surveys in Britain. JRSS(A) 150(3): Lepkowski (1988) Telephone sampling methods in the United States. In Groves et al (eds), Telephone Survey Methodology. John Wiley and Sons: New York Nicolaas & Lynn (2002) Random-digit dialling in the UK: viability revisited. JRSS(A) 165: Taylor, S. (2003) Telephone surveying for household social surveys: the good, the bad and the ugly. Surv. Methodol. Bull., 52: Tipping & Nicolaas (2001) Respondent selection procedures for telephone surveys. Survey Methods Newsletter, 21(1): 4-7, Nat. Centre for Soc. Res.