Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Increasing response rates Ineke Stoop SCP

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Is it possible?

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 and is it effective?

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Nonresponse research How to increase response rates? How to measure and correct for nonresponse bias?

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 SCP-survey Face-to-face questionnaire + long drop off for every 6+ years household member Use of social and cultural services, amenities, facilities (no values, opinions) Very low non-contact rate High rate of refusal conversion Final response rate %

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Response rates AVO

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Cumulative response rates

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Increase response rates Describe response process Contactability Reluctance

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Paradata Call records (nr, timing, outcome) Characteristics interaction Interviewer observations Sample frame data Low-level geographic data bases Interviewer characteristics

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Contactability TNC/NAH/CFC Call (nr, time, mode) First contact

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 At home, interviewer calls, contact rate

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Timing contacts Interviewers prefer working hours Contact rates higher in the evening Cooperation independent of timing Fewer calls if evening calls only Length fieldwork period/costs?

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Contactability Easy –Evening calls –Phone number available –Detached dwelling –Child at home Difficult –Big city dweller –Poor maintenance neighbourhood –Young –Single –Employed, student –High cultural participation

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Continuum of resistance Extrapolate from hard to reach respondents to noncontacts? –Socio-demographics –Core variables –Two types of hard to contact (short term/long term)

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Reluctance Respondent has been contacted

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Reluctance Request to cooperate Inter- view Cooperative respondent Refusal Establish first contact Follow -up Con- tact Inter- view Refusal Request to cooperate Soft converted refusal Inter- view Follow -up Con- tact Refusal Request to cooperate Refusal Hard converted refusal No Yes No

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 What happens in the AVO after a first refusal? Refusal at first contact Final refusal Re- approached Inter- view Final refusal

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Reluctant respondents Many among –Big city dwellers Few among –Men –Singles –Higher educated

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Cooperative and reluctant respondents

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Refusers, reluctant and cooperative respondents

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Region and refusal conversion

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Results refusal conversion Males, singles and higher educated underrepresented among converted refusals –Do interviewer strategies work for everybody? Are only promising cases re-contacted? –May worsen final sample composition Effect refusal conversion –Works only for honest burghers, Mr. and Mrs. Average?

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Conclusions regular AVO Contactability related to socio- demographics, being out of the house and fieldwork strategy Reluctance related to family composition, gender and education and persuasiveness

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Do nonrespondents differ? Are final refusals similar to converted refusals Sample frame information Neighbourhood information Fieldwork information –Late respondents (similar to non-contacts?) –Reluctant respondents (similar to refusers?) Socio-demographic information

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Response and nonresponse FrameFrame ObservationObservation Easy FieldworkFieldwork Hard to contact Reluctant Follow-up

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Follow-up survey Small subsample persistent refusers 1 person, 20 minutes, multi-mode Experienced, motivated, well paid interviewers Wide range of incentives 235 successful interviews 70% cooperation rate

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Why did refusers cooperate? High quality interviewers (and telling them they are the best) Extensive briefing Trust (money for incentives) and support (newsletter) Importance (newspaper article) Wide range of incentives Better payment Commitment sponsor and fieldwork organisation

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Idiosyncratic choice of incentives

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Final analysis Hard to contact respondents Reluctant respondents Refusers who cooperated in follow-up survey

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 Follow-up survey Sample of persistent refusals Socio-demographic differences (made up for composition regular respondents) Small differences in survey variables (mostly related to socio-demographics) Lower participation in classical culture Less PC ownership Fewer sports activities Slightly less active in many aspects Not at all similar to converted refusals

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005 refuserreluctanthard to contact Intercept-2,42-0,04-0,38 Age (years)-0,010,00-0,01 Age: absolute deviation mean (years) -0,01 Listed phone number0,50-0,10-0,26 Amsterdam, Rotterdam, The Hague-0,220,690,63 Good maintenance-0,21-0,26 Detached dwelling0,00-0,42-0,28 Male-0,16-0,76-0,05 Single0,14-0,370,29 Child at home-0,30-0,11-0,37 Employed, student0,17-0,130,31 Higher education-0,15-0,51-0,03 Cultural participation Both classical and popular-0,21-0,030,13 Classical-0,560,230,17 Popular0,17-0,070,15 No sports activities0,41-0,030,00 PC in household-0,870,180,00 Use of internet, 0,940,070,14 No religious affiliation0,38-0,110,00

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005

Reducing bias instead of increasing response rates Increasing response rates MAY not lead to better survey estimates Difficult respondents may not be similar to final nonrespondents –Obtain information on the process (cause of/reason for nonparticipation) Collect independent information on nonrespondents –Frame information + registers –Central question procedure –Follow-up survey among refusers

Quantitative Methods in the Social Sciences (QMSS) Lugano August 2005