RESULTS FROM AN EXPERIMENT TO PREVENT REFUSALS IN A CONTINUING LONGITUDINAL STUDY Presentation to PHSRN Workshop on Attrition in Cohort Studies, Royal Statistical Society, October 2009

The Research Team Principal Investigator: IAN PLEWIS, CCSR/SOCIAL STATISTICS, UNIVERSITY OF MANCHESTER. Co-Investigators: LISA CALDERWOOD, CLS, INSTITUTE OF EDUCATION, LONDON REBECCA TAYLOR, NATCEN, LONDON Research Officer: SOS KETENDE, CLS

Preventing Refusals Seeking to prevent/convert refusals in an ongoing cohort study – the Millennium Cohort Study – with two interventions applied in an experimental framework. Testing the hypothesis that devoting extra field resources to the problem of increasing cooperation will bring benefits in the forms of increased precision and less attrition bias. Guided by the view that failure to cooperate in later waves of a longitudinal study, conditional on initial cooperation, will be largely circumstantial.

Intervention 1: providing extra information in the form of a letter/leaflet that addresses previously reported concerns and reasons for not continuing to participate. Control condition: no leaflet Piloted in wave 4 dress rehearsal

Reasons for refusal, wave 3 1= “Too busy” (36%) 1= “Nothing has changed” (36%) 1= “Don’t see public benefit” (36%) 4 “Don’t want to bother” (23%) 5 “Stressful family situation” (15%) 6 “Survey too long” (10%) 7 “Looking after children” (8.9%) 8 “Don’t see personal benefit” (5.5%) 9 “Questions too personal” (4.8%) 10 “Survey not important” (4.3%)

Intervention 2: reissue all refusals to the interviewer (except ‘hard’ refusals), usually to a different interviewer. Control condition: standard NatCen reissue strategy (MCS, Wave 3 – just 4.4%).

Crossed design I1, I2. All initial and potentially convertible refusals reallocated to a different interviewer; leaflet sent in advance of the visit from the new interviewer. C, I2. All initial and potentially convertible refusals reallocated to a different interviewer; no prior leaflet. I1, C. Reissues of initial refusals follows NatCen standard practice of reissuing a proportion of all initial refusals; leaflet sent to all initial refusals prior to visit from new interviewer and to all non- issued and potentially convertible refusals with freephone number for those choosing to participate after all. C, C. Reissues of initial refusals follows NatCen standard practice of reissuing a proportion of all initial refusals; no leaflet. This group was likely to contain very few reissued cases.

Issued sample at wave 4 of MCS: n = (GB only) – this was allocated to each of the four experimental groups within the seven GB strata. ‘Intention to treat’ sample – all refusals at first issue: n = 1660 (11%).

Experimental Group by Stratum EnglandWalesScotland GROUP Ad.Disad.EthnicAd.Disad.Ad.Disad.Total Control (n) Leaflet (n) Reissue only (n) Reissue + leaflet (n) Total (%) Issued sample (%)

Outcome numbers by Intention to Treat Group ControlLeaflet Reissue only Reissue + leaflet Fully productive Partially productive Unproductive  No evidence to support any effect of the leaflet.  Reissuing appears to be effective.

Exclusions by Experimental Group Note * - p < 0.01 ControlLeaflet Reissue only Reissue + leaflet ‘Hard’ refusals * 67 (16%) 61 (14%) 80 (21%) 100 (24%) Other exclusions0958 Productive % (eligible)

Conversion attempt rate: 76% (for the two reissue groups). Burton et al. (2006), BHPS, waves 4 to 6: 40%. Productive rate (MCS) : 23% Productive rate (BHPS): 34% but only 13% for F2F interview.

Intensive reissuing reduced refusal rate from 11% to 10%. However, ratio of full to partially productive interviews = 2.4 compared with 8.0 for cases not refusing initially.

Conclusions  Intensive reissuing in an ongoing birth cohort study is effective, possibly more so for main respondents than for partners.  Leaflet addressing respondents’ reasons for refusing is not effective.

Further analyses 1.What was the previous response pattern for the converted refusers? 2.What are the characteristics of the converted cases? 3.Is refusal conversion cost effective?