Disability questions in censuses and surveys Methodological issues: proxy response, non-response and mode of administration September 2005 Montserrat López-Cobo Fifth Meeting of the Washington Group

Fifth Meeting of the WASHINGTON GROUP 2005  Are proxy-responses different from self-responses? Why?  Do all proxies provide the same quality responses? Who is the best proxy?  Is there an effect on the quality of data? Can we measure it? Can we correct it? Proxy response

Fifth Meeting of the WASHINGTON GROUP 2005 It is generally accepted that proxy- and self- responses differ  Many studies found that proxy-respondents underreport disability-related information:  Disability  Impairments of emotion and Pain  Limitation of activities, long-term disability, need for assistance with ADL  Chronic condition, days of activity restriction, bed disability days  General health events and conditions Proxy response Level differences 1

Fifth Meeting of the WASHINGTON GROUP 2005 ... While some other studies found that proxies overreport:  Cognitive status, Mild Cognitive Impairment  ADL disability  Functional health, when the self-respondent is demented Proxy response Level differences 2

Fifth Meeting of the WASHINGTON GROUP 2005  Differences in cognitive procedures (1/2)  Type and level of available information o Proxies report impairments or conditions that are more observable or mentioned to them o Proxy reports are more accurate for conditions that are serious, painful, persistent or potentially life threatening  If person under 65: proxies underreport  If person 65 and over: proxies overreport  Proxy information is more stable. Self information is more dynamic  Self-responses are less consistent across time Proxy response Understanding self-proxy differences 1

Fifth Meeting of the WASHINGTON GROUP 2005  Differences in cognitive procedures (2/2)  Proxies engage in less extensive cognitive process. Provide heuristic-based responses  Proxies rely more on inferences and estimations.  Proxies overreport disabilities aparently related to a previously reported disability and underreport disabilities seemingly unrelated to the previously reported.  Proxies are less affected by social desirability concerns  Proxies are more willing to report disability Proxy response Understanding self-proxy differences 2

Fifth Meeting of the WASHINGTON GROUP 2005  Social relationship with the selected person  The closest the relationship, the most accurate the proxy-response  Spouse’s responses are better than other proxies’  Proffesionals and caregivers provide more accurate information than lay proxies  Length of relationship  Positive effect in convergence self-proxy  Memory capacity of respondent  Affect to data quality Proxy response The best proxy-respondent

Fifth Meeting of the WASHINGTON GROUP 2005  Types of methods:  1) Traditional  First step: Measure the bias by comparing proxy- estimates with self- estimates  Second step: Introduce statistical control adjusting for demographic or health-related variables  2) New strategy  First: Set assumptions about the nature of differences between self- and proxy-responses  Second: Model and estimate these differences based on the assumptions Proxy response Measuring and correcting bias

Fifth Meeting of the WASHINGTON GROUP 2005 Mode of administration

Fifth Meeting of the WASHINGTON GROUP 2005  Differences in the sample  Population coverage  Response rates  Differences due to social context within which data are collected  Contact with interviewer  Non-verbal cues  Social desirability bias: Reduced with self- administered questionnaires Mode of administration Sources of differences among modes 1

Fifth Meeting of the WASHINGTON GROUP 2005  Differences intrinsic to the mode  Visual materials cannot be used by phone  Complexity of the questionnaire is limited by the mode  Stimuli in a visual mode  Primacy effects Stimuli in a hearing mode  Recency effects  Differences in responses involving long lists  Order effects  Effects of the mode on the interviewer Mode of administration Sources of differences among modes 2

Fifth Meeting of the WASHINGTON GROUP 2005 Telephone interviews  Do not underrepresent people with disabilities  Sometimes is not as well accepted by population as face-to- face interviews  Proved useful to assess mental health using recommended instruments (GHQ-12, CIS-R, CIDIS), the Expanded Disability Status Sacle Telephone vs Mail  Mail responses report poorer health and more chronic conditions than Telephone  Differential non-response rates by age: Elder: NR (Telephone) > NR (Mail) Young: NR (Telephone) < NR (Mail) General: Item NR (Telephone) < Item NR (Mail) Mode of administration Results from comparative studies 1

Fifth Meeting of the WASHINGTON GROUP 2005 Face-to-Face vs Mail  Health differences between respondents and non-respondents by mode: non-response bias  Non-respondents and late respondents to Mail are more cognitively impaired and more disabled than respondents (among elderly).  Non-respondents to F-t-F are similar to respondents. Mode of administration Results from comparative studies 2

Fifth Meeting of the WASHINGTON GROUP 2005 Telephone/CATI vs Face to Face  Differential non-response rates: NR (CATI) > NR (F-t-F)  Measurement bias: MBias (CATI) < MBias (F-t-F)  No differences between CATI and F-t-F for reports on chronic conditions, activity limitations and disability rates CATI vs CAPI  Differential non-response rates by age: Elder: NR (CATI) > NR (CAPI) Young: NR (CATI) < NR (CAPI) Mode of administration Results from comparative studies 3

Fifth Meeting of the WASHINGTON GROUP 2005 CASI  Reduces social desirability bias  Enhances the feeling of privacy  Respondents generally like CASI  Technological possibilities have a positive influence on data quality (minimizing errors)  Respondent’s self-disclosure is higher in CASI Mode of administration Results from comparative studies 4

Fifth Meeting of the WASHINGTON GROUP 2005  Disabilities: Physical, sensory, mental illness and mental retardation  Challenges addressed: communication, fatigue and cognitive issues  Questionnaire design  Eliminate soft consonant sounds (s, z, t, f and g) to overcome high-frequency hearing loss  Build in "breaks" for respondents to let them rest  Incorporate neutral encouragement to avoid drop-outs  Design checks for unexpected responses  Use structured probes for questions that might be difficult to understand Mode of administration An experience interviewing disabled people by telephone

Fifth Meeting of the WASHINGTON GROUP 2005  Interviewer training and supervision  Usual background and purpose of study  Training on challenges likely to face  Sensitive exercise regarding the treatment to the disabled person  Guidance to overcome each of the challenges  Support to interviewers and reduction of stress  Other recommendations  Interviews take longer  Multiple sessions may be required Mode of administration An experience interviewing disabled people by telephone

Fifth Meeting of the WASHINGTON GROUP 2005 Non-response

Fifth Meeting of the WASHINGTON GROUP 2005  Components of non-response  Non-contact  Non-cooperation  Non-contact: the household pattern of disabled  Are older and likely to live in non-metropolitan areas  higher probability of being contacted  Less likely to live with children  lower prob. of contact  May be more fearful of opening doors to strangers  lower prob. of contact Non-response and Disability

Fifth Meeting of the WASHINGTON GROUP 2005  Non-cooperation: exchange theory vs social isolation theory  Exchange theory: Persons who feel that the survey sponsor has provided (or could provide) benefits to them are more likely to cooperate  Persons with disabilities might be expected to be more cooperative with a government-sponsored survey  Social isolation theory: P eople who are isolated from the mainstream society feel less responsibility toward government and are less likely to cooperate  Persons with disabilities might be expected to be less cooperative Non-response and Disability

Fifth Meeting of the WASHINGTON GROUP 2005  Persons with severe disability are more likely than persons with less severe disability to be contacted and to cooperate, but when they are interviewed they are less likely to answer for themselves; instead, proxy and assistant respondents tend to answer for them.  Elder people with disabilities respond sooner than young and non-disabled to mail surveys.  These findings support Exchange theory  Self-perceived memory problems increase item non-response and “I don’t know” answers. Not preceived memory deficits implies inaccurate information on items requiring recall. Non-response and Disability Results from studies 1

Fifth Meeting of the WASHINGTON GROUP 2005  Health of non-respondents is worse than respondents’ in terms of: stroke, Basic ADL, mobility disabilities, self-rated health and mortality rates.  Respondents with poorer physical functioning and/or limiting long-term illness have higher non- response rates.  These findings support Social isolation theory Non-response and Disability Results from studies 2

Fifth Meeting of the WASHINGTON GROUP 2005 Recommendations for including disabled people in interview surveys

Fifth Meeting of the WASHINGTON GROUP 2005 Bias due to exclusion is amplified in surveys where disability is a key measure of interest  Recommendations:  Include institutionalised population in samples  Carry out the interview in a private environment  Provide adaptative technologies and procedures (of primary importance in visual, hearing and speech impairment).  Use simple and clear questions

Fifth Meeting of the WASHINGTON GROUP 2005  Recommendations:  Special training for interviewers  Provide alternative modes of administration  If proxy is used:  Respondent’s assessments of proxy’s answer is valuable  The proxy should be nominated by the selected respondent

Fifth Meeting of the WASHINGTON GROUP 2005  Proxy response validity depends on factors such as: proxy choice and distinction, topic investigated, health condition of the selected person...  Mode of administration (unique or a mix- mode) should be decided taking into account: topic investigated, population objective of the survey, expected acceptability of the mode by the population, technological possibilities... Final conclusion

Fifth Meeting of the WASHINGTON GROUP 2005  Different patterns of response might be indicative of bias which can affect estimates.  Non-response is one of the sources of data error. But not the unique... Emphasis should be given not only to minimazing non- response rates but also to estimation and control of measurement error. Final conclusion

21-23 September 2005 Fifth Meeting of the Washington Group