Response rates and disposition coding PHC 6716 June 8, 2011 Chris McCarty

RDD Telephone Disposition Codes I = Complete interview (1.1) P = Partial interview (1.2) R = Refusal and break-off (2.10) NC = Non-contact (2.20) O = Other (2.30) UH = Unknown if household/occupied HU (3.10) UO = Unknown, other (3.20) e = Estimated proportion of cases of unknown eligibility that are eligible

Types of rates Response Rate (RR) = Proportion of cases interviewed of all eligible cases Cooperation Rate (COOP)= Proportion of cases interviewed of eligible cases contacted Refusal Rate (REF) = Proportion of eligible sample that refused Contact Rate (CON) = Proportion of eligible sample where a household was reached

What gets reported? It is rare to see anything other than the response rate reported Occasionally people will report the cooperation rate This is often reported when the response rate is low

AAPOR Response Rates

I = Complete interview (1.1) P = Partial interview (1.2) R = Refusal and break-off (2.10) NC = Non-contact (2.20) O = Other (2.30) UH = Unknown if household/occupied HU (3.10) UO = Unknown, other (3.20) e = Estimated proportion of cases of unknown eligibility that are eligible

Calculating e e is an estimate of the proportion of non-contacts that are eligible Its calculation depends on survey design and execution There are a number of ways it can be calculated Some people just make assumptions about e

Example of calculating e for May 2011 CCI

Comparison of response rates for May 2011 CCI RR111.1 RR211.1 RR312.9 RR412.9 RR516.2 RR616.2

Exceptions to AAPOR Disposition Codes In some cases AAPOR codes may not be detailed or descriptive enough to allow for exceptional circumstances with sample If possible AAPOR codes should be used If new codes must be used they should aggregate to existing AAPOR codes

Last versus Final Dispositions Last Disposition is actually the “most recent disposition” Final disposition requires a rule for evaluating call history There are no firm standards for evaluating call history Research suggests that the difference is trivial

Last versus Final Dispositions -- Research RDD 1RDD 2RDD 3Listed 1Listed 2Listed 3 Sample released Call limit Average attempts per case Completed interviews Response rate using most recent disposition Response rate using ever eligible Response rate using all five final disposition rules Response rate using only Rule Response rate using only Rule Response rate using only Rule Response rate using only Rule Response rate using only Rule Response rate using only Rule 1 and Rule For most recent call: Rule 1: Business always coded as business Rule 2: No eligible respondent always coded as no eligible respondent Rule 3: Disconnected number previously coded as no answer or temporary phone problem always coded as disconnected number Rule 4: Fax/data line previously coded as no answer or temporary phone problem always coded as Fax/data line Rule 5: Non-working number previously coded as a temporary phone problem always coded as a non-working number

Council of American Survey Research Organizations (CASRO) Disposition Codes and Response Rates

Example response rates BRFSS Response rates by state (page 32): ftp://ftp.cdc.gov/pub/Data/Brfss/2010_Summ ary_Data_Quality_Report.pdf ftp://ftp.cdc.gov/pub/Data/Brfss/2010_Summ ary_Data_Quality_Report.pdf Quinnipiac polls: –

What affects Response Rates? Maximum call attempts Maximum number of refusal callbacks Fielding time Rotation of calls across day of the week and time of day The population surveyed Survey length Sample quality Listed versus random-digit dial Salience of survey topic to respondent

Variables in model used to predict response rate across 205 surveys Overall (R 2 =0.52) Listed Sample (R 2 =0.42)RDD Sample (R 2 =0.67) VariableCoeff.Prob>tCoeff.Prob>tCoeff.Prob>t Constant Salience level 2− − Salience level − Number of calls per case Minutes per piece of sample Refusal calls Survey length− − Fielding time Fielding time squared− − − Listed versus RDD− na

What would it take to get a 70% response rate?

Ways people overstate response rates Pre-screening samples incorrectly (e.g. removing persistently unavailable numbers) Unreasonable calculations of e Purchasing RDD sample from higher density banks

Response rates with lists When sampling from a list most of these rules must be modified The fundamental question is “Who is eligible?” If being on the list makes them eligible then they are

Analyze the effects of non-response If you have information about all potential respondents, compare respondents to non- respondents Compare converted refusals to non-refusals Compare early responders to late responders