The effects of panel conditioning on party support and political interest in Understanding Society Nicole Martin University of Essex

Motivation “Each time the interview comes around, it gives me a chance to assess my progress since the last meeting. The interviewer’s questions about my happiness levels at home, at work, and with friends and family are really the only time at which I am made to stop and think about each area of my life. This way, it feels like an informal annual report on the business that is my life. Here’s to many more years of self-administered personal progress reports!”

Motivation Two specific biases to panel data: (i) people leaving the study (attrition) in a non-random way. (ii) people changing their answers as a result of being surveyed repeatedly (panel conditioning). Panel conditioning isn’t always a negative thing. Political attitudes are: (i) an attitude not a behaviour. (ii) conceivably affected simply by asking questions (don’t think of an elephant). (iii) potentially sensitive information.

Motivation An Innovation Panel experiment asked people questions about their body weight and height different amounts of time. Both are known to be misreported in line with cultural norms. Those who were asked the questions more times gave answers that appear less biased. Uhrig (2014) interprets this as the survey process gaining the respondent’s trust. Fisher (2016) noticed differences in income reporting behaviour in Understanding Society, which appear to be panel conditioning where respondents mention more income components after they have been interviewed the first time.

Identity(fication) crisis The easiest way to identify panel conditioning is through an experimental design. But not normally feasible, so panel conditioning is usually subsumed into the effect of time and/or age, or into differences between different samples drawn at different points (with different patterns of attrition). Luckily, there is a quirk of Understanding Society fieldwork that permits the identification of panel conditioning.

Identification strategy Understanding Society has a new wave going into the field each January, but each wave lasts two years. The sample of each month of the (General Population Sample) were stratified in such a way that each month is a representative sample of Great Britain. Some questions repeated each year. ….So we can look at comparable samples being asked the same question at the same point in time, with their number of previous interviews the only difference between them. Deal with attrition by keeping a balanced panel i.e. people interviewed (not proxy) at each wave 1-5. The one exception is age.

Fieldwork Schedule 2008 (Sep-Dec) 2009 2010 2011 2012 2013 BHPS w18   GPS – GB w1 EMBS w1 GPS – NI w1 GPS – GB w2 EMBS w2 GPS – NI w2 BHPS w2 GPS – GB w3 EMBS w3 GPS – NI w3 BHPS w3 GPS – GB w4 EMBS w4 GPS – NI w4 BHPS w4

Respondent co-operation

Measuring party preference in Understanding Society Now I have a few questions about your views on politics. Generally speaking do you think of yourself as a supporter of any one political party? (Which one?) Do you think of yourself as a little closer to one political party than to the others? (Which one?) If there were to be a general election tomorrow, which political party do you think you would be most likely to support?

Combined measure: original sample

Combined measure: original sample

Combined measure: new entrants to the panel

Combined measure: new entrants to the panel Data: Pooled new entrants to UKHLS over waves 2-4 in their first two waves, who remained in the panel at least for the subsequent year. For a real 2 percentage point gap, sample size needed c6494

Which party benefits? Data: 2011 year, UKHLS GPS sample Wald tests show no difference in the effect for the destination party Data: 2011 year, UKHLS GPS sample

Highly-engaged or unengaged with politics? The effect of having one fewer interview is stronger for respondents with higher political interest Data: 2011 year, UKHLS GPS sample

Political interest No effect on political interest Data: UKHLS year 2011

Robustness checks Interaction with whether respondent received post-general election questions. Respondents in a shorter length balanced panel.

Conclusions In Understanding Society, there appears to be a positive effect of panel conditioning on the probability of expressing a party preference – despite this generally declining over the time period being studied. Short-lived? By the time the fourth wave had entered its first year, there was no difference. And out-weighed by other changes in this case. However, no effect seen on political interest. Poses a (small) threat to external validity of long panels. Poses a (small) threat to using panels as representative cross-sections.

What to do about it? In this case, it would be possible to use wave year as an instrument for number of interviews, and adjust accordingly. But (by default) this would assume that each additional interview has the same effect. Where surprising time trend noticed in panel data, should try to exclude panel conditioning as an alternative e.g. by looking at new entrants to a panel and evolution of responses (NB assumption about attrition over time).

Limitations Can’t measure the effect on actual behaviour e.g voting, due to rotating question modules. Maybe the two samples aren’t as balanced as they should be. No information on the mechanism – is this a change in reporting, or a change in attitudes? (Or both). Maybe treatment varied over waves. Fieldwork agency learning over the course of the wave.