1. Can online deliberation transform citizens? Preliminary findings from an internet field experiment in the UK Informing Public Policy Friday 24 th April 2009 Hisako Nomura, Research Associate Institute for Political and Economic Governance University of Manchester

2. The project One element of the Rediscovering the Civic research programme Funded by ESRC/CLG/NWIN Joint project of University of Manchester (Institute for Political and Economic Governance) and University of Southampton (Centre for Citizenship and Democracy Variety of experiments (RCTs, design experiments) on aspects of civil engagement (voting, recycling, lobbying, pledging etc) http://www.civicbehaviour.org.uk/research/

3. Background on online deliberation Attempts to encourage deliberation on public policy are often local and small scale (e.g. citizen’s juries) Information and communication technology (ICT) potentially offers new possibilities for public engagement –Opening a new “public sphere” of the sort envisioned by Habermas (1962, 1989) For the public, a new way to engage with others to discuss –Online discussion For the public sector, a new way to extend their reach –Online questionnaires –Discussion forums –Webchats, texting, e-mailing responses, ePetitions, Interactive voting, opinion meters etc.

4. Online deliberation The internet allows for an opportunity to ‘scale-up’ - to involve large numbers without the costs of physically bringing people together It is often claimed to promote honesty, directness in political exchanges and contact with people from different backgrounds which is appropriate for deliberation exercise (See Dahlberg 2001, Witschge 2004) On the other hand, anonymous encounters online can exert normative group influence (Postmes, Spears, & Lea, 1998) In that case, it runs the risk that the online deliberation experience may be diluted or even counterproductive

5. Information only vs Deliberation There is an ongoing debate in the literature on deliberative democracy as to whether it is information alone or information plus deliberation that has an effect on judgements Goodin (2003) argues that internal reflection (or ‘reflective deliberation’) is enough –interaction between citizen is not the important factor Others from public opinion theory argue that exposure to disagreement (i.e. interaction with others) does indeed contribute to more deliberative opinion formation (Price, Cappella and Nir, 2002)

6. RCT on deliberation Social scientist have evaluated deliberative forums to see if participants change their views, knowledge and opinions (Drysek 2002, Fishkin 1997, Delli Carpini et al 2004, Price, Nir and Cappela 2006) Few randomised control trials testing for the effect of deliberation (exceptions are Farrar et al 2003, Iyengar 2005) Moreover, existing studies have been confined to small numbers –mini publics – of participants who are able to participate in a face-to-face deliberation. No experiments testing the deliberative potential of online discussion forums

7. Research Questions To what extent does online engagement amongst large numbers lead to shifts in policy preferences and knowledge? To what extent can online engagement amongst large numbers be termed deliberative? What is the value of citizens’ interaction compared to the provision of information alone?

8. The RCT design 6,000 participants from Ipsos-MORI survey panel using random quota sampling. Randomly allocated to 4 treatment and 2 control groups 2 x deliberation groups –Goup 1: youth anti-social behaviour (ASB) followed by community cohesion (CC) –Group2: community cohesion (CC) followed by youth anti-social behaviour (ASB) –Access to discussion boards and background information 2x information-only groups (video clips and documents) 2 x control groups All participants completed three surveys –T1 (before experiment), T2 (after first issue), T3 (after second issue)

9. Control 1 ASB-CC N=825 /1001 Attrition=18% Control 1 CC-ASB N=825 Attrition=18% Survey 1 Full delib 2 CC-ASB N=742 /1002 Attrition=26% Full delib 2 ASB-CC N=695 Attrition=31% Info. 1 CC-ASB N=663 Attrition=34% Info. 2 ASB-CC N=642 Attrition=36% Full delib 1 CC-ASB N=714 Attrition=29% Control 2 ASB-CC N=797 Attrition=21% Survey 3 Anti-Social Behaviour (ASB) or Community Cohesion (CC) issues Total sample size N=6000 Info. 1 ASB-CC N=718/1002 Attrition=28% Info. 1 CC-ASB N=731 /1001 Attrition=27% Full delib 1 ASB-CC N=786 /1002 Attrition=22% Control 2 CC-ASB N=825/1001 Attrition=18% Survey 2 Anti-Social Behaviour (ASB) or Community Cohesion (CC) issues Deliberation Information Only CONSORT Flow Diagram

10. Hypotheses H1: An internet deliberative forum has an impact on participants’ views, knowledge and opinions. H2: Information alone has an impact on opinion change H3: The order of the topic can make an impact on opinion change. H4: The impact of deliberation is greater than information alone H5: Deliberation results in more opinion shift in youth anti-social behaviour than in community cohesion

11. The initial findings Just over 50% of the treatment groups took part in the online activities Completion of survey was much higher –77.5% who had anti-social behaviour topic completed T2 survey (73.2% of them completed community cohesion questionnaire in T3) –76.5% who had community cohesion topic completed T2 survey (71% of them completed anti-social behaviour questionnaire in T3) No obvious large-scale shifts in opinions. Many of the threads read like ‘speak your brains’ rather than exchange of opinions and considered reflection. –There was lack of responsiveness

12. Further analysis Application of propensity score matching of those who actually took part in online deliberation in the treated group with similar members of the control group for further analysis Application of adapted Discourse Quality Index for qualitative analysis