1. Using Crowdsourcing to Support Pro-Environmental Community Activism Elaine Massung, David Coyle, Kirsten Cater, Marc Jay, Chris Preist CHI 2013 Session: Evaluation Methods 1 Department of Computer Science, University of Bristol, UK

2. Outline Introduction Related Work The Close the Door Campaign Design & User Study Quantitative & Qualitative Results Recommendations Conclusions & Future Work 2

3. Introduction (1/2) Community activist groups, including pro- environmental groups, are typically driven by a core group of highly motivated individuals. Close the Door (CTD) aims to reduce energy waste by encouraging shop owners to keep their doors closed during cold weather. 3

4. Introduction (2/2) The CTD campaign faces the challenge: maintain their database of shops and monitor the behavior of shop owners on an ongoing basis. The aim is to increase the scalability of activist groups; What design strategies can we apply to motivate people to engage in pro-environmental data collection? 4

5. Related Work (1/3) Pro-Environmental Behavior Change – Prior HCI research on technology-supported environmental behavior change has been dominated by systems targeting individual behavior change. – The norm-activation model hypothesize that individual behavior is strongly shaped by community norms and everyday practices. 5

6. Related Work (1/3) Pro-Environmental Behavior Change Individual (rational choice) + Community (norm-activation) 6

7. Related Work (2/3) Citizen Science and Participatory Urbanism Citizen science (collect data in support of scientific research) + Participatory urbanism (municipal authorities seeking to engage people in monitoring and improving their urban environments) 7

8. Related Work (3/3) Motivating and Engaging Contributors intrinsic motivation (interest, curiosity, competence, enjoyment) + extrinsic motivation (financial pay-for-results & virtual reward) 8

9. The Close the Door Campaign On UK high streets it is common to see shop doors propped open -> energy costs However owners fear that they will lose customers if they close their doors whilst other shop doors remain open. The CTD campaign attempts to combat this problem by sending volunteers to shopping streets to record whether shops have their doors open or closed. 9 “If a shop will not Close the Door: don’t shop there, go elsewhere.”

10. Design (1/3) Control, Virtual reward and Financial app 10

11. Design (2/3) Data quality – A system of independent verification by a second user was used to validate new shops – Add new shops or to record doors within 200 yards of present location, 1 shop/day. 11

12. Design (3/3) By comparing the effectiveness of these apps we aimed to provide evidence of the effectiveness of different motivational strategies that can be applied to engage casual participants to collect pro- environmental data. 12

13. User Study (1/4) We recruited 48 participants: 60% female, aged 17-59, with a mean age of 27. Participants were told they could earn gift vouchers for taking part in a study that would involve using an iPhone app and was undertaken in collaboration with the CTD environmental campaign. 13

14. User Study (2/4) Participants were randomly assigned to 3 groups (Control, Virtual and Financial) and asked to record data on as many shop doors as possible while going about their everyday routines for a period of 2 weeks. Participants would receive a £50 voucher for participating in the study? 14

15. User Study (3/4) All the participants completed a preliminary online questionnaire to assess their attitude to environmental sustainability. We calculated an environmental disposition score for each participant by summing their responses to the behavior questions only. This provides a measure of their intrinsic motivation to engage in environmental activities. 15 five-point Likert scale

16. User Study (4/4) These apps kept track of the points and badges that the participants would have earned from their behavior to allow for comparison between the 3 groups. Based on total points, 2 high, 2 mid-range, and 2 low scorers from each group were selected for a semi- structured interview. This delved into further detail about app usage and motivational factors. 16

17. Quantitative Results (1/) Participants made 6674 individual recordings and added 1113 new shops to the CTD database. 17

18. Quantitative Results (1/5) Participants made 6674 individual recordings and added 1113 new shops to the CTD database. 18 many participants reported being unaware of how to earn the badges

19. Quantitative Results (2/5) Data quality was very high across all experimental conditions. – Using Google maps and local knowledge, we checked a random 10% of all added shops (including unverified ones) across the 3 groups. – All but one was found to be accurate. 19 ^ unverifiable

20. Quantitative Results (3/5) The non-parametric Kruskal-Wallis Test was carried out to test for differences between the groups. We then conducted post-hoc Mann-Whitney U Tests, with Bonferroni correction, to further assess the differences. 20 don’t follow a normal distribution

21. Quantitative Results (4/5) To investigate this further we plotted the Z- scores of all participants to test the relative comparative performance of individuals in each group. 21

22. Quantitative Results (5/5) Effect of existing environmental tendencies – We performed a correlation analysis within each group comparing participants’ environmental disposition scores, from the initial questionnaire, and their point scores. – The results showed no significant correlation. 22

23. Qualitative Results (1/3) The interviews were audio recorded and transcribed, and then undertook a thematic analysis. The Usability of the Apps – A vast majority of the participants felt that the app was intuitive and easy-to-use. – Some participants expressed frustration at the GPS receiver not accurately locating their position. 23

24. Qualitative Results (2/3) Motivators – Performance-Related Financial Incentives – Competition – Community Norms – Environmental Interest 24

25. Qualitative Results (3/3) Enablers – Lifestyle – Technology – Weather Behavior & Awareness As a way of drawing attention to the issue in a wider population and attracting more permanent volunteers to the organization. 25

26. Recommendations (1/2) 1.Seek those whose lifestyle is likely to enable them to participate. 2.Use passion for a cause as a threshold motivator, but do not assume it acts as an engagement motivator. 3.Make competition available, but easy to ignore. 26

27. Recommendations (2/2) 4.Provide information regarding ‘community norms’ in a way which motivates desired behavior. 5.Use financial motivation carefully. 27

28. Conclusions We have assessed the impact of different motivating factors and design strategies on the performance of subjects collecting data for the CTD pro-environmental campaign. We made recommendations for organizations wishing to design and use mobile apps to support community activism. 28

29. Future Work Examine motivation over a longer time period and what can be done to support continued engagement. Explore the motivating effect of competing with one’s own social group. Intrinsic pleasure; Self-gamification 29