1. GAMIFICATION – NOT ALL FUN AND GAMES September 2016
Bringing clarity, delivering breakthroughs
GAMIFICATION – NOT ALL FUN AND GAMES September 2016

2. Who we are and what we do

3. About this session
Objective: to understand the benefits of gamification through demonstrating Breaking Blue’s piloting and testing online surveys and discuss why progress towards improving web surveys has been patchy across the MR industry
Agenda
The challenge we are facing
Our evidence to support gamified surveys
Gamification - the reality
Q&A

4. The challenge we are facing

5. 25%? 50%? 20%? 40%? 15%? 30%? The challenge we face
Overall response rates are falling
50%?
40%?
30%?
Respondents are moving to mobile devices
Desktop
Smartphone
Tablet
Featurephone
Source: Trends Report: Mobile Participation in Online Surveys
Aaron Jue, Market Research Director, FocusVision (2015)

6. Drop out rates on mobile
MAXIMUM TIME DOING SURVEYS:
COMPUTER
TABLET
SMARTPHONE
5 minutes or less 2% 9%
27%
10 minutes or less
24%
45%
15 minutes or less
19%
42%
65%
20 minutes or less
34%
73%
25 minutes or less
71%
77%
30 minutes or less
81%
85%

7. The industry’s response
The main elements of a gamified survey:
State clear rules and goals for participants
Involve participants with a relevant and entertaining narrative
Maintain motivation by providing interesting and achievable tasks
Give feedback on their progress and reward accomplished tasks and answered questions
What role for visualisation?

8. Our evidence to support gamified surveys

9. Benefits of gamification
Less straight-lining: up to 80% lower
Lower neutral scoring: average 25% lower
Lower dropout: from 5% to 1%
“[Gamification is] the most powerful and effective means we have ever come across to encourage participants to put more thought and effort into taking part.”
(Puleston, 2012)

10. Why we decided to run our own experiment
Where gamification is weaker:
B2B surveys or B2B2C surveys
Surveys involving developing markets
Accurate behavioural data surveys
Pic to represent an experiment

11. Our experiment
Breaking Blue’s gamification survey experiment included:
A mixture of global consumer and B2B interviews
An objective to provide detailed behavioural information
A topic with limited interest amongst participants
Customer satisfaction measures
Measured survey experience

12. Our experiment The surveys Baseline survey
Optimised (visualised) survey
Gamified survey
The markets
USA: 100 interviews per survey
UK: 100 interviews per survey
India: 100 interviews per survey
Total: 900 interviews

13. Adding a timed task
QA. Which model of inkjet printer do you use in your home? OR
QB. We want to know the exact model of the inkjet printer you use. You have 120 seconds to check your printer and return back to continue the survey.

14. Visualising the question

15. Tracking study resistance – busted!

16. Customer satisfaction
How satisfied are you with the performance of the xxxxxxxxxxxxx e.g. in terms of speed to first print, print speed in pages printed per minute?
Please rate your satisfaction using the 10 point scale where 1 is not at all satisfied and 10 is completely satisfied.
Standard deviation of printer satisfaction ratings
Not at all satisfied 1 2 3 4 5 6 7 8 9
Completely satisfied 10
How satisfied are you with the reliability of your xxxxxxxxxxxxx ? By reliability we mean that the printer works every time you send something to print.
Please drag the handle to the appropriate position.

17. Satisfaction and understanding
Gamified
8.70
9.04
Visualised
8.59
8.50
8.95
Baseline
How much enjoyed filling in questionnaire
8.32
Rating of variety of question types
8.39
Agreement questions were easy to understand
8.87

18. Length vs time Gamified Visualised Baseline 15.0 17.0 8.51 8.57 16%
10%
Visualised
15.0
8.51
16%
Baseline
Questionnaire length (mins)
14.3
Questionnaire length (rating)
8.56
% unhappy with length
15%

19. Guessing games and quizzes
If you are doing this make sure you:
Complement the survey, do not distract from it
Maintain the look and feel of the survey
Remove if participants are rushing to get to the next guessing game or quiz

20. Open ended questions
Cancer and its treatment can affect all aspects of someone’s life. Before we begin the survey questions, we would be really grateful if you could tell us in your own words how cancer may have affected the day-to-day life of the person you care for in terms of their ability to carry out practical tasks or activities, and how it may have affected their relationships and emotional well-being. Please also tell us about the impact caring for them had on your life.
Participation rate = approx. 85 percent wrote at least one full sentence
Average word count = 63

21. Start as you mean to go on
Even survey introductions can make a huge difference
Version 2
We are speaking with organisations that are registered with the FCA. The main discussion will be about forthcoming changes that will affect your business. It is designed to improve the FCA’s communications with you and is likely to take about minutes.
Cooperation rate = approx. 65 percent
Version 1
We are conducting a random survey of organisations that are registered with the FCA and would like to ask you some questions relevant to consumer credit. Depending on your answers the survey is likely to take about 20 minutes.
Cooperation rate = approx. 50 percent

22. Recommendations from our experiment
Presenting questions better produces significant increases in participant satisfaction
Incorporating games can improve respondent satisfaction but doing so needs to be treated with caution
Possible changes to tracking data can be managed with parallel surveys and calibration of results
Attention needs to be paid to both the survey introduction and placement of open ended questions

23. Gamification – the reality

24. The situation Most surveys are not optimized for engagement
“There is no escaping the fact that [gamification] is a creative solution. There are no out-of-the-box techniques other than perhaps point-based scoring mechanisms.” (Puleston, 2012)
Puleston (2012)
Format like previous quote

25. The survey design process

26. Convincing the client Ways to convince your client:
Run experiments and share results
Conduct a pilot phase and gather feedback on experience
Run parallel gamified surveys alongside traditional

27. The industry needs to work together

28. Questions?