GAMIFICATION – NOT ALL FUN AND GAMES September 2016 Bringing clarity, delivering breakthroughs GAMIFICATION – NOT ALL FUN AND GAMES September 2016

Who we are and what we do

About this session Objective: to understand the benefits of gamification through demonstrating Breaking Blue’s piloting and testing online surveys and discuss why progress towards improving web surveys has been patchy across the MR industry Agenda The challenge we are facing Our evidence to support gamified surveys Gamification - the reality Q&A

The challenge we are facing

25%? 50%? 20%? 40%? 15%? 30%? The challenge we face Overall response rates are falling 50%? 40%? 30%? Respondents are moving to mobile devices Desktop Smartphone Tablet Featurephone Source: Trends Report: Mobile Participation in Online Surveys Aaron Jue, Market Research Director, FocusVision (2015)

Drop out rates on mobile MAXIMUM TIME DOING SURVEYS: COMPUTER TABLET SMARTPHONE 5 minutes or less 2% 9% 27% 10 minutes or less 24% 45% 15 minutes or less 19% 42% 65% 20 minutes or less 34% 73% 25 minutes or less 71% 77% 30 minutes or less 81% 85%

The industry’s response The main elements of a gamified survey: State clear rules and goals for participants Involve participants with a relevant and entertaining narrative Maintain motivation by providing interesting and achievable tasks Give feedback on their progress and reward accomplished tasks and answered questions What role for visualisation?

Our evidence to support gamified surveys

Benefits of gamification Less straight-lining: up to 80% lower Lower neutral scoring: average 25% lower Lower dropout: from 5% to 1% “[Gamification is] the most powerful and effective means we have ever come across to encourage participants to put more thought and effort into taking part.” (Puleston, 2012)

Why we decided to run our own experiment Where gamification is weaker: B2B surveys or B2B2C surveys Surveys involving developing markets Accurate behavioural data surveys Pic to represent an experiment

Our experiment Breaking Blue’s gamification survey experiment included: A mixture of global consumer and B2B interviews An objective to provide detailed behavioural information A topic with limited interest amongst participants Customer satisfaction measures Measured survey experience

Our experiment The surveys Baseline survey Optimised (visualised) survey Gamified survey The markets USA: 100 interviews per survey UK: 100 interviews per survey India: 100 interviews per survey Total: 900 interviews

Adding a timed task QA. Which model of inkjet printer do you use in your home? OR QB. We want to know the exact model of the inkjet printer you use. You have 120 seconds to check your printer and return back to continue the survey.

Visualising the question

Tracking study resistance – busted!

Customer satisfaction How satisfied are you with the performance of the xxxxxxxxxxxxx e.g. in terms of speed to first print, print speed in pages printed per minute? Please rate your satisfaction using the 10 point scale where 1 is not at all satisfied and 10 is completely satisfied. Standard deviation of printer satisfaction ratings Not at all satisfied 1 2 3 4 5 6 7 8 9 Completely satisfied 10 How satisfied are you with the reliability of your xxxxxxxxxxxxx ? By reliability we mean that the printer works every time you send something to print. Please drag the handle to the appropriate position.

Satisfaction and understanding Gamified 8.70 9.04 Visualised 8.59 8.50 8.95 Baseline How much enjoyed filling in questionnaire 8.32 Rating of variety of question types 8.39 Agreement questions were easy to understand 8.87

Length vs time Gamified Visualised Baseline 15.0 17.0 8.51 8.57 16% 10% Visualised 15.0 8.51 16% Baseline Questionnaire length (mins) 14.3 Questionnaire length (rating) 8.56 % unhappy with length 15%

Guessing games and quizzes If you are doing this make sure you: Complement the survey, do not distract from it Maintain the look and feel of the survey Remove if participants are rushing to get to the next guessing game or quiz

Open ended questions Cancer and its treatment can affect all aspects of someone’s life. Before we begin the survey questions, we would be really grateful if you could tell us in your own words how cancer may have affected the day-to-day life of the person you care for in terms of their ability to carry out practical tasks or activities, and how it may have affected their relationships and emotional well-being. Please also tell us about the impact caring for them had on your life. Participation rate = approx. 85 percent wrote at least one full sentence Average word count = 63

Start as you mean to go on Even survey introductions can make a huge difference Version 2 We are speaking with organisations that are registered with the FCA. The main discussion will be about forthcoming changes that will affect your business. It is designed to improve the FCA’s communications with you and is likely to take about 15-20 minutes. Cooperation rate = approx. 65 percent Version 1 We are conducting a random survey of organisations that are registered with the FCA and would like to ask you some questions relevant to consumer credit. Depending on your answers the survey is likely to take about 20 minutes. Cooperation rate = approx. 50 percent

Recommendations from our experiment Presenting questions better produces significant increases in participant satisfaction Incorporating games can improve respondent satisfaction but doing so needs to be treated with caution Possible changes to tracking data can be managed with parallel surveys and calibration of results Attention needs to be paid to both the survey introduction and placement of open ended questions

Gamification – the reality

The situation Most surveys are not optimized for engagement “There is no escaping the fact that [gamification] is a creative solution. There are no out-of-the-box techniques other than perhaps point-based scoring mechanisms.” (Puleston, 2012) Puleston (2012) Format like previous quote

The survey design process

Convincing the client Ways to convince your client: Run experiments and share results Conduct a pilot phase and gather feedback on experience Run parallel gamified surveys alongside traditional

The industry needs to work together

Questions?