1. Analytical Optimization Technologies for Games & Apps Analytics, A/B Testing, Segmentation & Dynamic Best-Fit Alan Avidan, Exec. Director & Chief BeezzzDev

2. Points We’ll Cover What is optimization What can be measured and optimized Optimization technologies for games and apps Analytics A/B Testing User Segmentation Dynamic Best-Fit Let’s get started!

3. Optimization Family Tree

4. What is Optimization? Data-driven efforts formulated and designed to maximize Key Performance Indicators (KPI) by enhancing in-game/app conversions Max Z {f(x)} ≡ f( Engagement, Retention, Monetization, Virality ) X s.t. g(x)=0, h(x)<0

5. Which Key Performance Indicators should you target for optimization? Monetization Engagement Retention Virality

6. Analytics and Optimization Companies

7. Optimization Results 88.9% improvement on landing page

8. Which Game Elements Can Be Optimized? New Features Arts (Creative) Message Wordings Game Mechanics Game Flow Landing Pages Promotions

9. Optimization Technologies We Use Analytics A/B Testing (Split Testing) User Segmentation Dynamic Best-Fit

10. The process of developing optimal or realistic decision recommendations based on insights derived through the application of statistical models and analysis against existing and/or simulated future data - Wikipedia Typical uses of Analytics Engagement Tracking Funnel Analysis Measure, Display, Analyze, Change, Repeat Analytics

11. Analytics - Bottom Line Upside Monitor, record, & display Key Performance Indicators (KPI) Measure effectiveness of game mechanics and monetization Efforts Access and display data to understand how users interact with game/app; decide where improvements are needed Downside The capture and storage of data, followed by analytics and visualization is tedious, provides retroactive information about the “Average User.”

12. A/B Testing Credit: Steve Collins, Swrve

13. A/B Testing Uses New features are introduced to a selection of users, and their reactions measured. Features remain only if users engage with them - Wooga Photo: Spencer Higgins; Illustration: Si Scott

14. Q: A/B Testing: What are the most unexpected things people have learned from A/B tests? Answer Wiki 1.Make sure that the test is statistically significant - run it for long enough, and with enough traffic to make it count 2.I have learned how dramatically, and ridiculously wrong my most basic assumptions were 3.It's empirically proven that you should let the data tell you what works or not and you should constantly be testing 4.That the devil is in the detail - a minor change can generate a significant result

15. A/B Testing – Bottom line Upside Simple; understandable; can achieve very good results Downside: One size fit all

16. User-Base Segmentation A Priori Segmentation: Geographic - states, regions, countries Demographic - age, gender, education Psychographic - lifestyle, personality, values Positive - similar wants or needs Clustering Segmentation: Behavioral - similarities of behavioral patterns and like-properties

17. Segmentation - Uses Cohort Analysis – Track over time users with common reference feature Targeting - Serve different treatments for each segment to maximize KPIs

18. Segmentation – the bottom line Upside Can be effective especially reaching out to groups identifiable by known attributes Downside: –Clusters are predefined and thus remain unchanged during the analysis –Requires storage of terabytes of data –privacy issues

19. Dynamic Best-Fit Real-Time Automated Action Optimization A predictive algorithmic technology used to serve each user the page option they are most likely to convert on at any feature point

20. DNA Signature Attributes Geo-Demographic attributes: age, gender, education, country Facebook attributes: Friends, Likes, Interests, Posts, Events Behavioral attributes: level, spending, score, progress, custom Session attributes: time of day, day, duration Proprietary attributes: novice, high-bidder, risk-averse 3 rd Party attributes: income level, education

21. How Dynamic Best-Fit Works Advanced statistical algorithms find strong correlations between user DNA data and past conversions

22. Best-Fit Wording

23. Best-Fit Options Payment Pages: Different Ranges

24. Best-Fit Options Payment Pages: Different Incentives

25. Best-Fit: Game Flows Option 1 Option 2 Open page Full tutorial Stage 1 Open page Short tutorial Stage 1 Option 3 Open page No tutorial Stage 2

26. Best-Fit: Payouts Only large and less frequent winnings* * The sum of all winnings are the same Mostly small but more Frequent winnings*

27. Best-Fit: Invite Friends - Different Layouts

28. Best-Fit: Promotions Go VIP Buy VIP card for 5 EUR and enjoy 30% more coins for all future buys Buy 1 Get 1 FREE Receive twice the amount of gold for regular price Triple your money Buy 100 Gold Get 200 Free

29. Best-Fit Arts

30. Dynamic Best-Fit: Results “ Total Friends ” attribute as conversion indicator in payment page Insight: users with less than 100 friends more readily reach the payment page, and moreover convert better “ Like ” attribute as conversion indicator in payment page Insight: users with more than 25% of Likes associated with apps monetize much better, and moreover clearly prefer Layout 2 Increases conversions and KPIs Gain Valuable new insights to improve app design and user targeting

31. Review Optimization is vital to your game/app’s success Retrofit existing games and plan for future games Match objectives with technologies: Different technologies have different uses; Require a different level of involvement; and produce different Uplift results Future? -- Lots and lots more data. Those that will learn to harness it will succeed

32. Alan@BeesAndPollen.com Q & A