1. Testing Overview Presentation

2. OPTIMIZELY TESTING TOOLKIT 2016 What is Testing? Why we do it, how we do it, and how to get involved YOUR COMPANY LOGO MM DD YYYY

3. FOR PRESENTERS ONLY Customize this template by replacing all [placeholder text] with information that is most relevant to your business and audience. Enjoy! OPTIMIZELY TESTING TOOLKIT 2016

4. What is testing? Why do we do it? What is our process? How can I get involved? After this presentation, you’ll be able to answer these questions:

5. What is testing? A method for improving the performance of [yourURL/app] and learning more about our audience through controlled experiments.

6. A/B Testing A/B/n Testing Split Testing Multivariate Testing Conversion Rate Optimization (CRO) Conversion and optimization testing and more! Testing is also known as: OPTIMIZELY TESTING TOOLKIT 2016

7. “ Conversion and optimization testing focuses on applying statistical hypothesis testing to multivariable systems.

8. We begin with an idea of what we would like to test and a hypothesis for why we think this change will perform better for our key goal In the experiment, visitors are shown two different versions of an experience, and their engagement with each is measured Whichever version performs best for the predetermined goal is then implemented, until a new hypothesis is tested In an A/B test, the following occurs: OPTIMIZELY TESTING TOOLKIT 2016

9. Site navigation: [Example test or hypothesis] Messaging: [Example test or hypothesis] Visuals: [Example test or hypothesis] Layout: [Example test or hypothesis] Add your own: [Example test or hypothesis] We’ll run experiments on experience components like: OPTIMIZELY TESTING TOOLKIT 2016

10. Which messaging does [audience] prefer? Should our calls to action be links or buttons? What’s the best action for a new customer? [Which steps in our checkout funnel will increase conversions?] [Which lead form style will generate more qualified leads?] [Which headline will increase pageviews?] We’ll run experiments to answer questions like: OPTIMIZELY TESTING TOOLKIT 2016

11. Why do we do it? We’re able to drive more [revenue/conversions/engagement] with our audience while making decisions based on data.

12. Informs key company decisions with data Generates cumulative improvements over time that drive business value Builds a deeper understanding of our audience Allows many more people to learn about and contribute ideas to our customer experience Building a Culture of Testing: OPTIMIZELY TESTING TOOLKIT 2016

13. What is our process? How we achieve testing success and uncover winning hypotheses.

14. Research Ideate Refine Plan & Run Review Testing Process:

15. Determine key goals for optimization Collect direct data: web analytics, heatmaps, user personas, usability tests, etc. Collect indirect data: competitor sites, best- in-class experiences, blogs, webinars, etc. Collect internal ideas from across the organization Research

16. Examine direct data to find areas that are under-performing or points of friction Research competitors’ websites to build awareness of current user experience trends Solicit direct customer feedback about our digital experiences Ideate

17. Prioritize test ideas based on predetermined requirements Evaluate the technical complexity of a potential experiment Document hypothesis and expected experiment outcomes Refine

18. Establish a testing roadmap and timeline Plan experiment duration based on needed sample size to reach statistical significance Set up experiments using Optimizely QA each test to ensure it will not interfere with the user experience Launch and wait (as patiently as we can!) Plan & Run

19. Call the test as winning, losing, or inconclusive Document results and revisit the hypothesis — what did we learn about our audience? Share results internally Decide whether to implement the variation Propose a follow up test based on learnings Review

20. How can I get involved? You can help us reach our testing and optimization goals with experiment ideas, weighing in on results, and much more!

21. Send us your experiment ideas! Vote on which variation you think will win in an upcoming or currently running experiment Share inspiring ideas from websites you visit (or our competitors) Keep asking questions and offering to help! Get Involved

22. Feedback? Questions? Reach out to [Contact Information] YOUR COMPANY LOGO OPTIMIZELY TESTING TOOLKIT 2016

23. Other testing and optimization processes from Optimizely customers. Make best practices your own OPTIMIZELY TESTING TOOLKIT 2016

24. Example: InTheSwim’s Testing Cycle Core team uses “Brainstorm, Test, Analyze, Repeat” METHOD Built a team of a strategist, developer, and designer Set a weekly cadence for working meetings Each meeting rotates focus RESULTS GOALS Move from many random ideas to a more structured process Build repeatable processes and a core team OPTIMIZELY TESTING TOOLKIT 2016

25. Example: InTheSwim’s Testing Flywheel Core team uses “Brainstorm, Test, Analyze, Repeat” to unlock testing wins METHOD Built a team of a strategist, developer, and designer Set a weekly cadence for working meetings Each meeting rotates focus to the next step of the testing phase RESULTS Moved from random test ideas to clear themes based on data Led to biggest win of the program so far! OPTIMIZELY TESTING TOOLKIT 2016 BRAINSTORM TEST ANALYZE

26. Example: TheNextWeb Scaling testing from 1-2 experiments per month to 5-6 per week! METHOD Expanded optimization toolset to include Google Tag Manager, Analytics, and heat mapping. Keep backlog of experiment ideas prioritized and ready to be built and launched. Coordinated with development team to ensure support for the testing program. Scaled as many steps of the optimization process as possible to maintain quick pace. OPTIMIZELY TESTING TOOLKIT 2016 RESULTS Scaled testing output 100x across TheNextWeb, TheNextWeb Deals and TheNextWeb Academy. Read the full storyRead the full story.