A/B testing aka split testing, bucket testing, and multivariant testing.

Slides:



Advertisements
Similar presentations
SEO IS NOT WHAT IS WAS Search Engine Optimization = WEB OPTIMIZATION
Advertisements

Welcome to Jestek Web Site SITE DELUX Making Web Site Changes: Site Add-Ons This tutorial is designed to help you make changes on your own with your SiteDelux.
DRUPAL How to create a website Summer Tech Academy 2010 Mercedes Conde.
”SEO” Search engine optimization Webmanagement training - Dar es Salaam 2008.
USING WORDPRESS. WEEK 1 1.Why WP? 2.Setting Up WP 3.Exploring the Admin screen 4.Page Organization 5.Posting 6.Polls.
Website Conversion Testing: Google Website Optimizer Brent Ludtke.
The Story of Past Presidents. What is PhotoStory?  PhotoStory is a free program from Microsoft that allows you to create a digital story.  You can personalize.
Digital Marketing Overview Tpugliese Adapted from Anton Koekemoer | April 2012.
WELCOME TO THE ANALYSIS PLATFORM V4.1. HOME The updated tool has been simplified and developed to be more intuitive and quicker to use: 3 modes for all.
The many uses of an iPad Here are a couple tips to keep in mind: Keep it simple: A website is a place for the exchange of information. It does not need.
Assignment: Improving search rank – search engine optimization Read the following post carefully.
Web Design, Usability, and Aesthetics 3
Web Page Design Tips & Tricks Layering Choose Insert – Layout Objects – Layer Click anywhere along the outline of the layer box to select it Click &
Web Design, Usability, and Aesthetics 2 Billboard Design 101 “Designing pages for scanning not reading” Notes from book “Don’t Make Me Think: by Steve.
There is a certain way that an HTML file should be set up. The HTML section declares a beginning and an ending. Within the HTML, there should be a HEAD.
Web 2.0 Testing and Marketing E-engagement capacity enhancement for NGOs HKU ExCEL3.
IStateLink 3.0 Help and Training Tutorial December 2010.
Accessible Word Document Training Microsoft Word 2010.
Welcome to Our Website Comments and Suggestions Welcomed! Presented by: Vicky Green.
Nick Donnarumma. We are either very lazy or very efficient because we tend not to read pages. We scan them instead to save time. We don’t figure out how.
Increasing Website ROI through SEO and Analytics Dan Belhassen greatBIGnews.com Modern Earth Inc.
Section 7.1 Identify presentation design principles Use a custom template Add pages to a navigation structure Section 7.2 Identify color scheme guidelines.
Our Focus is YOU! Getting Started with Resume Generator Plus If you are a first time Resume Generator Plus user: you will begin by going to the Employment.
Review of last Session Adding custom html Adding custom html HTML is the language that web servers understand, all web pages are created using HTML. HTML.
Creating a Blogger Account. Blogger This tutorial will show you how to create a Blogger account and how to publish your first blog post.  Set up the.
Color Theory in Web Design Web Design – Sec 2-2. Objectives  The student will: –Have a better understanding of effective use of color on the web. –Be.
1 PowerPoint Presentation Design Wednesday, September 02, 2015Ms. Wear Info Tech 9/10.
System Analysis & Design Chapter VII: User Interface Design Providing interactive and easy to use interfaces is an important task of system designer using.
The 3 Step Funnel. It’s All About the Cheese Give something of value in exchange for their address Make it clear what the value is QUICKLY Make.
Build a Free Website1 Build A Website For Free 2 ND Edition By Mark Bell.
+ Content Rich Websites Katie Tomaino. + What is a Content-Rich Website? “Organizes the online personality of your organization to delight, entertain,
Using a Web Browser What does a Web Browser do? A web browser enables you to surf the World Wide Web. What are the most popular browsers?
Moodle (Course Management Systems). Assignments 1 Assignments are a refreshingly simple method for collecting student work. They are a simple and flexible.
Google Adsense Tips and Tricks.... Every single website and its owner have the opportunity to make some great money by displaying Google Adsense ads on.
Overview In this tutorial you will: learn different ways to conduct a web search learn how to save and print search results learn about social bookmarking.
Plan My Move & MilitaryINSTALLATIONS May, 2008 Relocation Personnel Roles and Responsibilities MC&FP.
Welcome to the Interactive Survey System. 2 What Is the Interactive Survey System? Online, interactive version of the documents needed for NCQA accreditation,
Copyright ©: SAMSUNG & Samsung Hope for Youth. All rights reserved Tutorials The internet: Blogging Suitable for: Advanced.
INTERLEGES AGM KIEV THE “ESSENTIALS” OF LAW FIRM WEBSITES.
Another reason to post PLR content is to draw in search engine traffic. If you’re going to use your PLR content, then you will need to rewrite it! Rewrite.
4 Simple Web Design Rules. Your Web Site Should Be Easy To Read  Choose your text and background colors very carefully  Don't use backgrounds that obscure.
What makes a good interactive resume? Click for detailed information Multimedia Navigation Communication.
August 31, 2015 Ms. Fiorante CC Enrichment. * STEP #1 GO TO * YOU WILL NEED A GMAIL ACCOUNT TO SIGN UP.
Color Theory in Web Design Web Design – Sec 2-2. Objectives  The student will: –Have a better understanding of effective use of color on the web. –Be.
How to use LegalEasy - the Legal English & Corpus Consultation Website for Law students.
Animal Shelter Activity 2.
What is Voice Thread? VoiceThread is an application that runs inside your web browser, so there is no software to download, install, or update. VoiceThread.
Topics in Technology and Marketing Push Marketing: .
Computer-based Media Language Elements Understanding how we communicate through media Stewart.C. (2007). Media: New Ways and Means. John Wiley & Sons:
Online Journalism – Multimedia Package Writing Professor Neil Foote, University of North Texas.
Accessible Word Document Creation Using Microsoft Word 2010.
Web Page Design The Key to a Successful Web Page by Judy Bogdanets.
Testing Overview Presentation. OPTIMIZELY TESTING TOOLKIT 2016 What is Testing? Why we do it, how we do it, and how to get involved YOUR COMPANY LOGO.
Microsoft Expression Web 3 – Illustrated Unit G: Designing Site Navigation.
I really loved the idea of having a class blog. A place where students can get online accessibility to course information and updates and a place where.
Writing Accessible Electronic Documents with Microsoft® PowerPoint
Learning Aim A.  Websites are constructed on many different features.  It can be useful to think about these when designing your own websites.
Research Skills for Your Essay Where to begin…. Starting the search task for real Finding and selecting the best resources are the key to any project.
Use Your Social Profiles To get Search Engine Optimized.
Web Forms. Web Forms: A form allows our web visitors to submit information to us. Some examples uses for forms are to let the web user contact us, fill.
Designing for Usability OBJECTIVE: Learn “usability” principles related to design for interactive media.
Color Theory in Web Design
Web Design 1 Website Construction.
Krug Chapter 3 Billboard Design 101 and Chapter 4 Mindless Choices
Testing and Optimization (TAO) Overview
DELIGHT WITH UX DESIGN RACHEL KORPELLA
Tips on good web site Design
Getting to the Top of.
Best Helpful SEO Tips For Good Content Writing 2019 Presented By:- Abhinav Shashtri.
Edge Lights Tutorial Making the Light Shows.
Presentation transcript:

A/B testing aka split testing, bucket testing, and multivariant testing

Empirical Field study

What is it? Controlled experiments – Web site – Authentic users Split users into two groups: – Control – Test variant Cookies can ensure an individual gets consistent version. A metric (the overall evaluation criteria, or OEC) eg – Click-through rate – Purchases

Process Driving question: – eg Can I increase click through rates from my home page Hypothesis: – eg. Underlined links will outperform my current links Calculate time to run test: – eg A/B Test Duration Calculator.A/B Test Duration Calculator Run the test: – x% of users get underlined links – How to decide x? Review results

Holson, Laura M. [2009]. Putting a Bolder Face on Google. New York Times. 1st Mar http: // marissa.html

Mayer, Marissa [2009]. Google”. In: AIGA Design Conference AIGA. 9th Oct shades of blue … gmail blue was a little more green and the search blue was more blue Oct, 2015

More green, worse clicks on search and more abandonment More blue, brighter the blue, faster to search click

Adding red improves performance

usability-engineering/ usability-engineering/ A/B Testing, Usability Engineering, Radical Innovation: What Pays Best? by Jakob Nielsen on March 26, 2012Jakob Nielsen

interest-to-users.html There are dozens of A/B consultants, tools and compendiums of tips and tricks. Here are a few links to check out: Robin Johnson, writing earlier this year in Optimizely, gives you 71 different ideas on things to test, including website copy, visual elements and just plain common sense things such as whether you have a positive or negative spin on what you are trying to sell. Robin Johnson Uri Bar-Joseph, writing in SearchEngineWatch, has eight rules for A/B testing, including focusing on one variable, choosing your groups randomly and measuring the results carefully. Uri Bar-Joseph Brian Conklin of Vocus has his own nine rules in this blog post, including doing an % split: meaning start with ten percent of your users testing each of two options. Once you determine a winner, send the results to the remaining 80 percent of your audience. Brian Conklin And if you want to have some fun with A/B tests of the past, take a look at these 12 results that were somewhat counter-intuitive from WhichTestWon.com, a commercial A/B testing provider.12 results

12 results somewhat counter- intuitive A/B tests.

sliders-or-without/ sliders-or-without/ Oct 13, 2015 Key Performance Indicator (KPI): – clicks on the sponsorship Call To Action button Difference between versions: Version A: Sponsor a child page formatted with sliders Version B: Radically redesigned sponsorship page, without sliders

And the winner is Version B 60.4% increase in clicks on the sponsorship button! Used: – Optimizely used – 14 days 8,600 visitors. Key differences – Story, narrative – updated visual flow – clickable colored boxes were interactive

Final Page in a 4 Step Sequence

Both pages are well-designed and both, seemingly, doing a good job Version B. – video centrepiece, well-structured copy; it should convert Version A had 439% higher performance. Post-hoc rationalisation – text within the image brings focus – Form has fewer of fields looks shorter – “two images give the page a more authentic and trustworthy feel”

Test 11: A/B Button Color Test: Which Page Drove More Clicks to a Lead Generation Form?

Ah! The simple button color test. “Green is an affirming color that signifies positive action, in this case it’s been used with white text which completely washes the button out. It’s hard to know what the button is for at first glance.” “Version B’s yellow and black button may be ugly (and I mean ugly), but it is clear and led to a 14.5% increase in conversions.”

Summary Studying user’s actual behaviour Web as experimental platform Ease of testing A/B testing as an example of – Field studies…. Authenticity – Large scale empirical study cf Think-Aloud