Concept Validation Testing Jessica Richardson Digital Optimization Consultant, Accenture

Concept Validation: Testing to see if the concept makes an impact, and validate the priority Change that would normally require a large dev effort, but use testing to prototype the concept without actually implementing The design does not have to be perfect, just good enough to get the idea across

How are we organized? AB Team Located: Bangalore AB Lead (Yours Truly) Client Product Teams

106 tests per year 32% win rate Organized as Product Teams

Lesson 1:"Free" and “easy” doesn’t always mean you should build it.

“Let’s run a pilot” Will this additional information increase add to cart?

Result: Add to Cart Rate = not statistically different

Product CategoryDiff in Add to Cart Laptop+9.7% Monitor-11.3% SwitchInconclusive Keyboard/Mouse-6.6% SoftwareInconclusive Printer-36.3% Lets look at it by product category….

b9lqc5QFQVd6xQ/clips/714750?back=study_ We asked a few users No one expected to see the content that was actually behind the + icons. Users felt that the + icons had random information. Not one user said they would buy the product based on the info in icons.

Lesson 2: Real customer feedback is invaluable.

Run User Testing Discover User Issue Develop Hypothesis Users are leaving the site to find more information about the product, like videos. Providing videos to show the look and feel, will reduce exit rates due to users leaving the page in user testing.

1.Where are we going to get these videos? 2.How are we going to source this? 3.The dev effort to build is high…… “Let’s run a pilot to prove if its worth building at all. We can manually source the videos for a subset of SKUs.” Potential solution = a lot of questions

The Test…

-31% drop in bounce rate -27% drop in exit rate

Lesson 3: Powerful testing can be done when you work closely with your dev team.

Client vs. vendor recommendations: What will increase revenue gains?

Results: Total categories increased but the overall amount of items sold wasn’t different compared to the original What’s missing? Round 2: Added “add to cart” buttons

+150% increase in items sold +83% lift in Sales

Lesson 1:"Free" and “easy” doesn’t always mean you should build it. Lesson 2: Getting real customer feedback is invaluable. Lesson 3: Powerful testing can be done when you work closely with your dev team.