Title text Arthur Etchells Director of Product Management Imaging Resource

Our product looked like this

We added a cool product tour shortly after launch

‘Engagement’ was through the roof! People loved it!

“Lorem ipsum dolor sit amet, consectetur adipiscing elit, set eiusmod tempor incidunt et labore et dolore magna aliquam.” – First Name Last Name, Attribution Or did they? Was ‘engagement’ what we wanted?

Yes, engagement was great (Optimizely) Bounce rates were not (Google Analytics) Months pass Some internal feedback (that tour is annoying!) … Led to further research…

Title Text Crazy Egg In-page click analysis

Showed us something interesting…

Bullets only Up to five items One to four words First letter is capitalized Periods are not used Not the engagement we were looking for

The Question

*non-product tour content Do users interact more with our content* after viewing a product tour?

The Experiment

Two Variations

Two Metrics

Engagement

Clicks on non-product tour content

Results

Title Text Engagement Dropped by 20.4%

Title Text Clicks on non-product tour links Increased by 36.3%

Lessons Learned

Be careful what you wish optimize for Be careful what you wish for

Engagement may not be the best metric to optimize

Make sure product tours are actually doing what you hope they are There should be a high bar for anything you put between users and your content

Take a holistic approach to optimization efforts Subjective feedback can let you uncover issues earlier

User feedback sessions In-page feedback prompt HeatmapsAnalytics Biz goals: revenue, lead gen Objective Subjective Problem identified here

User feedback sessions In-page feedback prompt HeatmapsAnalytics Biz goals: revenue, lead gen Objective Subjective Hypothesis generated here

User feedback sessions In-page feedback prompt HeatmapsAnalytics Biz goals: revenue, lead gen Objective Subjective Could have identified the problem much earlier here

Those were the tools we used, but there are lots of options… Analytics (Omniture/Google Analytics/Mixpanel) Heatmaps (ClickTale/CrazyEgg/HotJar) In-page feedback forms (Qualaroo/HotJar) User feedback sessions (In-person/UserTesting)

Subjective feedback can allow you to uncover usability problems earlier What does that look like in practice?

UserTesting/HotJar/Optimizely Discussion…

“Lorem ipsum dolor sit amet, consectetur adipiscing elit, set eiusmod tempor incidunt et labore et dolore magna aliquam.” – First Name Last Name, Attribution Questions?