Prof. Panos Ipeirotis Search and the New Economy Session 2 Web Analytics

Frequency of Access Frequency of access decreases by distance^2 (Result from traditional library science) Result carries over to physical stores Result carries over to information environments Question: How do we understand how users/customers access what we offer?

Data-driven Decisions: Testing, testing, testing, testing Common scenario: –Boss, designer, employee “knows what works best” (for him/her) –Boss, designer, employee wants to do site design Common error: Think that we know what customers want 80% of the time we are wrong about what a customer wants or expects from site experience (e.g., consider reasons to visit Amazon.com) The only truth: Constant experimentation and testing improves customer “relevancy” and improves conversion

The objectives of this session Web analytics for own website –What customers look at –Where they come from –How to engage them Web analytics for monitoring competitors –How customers behave in general –Why they go to competitors –What they do there

Outline of today’s class How customers behave in our own website Micro / In-site / Quantitative (What) –Eyetracking –Clickstreams, log analysis Meso / In-site / Qualitative (Why) –Surveys –Lab-usability tests –In-situ tests How customers behave in other websites Macro / Across-site –Panel data (ComScore, Alexa) –ISP measurements (HitWise) –Search engine data (Google Trends, Microsoft adCenter)

Eyetracking monitors

Eyetracking studies

Eye Tracking Studies Golden Triangle –Top left corner –Not only in search engines Quick scan –For candidate Longer scan –For relevance

Analysis of Washington Post

Eyetracking (again) FDIC distrusts us * No Bank Quality * Will Lose Value Not ready to event an insurance? Tax group of our manager discussion free of funds. Get $25 to close an E*Trade Bank Money Market Plus Advice! Tax a gear cool and ATM access!

Eyetracking: The F-pattern Users don't read text thoroughly The first two paragraphs must state the most important information. Start subheads, paragraphs, and bullet points with information- carrying words Merge “foreign content” with page information

Clickstream / Log Analysis Eyetracking studies are limited to a lab Often we need to analyze how users behave when visiting our site

Sources of click data Web Server Logs Pros: You own the data, Capture search engine visits  Cons: Difficult to customize, Misses cached requests Web Beacons (1x1 pixel images) Pros: Easy to add  Cons: Bad reputation, often blocked Javascript Tags Pros: Capture real visitors, Customizable  Cons: ~5% of users have JavaScript off

Foundational Metrics Visitors / Unique visitors –Pay attention on definition of “unique” (cookie? date? IP?) Time on site –Tricky! Should consider the goal of the site Page views –Good for content/brand sites –Unclear for other sites –Increasingly outdated (blogs, Gmail, Flash, dynamic content) Bounce rate –Reveals real visitors –% of single page visits –(or % of <5 second visits)

Goal and Lead Metrics “Unique Visitors” tends to be THE metric to follow, BUT instead: Set up goals and measure conversion rate and goal value (Settings  Edit  Goal) Segment by: –Referring sites –Search engines + Keywords –AdWords campaigns Analyze for leads! –“Wikipedia referrals are more engaged and have low bounce rate” –Use Microsoft AdCenter Labs to analyze demographics (will get back to this)

Content Metrics Top content –Why users are coming –What they are looking for Top landing (entry) pages –First impression! –Polish and direct users to goals Click density analysis –Use CrazyEgg.com Funnel analysis –In multi-page processes, where users abandon? –Mortgage application at Agency.com  move personal information form later –Abandoned purchases at Lane Bryant  offer free shipping

Click Density Analysis Click OverlayClick Heatmap Where users click, and which users click in each link

Your own experience? Questions? Anything that you would like to add? Lessons from practical experience?

Redesign and Experimentation After detecting problems or opportunities: 1.Make a hypothesis 2.Redesign 3.Test for performance (Common error: Skipping step 1) Two common approaches for testing A/B testing Multivariate testing

A/B testing Version A Version B Run versions A and B and see which improves the target performance indicator Image on the left “add to shopping cart” bottom right Image on the right “add to shopping cart” top left

Multivariate Testing Modularize page and test variations for each module (see Google Website Optimizer, Offermatica, Optimost, SiteSpect, Kefta, …) Headline Image Text Call to action

Multivariate Testing 3 different headlines 3 different images …

Examine Results

Example: Dale & Thomas Popcorn company Variables: –Main layout –Order area headline (6) –Order area image (6) –Order area button –Popcorn flavors image (4) –“Free shipping” –“Sign-up for mailings” → 1.9 million variations possible +13% in sales, within a month

OK, we optimized our own site Is +13% good? …or lagging behind the competitors? What types of customers go to our competitors? Why? …

Outline of today’s class How customers behave in our own website Micro / In-site / Quantitative (What) –Eyetracking –Clickstreams, log analysis Meso / In-site / Qualitative (Why) –Surveys –Lab-usability tests –In-situ tests How customers behave in other websites Macro / Across-site / Competitive Intelligence –Panel data (ComScore, Alexa) –ISP measurements (HitWise) –Search engine data (Google Trends, Microsoft adCenter)

Need for Competitive Data Understand how competitors perform How competitors get visitors Where customers go after visiting competitor’s site Demographics

ISP-Based Data (HitWise) Benefits –Big sample size (~25M users) –Captures all types of traffic –Good for relatively small sites as well (~100K visitors) Concerns –(Relative) lack of depth of analysis –Lack of purchase / payment data (no https logging) Anonymous data, bought from multiple Internet Service Providers

HitWise: Upstream and Downstream

HitWise: Industry Statistics

HitWise: Keyword Statistics

Panel-Based Data (Alexa) Benefits –Free –Large sample size (>20M) Concerns –Percentage reporting (not absolute values) –Self-selection bias due to targeting (webmasters?) Useful mainly for comparing similar sites Users install toolbar and agree to have their traffic anonymously monitored

Panel-Based Data (ComScore) Benefits –Detailed demographics for users –Provides conversion rates and purchases –100% of traffic Concerns –Sample size (relatively) small, 100K-2.5M –Sample selection bias due to incentives –Mainly home usage, no work-based (but also avoid double counting?) –Not good for sites with less than 1M visitors Recruits users who agree to have their traffic monitored, in exchange for payment and benefits

What service would you use? Why?