Online Market Research © 2001 Ann Schlosser, University of Washington Business School

Agenda What accounts for differences in actual and forecasted numbers? –Survey panels vs. server log files –Response and sampling biases Online market research methodologies –VR/Simulated environments – surveys –ISP’s data warehouse (unobtrusive) –Quasi-experimental design Using the results

Who Has a Reliable Estimate of a Site’s Audience? Internal Server Third-Party Ratings Firms For pageviews, ratings-firm numbers were anywhere from 85% less than to 300% more than that reported by the site’s server.

Comparing Audience Measurement Firms Using Panel Data COMPANYU.S PANEL SIZE RECRUITMENT METHOD INCENTIVE Media Metrix 55,000Random-digit dialing, follow-up mail $50 annual maximum, giveaways, sweepstakes Nielsen NetRating s 65,000Random-digit dialing$50 U.S. savings bonds every six months PC Data120,000Random-digit dialing, advertising $40 annually, sweepstakes ComScore1.4 million Advertising, partnered with pollsters Provides faster Internet- page download service

The Trouble With Online Panels

Some Benefits of Panels Level of detail –Geography –Unique visitors (different people accessing the site) –Captures PC activity

Internal Server Records are Imperfect Tallies computer (not human) access –Over counts due to: Same user with home and work access Spiders and other bots ISPs that indiscriminately assign users to IP addresses –Under counts due to: Different users of same computer Cached pages –Especially problematic if done by ISPs and other content aggregators

Some Factors Influencing the Reliability of Data Response biases Sampling biases Incentives

Impact of the Internet on Market Research Kannan et al, 1998

Virtual Shopping Market Research

Comparing Electronic With Mail Surveys Mail Response rate6%-73%27%-56% Response speed7-8 days days CompletionLessMore Bad address notification10 minutesWeeks Percent of bad addresses24.5%2% Weible & Wallace (1998)

Collecting Data Through ISPs while Protecting Privacy ISP login/street address Anonymous ID, Geocode, Append Demog. Foveon Collection (now Plurimus) login Anonymous ID Data Warehouse street address = security key 3rd Party 50 participating ISPs and 3.5 million Web users

Raw Material: Collector Lines :04:30 com.egghead.www /store2/ent/eggs_ordstat.browse :30:57 com.egghead.www /store1/ent/images/space.gif :04:34 com.egghead.www /media/bnr/B_EggheadSpecials_ :04:42 com.egghead.www /store2/ent/images/space.gif :30:57 com.egghead.www /media/images/bdot.gif :30:11 com.egghead.www /media/images/but_previous.gif :30:11 com.egghead.www /store1/ent/eggs_shop.additem?s :30:55 com.egghead.www /store1/ent/eggs_shop.additem?s :30:32 com.egghead.www /media/images/portal_egg-header :30:57 com.egghead.www /media/images/recalculate.gif :30:08 com.egghead.www /store1/ent/eggs_prod.browse?se :30:07 com.egghead.www /store/ent/eggs_prod.browse?sec :31:03 com.egghead.www /media/images/but_proceed-order :04:34 com.egghead.www /media/images/but_nav_auction[ :04:34 com.egghead.www /media/images/but_search.gif :21:41 com.yahoo.rd /results/a/? :41:43 com.yahoo.rd /results/a/? :07:53 com.yahoo.rd /search/navbar/top/* FIDDate/TimeHostURL

Classification of Behavior OCT16:200012:04:30 com.egghead.www2 /store2/ eggs_shop.additem Duration Time of Day Calendar Day “Internet Computers” NAICS Code “Shopping Cart” Geography (BG ID) Demography Geocoding/Appending Calculation Site Classification URL Classification FIDDate/TimeHostURL

Log File Data User Start Date/ End Date/ Packets BytesHost URL ID Time Time To/From To/From

GBF vs. GIRF: Reasons for Dot-Com Failures Poor revenue, cost investment and profit models61% No competitive advantage44% No consumer benefit21% Organization, implementation and execution problems17% Channel conflict10% Ineffective fulfillment process7% Source: BCG

Various Performance Metrics 65% abandon shopping carts Overall average buyer conversion rate < 4% Low customer conversion, upselling and loyalty, even at best sites –At 10 leading e-commerce sites 20% converted from visitors to buyers 10% upsold 5% returned within 6 months

Retention Measures

KEY RESEARCH QUESTIONS –Are you reaching the right audience? –Do they remember your website? –Does it enhance the brand’s image? –Does it increase sales? What is the objective(s) of your website?

Web Site Report Page Views by Hour Site Navigation Summary Report

Disentangling the effect of your website A quasi-experimental design is the best means of isolating the impact of a website from the numerous external factors which may impact attitudes and opinions Online Marketing Traditional Marketing website Peer influencePress influence Pre-existing attitudes

Methodology = Website effect Experimental Effect First time users Experimental Group - Repeat visitors Control Group

User accesses website JavaScript sampling procedure Visitor continues surfing, unaware of the sampling script Not Sampled

Survey 1 Sampled Returned to the page originally requested Survey 2 invitation User accesses website JavaScript sampling procedure Visitor continues surfing, unaware of the sampling script Not Sampled

ONLINE QUESTIONNAIRES What should you ask? –Questions relating to your website’s objectives –Audience profile questions –Satisfaction questions –Relationship questions –Intent questions –Comparative questions

Type of Information Available 7.65 Type of Information Available 7.65 Detail of Information Available 6.98 Detail of Information Available 6.98 The Layout of the Site 7.84 The Layout of the Site 7.84 Navigation Aides on the Site 6.75 Navigation Aides on the Site 6.75 Speed of Loading Pages 7.10 Speed of Loading Pages 7.10 OverallSatisfaction I=.000I=.533I=.39I=.272I=.000 Customer Satisfaction Model Visit Again Recommend to others Recommend to others Spend more time Download Information I=.120 I=.320 I=.420

Customer Satisfaction Model CRITICAL IMPROVEMENTSTRENGTH LOWER LEVERAGE AREALOWER PRIORITY Detail of information available Navigation aides on the site Layout of the site Type of information available Speed of loading the pages SATISFACTION WITH FEATURE IMPACT ON OVERALL SATISFACTION

Strategic Improvement Matrix