1. Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall Conducting Research and Utilizing Tests to Measure Performance Chapter 10

2. Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall Please Note:

3. Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall Opening Vignette Omaha Steaks International

4. Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall Research Serves Direct Marketers Fact-Finding Information Gathering Problem-Solving Decision-Making

5. Need for Marketing Research Managers must have good, accurate, timely information with which to make decisions. Marketing research helps to gather the needed information. The results of research can be quantitative and/or qualitative. Valid research measures results … not opinions. Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

6. Problem Structure How much advertising is needed? How will the direct marketing mix be selected? How will our resources be utilized? Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

7. The Nature of Research Surveys vs. Experiments: A survey looks at things the way they are to measure product preferences or determine future buying intentions An experiment or test measures the effect of change Which one is used more widely by direct marketers? Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

8. Databased Research and Analysis “My mind’s made up. Don’t confuse me with facts.” A characteristic of database-driven and directed marketing is measurement and accountability for actions. Decisions are based on facts, not opinions. Direct marketers build databases from facts, relying not so much on responses derived from survey, but more on conclusions derived from experimentation. Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

9. Products and Services Media Offers/Propositions Copy Platforms Creative Formats Timing/Seasonality What Do Direct Marketers Test? Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

10. TEST THE BIG THINGS Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

11. Sourcing & Collecting Information  Secondary Data  Primary Data Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

12. Secondary Data Sources The Organization’s Internal Records Government Sources: Federal, State & Local U.S. Department of Commerce - Bureau of the Census U.D. Department of Labor - Bureau of Labor Statistics U.S. Department of Agriculture Other U.S. Government Sources: President’s Office, Congress, Treasury Department, Interior Department, and Health and Human Services Department State and Local Governments: economic surveys, license registrations, and tax records Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

13. Secondary Data Sources Trade, Technical, Professional and Business Associations Private Research Organizations Foundations, Universities and Other Nonprofits Libraries, Public and Private Advertising Media Financial Institutions and Utilities Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

14.  Behavior  Intentions  Knowledge  Socioeconomic Status  Attitudes and Opinions  Motivations  Psychological Traits Primary data, collected via survey, can yield information about: Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

15. PRIMARY DATA COLLECTION METHODS Survey MethodAdvantagesDisadvantages Personal Interview More complete and accurate sample, more complete information, greater flexibility in structuring questions, high response rate Cost, possibility of interview bias, and need for extensive interviewer supervision and control Telephone Interview Economy, speed, representative sampling, minimal nonresponse, simple callbacks, ability to make interview coincide with other activities Limited availability of information at time of contact, excludes those without phone service, increasing public resistance to phone calls, inability to use questions requiring demonstration or visualization. E-Mail Survey Ability to reach hard-to-reach consumer segments, speed, economy, respondent choice in when to reply Representative sample issues, higher degrees of nonresponse bias Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

16. PRIMARY DATA COLLECTION METHODS Survey MethodAdvantagesDisadvantages Mail Questionnaire Great versatility, low cost, no interviewer bias, no field staff needed, respondent choice in when to reply, respondent confidentiality High rate of nonresponse, need for follow-up, response bias factors On-line Surveys Ability to reach potential respondents at most opportune time, speed and convenience Similar to e-mail surveys - Representative sample issues, higher degrees of nonresponse bias Observation Removes respondent biasMore costly and time consuming than some of the other methods Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

17. How to Design An Experiment  Control  Randomization  Statistically-valid sample size Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

18. How to Track Responses Key Codes Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

19. Response rate & break-even analysis Control vs. experimental packages Direct marketers test or experiment with different offers and campaign themes to determine which one generates the greatest response rate Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

20. Promotion Cost ------------------------ = Break-even Number of Sales Unit Profit per Sale Break-even analysis Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

21. Samples & Estimations Random sample designs Simple Random Samples Systematic Random Samples Stratified Random Samples Cluster Samples Replicated Samples Sequential Samples Determination of sample size Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

22. Using Statistical Evaluation to Determine the Sample Size Confidence level - the number of standard deviations from the mean in a normal distribution (for example a 95% confidence levels corresponds to 1.96 standard deviations from the mean) Limit of error - the number of percentage points by which the researcher can miscalculate the actual response rate Expected (actual) response rate - the number of positive responses expressed as a percentage of the total Sample size - the number of observations in an experiment Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

23. Measurement of Differences Hypothesis testing Types of errors in hypothesis testing Statistical evaluation of differences Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

24. Hypothesis Testing Hypotheses are typically stated in negative terms; that is, a null hypothesis (H 0 ) versus an alternative hypothesis (H a ) in a form such as the following: H 0 : Direct mail response from the test promotion is at or below direct mail response from the control promotion. H a : Direct mail response from the test promotion is above direct mad response from the control promotion. The null hypothesis, then, states that direct mail response will not be better than the control. Our measurement sets out to disprove this null hypothesis. Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

25. Types of Error in Hypothesis Testing Type One: Results when the decision-maker rejects the null hypothesis even though it is, in fact, true... i.e., taking an action when one shouldn't Type Two: Results when the decision-maker accepts the null hypothesis when, in fact, not true... i.e., not taking an action when one should. Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

26. Assume that a sample has been properly selected and is of an adequate size. Assume further that an experiment has been designed and implemented in a valid manner. It now remains for the direct marketer to be able to recognize the difference in the response rate from a test and that from a control, with some degree of confidence and within an acceptable limit of error. Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

27. TestControlTotals Response A C A + C Non-response B D B + D Total mailed A + B C + D A + B + C + D = N The statistic  2 is computed as follows:  2 = N [  (A x D) - (C x B)  - N/2] 2 (A+B) x (C+D) x (A+C) x (B+D) Here is a sample calculation: TestControlTotals Response 200 100 300 Non-response 800 900 1700 Total mailed 1000 1000 2000  2 = 2,000 x [  180,000 - 80,000  - 1,000] 2 1,000 x 1,000 x 300 x 1,700  2 =38.4... which is significant at the 99++% level since it exceeds the critical value in the X 2 table for one degree of freedom for a significance level of 0.001, given as 10.83 Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

28. STRUCTURING and EVALUATING AN EXPERIMENT State the hypothesis Develop, by a priori analysis, the assumptions required and compute the appropriate sample size Structure and perform the experiment Develop, by a posteriori analysis, statistics for judging hypothesis validity Make the decision Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

29. CHAPTER CASE THE AMERICAN HEART ASSOCIATION Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall