Measuring the Effectiveness of the Promotional Program Chapter Nineteen Measuring the Effectiveness of the Promotional Program Measuring the Effectiveness of the Promotional Program © 2007 McGraw-Hill Companies, Inc., McGraw-Hill/Irwin © 2007 McGraw-Hill Companies, Inc., McGraw-Hill/Irwin

Marketer’s Views Toward Measuring Effectiveness Relation to text This slide relates to material on pp. 597-602 of the text. Summary Overview This slide presents marketer’s views toward measuring effectiveness, the result of two recent studies. Whereas advertisers know that it is important to measure effectiveness, with as many as 90 considering it a priority, many do not do so, or if they do, they are not confident with the results. Use of this slide This slide can be used as an introduction to the reasons companies give for not measuring the effectiveness of advertising and promotion strategies. © 2007 McGraw-Hill Companies, Inc., McGraw-Hill/Irwin

Reasons for and Against Measuring Effectiveness Advantages Disadvantages Avoid costly mistakes Avoid costly mistakes Cost of measurement Cost of measurement Evaluate alternative strategies Evaluate alternative strategies Research problems Research problems Relation to text This slide relates to the material on pp. 597-602 of the text. Summary Overview This slide shows some of the reasons why companies should measure effectiveness of their communications programs, as well as why they don’t. Reasons why such measures should be taken include: To avoid costly mistakes To evaluate strategies implemented To increase the efficiency of advertising in general To determine if objectives are achieved Many managers choose not to measure, citing the following reasons: The costs associated with measuring effectiveness Problems with research methods Disagreement as to what to test Objections of creative Time Use of this slide This slide can be used to discuss the reasons why managers should conduct research to measure the effectiveness of advertising and other components of the IMC program, as well as some of the reasons why they may resist doing so. Increase efficiency in general Increase efficiency in general Disagreement on what to test Disagreement on what to test Determine if objectives are achieved Determine if objectives are achieved The objectives of creative The objectives of creative Time © 2007 McGraw-Hill Companies, Inc., McGraw-Hill/Irwin

Measuring Advertising Effectiveness What to test Source factors Message variables Media strategies Budget decisions What to test Source factors Message variables Media strategies Budget decisions Where to test Laboratory tests Field tests Where to test Laboratory tests Field tests Relation to text This slide relates to material on pp. 602-607 of the text. Summary Overview This slide presents some of the issues that have to be considered in the measurement of advertising effectiveness. These include: What to test Source factors Message variables Media strategies Budgeting decisions When to test Pre-testing Post-testing Where to test Laboratory tests Field tests How to test Testing guidelines Appropriate tests Use of this slide This slide presents various factors that must be taken into consideration when measuring the effectiveness of advertising and other IMC elements. It can be used to introduce the material on the following slides which addresses many of these issues. When to test Pretesting Posttesting How to test Testing guidelines Appropriate tests When to test Pretesting Posttesting © 2007 McGraw-Hill Companies, Inc., McGraw-Hill/Irwin

Pretesting Methods Laboratory Field Consumer Juries Consumer Juries Dummy Ad Vehicles Dummy Ad Vehicles Portfolio Tests Portfolio Tests On-air Tests Physiological Measures Physiological Measures Theater Tests Theater Tests Relation to text This slide relates to material on pp. 603-605 and Figure 19-1 of the text. Summary Overview This slide discusses some of the various pre-testing methods available, further classifying them as to where they are to be conducted - laboratory or field. Pre-tests are those measures taken prior to the implementation of the campaign. Laboratory tests are those in which participants are brought to a specific location for testing, while field tests are those conducted in more natural viewing situations. Use of this slide This slide can be used to introduce and provide an overview of pre-testing measures as well as laboratory and field testing. Rough Tests Rough Tests Concept Tests Concept Tests Reliability Tests Reliability Tests Comprehension and Reaction Tests Comprehension and Reaction Tests © 2007 McGraw-Hill Companies, Inc., McGraw-Hill/Irwin

Field Posttesting Methods Recall Tests Recall Tests Tracking Studies Association Measures Association Measures Methods Relation to text This slide relates to material on pp. 603-605 and Figure 19-1 of the text. Summary Overview This slide provides examples of some of the various types of posttests available. All of these are field measures that are used to determine the effectiveness of the ads once the campaign has been implemented. Use of this slide This slide can be used to discuss the types of post-testing methods that are available to marketers to measure the effectiveness of their advertising programs. Recognition Tests Recognition Tests Single-Source Systems Single-Source Systems Inquiry Tests Inquiry Tests © 2007 McGraw-Hill Companies, Inc., McGraw-Hill/Irwin

Posttesting the Value of a Sponsorship Relation to text This slide relates to material on p. 606 and Exhibit 19-3 of the text. Summary Overview This slide shows a NASCAR race car that was sponsored by Lowe’s, the nation’s second largest home improvement center. Lowe’s recently conducted extensive field experiments to determine the effectiveness of its various forms of advertising including television spots, newspaper ads, circular catalogs and even its sponsorship of NASCAR auto racing. Use of this slide This slide can be used as part of a discussion of how companies such as Lowe’s use field testing to determine the impact of their IMC programs. You might ask students how a company might go about determining the value of a NASCAR sponsorship. © 2007 McGraw-Hill Companies, Inc., McGraw-Hill/Irwin

Positioning Advertising Copy Testing (PACT) 1. Provide measurements relevant to objectives of advertising 2. Require agreement on how results will be used before each test 3. Provide multiple measures (Single measures aren't adequate) 4. Be based on a model of human response to communications 5. Consider multiple versus single exposure to the stimulus 6. Require alternative executions to have the same degree of finish 7. Provide controls to avoid the biasing effects of exposure context 8. Take into account basic considerations of sample definition 9. Demonstrate reliability and validity Relation to text This slide relates to the material on pp. 606-607 and Figure 19-3 of the text. Summary Overview Because measuring effectiveness is not an easy task, twenty-one of the largest U.S. advertising agencies developed and endorsed a set of principles aimed at “improving the research used in preparing and testing ads, providing a better creative product for clients, and controlling the cost of TV commercials”. This set of nine principles, called Positioning Advertising Copy Testing (PACT), is designed to establish guidelines for good copy testing research. The nine PACT principles are shown here. Use of this slide This slide can be used to present the nine PACT principles that were developed to guide advertising copy testing. You might want to discuss each of these as they are essential to the development of good copy testing methods. © 2007 McGraw-Hill Companies, Inc., McGraw-Hill/Irwin

Occurs at Various Stages The Testing Process 1.Concept Testing 1.Concept Testing 2.Rough Testing 2.Rough Testing Occurs at Various Stages Relation to text This slide relates to material on pp. 607-617 of the text Summary Overview This slide provides an overview of the testing process that may occur at various stages throughout the development of an advertising campaign. The stages where testing may occur and types of testing that might be done include: concept testing rough testing finished art or commercial testing market testing (post testing). Use of this slide This slide provides an overview of the testing process and can be used to provide an overview of various forms of testing that are used. The subsequent slides provide specific information about each of these. 4.Market Testing (Post-testing) 3.Finished Art or Commercial Testing 3.Finished Art or Commercial Testing © 2007 McGraw-Hill Companies, Inc., McGraw-Hill/Irwin

© 2007 McGraw-Hill Companies, Inc., McGraw-Hill/Irwin Concept Testing Objective Explores consumers’ responses to ad concepts expressed in words, pictures, or symbols Explores consumers’ responses to ad concepts expressed in words, pictures, or symbols Alternatives are exposed to consumers who match the target audience Alternatives are exposed to consumers who match the target audience Method Relation to text This slide relates to material on pp. 607-608 and Figure 19-4 of the text. Summary Overview This slide summarizes what is involved in concept testing, which is conducted very early on in the research process. It shows the objective, methods and outputs associated with this form of testing. One of the more common methods of concept testing is through the use of focus groups, though field testing is often employed, and the Internet has resulted in an increase of concept testing online. Use of this slide This slide can be used to further explain concept testing which is one of the research methods that is used very early on in the campaign development process. Reactions and evaluations are sought through focus groups, direct questioning, surveys, etc. Reactions and evaluations are sought through focus groups, direct questioning, surveys, etc. Sample sizes depend on the number of concepts and the consensus of responses Sample sizes depend on the number of concepts and the consensus of responses Output Qualitative and/or quantitative data evaluating and comparing alternative concepts © 2007 McGraw-Hill Companies, Inc., McGraw-Hill/Irwin

Rough Art, Copy, and Commercial Testing Comprehension and Reaction Tests Comprehension and Reaction Tests Consumer Juries Control Advantages Disadvantages Relation to text This slide relates to material on pp. 608-611 of the text. Summary Overview This slide shows the methods of rough art, copy and commercial testing available to the marketer and the advantages and disadvantages associated with these. Because of the high costs associated with commercial development and production, many marketers attempt to measure the potential success of the commercial prior to completing the finished product. Use of slide This slide can be used to explain the use of the rough art, copy and commercial testing. It should be noted that for messages that do not involve high emotional content, these rough forms correlate highly with the finished product Control Consumer may become a self-appointed expert Consumer may become a self-appointed expert Cost effectiveness Cost effectiveness Number of ads that can be evaluated is limited Number of ads that can be evaluated is limited Endorsements by independent third parties Endorsements by independent third parties A halo effect is possible A halo effect is possible Achievement of credibility Achievement of credibility Preference for ad types may overshadow objectivity © 2007 McGraw-Hill Companies, Inc., McGraw-Hill/Irwin

© 2007 McGraw-Hill Companies, Inc., McGraw-Hill/Irwin Rough Testing Terms Animatic Rough Animatic Rough Terms Photomatic Rough Photomatic Rough Relation to text This slide relates to material on pp. 608-610 of the text and Figure 19-7. Summary Overview This slide shows the three broad categories into which rough testing can be classified including: Animatic rough - includes a succession of drawings/cartoons, rendered artwork, still frames and simulated movement Photomatic rough -successions of photographs often showing real people/scenery, etc. with still frames and simulated movement Live-action rough - employs live motion, stand-in involvement often with nonunion crews, limited props and opticals, and location settings. Use of this slide This slide can be used to show the various forms of rough copy testing. There are a number of options available for rough testing purposes and the choice of which to use will be based on the type of commercial to be tested. Live-action Rough © 2007 McGraw-Hill Companies, Inc., McGraw-Hill/Irwin

Pretesting Finished Print Ads A laboratory method A laboratory method Portfolio Tests Includes test and control ads Includes test and control ads Portfolio test have problems Portfolio test have problems Relation to text This slide relates to material on pp. 612-613 of the text that discusses pretesting. Summary Overview This slide shows a number of methods for pretesting finished print advertisements. The purpose of pretesting print ads is to attempt to determine how the finished product will perform, make corrections if needed, and fine-tune the message. As the slide shows, some of the options available for this type of testing include: Portfolio tests—a laboratory methodology designed to expose respondents to a portfolio consisting of both control and test ads. Readability tests—measures the communications efficiency of the copy in a print ad Dummy advertising vehicles—“dummy” magazines are created including articles, and test and non-test ads. The magazines are randomly distributed to homes in a specified area. Readers are interviewed to assess their reactions to both the editorial and the ads. Use of this slide This slide can be used to demonstrate some of the various research methods available to marketers for the pretesting of print advertisements. The determination as to which method will be used will be based on the needs and objectives of the advertiser Readability Tests Based on syllables per 100 words Based on syllables per 100 words Other factors also considered Other factors also considered Dummy Advertising Vehicles Distributed to random sample of homes Distributed to random sample of homes Product interest may still bias results © 2007 McGraw-Hill Companies, Inc., McGraw-Hill/Irwin

Pretesting Finished Broadcast Ads Theater Tests Measures changes in product preferences May also measure . . . Interest in and reaction to the commercial Reaction from an adjective checklist Recall of various aspects included Interest in the brand presented Continuous (frame-by-frame) reactions Theater Tests On-Air Tests Insertion in TV programs in specific markets Limitations are imposed by “day-after recall” Physiological Measures Relation to text This slide relates to material on pp. 613-615 and Figure 19-12 of the text. Summary Overview A variety of methodologies are available for pretesting broadcast commercials. A few of the most commonly used are presented in this slide including: Theater test - participants are invited to view pilots of new TV programs. Embedded in the programs (or following immediately after) are the test commercials. Theater tests can be used to take a variety of measures. On-air tests - testing of finished or rough commercials through actual TV programs in specified test markets. On-air tests offer some of the same advantages and disadvantages of theater tests but in a more realistic setting. Physiological measures - a laboratory method for measuring involuntary responses to commercials such as heartbeat, pupil dilation, eye movement, brain waves and galvanic skin response Use of this slide This slide can be used to discuss the various methods for testing reactions to finished broadcast commercials. While theater tests and on-air testing are commonly employed, physiological measures are used less often. © 2007 McGraw-Hill Companies, Inc., McGraw-Hill/Irwin

Physiological Measures Pupil dilation Pupil dilation Galvanic skin response Galvanic skin response Testing Relation to text This slide relates to material on pp. 615-616 and Figures 19-13 of the text. Summary Overview This slide shows the various physiological measures, a less common method of pretesting finished commercials. These measures include: Pupil dilation Galvanic skin response Eye tracking Brain waves Use of this slide This slide can be used to discuss how physiological responses are measured. These measures indicate the receiver’s involuntary response to the ad, theoretically eliminating biases associated with voluntary measures. Brain waves Eye tracking Eye tracking © 2007 McGraw-Hill Companies, Inc., McGraw-Hill/Irwin

Market Testing Print Ads Inquiry Tests Inquiry Tests Recognition Tests Recognition Tests Testing Relation to text This slide relates to material on pp. 616-619 and Figures 19-14, 19-15 and Exhibits 19-4 and 19-5 of the text. Summary Overview This slide shows the various methods that are used to determine the effectiveness of print advertising once the ads have been completed and put into the market. A number of methods are available including: Inquiry tests Recognition tests Recall Tracking studies Use of this slide This slide can be used to discuss the various options available to marketers for the testing of how well print ads are performing in the market. These methods are often used by advertisers to determine how well print ads are doing in terms of getting attention, being read, and being remembered. Tracking Studies Recall Tests Recall Tests © 2007 McGraw-Hill Companies, Inc., McGraw-Hill/Irwin

Starch-Scored Sports Illustrated Ad Relation to text This slide relates to material on pp. 617-618 and Exhibit 19-5 of the text. Summary Overview This slide shown an example of a Starch scored ad. The Starch method is one of the more commonly employed posttest measures of print ads. It uses three measures including: Noted score– the percentage of readers who remember seeing the ad Seen-associated score—the percentage who remember seeing or reading any part of the ad identifying the product or brand Read most score—the percentage of readers who report reading at least half of the copy portion of the ad. The Reflect.com ad did well in the first of these two categories, but not as well (10%) in the read most categories. Use of this slide This slide can be used to provide an example of a Starch scored advertisement. Starch measures have been used in the magazine industry for many years, and have become one of most commonly employed measures of advertising effectiveness © 2007 McGraw-Hill Companies, Inc., McGraw-Hill/Irwin

Market Testing Broadcast Commercials Day after recall tests Day after recall tests Persuasive measures Persuasive measures Tracking studies Diagnostics Diagnostics Testing Relation to text This slide relates to material on pp. 619-622, Exhibit 19-6 and 19-7 and Figure 19-16 of the text. Summary Overview This slide shows the variety of measures available for the market testing of finished broadcast commercials. These include: Day after recall test - the most commonly performed measure. Persuasive measures – attempting to measure persuasive impact of the commercial on brand selection. Diagnostic - measures to determine viewers’ perceptions of the commercial including evaluations, perceptions, etc. Comprehensive measures - a combination of each of the three previous measures. Test marketing - testing of finished commercials in specified test markets. Single source tracking - tracking of the impact of the commercial from TV set to the retail store through the use of scanner data Tracking studies - measuring the effects on various factors (recall, awareness, and attitudes) over time. Use of this slide This slide can be used to demonstrate the various options available for the marketer interested in measuring the effectiveness of a finished commercial. There are numerous research companies that offer post testing capabilities. Comprehensive measures Comprehensive measures Single-source tracking Single-source tracking Test marketing Test marketing © 2007 McGraw-Hill Companies, Inc., McGraw-Hill/Irwin

Comprehensive Testing by Ipsos-ASI Relation to text This slide relates to material on pp. 619-622 and Exhibit 19-6 of the text. Summary Overview This slide shows an example of a provider of research service, Ipsos-ASI. The company provides a variety of research services for print and broadcast measurement, including diagnostics and comprehensive measures. Use of this slide This slide can be used to provide an example of one of the many research companies whose services are available for commercial testing. © 2007 McGraw-Hill Companies, Inc., McGraw-Hill/Irwin

Problems With Current Research Methods Relation to text This slide relates to material on pp. 623-624 and Exhibit 19-3 of the text. Summary Overview This slide shows the principles important to good copy testing. It is clear that some of these principles are easily accomplished, whereas others require substantially more effort. Use of this slide This slide can be used to discuss problems with current research methods when compared to the criteria established by PACT. Principles 1 and 2: should require minimum effort providing measurements relative to the objectives sought and determining a priori how the results will be used. Principle 3: slightly more difficult, but largely in control of the researcher, providing multiple measurements Principle 4: one of the most difficult factors to control. Most current methods to little more than provide recall scores. Principle 5: slightly difficult, can be accomplished with proper research design. Principle 6: should require minimum effort, providing equivalent test ads. Principle 7: one of the most difficult factors to control. Lab measures are artificial and vulnerable. Field measures often lose control. Principle 8: slightly difficult, requires little more than sound research methodology. Principle 9: one of the most difficult factors to control due to concerns for reliability and validity. © 2007 McGraw-Hill Companies, Inc., McGraw-Hill/Irwin

Essentials of Effective Testing Use a consumer response model Use a consumer response model Establish communications objectives Establish communications objectives Use pretests and posttests Use pretests and posttests Testing Relation to text This slide relates to material on pp. 624 of the text. Summary Overview This slide shows the essentials of effective advertising testing. To be a truly effective measure of effectiveness, the testing method should: Establish communications objectives Use a consumer response model Use both pretests and posttests Use multiple measures Understand and implement proper research Use of this slide This slide can be used to provide an overview of the criteria for effective measurement. Solid research should meet these criteria. Research methodologies that meet these criteria will be the most effective for providing insight into the performance of advertising. Understand and implement proper research Use multiple measures Use multiple measures © 2007 McGraw-Hill Companies, Inc., McGraw-Hill/Irwin

Measuring the Effectiveness of Other Programs Sales promotions Shopping cart signage Shopping cart signage Non-traditional media Ski resort-based media Ski resort-based media Relation to text This slide relates to material on pp. 624-627 of the text. Summary Overview This slide shows a number of methods for measuring the effectiveness of other program elements not previously discussed. These measures include: Sales promotions—a number of organizations measure sales promotions, including MarketSource, Schnucks, Smitty’s Super Valu, Vons and Shopper Trak. Nontraditional media—examples include shopping cart signage, ski resort-based media, in-store radio and television, and other media. Sponsorships—essentially, measures of sponsorship can be categorized as exposure-based methods or tracking measures. Use of this slide This slide can be used to discuss the increased use of integrated marketing communications programs, which has led to more interest in determining the synergistic effects of all program elements. In-store radio and television In-store radio and television Other media Other media Sponsor-ships Exposure methods Exposure methods Tracking measures © 2007 McGraw-Hill Companies, Inc., McGraw-Hill/Irwin

Measuring the Effectiveness of Other IMC Program Elements Relation to text This slide relates to material on pp. 627-629 and Figure 19-23 of the text. Summary Overview This slide shows illustrates the importance of measuring effectiveness and efficiency. Based on the belief that integrated marketing communications improves both the efficiency and the effectiveness of a campaign, integration contends that most traditional measurement techniques focus only on the former of these. Use of this slide This slide can be used to discuss the increased demand for marketing managers to prioritize the media vehicles used to promote their brands, and to measure both the efficiency and the effectiveness of media used to establish contacts with consumers. © 2007 McGraw-Hill Companies, Inc., McGraw-Hill/Irwin