Evaluation of Effectiveness Advertising Principles and Practices Evaluation of Effectiveness

Questions We’ll Answer How well do you understand why and how advertising evaluation is conducted? Can you list and explain the stages of message evaluation? What are the key areas of media evaluation? How are campaigns and IMC programs evaluated?

Outstanding in the Field New Holland had to reinvent and reintroduce its tractor brand with sliding market share, a shrinking market, and limited budget. The campaign positioned New Holland as the better choice of an empowered consumer, and drove traffic to its Web site. Sales grew by 9%, market share by 21% and 36% for low and high horsepower categories, Web site visits were 68,000 over the campaign period. \ Visit the Site Prentice Hall, © 2009 19-3

Impact: How does it work? Many executive feel advertising is only successful if it produces sales. Others feel advertising should emphasize long-term brand building. If advertising delivers the desired communication effects, but sales don’t increase, was the advertising ineffective? Video Snippet AFLAC’s initial campaign created 89% brand recognition.

Evaluating Effectiveness Intuitive analysis is based on an experienced managers judgment. Measurement tracks consumer responses with structured feedback like response cards and calls. Formal evaluation is necessary: Financial stakes are high—production of :30 spot averages $200,000; national media costs several million. Advertising optimization—reducing the risk failure through testing, analyzing, tracking performance, and making changes to increase performance. Identify “best practices”—what works and what doesn’t, so brand advertising continues to improve.

Types of Evaluation Testing—to predict results Sample ads are tested before they run. Monitoring—to track performance Performance is tracked to see if anything needs to be changed. Measurement—to evaluate the results The results, or actual effects, are measured after the campaign runs.

Stages of Evaluation Developmental research Concurrent research Pretesting to see if an idea will work, or another is better. Concurrent research Tracking studies and test marketing to see how campaign is unfolding and how messages and media are working. Posttesting research Comparing the impact of campaign after it’s over against a benchmark, baseline, or other starting point. Diagnostic research Taking apart an ad to see what elements are working and which aren’t; examine frame by frame or piece by piece.

Facets: Measuring Responses It’s difficult to measure advertising’s effect on sales: Other factors affect sales (e.g., pricing, distribution, competition), making it hard to isolate impact. Effects are delayed; it’s hard to link sales to advertising. Communication effects an be measured as surrogate measures for sales impact: Awareness of the advertising, purchase intention, preference, liking. Principle: Good evaluation plans, as well as effective promotional work, are guided by a model of how people respond to advertising.

Table 19:1 Effectiveness Research Questions Perception Awareness/Noticed Attention Recognition (Aided ) Relevance Emotion Liking/Disliking Desire Cognition Interest Comprehension/ Confusion Recall (Unaided) Brand Recall/ Linkage Differentiation Research Questions What ads do you remember seeing? Which ads were noted? What caught your attention? Did the ad stand out among the other ads and content around it? What stood out in the ad? Have you seen this ad/this campaign? Sort elements into piles of remember/don’t remember. How important is the product message to you? Does it speak to your interest and aspirations? What emotions did the ad stimulate? How did it make you feel? Do you like this brand? This story? The characters (and other ad elements)? What did you like or dislike about the brand? The ad? Do you want this product or brand? Did you read/watch most of it? How much? Did it engage your interest or curiosity? Where did your interest shift away from the ad? What thoughts came to you? Do you understand how it works? Is there anything in the ad you don’t understand? Do the claims/product attributes/benefits make sense? Do you have a need for this brand or can it fulfill a need for you? What happened in the commercial? What is the main message? What is the point of the ad? What brand is being advertised in the ad? (In open-ended responses, was the brand named?) What’s the difference between Brand X and Y?

Table 19:1 Effectiveness Research Questions Persuasion Attitude Preference Intention Argument/Counter Argument Believability/Conviction Trust Association Personality/Image Self Identification Research Questions Do you have a favorable or unfavorable opinion of the brand? The ad? How excellent or weak is the brand? The ad? Do you respect it? In Category X (or product set), which brand would you choose? (Usually a pre- or posttest question). What brand do you prefer? Do you want to try or buy this product/brand? Would you put it on your shopping list? What are your reasons for buying it? Or for not buying it—or its competing brand(s)? How does it compare to competitor’s brands? Do you argue back to the ad? Do you believe the reasons, claims, or proof statements? Are you convinced the message is true? The brand is best? Do you have confidence in the brand? When you think of this brand, what (products, qualities, attributes, people, lifestyles, etc.) do you connect with it? Do you link this brand to positive experiences? What is the personality of the brand? Of whom does it remind you? Do you like this person/brand personality? What is the brand image? What does it symbolize or stand for? Can you see yourself or your friends using this brand? Do you connect personally with the brand image?

Message Evaluation Copy Testing Message Development Research During Execution: Concurrent Testing Posttesting: After Execution Research

Copy Testing Companies that conduct research and perform diagnostic methods to identify an ad’s strong and weak points: Ameritest: brand linkage, attention, motivation, communication, flow of attention and emotion through the commercial ARS: persuasion, brand/ad recall, communication Diagnostic Research: brand recall, main idea, attribute statements (importance, uniqueness, believability) IPSOS-ASI: recall, attention, brand linkage, persuasion, (brand switch, purchase probability), communication Mapes & Ross: brand preference change, ad/brand recall, idea communication, key message delivery, like/dislike, believability, comprehension, desire to take action, attribute communication Millward Brown: branding, enjoyment, involvement, understanding, ad flow, brand integration, feelings to ad, main stand-out idea, likes/dislikes, impressions, persuasion, new news, believability, relevance RoperASW: overall reaction, strengths and weaknesses, understanding, clutter-busting, attention, main message, relevance, appeal, persuasiveness, motivate trial, purchase intent

Ameritest Ameritest is an international advertising research firm that studies thousands of ads for some of the world’s largest advertisers to discern which ones work, which ones don’t, and which are close—and why. Visit the Site

Message Development Research Concept Testing Compares the effectiveness of various message strategies and their creative ideas (the big idea). Pre-testing Helps marketers make final go/no-go decisions about finished/nearly finished ads using photoboards or animatics. Diagnostics Designed to diagnose strengths and weaknesses of ideas to improve work still in development or to learn more in order to improve subsequent advertisements. Principle: Advertising effects are too complicated to be reduced to a single score.

During Execution: Concurrent Testing Coincidental Surveys In broadcast media, random calls to target market determine stations choices, ads they’ve seen/heard, brand perceptions. Tracking Studies Every 3 to 6 months, measure top-of-mind brand awareness. Brand tracking tracks the performance of the brand. Test Markets Evaluate product variations, campaign or media elements. Generally, two or more markets with markets as controls.

Posttesting: After Execution Research Breakthrough: Attention—interest, enjoyability, liking Engagement Tests—eye-tracking as readers scan ads Memory Tests—recognition test, recall tests, unaided recall, aided recall Emotion Test—MRI measures brain activity Likeability Tests—relevant, important, enjoyable, entertaining, fun Persuasion Tests—intention to buy, motivation Inquiry Tests—measures number of responses to an ad Scanner Research—tally up purchase and collect consumer buying info Single-Source Research—advertising and brand purchase data come from the same households, linking advertising to sales

Media Evaluation Evaluating Audience Exposure How did each media vehicle perform? Were reach and frequency objectives met? Services include Simmons-Scarborough, Arbitron, MediaMark. For outdoor, traffic counts don’t equal exposure. For Web or Internet advertising, what is measured and how does it compare to traditional media: hits, click-throughs, minutes spent? Alternative or guerilla marketing is even more difficult to equate to traditional media.

Arbitron Arbitron Inc. measures radio, TV, cable, online radio and out-of-home media audiences; surveys the retail, media and product patterns of local market consumers; and provides software for analyzing media audience and marketing information data. Visit the Site

Media Evaluation Advertising ROI and Media Efficiency Return on investment (cost to sales ratio) is hard to calculate because many factors affect sales. How do you determine if you’re overadvertising or underadvertising? Wearout—recall stabilizes or declines and irritation increases until there’s no or less response (can be a combo of creative impact and media buying). Media optimization—the goal is optimum media performance getting the most impact for the investment.

Evaluating Marketing Communications Campaigns Last, and perhaps most important, stage in the development of a campaign plan Determines whether the campaign’s message and media were effective Measures the overall impact on the brand, but the pieces are still evaluated to determine their individual effectiveness.

Marcom Tools Certain marketing communication functions such as public relations and sales promotion, do some things better than other areas. An integrated plan uses the best tools to accomplish the desired effect. Principle: Advertising is particularly effective in accomplishing such objectives as creating exposure, awareness, and brand image, and delivering brand reminders.

Table 19:2 Message Effectiveness Factors Key Message Surrogate Effects Measures Communication Tools Perception Exposure Adv Media; PR, PoP Attention Adv; Sales Promo, Packaging; PoP Interest Adv; SP; PR, Direct; PoP Relevance Adv; PR; Direct; PoP Recognition Adv; PR, Pkging; PoP, Specialties Cognitive Understanding Adv; PR; Sales; Direct Recall Adv; SP; PR, PoP, Specialties Adv; PR; Pkging Affective Emotions and Liking Adv; Sales Promo, Pkging; PoP Appeals Adv; PR; Sales; Events/Spnsrshps Resonate Adv; PR; Events/Spnsrshps Persuasion Attitudes Adv; PR; Direct Preference/Intention Adv; PR; Sales; Sales Promo Credibility PR Conviction PR; Sales; Direct Motivation Adv; PR; Sales; Sales Promo Brand Assoc Brand Image Adv; PR, Events/ Spnsrshps Action Trial SP; Sales; Direct, PoP Purchase SP; Sales; Direct Repeat Purchase Adv; SP; Sales; Direct, Specialties

Direct Response The objective is to generate an immediate behavior response (transaction, buy) Use toll-free numbers, mail-in coupons, Web site or email address, an offer in the copy Response is easy to measure in terms of effectiveness and ROI. Total responses divided by total mailed = Response per thousand (RPM)

Sales Promotion May be necessary to evaluate both trade and consumer promotions Payout analysis compares the costs of a promotion to the expected sales Breakeven analysis—finds the point at which the total cost of the promotion exceeds the total revenues

A Sales Promotion Breakeven Analysis At the breakeven point, where 30,000 premiums are delivered at a cost of $45,000, the sales revenues exactly cover, but do not exceed, total costs. Below and to the left of the breakeven point (in the portion of the diagram marked off by dashed lines) the promotion operates at a loss. Above and to the right of the breakeven point, as more premiums are sold and sales revenues climb, the promotion makes a profit. Prentice Hall, © 2009 19-25

Public Relations Measure the success in getting out the message in terms of output and outcomes Output: materials produced and distributed; how many press releases ran Input: acceptance and impact of materials; changes in public opinion Content analysis: Was coverage favorable? Public opinion studies: Have attitudes, behaviors, or knowledge changed?

Common Measures of Output and Outcomes in Public Relations Output Objectives Achieved Production. Number of PR messages, such as news releases or brochures, generated. Distribution. Number of media outlets (TV stations, newspapers) receiving PR products. Coverage. Number and size of clips, column inches, or minutes of time or space. Impressions. Media placements multiplied by circulation or broadcast reach. Advertising value. Equivalent ad costs for time or space. Content Analysis. Positive or valence (whether the story or mention seems to be more positive or negative), key messages (the idea in the story), sources, and prominence. Outcome Objectives Achieved Awareness. Aided and unaided recall. Attitudes. Perceptions, preferences, and intent to buy. Behavior. Did target do what you wanted them to do?

Web Site Evaluation Traffic volume Click-through rates Cost per lead Page views Site visitors Click-through rates Ads sold as pay-per click Cost per lead An attempt to measure ROI using a conversion rate (percent of visitors who complete desired action)

Special Advertising Situations Retail advertising B2B advertising International advertising Objective: Generate store traffic Simple counts of people at promotions and events Objective: Visibility Participation counts at events, or “how-to” classes Sign-up and fill-out forms Objective: Loyalty Participation in frequency clubs or loyalty programs

Special Advertising Situations Retail advertising B2B advertising International advertising Objective: Generate response/sales leads Lead count based on calls, emails, and cards returned to the advertiser Objective: Conversion rates—number of leads who make a purchase

Special Advertising Situations Retail advertising B2B advertising International advertising Difficult to evaluate because of the number of markets, distance, cost and variety of cultures. Evaluation should focus initially on pretesting to help correct big problems (due to unfamiliarity with the culture, language or consumer behavior) before they occur.

Campaign Evaluation It’s difficult to evaluate and estimate the impact of synergy. Brand tracking can measure campaign effectiveness by adding and taking away ingredients, and studying the effects of those changes The challenge: look at the big picture rather than individual pieces and parts. Advertisers seek an evaluation method that brings all the individual metrics together to efficiently and effectively evaluate and predict communication effectiveness.

ARS Group ARS Group is a marketing research firm that helps clients evaluate and optimize the effectiveness of their advertising messages within individual touchpoints and across integrated marketing campaigns. Visit the Site

Discussion Questions

Discussion Question 1 Many creative people feel that formal copy testing research doesn’t do justice to their ideas. In particular, they feel that research results are designed to reward cognitive approaches and don’t do a good job of evaluation for brand image ads and emotional ads. From what you have read in this chapter about copy testing, why do they feel that way? Do you believe that is a legitimate criticism of copy testing?

Discussion Question 2 Most clients want a quick and easy answer to the question of whether the ad works. Advertising professionals, however, tend to believe that a one-score approach to copy testing is not appropriate. Why would they feel that way? If you were helping an agency prepare for a presentation on its copy testing results, what would you suggest the agency say to explain away the idea that you can evaluate an ad with a single test?

Discussion Question 3 Three-minute debate: You are hiring a research consulting company to help a client evaluate the effectiveness of its advertisements. One of the consultants recommends using focus groups to evaluate their effectiveness. Another consultant suggests that focus groups aren’t very effective for posttesting and recommends other measures. Which viewpoint do you believe is most insightful? In class, organize into small teams with pairs of teams taking one side or the other. Set up a series of three-minute debates with each side having half of that time to argue its position. Every team of debaters has to present new points not covered in the previous teams’ presentations until there are no arguments left to present. Then, the class votes as a group on the winning point of view.

Copyright © 2009 Pearson Education, Inc. Publishing as Prentice Hall All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher. Printed in the United States of America. Copyright © 2009 Pearson Education, Inc.   Publishing as Prentice Hall