1. Evaluating an Integrated Marketing Program
Chapter 15

2. What should be evaluated?
Short-term outcomes (sales, redemption rates)
Long-term results (brand awareness, brand loyalty)
Product specific awareness (new product or extension, company)
Awareness of the overall company
Affective responses (liking the company, and positive brand image)

3. Evaluation
Pretest measures
Concurrent measures
Post test measures

4. Message Evaluation Techniques
Concept testing
Copytesting
Emotional reaction tests
Cognitive Neuroscience
On-line evaluation techniques
Behavioural techniques
Recall tests
Recognition tests

5. Difficulties in Evaluating Advertising
Influence of other factors on behaviour
Delayed impact of advertising
Consumers change their mind in the store
Whether or not the brand is in evoked set
Goal of ad may be to build brand equity, not increase sales so what you intend to measure is important to clarify.

6. Advertising Tracking Research
Track ads after launch
Nielsen IAG
Millward Brown
Monitors
Brand performance
Advertising effectiveness
Specific time test
Continuous tracking
A common method of evaluation involves tracking an advertisement by one of the advertising research firms, such as Nielsen IAG or Millward Brown. These tracking services monitor an ad’s performance and measure its effectiveness. Tests can be performed at specific times or continuously. Nielsen IAG, for instance, offers a syndicated database of real-time brand and ad performance based on more than 210,00 television program episodes and 250,000 commercials. Nielsen conducts thousands of surveys daily measuring viewer engagement with TV programs and effectiveness of ads.

7. Concept Testing Aimed at content of a marketing piece.
What is evaluated?
Verbal messages
Message and meaning
Translations for international copy
Product placement in ads
Value of a sales promotional efforts
Focus groups may be used
Concept testing instruments:
Comprehension and readability tests
Reaction tests

8. Copytesting Used when marketing piece is finished or in final stages
Portfolio test – print communication
Theater test – television/mass media communication
Methods used:
Focus groups
Can be mall intercept

9. Copytesting
Copytesting can be used to determine if viewers comprehend this ad and what their reaction to it is.

10. Copytesting Criticisms of copytesting Support of copytesting
Some agencies do not use them
Can stifle creativity
Focus groups may not be a good judge
Support of copytesting
Issue of accountability -
Majority support copytesting because clients want support for ad decision

11. F I G U R E
Copytesting principles of Positioning Advertising Copytesting (PACT)
Testing procedure should be relevant to objectives.
Researchers should agree on how the results will be used in advance.
Multiple measures should be used.
The test should be based on some model or theory of human response to communication.
Testing procedure should allow for more than one exposure.
In selecting alternate ads to include in the test, they should be at the same stage in the process as the test ad
The test should provide controls to avoid biases.
Sample used for the test should be representative of the target sample.
Testing procedure should demonstrate reliability and validity.
Source: Based on PACT document published in the Journal of Marketing, (1982) ,Vol. 11, No. 4, pp

12. What’s Happening?

13. Emotional Reaction Tests
Used for material designed to solicit emotions.
Difficult to measure emotions with questions.
Warmth Meters are usually used as am instrument

14. Sample Graph from a Warmth Meter 30-Second TV Advertisement
Ad section that elicited negative emotions

15. Cognitive Neuroscience
Measures brainwave activity
Better indicator of respondent’s true reaction
Does not rely on self-reporting, like emotional reaction tests
Companies experimenting with neuroscience
EmSense, NeureoFocus, Sands Research
Frito-Lay
Power of cognitive neuroscience
Reveals physiological reactions
Understand how information is being processed
Cognitive neuroscience measures brainwave activity and is the latest advance in advertising research. It is a better indicator of a person’s true reaction since it does not rely on self-reporting. A number of companies are experimenting with cognitive neuroscience. Frito-Lay used cognitive neuroscience to examine a particular ad that was rated inappropriate by a focus group. The neuroscience research showed women loved the ad. The power of neuroscience is that it reveals physiological reactions. It provides information on where the information is being processed in the brain and what consumers do with the information they receive. Instead of rely on self-reporting, cognitive neuroscience hopes to measure reactions accurately.

16. Online Evaluation Metrics
Click-throughs primary measurement
Length of engagement
Dwell time
Redemption rates
Response rates
Sales
Web chatter
The online environment provides a number of metrics that cannot be obtained with other media, especially so quickly. Web metrics can be obtained in almost real-time. The primary measure of online ads is click-throughs, which is the number of people who click on the ad and go to the Web site. Length of engagement is a newer metric that measures how long a person stays at a Web site, regardless of where they are on the site. Dwell rate measures the proportion of ad impressions that result in users engaging with an ad. Dwell time measures the amount of time users spend engaged with a particular ad. Redemption rates and response rates can be used to measure ad effectiveness, as well as sales. Companies can also monitor Web chatter to see how many times the brand name is being mentioned and what is being said about the brand.

17. Response rates to offers Test markets Purchase Simulation tests
F I G U R E
Behavioural Measures
Sales
Redemption rates
Response rates to offers
Test markets
Purchase Simulation tests

18. Direct marketing responses. Redemption rate of sales promotion offers.
F I G U R E
Responses to Marketing Messages that can be Tracked
Changes in sales
Telephone inquiries.
Response cards.
Internet inquiries.
Direct marketing responses.
Redemption rate of sales promotion offers.
Coupons, premiums, contests, sweepstakes

19. Test Markets Used to assess:
Advertisements (different versions in different markets)
Consumer and trade promotions
Pricing tactics
New products
Cost effective method of evaluation prior to large-scale launch.
Resembles actual situation.
Design test market to model full marketing plan.
Length of test market is a concern.
Competitive actions must be considered

20. Purchase Simulation Tests
Bias in purchase intention questions
Simulated purchase tests
Research Systems Corporation
Does not rely on opinions and attitudes
Another behavioral measure is purchase simulation tests. Purchase intention questions are biased with more people saying they will buy a product than actually do. Simulated tests are more accurate. Research Systems Corporation is a primary provider of simulation tests. It does not rely on opinions and attitudes. Research Systems Corporation uses a controlled lab and a simulated store to study consumer purchases. Subjects are given various scenarios or shown particular ads then asked to shop as they normally would at their store.

21. Recall Tests Day-after recall (DAR) Unaided recall Aided recall
Factors affecting recall:
Respondent’s age affects recall scores
Medium used also plays a factor

22. Recall Tests - Items tested
Product name or brand
Firm name
Company location
Theme music
Spokesperson
Tagline
Incentive being offered
Product attributes
Primary selling point of communication piece

23. Sample Recall Test 30-Second TV Advertisement for Pet Food

24. Recall Decay Magazine Ad vs. Television Ad
Source: Magazines Canada’s Research Archive