Pros and Cons of Measuring Advertising Effectiveness Reasons to Measure Ad Effectiveness Avoid costly mistakes Evaluate alternative strategies Increase efficiency of advertising in general Determining if objectives are achieved Reasons Not to Measure Effectiveness Cost of measurement Problems with research Disagreement about what to test Objections of creative personnel Lack of time

Evaluating Creatives* Concept Testing: Assessment of potential creatives Communication Research: A Look at the Advertising Creative Copy Testing: Assessment of creative achievement The word creative when used as a noun refers to an actual advertisement

Concept Testing A concept is similar to the actual ad, but more succinct without a lot of selling and posturing The concept states in clear terms the selling premise, unique position or benefit of the advertised product or service We test the concepts in order to determine which of a set of concepts would be the most useful in accomplishing our objective See handouts to see the difference between a concept and the corresponding creative

Types of Concept Tests New Product Concept Tests Generate concepts for multiple new product possibilities and test the concepts to assess which new product has the most potential for success Benefit Positioning Concept Tests Generate multiple concepts for one product to test which concept has the most potential for generating desired outcome, generally purchase intent Each concept will focus on a different benefit (position) Example: concept test for a new PDA concept 1 focuses on ease of use concept 2 focuses on technical capability

General Outline for Performing Concept Testing Generate Concepts Generate Questionnaire Administer the Test Organize, analyze, interpret data

Concept Generation Concepts are typically presented on 8.5”X11” cardstock or board, and are sometimes called “white card” tests Card contains: Headline Body copy Illustrations

White Card Headline This is NOT the same as an advertising headline A succinct summary of the main selling proposition or product benefit Leads into the body copy Gives consumer a clear understanding of the ideas that are to be presented in the body copy Use simple, single-minded declarative statements “The new GeoNET PDA integrates many wireless technologies so you can do more when you travel.”

Concept Body Copy Be direct and clear, using short paragraphs and active, declarative statements Do not use: Advertisingese; do not embellish Industry jargon that is unfamiliar to the audience Overwritten or long concepts If new product concept, you should also Identify product category membership List all primary benefits Identify differentiating attributes

Concept Body Copy The new GeoNET PDA integrates a wireless phone, GPS receiver, and Blackberry wireless technology with 512MB of internal memory and coast to coast Web access. With all of these features in one convenient device, working while traveling is easier than carrying multiple devices to accomplish the same tasks. The GeoNET PDA is available with either Palm OS or Windows Mobile for even more flexibility Suggested retail prices: $299 with black and white display; $449 with 16-bit color display

Concept Generation Considerations Using Brand names is generally accepted as concept perceptions are often impacted greatly by brand name presence Pricing information should be present If not in the concept itself, integrate it into the questionnaire Price does not need to be explicitly stated, but if it is, it should accurately reflect the actual expected price Illustrations and format between tested concepts should be as identical as possible

Generate Questionnaire Demographic and brand usage questions Diagnostic questions pertaining to specific characteristics of the concept Core questions Communication of the main idea Believability of the main idea Uniqueness of the main idea Personal relevance of the main idea Purchase Intent Purchase frequency Reasons for purchase intent and purchase frequency

Concept Test Administration Target definition Research methodology Questionnaire construction Concept exposure Data analysis and interpretation

Data Organization, Analysis and Interpretation Split sample by respondent characteristics (quota groups) and assess differences between concepts Split sample by concepts and assess differences between quota groups Crosstabs on core questions by purchase intent, brand usage, and communications Correlation matrix between believability, uniqueness, and relevance

Group Practice Time Concept Testing Handout Read through the description of the study and purpose of the research The actual concept and a variety of results tables are provided for you Answer the 4 questions at the end of the handout using the results provided

Communication Research After concept testing, we have identified the concept, that is, the main idea that is to be communicated in the advertisements Now we must develop the actual communications, the ads, and determine if we are communicating our ideas effectively Basic goal is to determine how consumers react to the ad and why they are reacting that way

General Outline for Communication Research Create test ads Specify Respondent Characteristics Select Methodology Generate Questionnaire Analyze and Report findings

Creating the Test Ads Print ads should be created to the semi-comp stage, meaning it is full size, with color and art in place and scaled to size May not be exactly how ad will look in print, but should be close Radio ads can be scratch tracks, meaning that all aural elements are there, but recording may not be professional quality yet TV ads may be storyboarded as long as the ad comes across to the audience in the correct way May use animatics, photoboards, or photomatics if needed to get across important visual cues In all cases, copy should be precise

Generate the Questionnaire Brand usage, demography, etc. Core questions to assess: Whether the respondent accurately processed the information presented Respondent attitudes toward the communication, i.e. the message was well received Should probe cognitive and emotional responses Respondent attitudes toward the ad itself Questions specific to the particular creative

Analysis and Interpretation of Communication Research Data Typically standard tests (t-tests, chi-square tests) are all that are needed to describe the results Relationships between variables can be interesting, but is more difficult and requires some creativity on the part of the researcher since no hypotheses are developed for testing

Copy Testing Communication research is performed during creative development and is diagnostic in nature Copy testing takes place after creative developments using actual ads, and is evaluative in nature Independent marketing research companies specialize in copy testing, and have established a set of norms upon which go/stay decisions are made The majority of copy testing is done on television advertising, although options exist for print, radio, and outdoor advertising as well Television advertising is far more expensive to create and run than other media, hence the abundance of tv copy testing

Four key decisions to make Place: in home viewing or out of home Naturalness: forced versus unforced exposure Number of exposures before data collection Key measures: Recall Message communication Attitude change Purchase intention Ad attitude Et cetera

Typical structure of Television Copy Test Show a sample of 200 respondents a 30 minute program with test ad imbedded Program is broadcast on an unused cable channel so respondents may view the program in their homes The day after viewing, respondents are contacted and asked to name the commercials they remember seeing during the program If unable to name the commercials, they are prompted with correct and incorrect answers This tests both aided and unaided recall Purchase intent can be ascertained with a pre-post exposure measurement Prior to watching, respondents are asked what brand they are likely to purchase next time After exposure, respondents are asked what brand they would prefer to win Asking the same preference question in a different context masks the purpose of the question to ensure accurate responses

Other Key Measures Intrusiveness: measure of the ads ability to break through the clutter and earn a place in the consumer’s mind Unaided recall – correct brand Proved unaided recall Proved brand aided recall Communication: identifies the ideas that consumers got out of the ad The Lead Idea is the idea that is most often cited by respondents Persuasion: ad’s ability to generate favorable brand attitudes Purchase intent Standard brand attitude measure Commercial Reaction Standard ad attitude measures

Typical Structure of a Magazine Copy Test 125 respondents are recruited through mall intercepts They are asked questions about media and product usage They are provided a fake magazine with test ad imbedded and asked to quickly flip through and give initial reactions Ask respondents what ads they remember seeing Have them look through magazine again, for a longer time this time Now ask recall, persuasion and communication questions Direct respondents to the test ad and ask them to read it Ask customized communication and diagnostic questions

Typical Measures in Print Copy Testing Recall/Recognition Idea Communication Brand Attitude Ad Attitude Purchase Intent Stopping Power Noted Readers: only remember seeing it Associated Readers: read part of it Read Most Readers: read more than half of it

Copy Testing Notes Methodology should be as close to “real world” as possible Using multiple exposures and collecting different data after each exposure allows identification of long term effects Responses are more valid when exposure occurs without respondent knowledge of participation

When Using Outside Firms Research Services Some measures between companies may have similar names, but measure completely differently. Understand the method completely so you can truly ascertain the meaning of the data Know what the “norms” are, and be sure that they are current and appropriate to the product Ensure that measures used by research firm are reliable and valid; if firm does not share, or claims to not know, reliability and validity diagnostics, be wary of results