Orientation Project Implementation Reporting & Sustainability Testing Efficacy To Ensure Applicability and Impact Module 3, Unit 3, Sub-Unit 1, Session 5
Failed in Focus Group Example: 2
Module 3, Unit 3, Sub-Unit 1, Session 5 3 Pretesting The process of systematically gathering target audience reactions to messages and materials before they are produced in final form
Module 3, Unit 3, Sub-Unit 1, Session 5 4 Use Pretesting To: Determine which of multiple alternative creative directions or messages may be most effective Identify strengths and weaknesses in a single execution (e.g., brochure, poster, etc.)
Module 3, Unit 3, Sub-Unit 1, Session 5 5 Reasons For Conducting Pretest Identify the benefit that resonates most with the target audience Ensure that audience understands the message Identify any “red flags” Strengthen your message Conduct “real world” check Make the materials more appealing Identify weak and strong points
Module 3, Unit 3, Sub-Unit 1, Session 5 6 Which Materials Do You Pretest? Pamphlets, posters, brochures, booklets, flyers, print ads, and TV and radio spots and dramas No need to pre-test: –Bumper stickers or badges –Derivatives of already pretested materials
Module 3, Unit 3, Sub-Unit 1, Session 5 7 Which Materials Do You Pretest? Multiple versions (3 to 4) of creative executions with core messages communicating various benefit exchanges More elaborated version of the materials – containing supporting facts and information
Module 3, Unit 3, Sub-Unit 1, Session 5 8 Criteria Evaluated During Pretesting Comprehension – does the Target Audience (TA) understand the materials? Relevance – Is the material and the message important to TA? Credibility – have trust in the message Acceptability – fitting with the target audience value and culture? Attraction – will it hold the audience’s attention? Persuasion – is it convincing?
Rate this Ad: Comprehension: Relevance: Credibility: Acceptability: Attraction: Persuasion: 9
Rate this Ad: Comprehension: Relevance: Credibility: Acceptability: Attraction: Persuasion: 10
Rate this Ad: Comprehension: Relevance: Credibility: Acceptability: Attraction: Persuasion: 11
12
Module 3, Unit 3, Sub-Unit 1, Session 5 13 Pretesting Process 1. Determine test objectives and choose research methods 2. Draft research instruments (e.g., discussion guides, questionnaires) 3. Secure interviewers and space for conducting research (if required) and identify, screen and recruit research participants 4. Conduct testing and analyze results 5. Based on consumer feedback revise concepts and materials
Module 3, Unit 3, Sub-Unit 1, Session 5 14 Determine Test Objectives and Choose Research Methods Sample Objectives: Ensure that the main message of a [poster, brochure, etc.] is readily understood Ensure that the language and images used in a [poster, brochure, etc.] are clear Select the concept that best communicates the intended message Determine what was unclear or hard to understand Identify the concerns of respondents that need to be addressed by the materials
Module 3, Unit 3, Sub-Unit 1, Session 5 15 Determine Test Objectives and Choose Research Methods Methods: –Expert and gatekeeper review –Focus groups –In-depth interviews –Central location intercept interviews –Self-administered questionnaires
Module 3, Unit 3, Sub-Unit 1, Session 5 16 Expert and Gatekeeper Review Purpose: –Ensure technical accuracy and acceptability To Be Tested : –Content and artistic elements –Print and audio-visual Number: –5 to 10 Pros and Cons
Module 3, Unit 3, Sub-Unit 1, Session 5 17 Focus Groups Purpose: –Group interaction; idea generation; materials evaluation; problem identification To Be Tested : –Print, radio, or TV message concepts, slogans/- tag-lines –Logos and rough artwork Number: –8 to 10 per group –2 groups per segment Pros and Cons
Module 3, Unit 3, Sub-Unit 1, Session 5 18 In-depth Interviews Purpose: –No peer pressure; sensitive issues or complex draft material; hard-to-reach audiences To Be Tested : –Draft brochures/pamphlets, including on sensitive or emotional subjects –Message concepts Number : –Minimum 12 to 24 Pros and Cons Variations
Module 3, Unit 3, Sub-Unit 1, Session 5 19 Central Location Intercept Interviews Purpose: –Short time period and large number of respondents To Be Tested : –Message concepts; print materials (e.g., leaflets, fact sheets, posters); print or radio ads Number: –Minimum Pros and Cons
Focus Group on Tape: 20
Module 3, Unit 3, Sub-Unit 1, Session 5 21 Self-Administered Questionnaires Purpose : –Reactions to materials; large number of respondents To Be Tested : –Print materials such as book- lets, brochures, pamphlets Number : – Pros and Cons
Module 3, Unit 3, Sub-Unit 1, Session 5 22 Draft Research Strategy Outline main features of pretest: –Materials to be tested –Variables to be measured –Method –Sample –Estimated length of each interview –Description Control for presentation Pretesting brochures and radio spots
Module 3, Unit 3, Sub-Unit 1, Session 5 23 Secure interviewers and space for conducting research (if required) and identify, screen, and recruit research participants. –“Facilities” –Moderators Pretest Logistics
Module 3, Unit 3, Sub-Unit 1, Session 5 24 Conduct Testing and Analyze Results Observers Note taking Data analysis Pitfalls of data analysis Report writing
Module 3, Unit 3, Sub-Unit 1, Session 5 25 Revise Concepts and Materials Change Possibly change Keep
Module 3, Unit 3, Sub-Unit 1, Session 5 26 Reporting the data Background: research objectives and description of materials tested. Method: include sample size, sample description, how the respondent were recruited) Findings: included reactions and qoutes Conclusions and Recommendation:
Module 3, Unit 3, Sub-Unit 1, Session 5 27 A Word of Caution About Pretesting No guarantees Controlled environment Depends on execution Biases introduced by participants Statistically imprecise Bad advice?
Module 3, Unit 3, Sub-Unit 1, Session 5 28 Myths and Misconceptions About Pretesting “I don’t have the time or money” “I can tell the difference between good and bad materials, so I don’t need to pretest” “Our artist/producer says that pretesting can’t be used to judge creativity”
pdY 29