COM215 Taejin Jung, Ph.D. Week 12: Evaluation

The Purpose of Evaluation The widespread adoption of the MBO (Management-by-Objectives) system by clients and employers of PR specialists The measurement of results against established objectives set during the planning process “The systematic assessment of a program and its results. It is a means for practitioners to offer accountability to clients-and to themselves” During the campaign, constant research must be used to determine the strategies and tactics are working Shared Values Campaign (

Objectives: A Prerequisite for Evaluation It is important to have a clearly established set of “measurable” objectives - PR personnel and management (clients) should agree on the criteria that will be used to evaluate success in attaining objectives - Pre-determined: Don’t wait until the end of the public relations program to determine how it will be evaluated Checklist for the basic evaluation questions - Was the activity or program adequately planned? - Were all primary and secondary audiences reached? - Did the recipients of the message understand it? - Was the desired organizational objective achieved? - What unforeseen circumstances affected the success of the program or activity? - Did the program or activity fall within the budget set for it? - What steps can be taken to improve the success of similar future activities? - How could the program strategy have been more effective?

Current Status of Measurement and Evaluation PR personnel use a mix of evaluation techniques, many borrowed from advertising and marketing, to provide more complete evaluation Sophisticated techniques - Computerized news clip analysis - Survey (quasi-experimental designs) PR budget devoted to measurement and evaluation is now more than 5 percent Evaluation goals for PR programs Advanced (Motivation) Behavioral Change Attitude Change Opinion Change Intermediate (Awareness) Retention Awareness/Comprehension Basic (Exposure) Message distributions Impressions Media Placements

Exposure: Measurement of Production Count how many news releases, feature stories, photos, letters, and the likes are produced in a given period of time - Not meaningful because it emphasizes quantity instead of quality Specify what the PR person should accomplish in obtaining media coverage - Unrealistic and impossible to guarantee Quantify the distribution - Ex) A total of 756 news releases were sent to 819 daily newspapers, 250 weeklies…

Exposure: Measurement of Message Exposure (placement) The compilation of press clippings and radio-television mentions - Most widely practiced form of evaluating PR programs - The publicity effort accomplish the first stage of the adoption process by making people expose to it. - Ex) Burrelle’s monitors nearly 400 local TV stations in 150 cities/ Web tracking systems and firms - Count “placement” and calculate “impressions”

Measurement of Message Exposure Media impressions (← media placement) - How many people may have been exposed to the message - The potential audience reached by a periodical or a broadcast program (circulation x placements) - They don’t disclose how many people actually read or heard the stories and how many absorbed or acted on the information Hits (visits) on the Internet - Cyberspace version of media impressions - The number of people reached via an organization’s WWW or home page

Measurement of Message Exposure Systematic Tracking - New advances in computer software make it possible to track media placements - Computer database analyze the content of media placement a. market penetration b. type of publication c. tone of coverage d. sources quoted - Continuing, regular feedback during a campaign in terms of placements and mention of key messages - To compare its media coverage with the competitors - ROI (Return on Investment): Comparing the number of news release sent with the number actually published and in what kinds of periodicals Total CoverageTotal Total impressions % of positive impressions % of negative impressions Total articles % of positive articles % of negative articles 89,641, % 19.85% 1, % 16.02%

Measurement of Message Exposure Cost per Person - To determine the cost of reaching each member of the audience - Cost-effectiveness (CPM: Cost- per-Thousand) The cost of publicity / the total media impressions - A 30-second commercial during the 2004 Super Bowl cost $2.4 million ($2.4 million/145 million = less than a half cent) Audience Attendance - Counting attendance at event is a relatively simple way of evaluating the effectiveness of pre-event publicity - Better objective evaluation is needed

Measurement of Audience Awareness To determine whether the audience actually became aware of the message and understood it - The second level of PR evaluations - Whether they paid attention (understood, have retained) the messages in any shape or form “Survey” is needed to answer such questions - Target audience be asked about the message and what they remember - Public awareness of sponsoring organization - Day-after recall

Measurement of Audience Attitudes Quasi-experimental study - A measurement of audience attitudes and opinions before, during, and after a public relations campaign - Show the percentage difference in attitudes and opinions as a result of publicity - Considering intervening variables (e.g., coefficient and R 2 ) “The only way to determine if communications are making an impact is by pre- and posttest research”

Measurement of Audience Action The objective of “Greenpeace” is not to get editorials written in favor of whales, but to motivate the public… a. to write elected officials b. to send donations for its preservation efforts c. to get protective legislation passed Public relations efforts ultimately are evaluated on how they help an organization achieve its objectives

Example: The Truth Campaign Florida "truth" Campaign