1. PUBLIC RELATIONS PROCESS
EVALUATION OF PR EFFECTIVENESS

2. Today’s Learning Objectives
Understanding the need for evaluation
Learning the ways in which evaluation is conducted
Knowing closed system and open system evaluation techniques
We will learn the different ways to evaluate the results of a campaign and how to measure those results in order to prove a campaign’s value.
Why learn about evaluation?

3. Evaluation Is Used To Answer…
What have we accomplished?
What remains to be done?
What benchmarks can help us next time?

4. Evaluation Proves PR Value
Competent evaluation demonstrates the value of public relations to management.
That means your value, at least to the company.
Merging research and evaluation…

5. Research Results Assist in Evaluation
Communication To
Target Publics
Measurement Techniques
Objectives and
Strategies
Response
Results

6. Evaluating Results Evaluating research can tell:
How many audience members received messages.
How many changed an attitude or opinion.
How many changed behavior.
These could equally be analyzed using various web bases media tools like Neilsen, MRI, HookMedia

7. Using Research Results
The public relations practice is cyclical in nature. Research data from the first cycle form the basis for planning subsequent efforts. Changes may be made in:
Program strategies and/or tactics
Targeted audiences
Media and messages
The organization’s products, services, policies, or procedures

8. The National Youth Anti-Drug Media Campaign
Click on the image to see how the National Youth Anti-Drug Campaign conducted its evaluation or this link

9. Evaluating Implementation
Is implementation being conducted as planned?
After variations are analyzed and explained …
Should the plan be modified or …
Can the discrepancies otherwise be corrected?
In-progress monitoring is similarly handled…

10. Evaluating Program Progress
Determine program effectiveness.
Identify unanticipated results.
Assess their impact on overall outcome.
Make adjustments as necessary.
Outcome evaluation is the final need…

11. Evaluating Outcomes Compare outcomes with objectives.
Interpret results.
Report results to management.
Make recommendations concerning subsequent programs or program cycles.
Measuring HIV campaign in Zaire…

12. Four Measurement Strategies …
The extent audiences are covered
Levels of audience response
The impact of the campaign as a whole
The extent of environmental influences
We’ll examine them individually …

13. Audience Coverage Is Measured By …
Maintaining complete records of messages sent and media usage to determine the extent that…
Targeted audiences are exposed to messages.
Non-targeted audiences receive messages.
Now we’ll look at audience response …

14. Messages Can Be Measured By …
Pre-testing messages using…
The Flesch Formula
The Gunning Fog Index
The Dale-Chall Formula
The Cloze Procedure
Post-testing messages by using survey techniques to determine change in attitudes or behaviors
Now we’ll look at environmental factors …

15. External Influences on Campaigns
External factors can influence campaign outcomes by…
Magnifying success or failure
Measurement error can compound problems …

16. Guidelines for Measurement
Trust me, men. I had 500 leaflets passed out today. There will be no strike tomorrow.
Volume of output does not determine results.
The impact of news releases does not increase with their number.
Estimate is not measurement.
Experience and intuition are not sufficient to gauge effectiveness.

17. Sampling Selection Samples should be carefully selected.
Wrong decisions often result from volunteered or unsystematically collected comments.
More common errors …

18. Effort = Knowledge The Effort Error
The amount of effort expended does not equal understanding of the audience’s response.

19. Knowledge = Favorable Attitude
The Knowledge Error
Knowledge = Favorable Attitude
What audiences know about a topic does not necessarily produce favorable reactions.

20. Attitude = Behavior The Attitude Error
Favorable perceptions may halt opposition but not produce support.

21. PR Evaluation Traditional Evaluation Output measurement
Survey of distribution system
Number of potential audience impressions

22. Weaknesses of Traditional Approach
No Way to assess quality
Cost Effectiveness isn’t measured
No opportunity for improvement

23. Measurement should include:
Behavior
Attitudes
Knowledge
Awareness
Interest/Expectation

24. Levels & Criteria for Evaluation
Preparation
Adequacy of background information
Appropriateness of message content & organization
Quality of message presentations

25. Implementation Number of messages sent—distribution
Number of messages appearing in media
Number of people receiving message
Number of people attending to the message

26. Impact Number who learn message content Number who change opinions
Number who change attitudes
Number who behave in desired fashion
Goal achieved/problem solved
Social and Cultural change

27. Evaluation Techniques May Include …
Impact analysis
Audience coverage
Audience response
Campaign impact
Environmental assessment

28. Measurement Techniques
Closed and open systems of evaluation help determine the effectiveness of programs

29. Closed System Evaluation …
The closed system is limited to planned messages and events and their effects on publics.

30. The Closed System Has Two Tests
Pre-test
of Message
and
Media
Post-test
of Message
and
Media
Campaign
Time

31. Pre-test/Post-test Design Examines …
Amounts and costs of public relations tools used
Channels used to convey messages
The extent audiences heeded messages
Whether audiences received messages
Readability of messages transmitted
The process has limitations…

32. Weaknesses In Closed System Method
Successfully transmitting messages does not equal campaign success.
Potential for success is influenced by external as well as internal factors.
Pre-test/post-test techniques measure only the extent attitudes have changed.
Now we’ll look at the open system method…

33. Open System Evaluation …
Is most needed in connection with continuing or long-range programs, which require feedback throughout the process.
Attempts to account for factors outside practitioner’s ability to control.
Emphasizes organizational and external influences on the process.
Turning to techniques…

34. Open Systems Evaluation Has Two Tools
Environmental monitoring and social audits are used as tracking tools.
Social Audits
Environmental Monitoring
Slide 34 of 50

35. Environmental Monitoring
Environmental monitoring can be quite extensive and expensive.
There are companies that provide environmental monitoring--another item that can be outsourced.

36. Open Systems Evaluation Can …
Assess the program’s impact on the environment, and vice versa.

37. Open Systems Evaluation Can …
Track both internal and external influences.
An open system approach in practice …

38. AT&T’s Open System Includes:
Administrative processes
Employee publications
Media relations

39. Another Way to Classify Evaluation
Summative:
Evaluation completed after the campaign or program is over
“how did the campaign do?”
Formative:
Ongoing evaluation of in-progress programs or campaigns
“how are we doing?”

40. Five Publication Measures
Readability
Gunning readability
Understanding
Average question response
Recall and comprehension percentage
Reliability
Awareness
Average recall
Percentages of issues delivered within 38 hours
Distribution

41. Publication Measures Explained …
Distribution – Percentages of issues delivered within 38 hours
Awareness – Average recall of data
Reliability – Average response to questions about comprehension, balance and information content
Understanding – Percentage recalling and comprehending a story
Readability as measured with the Gunning scale

42. In Summary … Evaluation is a new beginning rather than a final step.
The process provides new direction for on-going programs.
Evaluation establishes the value of public relations programs.