Chapter Eight: Evaluation

Overview The Purpose of evaluation The Purpose of evaluation Objectives Objectives Status of evaluation and measurement Status of evaluation and measurement Measurement Measurement  Production  Exposure  Awareness  Attitudes  Action  Supplemental activities

The Purpose of Evaluation The fourth step of the public relations process is evaluation – the measurement of results against established objectives (194).

Continued --- _ Evaluation is the measurement of results against objectives, this can enhance future performance and also establish whether the goals of management by objective have been met. _ Evaluation is well described as ( the systematic assessment of a program and it ’ s result, it is a mean for practitioners to offer accountability to Clients and to themselves.

Objectives: A Prerequisite for Evaluation Before any public relations program can be properly evaluated, it ’ s important to have a clearly established set of measurable objectives, these should be part of the program plan but first some points need to be reviewed. Before any public relations program can be properly evaluated, it ’ s important to have a clearly established set of measurable objectives, these should be part of the program plan but first some points need to be reviewed. 1. Public relations personnel and management should agree on the criteria that will be used to evaluate success in attaining objectives.

Continued … 2. Don ’ t wait until the end of public relations programs to determine how it will be evaluated. ( Evaluating impact starts in the planning stage, you break down the problem into measurable goals and objectives then after implementing the program, you measure the results against goals ) _ If the objective is informational, measurement techniques must show how successfully information was communicated to target audiences.

Continued … _ Motivational objectives are more difficult to accomplish, if the objective is to increase sales or market share, it ’ s important to show that public relations efforts caused the increase rather than advertising or other marketing strategies, But if the objective is to change attitude or opinion, research should be done Before and after public relations activity to measure the percentage of Change

Objectives: A Prerequisite for Evaluation Evaluation Questions Adequately planned? Adequately planned? Message understood? Message understood? How more effective? How more effective? All primary and secondary audiences reached? All primary and secondary audiences reached? Organizational objective achieved? Organizational objective achieved? Unforeseen circumstances? Unforeseen circumstances? Within budget? Within budget? Steps to improve? Steps to improve?

Current Status of Measurement and Evaluation _ The proportion of public relations Budgets devoted to measurement and evaluation grew over the 1990s to about 5 percent. On the most basic level, practitioners can measure message distribution and media placements. The second level would be measurement of audience awareness, comprehension and Retention. The most advanced level is the measurement of change in attitudes, opinions and behaviors

Current Status of Measurement and Evaluation Sophisticated Techniques Computerization Computerization Clips as evaluation supplanted by many methods Clips as evaluation supplanted by many methods Management demands more accountability to the bottom line Management demands more accountability to the bottom line

Measurement: Production _ One elementary form of evaluation is simply to count how many news releases, feature stories, photos, letters and the like are produced in a given period of time _ This kind of evaluation is supposed to give management and idea of staff ’ s productivity and output. _ P.R professionals, don ’ t believe that this evaluation is very meaningful, because it emphasizes quantity instead of quality. It may be more cost effective to write fewer news releases and spend more time on the few that are newsworthy.

Measurement: Production _ Another side of production approach is to specify what the public relations person should accomplish in obtaining media coverage. _such evaluation Criteria not only are unrealistic, they are also impossible to Guarantee because public Gatekeepers not the public relation make such decisions. Management may argue that such placement goals provide incentives to public relation staff and are tangible criteria in employee performance Evaluation

Measurement: Exposure The most widely practiced form of evaluating public relation programs is the compilation of press clippings and radio – television mentions. Public relation firms and company departments working primarily on a local basis often have a staff member scan and clip the area newspaper. Large companies with national and international outreach hire clipping services to scan large numbers of publications, it ’ s also possible to have electronic clipping services monitor.

Measurement: Exposure Measurement of exposure relies on:  Media impressions  Hits on the internet  Advertising equivalency  Systematic tracking  Requests for 800 numbers  Cost per person  Audience attendance

Media Impressions _ In addition to the number of media placements, P.R departments and firms report how many people have been exposed to the message, these numbers are Described as Media impression, the potential audience reached by a periodical or a broadcast program. _ National news story can generate millions of impressions by simple multiplication of each placement by the audience of each medium

Continued _ Media impressions are commonly used in advertising to document, this figure give a rough estimate of how many people are exposed to a message, they disclose how many people read or heard the stories and more important how many absorbed or acted on the information.

Hits on the Internet _ A cyberspace version of media impression is the number of people reached Via an organization ’ s world wide web site or home page. Each instance of a person accessing a site is called a Hit or a Visit.

Advertising Equivalency _ Another approach is to calculate the value of message exposure, this is done by converting stories in the regular news columns or on the air into equivalent advertising Costs. _ some practitioners even take the approach of calculating the cost of advertising for the same amount of space and then multiplying that total three to six times to reflect a number of research studies that show that a new story has Greater Credibility than an advertisement.

Systematic Tracking As noted earlier, message exposure traditionally has been measured By Sheer Bulk, new advances in computer software and database, now make it possible to track media placements in a more sophisticated way. _ Computer databases can be to analyze the content of media placements by such variable as market penetration …

Systematic Tracking _ The value of systematic tracking is manifested in several ways one is continuing regular feedback during a campaign to determine if an organization ’ s publicity efforts are paying off in terms of placements and mention of key messages, tracking coverage and comparing it over a period of time is called Benchmarking.

Requests and 800 numbers _ Another measure of media exposure is to compile the number of requests for more information, a story in a newspaper or an appearance of a company spokesperson on a broadcast often provides information as to where people can get more information about a subject. _ Requests for materials also show the effectiveness of a public relation program

Cost Per Person _ A way to evaluate to the message is to determine the cost of reaching each number of the audience, the technique is commonly used in advertising to place costs in perspective. _ Cost effectiveness, as this technique is known, also is used in public relations. Cost-per-thousand (CPM) is calculated by taking the cost of the publicity program and dividing it by the total media impressions (discussed earlier). Cost-per-thousand (CPM) is calculated by taking the cost of the publicity program and dividing it by the total media impressions (discussed earlier).

Audience Attendance _ Counting attendance at events is a relatively simple way of evaluating the effectiveness of pre-event publicity. _ Poor attendance at a meeting or event can indicate inadequate publicity and promotion. _ Another major cause is lack of public interest, even when people are aware that a meeting or event is taking place. _ Low attendance usually results in considerable finger-pointing, thus an objective evaluation of exactly what happened – or didn ’ t happen – is a good policy.

Measurement of Audience Awareness _ Techniques of measuring audience exposure and accurate dissemination have been discussed. _ A higher level of evaluation is to determine whether the audience actually became aware of the message and understood it.

Continued … _ The tools of survey research are needed to answer such questions. _ Members of the target audience must be asked about the message and what they remember about it. _ Public awareness of what organization sponsors an event also is important. _ Another way of measuring audience awareness and comprehension is day-after recall. _Under this method, participants are asked to view a specific TV programme, the next day they are interviewed to learn which message they remembered.

Measurement: Attitudes _ Closely related to audience awareness and understanding of a message are changes in audiences perception and attitudes. A major technique determine such changes is the baseline study.

Measurement: Attitudes Measurements of audience attitudes can be accomplished through: Baseline Studies – a measurement of attitudes before, after, and during a campaign. It ’ s also called benchmark studies

Measurement: Audience Action Measurement of audience action is determined by a campaign objectives … … where they met?

Measurement: Supplemental Activities Measurement of supplemental activities includes: –Communication audit –Pilot tests and split messages –Meeting and event attendance –Newsletter readership –Content analysis –Readership interest surveys –Article recall –Advisory boards

Communication audits _ communication activity of an organization should be evaluated at least once a year to make sure that every primary and secondary public is receiving appropriate messages _ A communication audit could include the following :

Continued Analysis of all the communication activities 2. Informal interviews with rank and file employees 3. Informal interview with community leaders.

Pilot tests and split messages _ A variation of Pre-testing is the pilot test, before going national with the P.R messages, companies often test the message and copy points in selected cities to learn how the media accept the message and how the public reacts. _ The split message approach is common in direct mail campaigns

Pilot tests and split messages _ With perception analyzer systems, some times called theater system, selected subjects are asked to view speeches. _ The think- aloud method can be used to asses with content in a similar fashion to the perception analyzers.

Meeting and event attendance _ P.R practitioners use a number of information methods to evaluate success of the meeting, but they also employ more systematic methods. The most common technique is an evaluation sheet that participants fill out at the end of the meeting.

Newsletter readership _ Editors of newsletters should evaluate readership annually, such an evaluation can help ascertain 1. reader perception 2.the degree to which stories balanced 3. the kinds of stories 4. topics that should be covered 5. the credibility of the publication 6. the extent to which the newsletters is meeting organizational objectives

Continued … _ A newsletters can be evaluated in a number of ways : 1. Content analysis 2. readership interest surveys 3. Article recall 4. Advisory board

Content analysis _ Based on representative samples of past issues _ this systematic analysis will show the percentage of the publication is devoted to each category _ stories may be categorized under general headings such as New product developments, Job related information

readership interest surveys _ The purpose of these surveys is to get feedback about the types of stories employees are most interested in reading, the most common survey method is simply to provide along list of generic story to picks and have employees rate each as 1.important one 2.to some what important 3.or not important

Article recall _ The best kind of readership survey occurs when trained interviews ask a sampling of employees what they have read in the latest issue of the publication.

Advisory board _ Periodic feedback and evaluation can be provided by organizing an employee advisory board meet several times a year to discuss the direction of the publication this is a useful technique because it expands the editor feedback net work and clear comments that employees might be hesitant to tell the editor face to face.

Thank you Thank you