Measurement & Evaluation in PR: 2007 to 2014 Professor Tom Watson, Bournemouth University

Agenda From 2nd to 3rd edition (2007 to 2014) Principles Research priorities Barcelona Principles ROI debate Communication Performance Management

Evaluating Public Relations, 3rd edition Published in June; Available in Australia in August London: Kogan Page ISBN: 9780749468897

Aircraft instruments

Defining evaluation “…any and all research designed to determine the relative effectiveness of a public relations program, strategy, or activity, by measuring the outputs and/or outcomes of that PR program against a predetermined set of objectives” (Lindenmann) Management-by-Objectives (MBO) framework

Measurement concepts Input: expenditures on communication-related services; use of financial and human resources Output: Presentation and dissemination Out-take: Awareness; processing of messages Outcome: The desired result Arise from Lindenmann’s typology; Widely applied

Outflow -1 Outflow is the economic impact that results from influence on stakeholders exerted by corporate/organizational communications “It becomes visible what communications have actually contributed to achieving the financial and strategic goals of the organization” (DPRG 2011:14)

Outflow -2 Corporate communication can add value by supporting service provision processes of other corporate functions or by creating intangible assets Both contributions depend on specific organisational goals and strategy Performance indicators include business-related metrics such as sales, innovations, productivity, etc. or intangible capital such as monetary brand value or reputational capital

‘Value’ as a PR research priority: Delphi study (Watson (2008) PR’s role in contributing to strategic decision-making, strategy development and realization and organizational functioning The value that PR creates for organizations through building social capital; managing key relationships and realizing organizational advantage The measurement and evaluation of public relations both offline and online

Barcelona Principles Seven principles of measurement Set by AMEC – adopted worldwide by PR professional bodies Changed thinking away from AVE

Barcelona Principles

ROI – misapplied language PR sought to use business language Return on Investment widely expressed, inappropriately Related to value created over time by investment in capital equipment Not to immediate results of a PR activity Many PR actions can’t express a financial result (Gov’t; NfP) “loose and fuzzy” (Watson & Zerfass, 2011, 2012)

Jim Grunig speaks “I talk more about the value of public relations than about ROI. As I said, you can explain the value of relationships; but you really can’t measure a financial return to compare with the money invested in it. I tend to use the term ROI because PR people want to hear it used. I will now cease and desist from using it” (Likely & Watson, 2013: 153)

‘Communication Controlling’ Controlling = auditing, performance management Central European approach to link corporate objectives with communication objectives Each level can be monitored: Input > Output (2 stages) > Outcome (2 stages) > Outflow Matrix of measurements, appropriate to each level Used by BASF, Siemens, Henkel, Commerzbank, etc

Levels of Impact and Evaluation of Communications Outflow Outflow Outcome Value Creation Impact on Strategic and/or Financial Targets (Value Chain) Impact on Tangible and/or Intangible Assets (Capital Accumulation) Sales No. of Project Agreements Cost Reduction Reputation Capital Brand Value Employee Performance … Direct Outcome Perception Utilization Knowledge Awareness Unique Visitors Session Length Reader per Issue Recall Recognition … Indirect Outcome Opinion Attitudes Emotion Behavioral Disposition Behavior Reputation Index Brand Image Strategic Awareness of Employees Purchase Intention Leads Innovative Ideas Project Participation … Levels of Impact Output Internal Output Process Efficiency Quality of Works Product Budget Compliance Throughput Times No. of Shortcomings Readability/Fogg-Index Satisfaction of Internal Clients … External Output Media Coverage Content Clippings Visits Downloads Impact Ratio Share of Voice … Input Ressources Employee Assignment Financial Expenses Personnel Costs Outsourcing Costs … Measure-ment Range Indica-tors (e.g.) Let’s take a first look at the Impact model that connects closely with Communication Controlling. HAND OUT COPIES OF MODEL You’ll see it is not a single , simplistic model but has are four stages – Input, Output, Outcome and Outflow. DISCUSS TEXT ON SCREEN Next we’ll look at the definitions ... MEA-SURED OBJECT ORGANISATION MEDIA/CHANNELS STAKEHOLDERS ORGANISATION Initiation of Communication Processes Communication Processes Results of Communication Processes Low Impact on Value Creation Strong Influence of Communications Management High Impact on Value Creation Weak Influence of Communications Management 15

Implementation Communication Controlling has four phases Analysis Conception Operationalization Steering / Reporting The development and implementation of a communication controlling system follows a process flow consisting of four phases: analysis, conception, operationalization and steering/reporting

Implementation steps of Communication Controlling Analysis Conception Operationalization Steering/Reporting 1 2 3 4 5 6 Analysis Communi- cation Objectives Steering and Reporting Corporate Goals Metrics & Indicators Data Collection Key Questions Key Questions Key Questions Key Questions Key Questions Key Questions Which data already exist in the company? Which data can be obtained with reasonable cost and time expenditures? What is the current status of corporate communications? Which are the goals of the company for the current planning period? Are these goals clearly defined and prioritized? What are the strategic goals and what can the company do to achieve those? The achievement of which of these goals requires the support of the communications function? What do communications need to contribute in terms of stakeholder impact and economic outflow? Which communications objectives are aimed at which stakeholder groups? Which new potential assets need to be developed to address these objectives? What do we have to do for that and what kind of resources will we have to invest? How shall the success of communications be measured? Which indicators and KPIs can be defined for these metrics? Which data will be collected at which point in time? Which evaluation methods and instruments are to be applied? Who is to be informed at which point in time or event? In which form and way are the findings to be reported? What kind of recommendations are expected to support decision-making? The analysis phase comprises the collection, systematization and evaluation of survey methods, studies and data sources. To avoid duplication, it is helpful to start with an audit of the existing monitoring, evaluation and steering instruments. Most companies already measure a variety of communication impacts. To find these data, evaluate the applied methods and integrate the useful material in the future system not only helps saving cost and time expenditures but winning the support from the research owners. The aim of the conception phase to develop a practical and company-specific framework for communication controlling. This can be summed up in a three-step process. The first task is to identify those corporate goals that can only be achieved through the support of communication. These goals are then translated into communication objectives related to those stakeholder groups whose behaviour is critical for enabling the corporate strategy. The last step is to develop and agree on (key) performance and result indicators for monitoring progress and evaluating target achievement. A detailed documentation of the management system helps communication executives to handle its complexity and adapt it to future challenges.

Making it work Has organization defined its corporate objectives? Are “scorecards” used? Is measurement and evaluation part of corporate culture? How is the communications function organized? “Strong and visible” top management commitment is needed Close cooperation with management accountants Text for second half previous slide During the operationalization phase, data collection and analysis processes are established. In cooperation with the communication managers who will later be held accountable, roles and responsibilities are assigned to all functions involved. Based on the defined indicators, actual values are confirmed and target values agreed. These include documented clarification as to which data are to be collected, who will be in charge, timescale, methods to be applied, and the instruments used. The final phase deals with the steering and reporting of corporate communication. Communication scorecards are a tried-and-tested steering tool, best suited for integrating the key result and performance indicators of the various areas and levels of corporate communication. Communication executives and management accountants also need to agree how the relevant indicators and developments within corporate communication are to be processed and regularly reported. A trial period will provide first learnings regarding the usability of the management system and ideas for further optimization. Experiences made during the implementation will enable communication professionals to adapt their steering and evaluation methods, thus initiating the process of on-going improvement.

Summary Multiple measurement needed for evaluation Value can’t always be measured but can be evaluated Communication Performance Management links PR with objectives Read all about it!