Sigried Caspar European Commission, DG Employment & Social Affairs Moderator

Workshop A7: how to evaluate your communication activities December 10, 2013 – Brussels Céline Mas Occurrence est certifiée ISO 9001 depuis 2004

"Half the money I spend on advertising is wasted; the trouble is I don’t know which half." John Wanamaker Inventor of mass retailing in the United States Occurrence - Workshop A7 - 10/12/2013

Key issue >Communications are under pressure: -How much does it cost? >View it as an investment and not as an expense > Provide the resources to prove your effectiveness: 5% to evaluation. >"How much does overlooked inefficiency cost?“ : if you can not assess it, you can not improve it! € How much does it contribute?  What does it contribute? 4 Occurrence - Workshop A7 - 10/12/2013

Communication plan Actions A goals and resources « contract » with company management Assessment Most of the time, evaluation is occasional or partial. It rarely shapes dialogue between the communications team and the other decision makers. The missing link in a virtuous circle A goals and resources « contract » with institution or company management Reporting on the achievement of the targets, or on the progress and effectiveness of the implementation Occurrence - Workshop A7 - 10/12/2013

Ongoing improvement Behaving like any other function >Communications must not exclude itself from Quality and Operating Excellence systems >Communications is a job and a skill; it must include ongoing improvement procedures The Deming Wheel  Quality management system 6 Occurrence - Workshop A7 - 10/12/2013

The 4 main benefits of an evaluation 1. PROMOTING Circulating results and performances to other departments/teams Achieving investment choices that are based on targets, and not on expenditures on resources 2. MANAGING Allocating resources in accordance with performance indicators Identifying the most effective initiatives for achieving your various objectives 3. SAVING TIME Prioritizing/Sorting initiatives by order of effectiveness Concentrating your efforts and budget on the most effective initiatives 4. SHARING Gathering all the activity and effectiveness data Highlighting best practices 7 AND SPECIFIC TO THE PUBLIC SECTOR Giving evidence of a sound use of public money Reinforce citizen’s trust Occurrence - Workshop A7 - 10/12/2013

2 main categories of performance (KPI) indicators >A third, highly practical approach is possible: assessing the satisfaction of (internal) customers >Defining performance thresholds for each indicator e.g. Number of initiatives/tools, type of initiatives/tools, assessment of the content issued (Press releases, and internal communications), Ressources: Who? How much? How long? Activity including Resources (What?) e.g. Memorization, Understanding, Buy-in, Incentive, Transformation, Satisfaction, Improving the brand's image, and satisfying internal customers Audience: How many people attended? How many Likes ? How many readers? … Effectivenes s including the Audiences (For what purpose?) 8 Occurrence - Workshop A7 - 10/12/2013

Process : global picture    1. Define 2. Count 3. Contact details Yes - "ABC respects environment” - … 60% Indicators Target audienceCommunications objectives Segmentation  x N [Sample survey] Opinions/perceptions to share - - "ABC respects environment” Yes :  No :  Don’t know :   Targeted score : 90 % Current score: 60 %    x 1 [File qualification] Gap between results and objectives OK!!! Surveys, research media analysis, observations Do you agree with the following assertion ?  Management Allocate Understand Adjust Maintain Etc… Communications Dashboard communications plan Occurrence - Workshop A7 - 10/12/2013

Advice >Efficiency means producing the desired effects on the desired target audiences >Therefore, you need to define the target audiences that you want to reach with which effect, prior to the initiative, and ideally to define the performance threshold >As a starting point: KISS (Keep It Simple, Stupid) -Keep things simple at the beginning >BUT assess them regularly >And don't change the assessment system for each evaluation >Do not restrict the evaluation to the activity, in order not to limit communications to initiatives and tools >Design SMART (Specific, Measurable, Achievable, Realistic, and Timed) tools and goals for each initiative >Share the results and the decisions they help to take in order to enhance the value added of evaluation 10 Occurrence - Workshop A7 - 10/12/2013

Thank you >Contact : Céline Mas >Partner & Research Director Occurrence Occurrence - Workshop A7 - 10/12/2013

Brussels, 9-10 December PPT

Brussels, 9-10 December

Workshop B7: How to evaluate communication activities UK Government perspective December 9, 2013 – Brussels Paul Njoku Cabinet Office, UK

Brussels, 9-10 December Agenda A Why evaluate (context)? Barriers to evaluation B How to go about it - PROOF guiding principles 4 stage evaluation process - The Big IDIA Main performance categories - KPI indicators C Top tips

16 Why evaluate? A1

Brussels, 9-10 December Context Austere times  Need to make every € count  View as an investment not an expense Media landscape & consumption patters  Evidence of what works and what does not  Optimise use of scarce resources The role of communications  How it supports achievement of policy outcomes  Business planning & activity prioritisation A2

COMMUNICATION OBJECTIVE Role that communication will play in achieving departmental objective DEPARTMENTAL OBJECTIVE To address specific issue Sub-objective Communication plan Specific activities, channels, target audiences Overall communication strategy - how communication will achieve its objective Policy development, policy delivery, reputation management Strategic alignment A3

19 Barriers – stopping it happening Insufficient time / resource / budget Lack of SMART or unrealistic policy objectives / targets Difficulty accessing the right data / tools Culture & entrenched behaviours Gaps in evaluation standards & capability A4

Brussels, 9-10 December How to go about it Five key principles Pragmatic – best available within budget, not best ever Realistic – prove what you can, acknowledge what you can’t Open – record and share as much as possible Objective – be honest & constructive about results, to inform future learning Fully integrated – integral part of planning & delivery, not an add-on P R O O F B1

Brussels, 9-10 December Evaluation stages –The Big IDIA B2 Identify The scope of your project Develop Your evaluation plan Implement Gather data to measure performance Analyse & report Performance against plan Task 1: Define what you need to evaluate by asking: What activity am I evaluating? What do I know & what factors could affect the outcome? What is my evaluation expected to achieve? Output: Summary of your proposed evaluation approach Task 1: Define what you need to evaluate by asking: What activity am I evaluating? What do I know & what factors could affect the outcome? What is my evaluation expected to achieve? Output: Summary of your proposed evaluation approach Task 2: Define how you’ll measure success: Set SMART objectives & defining your target audience Map out how activity will work Set performance metrics (KPIs) & agreeing targets Output: Draft evaluation plan Task 2: Define how you’ll measure success: Set SMART objectives & defining your target audience Map out how activity will work Set performance metrics (KPIs) & agreeing targets Output: Draft evaluation plan Task 3: Identify and gather evaluation data: Make most of existing data Gather additional data (research, feedback & proxies) Review data gaps (more budget ?) manage expectations Output: Completed evaluation plan Task 3: Identify and gather evaluation data: Make most of existing data Gather additional data (research, feedback & proxies) Review data gaps (more budget ?) manage expectations Output: Completed evaluation plan Task 4: Assess the success of your activity: Analyse effectiveness & provide insights for future Demonstrating efficiency and value for money Demonstrating role of communications in supporting the achievement of policy objective (outcome) Output: Final evaluation report Task 4: Assess the success of your activity: Analyse effectiveness & provide insights for future Demonstrating efficiency and value for money Demonstrating role of communications in supporting the achievement of policy objective (outcome) Output: Final evaluation report

Brussels, 9-10 December Key performance indicator categories B3 ActivityEffectivenessResult

Brussels, 9-10 December 1.Strategic alignment – Ensure activity objectives are SMART and supports policy delivery. 2.Business impact – Aim to measure true business impact (outcome) rather than for example, the perceived quality of specific channels. 3.PROOF the big IDIA – Try to adopt the suggested guiding principles and follow the big IDIA stages. 4.Continuous improvement – Ensure results drive appropriate actions and any learnings inform future activities. 5.Best practice – Be objective, share results and make evaluation an integral part of your communications planning process. Top tips

24 Thank you! Contact: Paul Njoku Web link to guide :

Brussels, 9-10 December Appendix

Products Customer Service Investments Employment Branding Public Relations Marketing Social Responsibility MEDIA (Traditional, Social) Topic Experts, Leaders, Friends/Family Perceptions & expectations Supportive Behaviour Results What your department Says/Does Direct Experience What Others Say How reputation is created

The attributes and dimensions have different meanings and importance for different stakeholders. Beneath the 7 dimensions, 32 attributes underpin the individual dimension themes. Different stakeholder groups typically have unique attributes that are found more important than others (reputation drivers). Reputation Attributes Reputation Dimensions The seven dimensions specify at a more operational level, which aspects are most important for stakeholders’ perceptions and expectations – i.e. what’s driving a company’s reputation Reputation Strength A measure of the emotional connection. Reputation has a positive/negative impact on support. An increase in reputation = an increase in support. Support (such as buying products and services, saying something positive, giving the benefit of doubt in times of crisis (etc.) leads to increased business results Supportive Behaviour Reputation drivers & dimensions

Brussels, 9-10 December PPT

Brussels, 9-10 December