Presentation is loading. Please wait.

Presentation is loading. Please wait.

Sigried Caspar European Commission, DG Employment & Social Affairs Moderator.

Similar presentations


Presentation on theme: "Sigried Caspar European Commission, DG Employment & Social Affairs Moderator."— Presentation transcript:

1 Sigried Caspar European Commission, DG Employment & Social Affairs Moderator

2 Céline Mas Occurrence communication research & consulting company, France

3 Workshop A7: how to evaluate your communication activities 2014-2020
December 10, 2013 – Brussels Céline Mas Occurrence est certifiée ISO depuis 2004

4 John Wanamaker Inventor of mass retailing in the United States
"Half the money I spend on advertising is wasted; the trouble is I don’t know which half." John Wanamaker Inventor of mass retailing in the United States Occurrence - Workshop A7 - 10/12/2013

5 €  Key issue How much does it contribute? What does it contribute?
Communications are under pressure: How much does it cost? View it as an investment and not as an expense Provide the resources to prove your effectiveness: 5% to evaluation. "How much does overlooked inefficiency cost?“ : if you can not assess it, you can not improve it! How much does it contribute? What does it contribute? 5 Occurrence - Workshop A7 - 10/12/2013

6 Most of the time, evaluation is occasional or partial.
The missing link in a virtuous circle Communication plan A goals and resources « contract »  with institution or company management Most of the time, evaluation is occasional or partial. It rarely shapes dialogue between the communications team and the other decision makers. Actions A goals and resources « contract »  with company management Assessment Reporting on the achievement of the targets, or on the progress and effectiveness of the implementation Occurrence - Workshop A7 - 10/12/2013

7 Behaving like any other function
Communications must not exclude itself from Quality and Operating Excellence systems Communications is a job and a skill; it must include ongoing improvement procedures The Deming Wheel Ongoing improvement Quality management system 7 Occurrence - Workshop A7 - 10/12/2013

8 The 4 main benefits of an evaluation
1. PROMOTING 2. MANAGING Allocating resources in accordance with performance indicators Identifying the most effective initiatives for achieving your various objectives Circulating results and performances to other departments/teams Achieving investment choices that are based on targets, and not on expenditures on resources 3. SAVING TIME 4. SHARING Prioritizing/Sorting initiatives by order of effectiveness Concentrating your efforts and budget on the most effective initiatives Gathering all the activity and effectiveness data Highlighting best practices AND SPECIFIC TO THE PUBLIC SECTOR Giving evidence of a sound use of public money Reinforce citizen’s trust 8 Occurrence - Workshop A7 - 10/12/2013

9 2 main categories of performance (KPI) indicators
Activity including Resources (What?) e.g. Number of initiatives/tools, type of initiatives/tools, assessment of the content issued (Press releases, and internal communications), Ressources: Who? How much? How long? Effectiveness including the Audiences (For what purpose?) e.g. Memorization, Understanding, Buy-in, Incentive, Transformation, Satisfaction, Improving the brand's image, and satisfying internal customers Audience: How many people attended? How many Likes ? How many readers? … A third, highly practical approach is possible: assessing the satisfaction of (internal) customers Defining performance thresholds for each indicator 9 Occurrence - Workshop A7 - 10/12/2013

10 Process : global picture
Target audience Communications objectives Communications Dashboard communications plan Segmentation Opinions/perceptions to share Do you agree with the following assertion ?  1. Define 2. Count 3. Contact details - "ABC respects environment” - … Surveys, research media analysis, observations Yes Indicators Yes : No :  Don’t know : x 1 [File qualification] -- "ABC respects environment”  x N [Sample survey]   60% Current score: 60 % Targeted score : 90 % Management Allocate Understand Adjust Maintain Etc… Gap between results and objectives OK ! !! Occurrence - Workshop A7 - 10/12/2013

11 Advice Efficiency means producing the desired effects on the desired target audiences Therefore, you need to define the target audiences that you want to reach with which effect, prior to the initiative, and ideally to define the performance threshold As a starting point: KISS (Keep It Simple, Stupid) Keep things simple at the beginning BUT assess them regularly And don't change the assessment system for each evaluation Do not restrict the evaluation to the activity, in order not to limit communications to initiatives and tools Design SMART (Specific, Measurable, Achievable, Realistic, and Timed) tools and goals for each initiative Share the results and the decisions they help to take in order to enhance the value added of evaluation La Communication peut et doit s’évaluer La seule contrainte est de définir des objectifs clairs et partagés en amont L’évaluation nécessite un minimum d’effort : 5% mesure L’évaluation avance sur deux pieds : Les indicateurs d’activité (dont les ressources) moins coûteux à produire Les indicateurs d’efficacité (dont les audiences) plus coûteux à produire L’évaluation permet aux communicants de Crédibiliser / légitimer / valoriser la fonction communication Démontrer son rôle et sa création de valeur pour l’entreprise Piloter de manière plus rationnelle les investissements (arbitrage éclairé) Gagner du temps et du budget en concentrant ses moyens sur les actions les plus efficaces 11 Occurrence - Workshop A7 - 10/12/2013 11

12 Thank you Contact : Céline Mas Partner & Research Director Occurrence
Occurrence - Workshop A7 - 10/12/2013

13 PPT

14

15 Ben Ward Head of Evaluation and Research, Coffey International Development, UK

16 Instructions for use Evaluating communication
This template has been built with custom styles and formats aligned to the new brand identity. All charts, org charts, tables and smart graphics have the custom colour palette and font styles embedded. Choose the style that best suits the presentation of your information. Inserting charts as images and including photography will increase the file size – include lower resolution files. The Coffey orange should only be used in headers, sub headers and bullets – avoid using it in charts, tables and graphs. Secondary colours are only used in charts, tables and graphs – avoid using as headers, sub headers and bullets. Bolder tones are best used for charts – as they appear clearer on screen. Please do not change the primary colour set ups.   Headlines on pages: if they are one line in length please ensure they are presented in black.  If the headline/title splits across two lines – please retain the first line in black and put the second line in the Coffey Orange (select from custom colour palette). Evaluating communication Ben Ward December 9th 2013 IDEV Executive Workshop 4 November 2013

17 Evaluating communication activities
Contents Who we are and what do we do? Challenges faced by public sector communicators Evaluation - hints and tips Discussion Telling the story conference December 2013

18 Evaluating communication activities
Coffey Evaluation and Research Example I&C assignments Founded in 1990, acquired by Coffey International Development in 2009 Team of 30 international and multilingual M&E professionals Work in EU, UK domestic, and international development evaluation markets Evaluation on EU Regional Policy communication (DG REGIO) Ex ante evaluation of campaign on violence against women (DG JUSTICE) Evaluation of ESF communication activities (DG EMPL) Evaluation of DG TRADE’s communication strategy Ex-ante, Mid-Term and Ex-post Telling the story conference December 2013

19 Evaluating communication activities - Challenges
The communication challenge in the public sector is greater than for business communicators (Adapted: Liu and Horsley, 2007) Public sector comms Greater resource constraints Legal constraints Political agendas Extreme media scrutiny Telling the story conference December 2013

20 Evaluating communication activities - Challenges
Telling the story conference December 2013

21 Evaluating communication activities – Hints and tips
Take an evidence based approach to communications design Design of any communications campaign should be driven by good evidence The evidence base: Primary and secondary Quantitative and qualitative Research areas: - Build deep knowledge around what you are communicating – e.g. the problem itself - Defining target audience(s) - Appropriate communication tools and activities Research primary and secondary: doesn’t have to be expensive Research areas: the issue itself – what are the facts? Telling the story conference December 2013

22 Evaluating communication activities – Hints and tips
Evaluation should start at the very beginning Set the objective(s): What are you trying to achieve? Develop an intervention logic and SMART indicators Mapping how you will get to your desired outcomes (use an evidence base to inform this) Workshop - communications team and other relevant stakeholders “He who fails to plan is planning to fail”. Communicators say to the product development team to the policy team…..comms need to be involved from the beginning Evaluators say to communicators ….evaluation needs to be considered from the beginning Telling the story conference December 2013

23 Evaluating communication activities – Hints and tips
Effective evaluation: Focus on measuring the outcomes Measure as much as you can with the resources available Outputs and outtakes both have their place (So what?) However, achieving outcomes ultimately help to justify the use of communications Producing a lot of material, such as leaflets and posters, and getting out the right messages is no good if you fail to get the final result!! “Measurement isn’t about simply monitoring outputs (i.e. the quantity of media coverage achieved), but instead effective measurement is about assessing outcomes (i.e what attitude shifts have occurred) Outputs – just a case of counting the number of outputs which are the result of some form of communications activity. SO WHAT? Out-takes – an out-take is something the public or your target group will take away as a result of a particular piece of communications activity. For example a key message, a perception of understanding of a concept or issue. Outcomes – outcomes are quantifiable changes in attitudes, behaviours or opinions. With a campaign aimed to encourage greater recycling, an out-take could be an individual taking on board the messages, and able to recall hearing or seeing the message if asked. However, an outcome would be that the person has actually changed their recycling habits for the better as a result of the campaign. Fundamental to behaviour change is being able to use insight to understand your audience and the behaviours you are seeking to influence RBM concept! In management and comms! Telling the story conference December 2013

24 Evaluating communication activities – Hints and tips
A few final tips…….. Recognise what you can and you cant do with your evaluation (but also with your comms!) Take a proportionate approach to evaluation Size of intervention Repeatability / Scalability factors Use mixed methods (Quant and Qual) Limited resources (Time and money) : Recognise what you can and you cant do Don’t promise the world! Put yourself in a position to walk the talk! Give yourself a chance! The what but also the how and the why? Quant data showing crime in NY on decline not really understanding why? Telling the story conference December 2013

25 Evaluating communication activities
Thank you Telling the story conference December 2013

26 PPT

27


Download ppt "Sigried Caspar European Commission, DG Employment & Social Affairs Moderator."

Similar presentations


Ads by Google