1. Monitoring and Evaluation in Communication Management

2. Terms used:
Monitoring: a systematic and continuous collection, analysis and using of information for the purpose of management control and decision making.
Evaluation: This is the assessment of whether the objectives of communication have been achieved and what impact has been made.
Decisions are made on:
Continuation of the communication;
Draw up lessons that can be applied
Review implementation strategies.

3. Criteria that guide M&E practices in communication
Relevance
Efficiency
Effectiveness
Outcome
Sustainability
Impact

4. Value of M&E to Communication Management
Facilitate learning and to inform decision making within the organization.
Timely information
Improving communication management
Improves the relevance, outcome & overall contribution of the communication Management to the beneficiaries.
Strengthens the capacity of the implementing agencies
Meeting donor requirements and ascertaining the capacity of the implementing organization. Enhances trust.

5. Steps to Designing, Building and Sustaining a Results-Based M&E System
Conducting a Readiness Assessment
Agreeing on Outcomes to Monitor and Evaluate
Selecting Key Indicators to Monitor Outcomes
Baseline Data on Indicators—Where Are We Today?
Planning for Improvement — Selecting Results Targets
Monitoring for Results
The Role of Evaluations
Reporting Your Findings
Using Your Findings
Sustaining the M&E System Within Your Organization 1 2 3 4 5 6 7 9 8 10

6. Characteristics of Quality Evaluations
Feedback/ dissemination
Technical adequacy
Value for money
Stakeholder involvement
Usefulness
Impartiality

7. Communication Areas for Monitoring
Costs
Activities
Time/schedule
Feedback
Channel
Actors

8. Who does the Evaluation?
Built in self-evaluation: conducted by those directly involved in the implementation;
Participatory evaluation: staff and external evaluators consult with the beneficiaries;
External evaluation: carried out by individuals outside the implementing team.

9. The Principles of M&E in Communication
The Core Principles here are;
Primary stakeholders are active participants – not just sources of information
Building capacity of local people to analyze, reflect and take action 
Joint learning of stakeholders at various levels 
Catalyzes commitment to taking corrective actions

10. Using M&E Indicators in Communication
What is an M&E indicator?
Measurable parameter that provides an overall summary of a situation
Usually expressed as a rate/ratio –
Defines data to be collected – if not used for an indicator, then don’t collect it!

11. Why Indicators?
Ability to capture a situation in a single or few objective measures
Enables overview of communication outcome
Identifies outliers where management attention is needed
Provides assurance of adequate performance
Enables comprehensive oversight of entire communication spectrum

12. Characteristics of A Good Indicator
Validity: Measures in fact what it intends to measure conceptually;measures what it is intended to measure e.g. number of individuals vs. number of visits/services
Reproducible - (all observers get the same result)-a reliable measure, uses consistent methods of measuring.
Reliability: Minimizes measurement error
Precision: Is operationally defined in clear terms

13. Characteristics Cont..
Sensitive -(changes noticeably with changes in the situation being measure
Timeliness: Provides a measurement at time intervals relevant and appropriate in terms of program goals and activities
Comparability: Generates corresponding or parallel values across different population groups and program/project approaches

14. Examples of Indicators at Different Levels
Goal (Impacts)
Improved academic status of Universities with better teaching staff
Outcomes
Percent reduction in student failure
Proportion of youths attending school
Proportion of literate adolescent orphans
Outputs
No. of families with educated children
No. of families that appointed standby guardians
Proportion of school children with textbooks
Proportion school children receiving meals at school
Inputs
No. of families taught in will-writing
No. of text books distributed
No. of meals provided at school

15. Use of Targets in M&E for Communication
Targets Are the quantifiable levels of the indicators that an organization wants to achieve at a given point in time Target setting
Target Performance
Desired level of performance to be reached within a specific time + =
Baseline Indicator Level
Desired Level of Improvement
Assumes a finite and expected level of inputs, activities, and outputs

16. Factors to Consider When Selecting Indicator Targets
Clear understanding of baseline starting point
Funding and level of personnel resources expected throughout the target period
Amount of outside resources expected to supplement the communication’s strategies
Political concerns
Institutional capacity

17. Using Indicators for Communication Decisions
Progress report
Comments on outliers
Validity checks – absolute, expert
Outlier attention – solving problems, lessons to share
Monitor progress – local targets at achievable levels

18. Using Indicators for communication Decisions
Feedback – immediate – before next month (this is not a history exercise!)
Local graphing –
limited indicators, 4-5 only
Up to date
numbers of services – not percentages
know your denominators
cumulative coverages, for preventive services
targets for each, set locally
annual percentages – overall accomplishments

19. Data Quality
Without sound and reliable data, the best-designed indicators will be useless.
Data Quality Issues:
Will the data cover all of the elements of interest? (Coverage)
Is there a complete set of data needed for each element of interest? (Completeness)
Have the instruments been tested to ensure validity and reliability of data? (Accuracy)

20. Data Quality
Are the data collected as frequently as needed? (Frequency)
Does the available data reflect the time periods of interest (Reporting Schedule)
Can the data needed from each source be collected/retrieved? (Accessibility)

21. Qualitative and Quantitative Data
Quantitative data are necessary for tracking trends accurately
Qualitative data are useful for understanding the context in which the trends occurred and to interpret the quantitative data accurately

22. Developing Indicators
Consider the element for which the indicator is necessary
Decide on the indicator
Identify source of information
Identify who collects
How often
For what purpose?

23. “CREAM” & “SMART” Indicators
Memorizing characteristics of good indicator:
Clear
Specific
Relevant
Measurable
Economic
Adequate
Reliable
Monitorable
Time bound

24. When Selecting Your Communication Indicators
Select several for any one outcome
Make sure the interest of multiple stakeholders are considered
Know that over time, it is ok (and expected) to add new ones and drop old ones
Have at least three points of measurement before you consider changing your indicator

25. Developing A Set of Outcomes Indicators for a given communication channel
Example: Activity 1
Outcomes/ Goals
Indicators
Baseline
Target 1 2 3

26. Checklist for Assessing Proposed Indicators
Activity 2
Outcome to be measured: ______________________________
Indicator selected: ____________________________________
Is the Indicator… 1
As direct as possible a reflection of the outcome itself? 2
Sufficiently precise to ensure objective measurement? 3
Calling for the most practical, cost-effective collection of data 4
Sensitive to change in the outcome, but relatively unaffected by other changes? 5
Disaggregated as needed when reporting on the outcome?

27. Communication by Indicators
A few carefully defined and selected indicators will make everyone a better Communicator