1. Developing Indicators
Workshop for
Børne - og ungdomsnetværket
Copenhagen
Wednesday 13 August, 2003

2. During this workshop we will
Place the development of indicators in the context of M&E and LFA
Clarify what we understand by indicators and their importance to improving organisational performance
Examine the characteristics of a good indicator and the process of developing them
Explore the particular challenges of developing indicators for rights-based work with children
Identify ways in which the network can support members in developing indicators for their work with children

3. Monitoring and evaluation
“Monitoring is the systematic and continuous assessment of the progress of a piece of work over time.”
“ An evaluation is the assessment at one point in time of the impact of a piece of work and the extent to which the stated objectives have been achieved.”

4. Why monitor and evaluate?
To improve performance
To improve day-to-day decision-making
To enhance impact
To provide early warning of problems
To empower stakeholders
To build understanding and capacity
To stimulate learning
To demonstrate & strengthen accountability

5. The programme management cycle
Situation analysis
Planning
Implementation
Monitoring
Review and evaluation

6. LFA and indicators
LFA is a commonly used approach for project and programme planning, implementation and evaluation
LFA structures thinking about links between objective setting, monitoring and evaluation
A Logframe matrix is used to map out the ‘objective hierarchy’ Goal, Purpose, Objectives, Outputs and Activities
Requires the identification of indicators and means of verification for each ‘level’

7. Key stages in the Logical Framework Approach
Establish the scope of the work
Agree on the planning framework
Undertake a situation analysis
Problem analysis
Stakeholder analysis
Objectives analysis
Develop the implementation strategy
Complete the Logframe matrix
Establish indicators and MoV
Identify & analyse assumptions & risks
Develop the M&E framework

8. Logframe terms and example
Goal
The broader development impact to which the project contributes
Improved community health on a sustainable basis
Purpose
The development outcome expected at the end of the project
A clean, reliable and sustainable supply of water adequate for community needs
ComponentObjectives
The expected outcomes for each component of the project
Reticulated water supply established by (date)
Competent village water supply technicians in place by (date)
Outputs
The direct measurable results of the project (largely under management’s control)
1.1 plan for water supply
1.2 suitable header tank
1.3 operational pump
Activities
The tasks carried out to implement the project & deliver the outputs
Conduct site survey; identify local labour sources; build header tank; construct pump.

9. The Logframe matrix Project Description Indicators
Means of Verification
Assumptions
Goal
Purpose
Objectives
Outputs
Activities

10. What is an indicator?
An indicator is the quantitative or qualitative evidence that will be used to assess progress towards an objective. Indicators provide the basis for monitoring progress and evaluating the achievement of outcomes. An objective may have more than one indicator.

11. Different types of indicators
Process indicators
Impact indicators
Quantitative indicators
Qualitative indicators
Proxy indicators
Indices
PROCESS - show whether the activities that were planned are actually being carried out, and carried out effectively.
IMPACT - to assess what progress is being made towards reaching the objectives, and what impact the work has had on the different groups of people affected by the work.
QUANTITATIVE indicators involve the definition of numerical measures eg number of meetings attended.
QUALITATIVE indicators refer to defining characteristics that cannot be quantified. For example, changes in behaviour or people’s perceptions.
PROXY indicators measure things that represent (or approximate) changes that cannot be measured directly. Eg the percentage of households with bicycles may approximate a level of economic well-being in communities where bicycles are desirable but expensive.
INDICES combine a number of different indicators to enable comparisons. For example UNDP’s human development index.

12. SMART Properties of indicators
Specific
Measurable (and unambiguous)
Attainable (and sensitive)
Relevant
Time-bound
SPECIFIC -
Must relate to those things that the project intends to change, avoiding measures that are largely subject to external influences
MEASURABLE -
Indicators must be precisely defined so that their measurement and interpretation are unambiguous.
Quantifiable indicators are more precise but because development and rights-related work may be difficult to quantify, qualitative indicators should also be used.
ATTAINABLE -
The information required by an indicator must be achievable at a reasonable cost using an appropriate collection method and sensitive to the changes the project wishes to make.
RELEVANT -
Indicators should be relevant to the work of the project.
TIME-BOUND -
Indicators should describe by when a certain change is expected.
It is not always easy to meet all these criteria!

13. The Logframe matrix Project Description Example Indicators
Means of Verification
Goal
Improved community health on a sustainable basis
Reduced water-borne disease rates.
Health records
Interviews with community members
Purpose
A clean, reliable and sustainable supply of water adequate for community needs
Water cleanliness at acceptable standards
Water availability with no supply shortages of more than (specified time).
Supply breakdowns fixed within agreed standards.
Water cleanliness tests
Site visits to examine supply
Objectives
Water supply established by (date)
Competent village water supply technicians in place by (date)
Water supply in place and being used by (date)
Village technicians able and dealing with supply maintenance & problems.
Site visits
Records of supply use.
Testing of technicians’ ability to deal with problems.
Technicians maintenance logs
Outputs
1.1 plan for water supply
1.2 suitable header tank
1.3 operational pump
Workable plan produced and used for construction
Tank and pump constructed to agreed standards
Examination of plan
Site visits to examine ‘hardware’
Activities
Conduct site survey; identify local labour sources; build header tank; construct pump.
Work plan targets met
Work plan
Construction log

14. Developing indicators that address performance issues
Establish goal, purpose and objectives in the Logframe
Identify performance questions for all levels in the objective hierarchy
Develop the indicators needed to answer the performance questions
Develop the means of verification

15. Criteria for selecting indicators
Does the indicator measure what you want to measure?
Does the indicator yield data that it is essential to know (rather than just nice to know)?
Does the indicator yield data that is useful for programme planning and management?
Is the indicator worth the time and effort to measure?

16. Dimensions of change in child rights programming
Source: Theis, Joachim (2003) Rights Based Monitoring and Evaluation: A Discussion Paper, Save the Children

17. Evaluation and rights based programming
Must gather the views & opinions of children & young people whilst recognising the ‘child’s best interests’
Should be designed with and involve children and young people whenever possible
Must address the ‘Three Pillars’, as appropriate

18. Evaluating children and young people’s participation
Impact of children & young people’s participation
Impact on the stated objectives
Impact on children
Impact on adults, communities and institutions
Quality of children & young people’s participation
Effectiveness of children & young people’s participation

19. Demonstrating a commitment to children’s participation
Transformed power relations between children and adults
Raised awareness and developed skills in children’s participation among children and adults
Children’s civil rights (information, expression and association) actively promoted throughout the organisation
Obstacles overcome and space increased for decision making by children at all levels of society and in all institutions
Children and adults supported to claim children’s rights

20. Capacity building - some ideas
Arrange training courses and workshops
Organise mentoring support
Support development of organisational strategies, policies & standards
Support development of organisational systems and procedures
Arrange exchange visits & info exchange
Produce and distribute case-studies, annotated reading lists, guidelines, tools, manuals & practice standards
Facilitate collaborative working on common issues
Develop a list of resource persons

21. Useful documents used in developing this presentation
Theis, Joachim (2003) Rights Based Monitoring and Evaluation: A Discussion Paper Save the Children. Available online at
Kirby, Perpetua and Sara Bryson (2002) Measuring the Magic?: Evaluating and researching young people’s participation in public decision making, London: Carnegie Young People Initiative. Available online at
Children and Participation: Research, monitoring and evaluation with children and young people. Available online at