Indicators Dr Murali Krishna Public Health Foundation of India

Lesson plan Objective: At the end of session every participant should be able to – Define indicators – say the importance of indicators in M & E – Able to choose appropriate indicators Time: 1 hour 30 minutes Mode of lecture: Interactive session Media: Power point presentation Evaluation: in group work

Introduction Essential part of a monitoring and evaluation system as they are what you measure and/or monitor. Through the indicators you can ask and answer questions such as: – Who? – How many? – How often? – How much?

Definition a quantitative or qualitative factor or variable that provides a simple and reliable means to measure achievement or to reflect the changes connected with an intervention. Indicators are compared over time in order to assess change.

E.g How did your class went on?

Characteristics of indicators The desired properties of indicators, also known as variables, will depend on the approach adopted and the nature of the programme or project being evaluated. All indicators have specific characteristics: – Numeric = the values are numbers – Nominal = the values have names (e.g. male and female) – Continuous = the values are infinite or very large – Ordinal/categorical = the values have a known order (e.g. low to high)

Characteristics of good indicator Indicators will vary from one project to another, according to the work and its context, but in general they are often expected to be: – SMART (specific, measurable, attainable, relevant and time- bound) – SPICED (subjective, participatory, interpreted, cross-checked, empowering and diverse) – CREAM ((Schiavo-Campo 1999, p. 85) Clear : Precise and unambiguous, Relevant : Appropriate to the subject at hand, Economic: Available at a reasonable cost, Adequate : Provide a sufficient basis to assess performance Monitorable: Amenable to independent validation

Practicality of indicators include  Measurability: Is the indicator measurable? Is it sufficiently sensitive to an improvement or deterioration in conditions?  Ease and cost of collection: How easy is it to obtain the information required? How costly will this be? Can the community participate? Are some relevant data already collected?  Credibility and validity: Are the indicators easy to understand, or will people end up arguing over what they mean? Do they measure something that is important to communities as well as implementing organizations?  Balance: Do the selected indicators provide a comprehensive view of the key issues?  Potential for influencing change: Will the evidence collected be useful for communities, implementers and decision- makers?

Common Indicator Metrics Counts –Number of providers trained –Number of condoms distributed Calculations: percentages, rates, ratios –% of facilities with trained provider –Maternal mortality ratio, Total fertility rate Index, composite measures –Quality index comprising the sum of scores on six quality outcome indicators –DALY (Disability Adjusted Life Years) Thresholds –Presence, absence –Pre-determined level or standard

Types of indicators – processing of information: Elementary, derived and compound indicators – comparability of information: Specific, generic and core indicators – phases of completion of the programme: Resource, output, result and impact indicators – evaluation criteria: Relevance, efficiency, effectiveness and performance indicators – mode of quantification and use of the information: Monitoring and evaluation indicators

Developing indicators Step 1: Identify the problem situation you are trying to address. The following might be problems: 1. Economic situation (unemployment, low incomes etc) 2. Social situation (housing, health, education etc) 3. Cultural or religious situation (not using traditional languages, low attendance at religious services etc) 4. Political or organizational situation (ineffective local government, faction fighting etc)

Step 2: Develop a vision for how you would like the problem areas to be/look. This will give you impact indicators. a. What will tell you that the vision has been achieved? b.What signs will you see that you can measure that will “prove” that the vision has been achieved? c.For example, if your vision was that the people in your community would be healthy, then you can use health indicators to measure how well you are doing. a.Has the infant mortality rate gone down? b.Do fewer women die during child-birth? c.Has the HIV/AIDS infection rate been reduced? If you can answer “yes” to these questions then progress is being made. Developing indicators

Step 3: Develop a process vision for how you want things to be achieved. This will give you process indicators. Example: you want success to be achieved through community efforts and participation, then your process vision might include things like – community health workers from the community trained and offering a competent service used by all; – community organizes clean-up events on a regular basis, and so on Developing indicators

Step 4: Develop indicators for effectiveness. For example: if you believe that you can increase the secondary school pass rate by upgrading teachers, then you need indicators that show you have been effective in upgrading the teachers e.g. evidence from a survey in the schools, compared with a baseline survey. Developing indicators

Step 5: Develop indicators for your efficiency targets. Here you can set indicators such as: planned workshops are run within the stated timeframe, costs for workshops are kept to a maximum of US$ 2.50 per participant, no more than 160 hours in total of staff time to be spent on organizing a conference; no complaints about conference organization etc. Developing indicators

Common Pitfalls in Indicator Selection Indicators not linked to program activities Poorly defined indicators Indicators that do not currently exist and cannot realistically be collected Process indicators to measure outcomes & impacts Indicators that are not very sensitive to change Too many indicators