UNFCCC Meeting on experiences with performance indicators for monitoring and evaluation of capacity building in developing countries Rio De Janeiro, Brazil 6-7 November, 2008 Jamil H.Chowdhury Bangladesh M & E Network Dhaka, Bangladesh

Thing to be remember 1 “ In the evaluation of capacity development the impact metaphor should be avoided. The militaristic impact metaphor fails to capture the essential features of capacity development which is a process of change and growth.” (Horton, D. 2002)

Common confronting question in M & E of capacity building 2 Has the capacity building been well defined? Are the widespread examples of M & E of capacity building? What tools we should use for accessing national capacity in a integrated way? Do we have any standardized approaches to monitoring and evaluating capacity building interventions? Do we have standardized logical framework or gold standard for assessing country capacity? Are we clear about the relationship between capacity and performance i.e. what elements of capacity are critical to performance? Do we understand the difficulty in quantifying many elements of capacity before attributing capacity change to any single intervention? Can we separate capacity measurement from the process of building capacity itself?

3 Identifying existing needs and gaps in the process of implementation. Understanding the status of implementation of multilateral decisions. Identifying needs for technical and financial support. Assessing prioritization of climate change issue in national and local planning and decision making. Assessing arrangement for mainstreaming climate change related capacity building within the context of wider sustainable development. Determine the level of country commitment and ownership. Rationale for M & E of capacity building

Issues of consideration for evaluating country capacity 4  Country level performance monitoring for assessing capacity may be a preferred choice and easy to implement on a regular basis.  Evaluating a country capacity is complex and methodologically challenging.  Evaluation must be done for specific desired changes or outcome variables in performance with an interval of every three-five years.  Capacity evaluation tend to get mix up with performance measurement.  Capacity indicators must be well identified and well defined before they are evaluated.  Quality of capacity building remain usually unaddressed.  Measuring cost of capacity building at any point of time remains a great challenge.

Suggested indicators for assessing country capacity There are always unlimited range of national capacity that may include:  Level of national response or political commitment.  Policy and regulatory changes.  Budgetary provisions for capacity building.  Ability and level of procurement of external resources for enhancing capacity.  Building organization or strengthening institutions both at public and private sector.  Capacity of vulnerability and adaptation assessment and implementation of mitigating options. 5

Suggested indicators of country capacity (contd..) 6  Functional availability of climate change secretariat and focal points.  Coverage of climate change issues by the leading news papers.  Institutional arrangement for technology transfer.  Level of implementations of adaptations measures.  Level of interministerial or multi sector co-ordination to address climate change issues.  Type of bi-lateral and multilateral agreement signed for strengthening capacity.  Availability of baseline data on current capacity/capacity needs assessment for monitoring future progress.

Suggested indicators of country capacity (contd..) 7  Instances of developing/strengthening research and training institutions.  Level of involvement of civil society, academician and other stakeholders.  Availability of national plan of action and national strategies (inclusions of climate change agenda in the national development plan).  Plan for development of technical human resources (metrological, hydrological land, water & forest management).  Training & awareness of print and electronic media journalists for expanded advocacy.

Common barriers to measurement of capacity 8  Capacity building is rarely considered an important management issue and always seen in isolation.  Most people seem uncertain as to what, when and how to monitor when it comes to capacity building.  Absence of baseline data to compare the level of progress.  High staff turnover and repeated transfer of technical/ skilled human resources especially at the public sector.  Lack of availability of quality data or failure to identify data sources.  Lack of proper documentation of best practices and lessons learnt.

Suggested methodologies 9 Combine qualitative and quantitative methods. Review national and policy and plan documents. Review organizational records/documents (if available) Interview key professionals/focal points or conduct focus groups with them. Review research reports/secondary data. Review leading newspaper reports for last 1 year for monitoring change and media coverage.

Suggested methodologies (contd…) 10 Use scoring system for relevant indicators. Engage external evaluators. Consider participatory self assessment methods. Make separate capacity assessment of organizations/institutions as part of assessment of national capacity. Try Appreciative Inquiry (AI) for relevant organizations/institutions for identifying strengths.

The way forward 11 For appropriate methods of M & E of capacity building we need to:  Assess the current level of capacity.  Identify the gaps in performance and capacity.  Map what inputs are available to support the current capacity.  Determine which interventions will address the gaps in capacity and performance.  Define performance objectives and determine the outcome measures to evaluate progress.

The way forward (contd…..) 12  Plan long term financial support to develop capacity building / training organizations.  Plan long term support for capacity building of research organizations for systematic research and monitoring of climate change issues.  Develop a time frame for periodic evaluation of outcome.

Example of a success story 13 Best known example of capacity building is in the EPI programme in the Bangladesh health sector. Reasons for success It started as a project with long time continued support from UNICEF, WHO & other few INGOs The project was mainstreamed into the health system after 5-7 years. The external and technical support continued even after integration into the system. Dynamic leadership at the initial stages brought success. Strong BCC campaign reinforced the effort. Successful community involvement and strong community support helped sustain the mainstream.

Example of a failure case Bangladesh bureau of statistics has been generating data on various development indicators for more than 30 years, but with little institutional capacity attainment. Reasons for failure  It failed to produce quality data to satisfy national & international users.  Lack of institutional leadership and ownership at any point of time (the head of the institutions frequently changed).  Lack of long time technical support from international agencies.  Lack of accountability and good governance.  Misdirected energy and waste of resource without institutional capacity building goal.  Lack of institutional commitment. 14

End