1. M&E and Adaptive Management
For complex integrated
conflict-sensitive programming

2. Definitions M&E Monitoring Evaluation Adaptive management
Project/Program
Scientific
Monitoring, Evaluation and Adaptive Management are particularly crucial in complex programs involving conflict, LTPR and issues of environmental degradation.
Monitoring is the periodic gathering of data to show trends and patterns (ideally) related to investments
We have some pretty good standard indicators that get cross sectoral impacts:
Number of proposed improvements in laws and regulations affecting property rights of the urban and rural poor drafted with USG assistance
Number of USG-supported initiatives/mechanisms designed to reduce the potential for violent conflict over the control, exploitation, trade or protection of natural resources
Number of people attending facilitated events that are geared toward strengthening understanding among conflict-affected groups that were supported with USG assistance
But these are largely process indicators and not outcome or impact indicators
There are a few:
Number of people with increased economic benefits derived from sustainable natural resource management and conservation as a result of USG assistance. Number of hectares in areas of biological significance showing improved biophysical conditions as a result of USG assistance
Number of USG-supported activities that demonstrate the positive impact of a peace process through the demonstration of tangible, practical benefits
Of course we have the option (and obligation) to design custom indicators that capture site specific processes and impacts
Monitoring can only do so much which is why we need evaluation
USAID has a new valuation policy as of Jan 2011 that encourages rigor, transparency and usefulness of evaluations
It tells us to budget 3-5% of our programs for evaluation
We can use our budgets for formal evaluations but also for shorter assessments and participatory stakeholder and project level evaluations as needed
Which gets us to Adaptive Management
Last week we had a very fruitful discussion about adaptive management in terms of ENRM projects and we can provide the materials on that to those who did not attend the course. Adaptive management is our ability to use multiple data streams (monitoring, evaluation, project reports, site visits, etc.) to adjust our activities as needed. We will take more about how to do it. It is used in both project management and in the scientific world. As an example, satellite imagery in West Africa is showing that there are pockets of healthy forest managed by communities that are not on the radar screen for conservation while some protected areas are heavily degraded. This helps us to adapt our forestry and conservation investments towards saving what is there.

3. Principles Evidence-based programming and importance of baselines
Accountability and risk management
Sensitivity
Triangulation
Appropriateness and utility
Participation and capacity building
PPL is calling for evidence based programming. We’ve had a couple of “evidence summits” in DC
The first was on on tacking tough problems in counter insurgency and counter terrorism
And second on broad based economic growth
They are using academic and practitioner data to lay out evidence for different pathways that we might take. For instance, at the broad based economic growth summit, I talked about the evidence for different investments in ENRM that yield highest economic benefits.
At the project level, we won’t have much good evidence about what is working (and to test our theory of change) if we don’t have baselines that measure key components of conflict, land tenure systems and ENRM.
We will talk about how to measure some of these variables when the issues and context are sensitive and complex
M&E is essential for accountability and risk management. Data quality is auditable (more on that later). Things can quickly go awry in conflict situations and we have to be on top of them.
We have to be alert to sensitivities of local people, government and other partners especially in terms of privileged information BUT to get good information we can’t rely on one source, hence the need for “triangulation” or gathering data from at least 3 sources
Our M&E has to be useful to our partners and other key stakeholders, build their capacity and enhance positive participation

4. Integrated M&E in USAID
Results Framework & PMP
Standard and custom indicators
Indexes
DQAs (Data Quality Assessments)
Formal reporting (workplans, reports)
USAID’s formal M&E and reporting system can be worked for maximal learning benefit as well as to monitor and report on integrated programs. Creating a Results Framework demands the development of causal chain (development hypothesis or Theory of Change) with each Development Objective and IR logically linked and measured by an indicator. Creating an integrated RF is a great task for the team to think about interrelationships: conflict mitigation and sound NRM, clarification of property rights and conflict reduction.
There are also particular challenges in programs with multiple objectives—are you measuring components separately, measuring synergies among elements or the whole program? There is a danger in relying on indicators alone because they lead to fragmented thinking UNLESS you are willing to put the time and energy into more complex indicators such as policy indices or multivariate indicators (e.g., terms of trade, good governance indices)
PMP is required and allows USAID to develop with partners and contractors on a framework that measures key results over time
Standard indicators can be useful—example of “hectares under improved mgt” as a way of describing and defining the action space.
Custom indicators are tailored to specific projects. They can be refinements of standard indicators, for instance measuring capacity building or benefit to a specific group of stakeholder, conflict dimensions of one natural resource (diamonds, timber)
DQAs are a way to discuss the process of data collection and quality of the data, including building local capacity and use of the data
Formal reporting can be very useful to track changes because it puts flesh on the bone of indicators with narratives about results and changes. Watch for generic vs specific language about results and changes. Push for specificity and detail. Read the reports and use them!

5. Tough but critical M&E in conflict settings
Proxies for sensitive measures
Monitoring illegal and clandestine activities
Getting information remotely when there is ongoing conflict. How can you do that?
Example of bonobo monitoring in Congo Basin using Baptist Church network, providing them with school notebooks with cartoons of bonobos and asking for sightings, perceptions
One partner continued to enter into war zone to work in protected area of DRC (WCS)
How do we measure sensitive and complex phenomena such as conflicts, land distribution/ownership, NR ownership, elite capture?
Sensitive measures could include things like number of conflicts, ownership of key assets, income and revenue from illegal or semi-legal activities. It is often difficult for USAID and partners to directly measure these dimensions yet they are critical to understanding change. Sometimes proxies can be used, for instance in assessing changes in economic status you can look at changes in material goods, housing, infrastructure. Court records may show shifts in number and type of disputes but this is only in the formal sector. Interviews with key informants in refugee camps, humanitarian orgs, school teachers, nurses, etc can yield information on existing and brewing conflicts.
Market data can be used to track some conflict resources such as bushmeat, timber, diamonds but getting this data is not at all easy if the markets are clandestine. Satellite imagery can reveal some NRM impacts such as deforestation. I am sure DOD has more sophisticated tools!
Watchdog groups such as Global Forest Watch and Global Witness do this kind of work. In DRC and Liberia GW measured the relationship between corruption in the timber sector and government actors. GFW mapped extent and management of timber concessions.

6. Conflict sensitive M&E
Trust
Diversity
Protection of sensitive information
When he gets out of his cage he likes to fight with other lions
Conflict sensitive M&E should be employed even if there is no overt conflict
Establishing trust of key partners and communities is essential
Having a diverse M&E team helps to accommodate different viewpoints
Sensitive information must be protected, passworded, labeled SBU

7. Monitoring LTPR MCC Land Rights and Access Indicator
An index of property rights security for vulnerable groups
Conflicts over land and resources
How do you monitor “progress” as well as conflict n LTPR?
We looked at a couple of the standard indicators…
MCC indicator: This indicator evaluates whether and to what extent governments are investing in secure land tenure. This composite indicator is calculated as the weighted average of three indicators. Access to Land is weighted 50% and Days and Cost to Register Property are each weighted 25%.
Custom indicators could include:
An index of property rights security for vulnerable groups
Awareness of problem
Agreement to address it
Delineation of communal area
Communal area accorded provisional tenure
Communal area rights enshrined in law
Individual property rights delineated within communal area
Conflicts over land and resources
Court records
Mapping as was done in eastern DRC

8. NRM & biodiversity Biodiversity conservation is not a neutral activity
Participatory threats analysis
Know your hectares!
Think before you demarcate
Participatory threats analysis and reduction plan
What is inside the hectares you are monitoring? What biodiversity, what peoples, what LTPR systems?

9. Going grassroots 
“Grassroots” indicators of conflict, social and ecological change
Validate local knowledge and builds capacity
Grassroots indicators are ways that local people monitor the situation. For instance in Melanesia, people kept track of conflict among local clans by tracking the rates of clan intermarriage. I learned about a creative multivariate measure in Indonesia where ownership of plantations was measured in relation to closeness to Suharto and to NRM status. The researchers found that those owners with closest links to Suharto had the worst managed plantations, thus indicating the perverse effects of patronage. This result was published. With global climate change it is going to be increasingly important to use scientific research and indicators to help us program activities that relate to use of natural resources and the environment. We need to keep track of changes in earth systems and how these changes impact stakeholders. For instance, a shift in growing range for key crops will mean that populations have to change their cropping systems and populations may migrate.
This photo is of an object that farmers put in their fields in Sierra Leone to indicate that the field is protected by magic and to ward off those who would steal the crops; this is a post-conflict area of SL
It is a very interesting indicator of how people wish to avoid conflict by using culturally relevant symbols and tools

10. Leveraging science Using scientific research and indicators
Integrating science and local knowledge
With global climate change it is going to be increasingly important to use scientific research and indicators to help us program activities that relate to use of natural resources and the environment. We need to keep track of changes in earth systems and how these changes impact stakeholders. For instance, a shift in growing range for key crops will mean that populations have to change their cropping systems and populations may migrate.
Scientists are monitoring these shifts and are able to predict some changes. For instance the cocoa belt in West Africa is shifting northward which will have huge implications for LTPR, conflict and economic growth/livelihoods.
Other huge impacts being monitored: glacial melt, sea-rise, loss of biodiversity including economically valuable species
USAID invests in research and we need to capitalize on it:
Example: new CGIAR mega-initiative on climate change, agriculture and food security is working with world class climate scientists to translate scientific data and gather local knowledge for interactive knowledge sharing on climate change

11. Beyond indicators Indicators can tell you what but not why
Scenario planning: expecting the unexpected
Informal networks and information flows
Site visits: what to look for
 Triangulation, again
Indicators can only do so much and are only ONE part of Adaptive Management. As discussed, AM is a mindset. It doesn’t get turned on and off at particular moments in the project cycle. There are always opportunities to learn and adapt. Scenario planning should be done at the outset. It is linked to development hypothesis and assumptions. One ASSUMES that the project will roll out as planned but WHAT IF? The WHAT IF can include changes in environmental conditions, reactions to project activities, political changes or even staff changes, turnover, burnout. When USAID started back in DRC in 1999 after 8 year hiatus we developed a 3 choice scenario—business as usual, deterioration or improvement and developed sample activity plan for each of these scenarios.
On a project level use site visits to advance AM. Look for things that are going well that can be enhanced and areas that are creating tension, confusion or conflict. Best practice on site visits is to make it informal and as part of normal work flow. Don’t do “state visits” that emphasize your role as a donor. You will not get good information. If you don’t speak the language bring your own translator. Otherwise you will get the translation your partners want you to hear. Talk informally after work, over a beer (if culturally appropriate!). Try and have both men and women on a site visit. Triangulate your findings.

12. Methodologies Participatory and appreciative approaches
Key informants and focus groups
Process documentation and informal reporting
Science and surveys
New technologies
When doing an evaluation or even a site visit, consider a range of methodologies to gather information, appropriate to conflict environments
Participatory and appreciate inquiry approaches can be empowering but appropriate mostly when you are committed to a community in the long-term. They can help elicit grassroots indicators of change. Ask Dan for example from Liberia assessment.
Sensitive information can be collected using key informants (but triangulate!) and focus groups of peers that feel comfortable speaking in a group (but there are cultural dimensions of peer pressure as discussed in the Fiji example below.
Process documentation of meetings and workshops is often informative about how people responded to an action. It allows for verification that a group is actually meeting, that its meeting agenda is actually tackling issues and the attendance roster is inclusive. But even this might be deceptive. I spent 2 years going to packed PRA meetings in Fiji where everyone was in agreement on the project but then we did a household survey and found that women and non-elites were not included and in fact over 80% of households didn’t even know about the project. So sometimes you just need to delve deeper.
It is really important to integrate or even commission research that might not be possible under your formal project. If you are using AM you will be alert to situations that need clarification and review. Is a process of land reform going well for key, including vulnerable stakeholders? Is decentralized forest management leading to improved management?
My dream of “best practice” in NRM-LTPR-Conflict programming would be to train a cadre of “local ethnographers” at the beginning to keep track of how things are going using both informal and formal methods. I tried this in Solomon Islands where we trained a group of women to undertake these interviews but the project management never read their reports, leading ultimately to disregarding serious issues related to equity and misuse of project equipment.