Carol J. Pierce Colfer Center for International Forestry Research & Cornell International Institute for Food, Agriculture and Development

Why do qualitative M&E? Three Examples of M&E in action: 1. Quilcene, WA, educational research 2. Global comparative research on assessment of sustainable forest management 3. Global comparative action research with local communities Some Dangers and Conclusions

Need for holistic understanding of a situation & how it has changed. An unexpected finding emerged that hadn’t been measured initially. An external, post-facto analysis is demanded, on issues not initially assessed. The team has qualitative, not quantitative skills.

[Typically] involves Participant observation Long term residence in the research context A holistic, inductive, open-ended orientation Varying degrees of independence from project being evaluated

10 US rural schools, given grants to experiment with their local schools External, long term M&E, both qualitative (fieldwork) & quantitative (cross-site) Field researchers lived in communities for ~3 years, documenting what happened and helping with cross- site studies – teasing out what went right, what went wrong.

A central idea has been that C&I can be used to monitor, assess, & even define a subject of interest. Hierarchy of Principles, Criteria, Indicators, Verifiers Ideal ones are SMART (Specific, Measurable, Achievable, Realistic, Timely) BUT some topics are difficult (impossible?) to quantify:

1. Greater self-confidence among women & other marginalized groups 2. Improved knowledge of regulations among groups previously uninvolved 3. Involvement in enforcing sanctions, by a broader spectrum of stakeholders 4. Closer links between communities & outsiders (government officials, industry, projects, academics)

Aim was to develop widely agreed-upon C&I to define SFM, & for use in monitoring & assessing it (initially, in certification of timber). Series of 1-month, interdisciplinary, international field visits to compare & hone existing sets of C&I that would work in each country studied, using a series of filtering steps (described in CIFOR Toolbox No. 1).

A long term, learning-based approach involving (facilitated) community groups identifying shared, future goals Analyzing, planning, & implementing what is needed to reach those goals Monitoring progress & revising plans accordingly Linking productively with relevant external actors

Facilitator/researchers worked with communities in 11+ countries (vertically, horizontally & iteratively) Teams assessed community progress in ways that worked in their contexts: C&I that local people developed repeated ‘reflection’ meetings use of ethnographic observations

Ongoing M&E by community members – are we reaching our community/group goals? M&E by researcher/facilitators---to what degree is ACM actually empowering, enriching people, &/or enhancing their well being or environments? Result: A complicated life for researcher/facilitators; a qualitative, cross-site assessment, examining/comparing site experiences (7 dimension framework)

Getting too complex - e.g., Landscape Mosaic project’s 4 levels of monitoring Producing something so holistic, ‘deep’ and long that no one will ever read it (cf. quantitative baseline surveys so long & complex that data never get entered, let alone analyzed). [as with any method] Being swayed by your own ideological biases, or someone’s (donors’, employers’) desire for evidence of success

Conducting qualitative M&E can be an uphill battle – donors, policymakers, & many researchers prefer quantitative assessments BUT Qualitative M&E can often provide valuable insights, unexpected findings, not available with conventional quantitative approaches. IDEALLY: Combine the two!