VVSG Vorming Event Most Significant Change (MSC) Technique 1 St 2010 Introduction to MSC Technique Cecile Kusters, Centre for Development Innovation, Wageningen UR, The Netherlands

Adapted from: Rick Davies - MandENEWS Jessica Dart – Clear Horizon

Most Significant Change (MSC) Form of qualitative, participatory M&E Storytelling technique Based on stories of significant change Developed by Rick Davies Bangladesh Now used in numerous development programs and in the public sector

MSC – exercise (1) Each person to think about a story (beginning-middle-end) that illustrates the most significant change that has taken place in the way you view and carry out (or support) monitoring and evaluation as a result of being engaged in the city-to-city linkage (or if this is difficult as a result of your engagement in international development). Draw a story board (on cards) to develop your story: Draw 4-6 boxes or use 4-6 cards How would you introduce your story? Draw a sketch in the first box/card. Draw a simple image (stick figures, quick sketch, no detail needed, no words) Continue rapidly with the outline of the story. Draw these in the boxes/cards.

MSC - exercise (2) Turn to your neighbor and ask: Tell me how you (the storyteller) first became involved with monitoring and evaluation (especially more qualitative M&E methods or methodologies) and what your current involvement is with these methods/methodologies is From your point of view, describe a story (beginning-middle-end) that illustrates the most significant change that has taken place in the way you view and carry out (or support) monitoring and evaluation as a result of being engaged in the city-to-city linkage (or if this is difficult as a result of your engagement in international development). Why was this story significant for you? (Get the details) Document the MSC story (use the story collection format): Contact details, incl. who documented the story (name, position location, date) exposure/engagement with more qualitative methods/methodologies description (who, what, where, when) explanation (why is it significant)

Then…. Then let your neighbor ask the same questions to you. Then come to a decision about which of the two stories you both think is most significant, and identify why you both think so. You may have a number of reasons. Do the same with another pair: share each your 1 selected story and select one out of the 2 MSC stories. Document why you chose this as the most significant of the 2 MSC stories. Do the same with another group of 4 people. Share your selected stories plenary

Plenary selection of stories Share: name; title of MSC story; what was the MSC and why was this selected as the most significant of stories; comments about the story Select: secret balloting – write name of the person and explain why you choose this as the most significant of all selected stories.

Selection chart NameTitle of MSC story What the MSC is about Comments about the story

Selection process

Brief reflection after the group work How do you feel about the process? What have you learned? What did you like? What was difficult? Link to Zanzibar experiences (e.g. facilitating storytelling and voting)

Most Significant Changes

Why stories? People tell stories naturally - indigenous Stories can deal with complexity and context People remember stories Stories can carry hard messages /undiscussables But stories not known for accuracy/truth

Use of stories in MSC Collection of stories + systematic, collective interpretation = story telling can be effectively harnessed for participatory monitoring and evaluation and learning Because interpretations tell another story & process has beneficial outcomes for evaluation utilisation

MSC Form of qualitative, participatory M&E Based on stories of significant change Developed by Rick Davies Bangladesh Now used in numerous development programs and in the public sector

MSC Creates space for stakeholders to reflect, to make sense of complex changes Provides dialogue to help make sense of each others values Facilitates dynamic dialogue ie. what do we really want to achieve and how will we produce more of it? Excellent for participatory programs with diverse, complex outcomes & multiple stakeholders

The core of MSC A question: In your opinion what was the most significant change that took place in ….over the … months [describe the change and explain why you think it is significant] Re-iteration of the same kind of question Which of these SC stories do you think is the most significant of all? [describe the change and explain why you think it is significant]

Explaining MSC in stages 1. Defining Domains of Change 2. Define reporting period 3. Collecting SC stories 4. Selection of collected SC stories 5. Feedback of the choices made 6. Verification 7. Quantification 8. Meta-monitoring and secondary analysis 9. Re-settings of MSC system

Before you begin – questions for fit and purpose Why have you decided to use MSC? In what ways does your programme lend itself to MSC? How will it complement your existing M&E system? What are the main benefits you hope to get from using MSC? Is there management support and/or an organisational learning culture?

1. Defining domains Opposite of SMART indicators? Like newspaper sections: sports, finance, leisure, business, etc Defined by how people use them Examples: changes in peoples lives changes in relationships with our partners changes in government policy on HIV/AIDS

Defining domains… Not essential but Can help structure the selection process Can help focus on goals of concern Their use tells us how what goals mean to participants Options Open window domain Negative changes domain

Zanzibar case Change in peoples lives as a result of being engaged in the program (agriculture and livestock) Example Before training my cassava tubers were very thin, light and small in size. After training, cassava tubers are big, heavy and many per plant. Banana and potatoes also are bigger. Indeed my income has increased to the extent that I have started building a new house and I am planning to buy a milking cow. Example of negative change First I could use the land of my brother for farming. Now that both he and I are engaged in the program (farmer-field-school) he wants his land back so he can practice his new knowledge and skills

2. Set the reporting period In your opinion what was the most significant change that took place in ….over the … months Period used by NGOs varies from 2 weekly, to monthly, to three monthly, and yearly. Three monthly is most common Time demands on staff is the main constraint on frequency

Zanzibar case Annually, spread over the year.

3. Collecting SC stories From those closest to the events of concern. But do not exploit peoples unpaid time Basic format: Description (who, what, where, when) Explanation (why is it significant) Who documented the story (name, position location, date) Option: Recommendation

Collecting SC stories… Reminder: Key parts of the question Looking back over the last month… …what do you think was… …the most significant… …change… …in the quality of peoples lives… …in this community?

Zanzibar case Story collection format

Zanzibar case- overview field work

4. Selecting SC stories

Funder meeting State meetings Region 1Region 2 Region 3 Region 4 Story tellers feedback flow of stories

Selecting SC stories… Task is to read through and identify the most significant of all the submitted SC stories. Take one domain at a time Need to decide who to involve: story providers, their superiors, their peers,.. Need to decide whether to predefine selection criteria, or let them emerge through discussion of SC stories

Selecting SC stories… Must (not optional) Document what SC story was selected Why it was selected Process used to make the selection Participants Their preferences [Subjectivity is made accountable through transparency]

Zanzibar case

5. Feedback To immediate providers of SC stories, on what was selected, why selected and process used Enables adjustment of focus of MSC next time around A motivational factor Weakest point in all M&E systems, including MSC

Zanzibar case Feedback: from ASFT (Agric Services Facilitation Team) with national farmer forum to DFT (distrcif Facilitation Team) & district farmer forum to FFS (Farmer Field Schools). Also directly from district level back to farmers once a story has been selected. Resource centre can be used for sharing. Representatives at district level can provide the feedback to farmers. Quarterly newsletters for farmers can also be used.

6. Verification of SC stories What Factual content & interpretation of facts Why Encourages some discipline in reporting Enables elaboration and further learning When When SC story first enters system When selected as MS of all SC When SC stories are publicly used

Zanzibar case All selected stories will be verified Verify: the source of the story; the most significant change; reasons for choosing this as most significant of changes for this person; comments on the story itself. Who: Storyteller and FFS facilitators. When: verification at farmers level – after selecting 1 per group. It has to be verified before going to the next level. Visit the family of the story teller, have a discussion around the most significant change indicated by the farmer and the story behind it. Capture photos, video of e.g. the new house, the chickens.

7. Quantification Within the SC story Number of people, events, etc involved As once–off follow-up to SC story How many other cases like this known Within meta-monitoring (see next) How many other SC stories like this

8. Meta-monitoring and secondary analysis Keep all SC stories on record Meta-monitoring (Recommended) of Changes in numbers of SC stories, who provides them, whose SC stories are selected, changes in percentage of negative stories Secondary analysis (Optional) by Categorising and counting of types of changes reported, and types of explanations given, at different levels

Zanzibar case on secondary analysis and quantification Will feed into the M&E system – link to logframe/theory of change. Add domains to the story collection format. The facilitator can tick the most significant change in one of these domains. The domains include: income, food quality, food quantity, housing, transport, health, education, decision making, leadership. Additional domains to be added by the district (define these before starting the whole process) plus open domains e.g. to capture negative changes.story collection format Combine with the selection process at district and national level: quantifying the number of stories on e.g. improved income as the most significant change. A format is to be developed for the facilitator to fill in the number of stories per domain. Should be in line with selection process at district and national level.

9. Re-setting of MSC process Frequency of reporting Definition of domains to use Who sorts SC stories into domains Selection process design: participants & process used Feedback and follow up

Where to use MSC Not as a stand-alone method Alongside indicator based systems To identify unexpected changes To engage people in analysis of change To involve a wide range of people To focus on outcomes rather than outputs

Questions about starting How will you get buy in from the people who will be involved in creating/selecting SC stories? How will you expose people to MSC – what training, if any, is needed? Where can you begin – is there a small pilot that you can test first? Who are the best people to capture the first SC stories from?

Finding out more about MSC Original MSC paper (nth version) is at MSC Mailing list is at Rick Davies at

Combining MSC with PV? Participatory Video – see also evaluation evaluation It takes training and commitment to engage in MSC-PV Webbased support is needed to ensure that videos with stories are shared. Sharing can be done easily e.g. youtube or bliptv. Important that it al fits within an overall approach for monitoring and evaluation!

© Wageningen UR Thank you for your attention!