© SEESAC, 2006 Name, Organisation, Event Location, Date Monitoring and evaluation during SALW Awareness programmes

© SEESAC, 2006 Part 1 – 30 minutes 1. Define monitoring and evaluation 2. Explain the differences between them 3. Explain why they are important

© SEESAC, 2006 Part minutes 1. Describe the different levels at which monitoring and evaluation can be carried out

© SEESAC, 2006 Part 3 – 40 minutes 1. List some common tools for doing monitoring and evaluation 2. Explain why it is best to use a mix of tools 3. List some common criteria for selecting monitoring and evaluation tools

© SEESAC, 2006 Part 1

© SEESAC, 2006 Exercise 1 1. What is monitoring? 2. What is evaluation? 3. What are the similarities and differences? 4. Is it important to do M&E, and if so, why?

© SEESAC, 2006 An informal explanation: Monitoring and evaluation are two forms of activity which aim to measure how your work is progressing Measurement during a project = monitoring Measurement after a project = evaluation What are monitoring and evaluation?

© SEESAC, 2006 A formal definition (SASP 2)… Monitoring and evaluation (M&E) = Collecting and analysing information…. …to determine whether those groups engaged by a SALW Awareness programme have,….. …as a result of the intervention,…. …changed their awareness of, and attitudes and behaviour towards, SALW,…. … in line with the stated programme objectives.

© SEESAC, 2006 Table 9, (SASP 2)… WHAT IT INVOLVESWHEN IT OCCURS WITHIN THE PROGRAMME CYCLE Monitoring Tracking progress towards the achievement of objectives, in order to identify what is working and what isnt working so well, allowing a degree of adaptability in strategy and tactics as appropriate. Ongoing. Evaluation A more structured and formal process of reviewing achievements, in order to make judgements about past effectiveness and learn from experience to improve future practice. At fixed times – for projects lasting more than 18 months, this would normally include a mid-term review as well as an evaluation conducted at the completion of the project.

© SEESAC, 2006 Why are they important? Test for effectiveness Better appreciation of area where working Learn how events have affected the work Adapt programme both during project life- cycle (M) and before next phase (E) Identify good practice to use elsewhere Information to share with others

The programme cycle Feasibility study Analysing and planning Evaluation Designing activities and materials Field-testing Implementing activities Monitoring and reviewing Needs and capacity assessment Planning for monitoring and evaluation

© SEESAC, 2006 The time difference… RESEARCH + ANALYSIS DESIGNIMPLEMENTEVALUATE - Monitoring -- Evaluation -

© SEESAC, 2006 Part 2

© SEESAC, 2006 Imagine you are doing SALW Awareness …

© SEESAC, 2006 Levels of M&E (SASP 2) LEVEL OF EVALUATION PURPOSEKEY QUESTIONS Activities To assess how well the programme has been organised and whether resources have been used efficiently. Outcomes To identify changes in knowledge, attitudes and behaviour among target groups that can be reasonably attributed to the programme. Impact To explore how a particular programme may have made a difference to the lives of specific groups of people, e.g. better security. - Are we sending people the correct messages? - Are the messages reaching the right people? - Are there any signs that knowledge, attitudes and beliefs are changing? - Are there any signs that behaviour is changing? - What impact has the programme had in terms of security / casualties etc.?

© SEESAC, 2006 Activity monitoring example.. Monitoring a TV spot Watch TV! Monitoring a community meeting Phone the local organiser afterwards

© SEESAC, 2006 Outcome monitoring example… Balkan Youth Union (BYU) 5 th April 2003, central Belgrade BYU and children destroyed 500 toy weapons Puppet show T-shirts MUP information leaflets to support collection

© SEESAC, 2006 RTS and RTV B-92, as well as to the journalists of dailies DANAS, BLIC and POLITIKA

© SEESAC, 2006 Outcome monitoring example… Outcome evaluation by: Letters to BYU (hundreds) Media coverage of interviews with the public

© SEESAC, 2006 Impact monitoring and evaluation… Difficult Other factors Casualty figures Crime levels Observe weapons visibility

© SEESAC, 2006 Six questions… 1. How has your awareness programme reduced the number of weapons casualties resulting from weapons in target communities? 2. Are the messages being promoted reaching the right people? 3. Are there any signs of changes in practice or behaviour? 4. How has your awareness programme changed security in targeted communities? 5. Are there any signs that knowledge, attitudes and beliefs are changing? 6. Are the messages being promoted the right ones?

© SEESAC, 2006 Part 3

© SEESAC, 2006 Things to consider: Cost Staffing Skill-levels Representativeness Geographic coverage Depth of explanation Access to social groups Level of participation

© SEESAC, 2006 Common M&E Tools Interviews Focus groups Questionnaires Secondary (desk) research Participatory methods

Map exercise…

© SEESAC, 2006 Group Exercise… Group 1: Cheap Give a deep understanding of target groups feelings about SALW Capture womens views well Group 2: Allow generalisations to be made about the wider population Allow respondents to participate Group 3: Build the skills of respondents Capture information about unexpected impacts