The Value of Evaluation “Stronger Together! How to access future ESF funding” Action with Communities in Rural Kent The Great Danes Hotel, Hollingbourne

WHY WE ARE CONCERNED WITH EVALUATION More accountability for public expenditure Governments are increasingly concerned with micro-managing initiatives and in controlling the activities of new agencies and groups Crisis of credibility of professionals, insitutions and people in authority What counts is what works (C) Antonio Sama2

OTHER RELATED ACTIVITIES Audit is the verification of compliance of the use of resources (mostly financial) with the binding legal regulations Control, like audit, can refer to the financial and legal aspects of a given project implementation Monitoring is the regular gathering and examination of quantitative and qualitative data on projects and whole programme implementation (C) Antonio Sama3

ELEMENTS OF AN EVALUTIONS (EU) Be analytical; i.e. based on recognised research techniques Be systematic; involving careful planning and consistent use of the chosen techniques Be reliable; this means that a different evaluator with access to the same data and using similar techniques of data analysis would arrive at similar findings Be issue-oriented; they should address important issues such as the relevance, efficiency and effectiveness of the programme Be user-driven; meaning that evaluations should be executed in ways that provide useful information to decision-makers (C) Antonio Sama4

FORMS OF EVALUATION Formative Evaluation: To enable people and agencies make judgements about the work undertaken; to identify their knowledge, attitudes and skills, and to understand the changes that have occurred in these; and to increase their ability to assess their learning and performance Summative Evaluation: To enable people and agencies to demonstrate that they have fulfilled the objectives of the programme or project, or to demonstrate they have achieved the standard required (C) Antonio Sama5

TYPES OF EVALUATIONS Ex-ante evaluation - is performed before implementation and its objective is to assess whether the planned intervention is accurate with regard to needs Mid-term evaluation is the evaluation performed towards the middle of the implementation of an intervention. This evaluation critically considers the first outputs and results, which enable assessing the quality of implementation Ex-post evaluation - the evaluation of an intervention after it has been completed. It should be carried out some years after the end of the implementation period. The ex-post evaluation aims at examining long-lasting effects of a programme or intervention and their sustainability. On-going evaluation - has a supplementary character for all above-mentioned evaluation types and can be conducted independently. The on-going evaluation is carried out throughout the period of implementation of an intervention; however it can not be taken for monitoring, as it consists of the in-depth assessment of the chosen problems that have appeared during other evaluations. (C) Antonio Sama6

TRADITIONAL (BANKING) APPROACH TO EVALUATION 1) A search for objectivity and a ‘scientific approach’, through standardised procedures. The values used in this approach… often reflect the priorities of the evaluator (2) An over-reliance on quantitative measures. Qualitative aspects…, being difficult to measure, tend to be ignored (3) A high degree of managerial control, whereby managers can influence the questions being asked Other people, who may be affected by the findings of an evaluation, may have little input, either in shaping the questions to be asked or reflecting on the findings (4) Outsiders are usually contracted to be evaluator in the belief that his will increase objectivity, and there may be a negative perception of them by those being evaluated’ (C) Antonio Sama7

THE ALTERNATIVE (DIALOGICAL) APPROACH TO EVALUATION 1) Evaluation is viewed as an integral part of the development or change process and involves ‘reflection-action’. Subjectivity is recognised and appreciated (2) There is a focus on dialogue, enquiry rather than measurement, and a tendency to use less formal methods like unstructured interviews and participant observation (3) It is approached as an ‘empowering process’ rather than control by an external body. There is a recognition that different individuals and groups will have different perceptions. Negotiation and consensus is valued concerning the process of evaluation, and the conclusions reached, and recommendations made (4) The evaluator takes on the role of facilitator, rather than being an objective and neutral outsider. Such evaluation may well be undertaken by ‘insiders’ - people directly involved in the project or programme (C) Antonio Sama8

THANK YOU Contacts Antonio Sama Senior Lecturer KE Support Lead The Business School North Holmes Road (Rg.10) Canterbury Kent CT1 1QU (C) Antonio Sama9