1. Seminar on Mid Term Evaluation in Objective 1 and 2 Regions Lessons from the Mid Term Evaluation of Merseyside Objective One

2. Overall Evaluation Approach  Comprehensive approach, responding directly to the evaluation questions  Mix of top down and bottom up analysis  Evaluation highlighted value, but also limitations, of original research

3. Bottom up Analysis  Need to understand nature and performance of projects, in terms of:  Benefiting target groups  Programme performance  Overall socio-economic performance  Analysis therefore included:  Detailed project reviews  Surveys of business and individual beneficiaries

4. Project Reviews  Purpose:  Reality check for progress ‘on ground’  Check validity of monitoring data  Explore emerging impacts  140 reviews in total:  Covering 50% of committed spend  Mix by priority, measure, fund, delivery agency, location and size  Use of a structured questionnaire, gathering both quantitative and qualitative data  Performance data collected before hand Review coverage:  Rationale and purpose  Value added by SF  Project progress  Financial performance  Output/Impact performance  Cross cutting issues  Programme management

5. Project Reviews (cont)  Reviewers trained to undertake a rigorous assessment, including judgements on:  Adequacy of project rationale and approach  Articulation of project with the Programme Objectives  Appropriateness and realism of targets and outputs achieved  Value added by Structural Fund support

6. Surveys of Beneficiaries  Two separate surveys:  Individual beneficiaries  Business beneficiaries  Purpose:  Explore the effectiveness of the assistance for the individual or business (gross impact)  Establish the extent of deadweight and displacement (net impact upon the economy)  Investigate participants views of the support and opportunities to improve provision  But there are limitations

7. The Surveys Individual Beneficiary Survey  Postal survey with 100% coverage  1,500 completed questionnaires – 18% response rate  Potentially powerful tool, but limited by:  Ability to capture detailed information on impacts  Ability to secure a representative sample Business Survey  Telephone survey of 200 SMEs (c20% of participants)  Richer source of data due to use of telephone survey  Survey issues:  Exclusion of SMEs with minor interventions  Ability to locate most appropriate contact  Too early to draw firm conclusions on impacts

8. Survey Issues  Suitability of survey methods  Representativeness of sample  Timing of survey  Design and testing of questions  Training of interviewers

9. Conclusions  Value of original research in answering evaluation questions  Success dependent upon:  Appropriate techniques  Well designed and tested research instruments  Accurate beneficiary data  Timing of research  But need to be aware of limitations of approach