‘tools’ KICK-OFF MEETING BRUSSELS January 2012 External evaluation ALB CONSEIL PARIS – EVALUATION SERVICES FOR EU PROJECTS

Sunday, April 21, 2019 ALB Conseil - TOOLS Project WHO WE ARE ALB CONSEIL , Paris, is a consortium of 3 professionals from Belgium, France and Italy, with proved experience in EU projects and LLP projects evaluation: LEONARDO, GRUNDTVIG, ERASMUS, COMENIUS KA2 LANGUAGES, KA3 ICT, E-LEARNING They teach, translate and work in total 6 languages: English, Italian, French, German, Romanian, Spanish. Angelica Bucur and Lucia Leporati were in 2011 Quality experts for Languages (applications assessment) and for Comenius (e-Twinning Pilot Assessment). External evaluation of 7 LLP projects and contribution to their successful implementation and quality of results. Two of these projects won EU or national awards (Germany NA Award 2011 for CyberTraining ) or/and are in Compendium of the EU education best practices projects (KEMP GRU Project). Experience in evaluation methodologies and models for content and quality of educational/learning results AND work processes. Experience as independent assessors of LLP project applications, Progress and Final Reports, with insight of the specific challenges and the “state of the art” of Lifelong Learning projects.

WHO IS WHO? TOOLS’ E.E. TEAM Sunday, April 21, 2019 ALB Conseil - TOOLS Project WHO IS WHO? TOOLS’ E.E. TEAM Angelica Lumi Bucur Manager ALB CONSEIL Economist, MBA, MD in EU Affairs – Belgium 25 years of experience in management, planning, strategy, business, e-business, ICT, education and training + 8 years assessment of EC projects & LLP applications, Progress and Final Reports. External (and co-) evaluator of 7 LLP projects LEO and GRU Lucia Leporati Teacher ALB Consultant MD in Modern Foreign Languages Literatures, with specialization in German, English, and Spanish 19 years of experience in teaching in Italy, and from 2010 in Switzerland; CLIL expert Participated to Comenius 1, 2.1 and 2.2) and to European Network Comenius3 + 6 years assessment of EC projects & LLP. Ange: Also evaluation of appeals. Lucia has participated in many in-service teachers training and to numerous professional qualification courses. Her didactics and methodology applied for Foreign Language Teaching include learning methods and techniques to increase pupils’ motivation such as using ICT and web 2.0 in education or creating CLIL lessons about innovation in Germany for the teaching of German and sciences. To help students become more independent and autonomous learners, Lucia develops Task-Based-Learning, role-play activities for the annual study voyages organized by her with the classes of Italian speaking students in Germany or in Switzerland. An independent expert for European Commission, Lucia was invited by TAO and then EACEA in 2005, 2006, 2007, 2008, 2009, 2010 to the assessment of proposals submitted in the framework of the SOCRATES and LIFELONG LEARNING PROGRAMME, in particular COMENIUS, E-TWINNING (school education).

TOOLS partners: What role FOR THE external evaluatorS (e.e.)? Sunday, April 21, 2019 ALB Conseil - TOOLS Project TOOLS partners: What role FOR THE external evaluatorS (e.e.)? WHAT EXPECTATIONS DO YOU HAVE? HOW DO YOU WANT TO COLLABORATE WITH E.E.? YOUR OPINIONS WILL BE INCLUDED IN THE E.E. STRATEGY TO BE FINALIZED AFTER KICKK-OFF MEETING

EXTERNAL EVALUATORS of TOOLS: WHAT WOULD WE LIKE TO DO? Sunday, April 21, 2019 ALB Conseil - TOOLS Project EXTERNAL EVALUATORS of TOOLS: WHAT WOULD WE LIKE TO DO? Go beyond the paradigm of evaluation as assessment: We aim to bring out the value of the project as a LLP initiative and highlight the strengths of outcomes for learning purposes and those ensuring outcomes ‘ sustainability. To act as critical friend of the TOOLS project and partnership Find evidence of the quality achieved by the project, according to the quality indicators defined in the application and agreed by the whole TOOLS partnership

METHODOLOGY & EVALUATION AREAS Sunday, April 21, 2019 ALB Conseil - TOOLS Project METHODOLOGY & EVALUATION AREAS We aim to provide a comprehensive model and external evaluation program of the project achievements through a range of activities and assessments External evaluation is an integral part of quality evaluation of TOOLS project work, achievements, contribution to original aims and specific objectives Three areas: operational, summative, learning.

Sunday, April 21, 2019 ALB Conseil - TOOLS Project 3 EVALUATION AREAS Operational formative evaluations will provide information and recommendations /eventual corrective measures on how the project is being developed Summative evaluations for PR and FR time are based on formative evaluations, and address the correlation between expected results and achieved results Additional evaluation for learning purposes at end of project will take place in month 30 : the updated assessment of all the project’s specific achievements to LLP and KA2, to end-users, the added-value and EU-level contributions that can be drawn from the project at the end of its EU-funded lifetime. FORMATIVE E.E. COVERS EACEA ASPECTS: how the project is being developed and cover: the efficiency of project management, the quality and effectiveness of partners’ participation, the work-progress and respect of deadlines, the modalities according to which work is being carried out and additional solutions/ products, the respect of all quality indicators, with special attention on software developed and on-line learning units, the effectiveness and impact of activities involving end-users groups in testing and using the products, the quality and reach of dissemination exploitation activities. 2. The process and resources used and their efficiency to reach application objectives the impact and contribution of TOOLS outputs to teaching/learning the 8 languages (LWULs, neighbour, regional, minority, mainstream) the CLIL benefits for the teachers from the 3 target sectors- Erasmus, Grundtvig and Leonardo, in participating countries the critical factors at the origin of the success and/or delays of the project activities, added-value results and sustainability.

IMPLEMENTATION APPROACH AND ACTIVITIES Sunday, April 21, 2019 ALB Conseil - TOOLS Project IMPLEMENTATION APPROACH AND ACTIVITIES Two-fold approach: ON-GOING MONITORING AND EVALUATIONS ACTIVITIES: Participation to project meetings, quantitative and qualitative tools/questionnaires and interviews to analyse and record progress against key indicators, taking in account at what extent they match on the project own internal evaluation and quality management strategy.   REFLECTIVE REPORTS: Summative and/or formative evaluations of the overall progress of the project, partners' contributions in relation to end project achievements, innovative aspects, range/means of communication and dissemination activities, external impact, exploitation and sustainability.

MONITORING AND EVALUATION MODEL -TOOLS Sunday, April 21, 2019 ALB Conseil - TOOLS Project MONITORING AND EVALUATION MODEL -TOOLS CLIL in ERASMUS, LEONARDO, GRUNDTVIG Teachers and Student testing / using TOOLS in 8 target languages Teachers of other LWULs, VOLL in ALL 4 EDU sectors Quality Control Group All TOOLS members Users Participants to pilots, Partners Peer reviews EACEA KA2 monitoring / control Complement Internal QA, Objective evaluation of Quality of Work-process and results Added-Value of Content and learning Quality Plan, monitoring QA Activities, Reports, Deliverables Quality indicators & Success criteria Exploitation QA INTERNAL EVALUATION EXTERNAL EVALUATION EXTERNAL STAKEHOLDER USERS INTERNAL STAKE HOLDERS

‘TOOLS’ EXTERNAL EVALUATION OVERVIEW Sunday, April 21, 2019 ALB Conseil - TOOLS Project ‘TOOLS’ EXTERNAL EVALUATION OVERVIEW EVALUATION SCOPE: Results (assessment in line with Lifelong Programme framework) and the education learning processes and relations between the actors involved (partners /users /stakeholders and results). KEY AIMS: Measure, forecast, control, support; highlight value /changes. JUDGEMENT: Factual and quality judgements (based on the EACEA criteria and/or judgement), and value judgments based on project success indicators and other criteria during implementation. METHODS: Quantitative AND Qualitative Evaluation methods ACTION-PLAN: pre-established (in line with TOOLS plan work-and LLP/EACEA assessment criteria) and participatory during the whole duration of 2,5 years. Questions , feedback about the proposed strategy, activities and areas by the external evaluation?

Sunday, April 21, 2019 ALB Conseil - TOOLS Project Questions , feedback about the proposed strategy, activities and areas by the external evaluation? This discussion can be linked with the internal evaluation and Quality Plan

Internal evaluation: TOOLS quality indicators Sunday, April 21, 2019 ALB Conseil - TOOLS Project Internal evaluation: TOOLS quality indicators The application set-up for each WP clear quality indicators and ambitious quantitative indicators Need to review and agree on the key indicators

DISCUSSIONS and AGREEMENT? Sunday, April 21, 2019 ALB Conseil - TOOLS Project DISCUSSIONS and AGREEMENT? Internal evaluation Main aspects for Kick-off Meeting External evaluation Complementarity Who is in charge in each partner organisation WPs internal indicators of performance Reports Frequency Etc. Collaboration Data gathering Frequency Means: e-mails, skype, participation to meetings Reports Recommendations

WE THANK YOU ! Angelica L. Bucur Lucia Leporati ALB CONSEIL Manager Sunday, April 21, 2019 ALB Conseil - TOOLS Project WE THANK YOU ! Angelica L. Bucur ALB CONSEIL Manager lumiangeli@yahoo.fr Lucia Leporati Consultant lucia.leporati@gmail.com ALB CONSEIL PARIS - EVALUATION SERVICES FOR EU PROJECTS