1. ‘tools’ KICK-OFF MEETING BRUSSELS January 2012 External evaluation
ALB CONSEIL PARIS – EVALUATION SERVICES FOR EU PROJECTS

2. Sunday, April 21, 2019
ALB Conseil - TOOLS Project
WHO WE ARE
ALB CONSEIL , Paris, is a consortium of 3 professionals from Belgium, France and Italy, with proved experience in EU projects and LLP projects evaluation:
LEONARDO, GRUNDTVIG, ERASMUS, COMENIUS
KA2 LANGUAGES, KA3 ICT, E-LEARNING
They teach, translate and work in total 6 languages: English, Italian, French, German, Romanian, Spanish.
Angelica Bucur and Lucia Leporati were in 2011
Quality experts for Languages (applications assessment) and for Comenius (e-Twinning Pilot Assessment).
External evaluation of 7 LLP projects and contribution to their successful implementation and quality of results. Two of these projects won EU or national awards (Germany NA Award 2011 for CyberTraining ) or/and are in Compendium of the EU education best practices projects (KEMP GRU Project).
Experience in evaluation methodologies and models for content and quality of educational/learning results AND work processes.
Experience as independent assessors of LLP project applications, Progress and Final Reports, with insight of the specific challenges and the “state of the art” of Lifelong Learning projects.

3. WHO IS WHO? TOOLS’ E.E. TEAM
Sunday, April 21, 2019
ALB Conseil - TOOLS Project
WHO IS WHO? TOOLS’ E.E. TEAM
Angelica Lumi Bucur Manager ALB CONSEIL Economist, MBA, MD in EU Affairs – Belgium
25 years of experience in management, planning, strategy, business, e-business, ICT, education and training
+ 8 years assessment of EC projects & LLP applications, Progress and Final Reports.
External (and co-) evaluator of 7 LLP projects LEO and GRU
Lucia Leporati
Teacher ALB Consultant
MD in Modern Foreign Languages Literatures, with specialization in German, English, and Spanish
19 years of experience in teaching in Italy, and from 2010 in Switzerland; CLIL expert
Participated to Comenius 1, 2.1 and 2.2) and to European Network Comenius3
+ 6 years assessment of EC projects & LLP.
Ange: Also evaluation of appeals. Lucia has participated in many in-service teachers training and to numerous professional qualification courses. Her didactics and methodology applied for Foreign Language Teaching include learning methods and techniques to increase pupils’ motivation such as using ICT and web 2.0 in education or creating CLIL lessons about innovation in Germany for the teaching of German and sciences. To help students become more independent and autonomous learners, Lucia develops Task-Based-Learning, role-play activities for the annual study voyages organized by her with the classes of Italian speaking students in Germany or in Switzerland. An independent expert for European Commission, Lucia was invited by TAO and then EACEA in 2005, 2006, 2007, 2008, 2009, 2010 to the assessment of proposals submitted in the framework of the SOCRATES and LIFELONG LEARNING PROGRAMME, in particular COMENIUS, E-TWINNING (school education).

4. TOOLS partners: What role FOR THE external evaluatorS (e.e.)?
Sunday, April 21, 2019
ALB Conseil - TOOLS Project
TOOLS partners: What role FOR THE external evaluatorS (e.e.)?
WHAT EXPECTATIONS DO YOU HAVE?
HOW DO YOU WANT TO COLLABORATE WITH E.E.?
YOUR OPINIONS WILL BE INCLUDED IN THE E.E. STRATEGY TO BE FINALIZED AFTER KICKK-OFF MEETING

5. EXTERNAL EVALUATORS of TOOLS: WHAT WOULD WE LIKE TO DO?
Sunday, April 21, 2019
ALB Conseil - TOOLS Project
EXTERNAL EVALUATORS of TOOLS: WHAT WOULD WE LIKE TO DO?
Go beyond the paradigm of evaluation as assessment:
We aim to bring out the value of the project as a LLP initiative and highlight the strengths of outcomes for learning purposes and those ensuring outcomes ‘ sustainability.
To act as critical friend of the TOOLS project and partnership
Find evidence of the quality achieved by the project, according to the quality indicators defined in the application and agreed by the whole TOOLS partnership

6. METHODOLOGY & EVALUATION AREAS
Sunday, April 21, 2019
ALB Conseil - TOOLS Project
METHODOLOGY & EVALUATION AREAS
We aim to provide a comprehensive model and external evaluation program of the project achievements through a range of activities and assessments
External evaluation is an integral part of quality evaluation of TOOLS project work, achievements, contribution to original aims and specific objectives
Three areas: operational, summative, learning.

7. Sunday, April 21, 2019
ALB Conseil - TOOLS Project
3 EVALUATION AREAS
Operational formative evaluations will provide information and recommendations /eventual corrective measures on how the project is being developed
Summative evaluations for PR and FR time are based on formative evaluations, and address the correlation between expected results and achieved results
Additional evaluation for learning purposes at end of project will take place in month 30 :
the updated assessment of all the project’s specific achievements to LLP and KA2, to end-users,
the added-value and EU-level contributions that can be drawn from the project at the end of its EU-funded lifetime.
FORMATIVE E.E. COVERS EACEA ASPECTS: how the project is being developed and cover:
the efficiency of project management, the quality and effectiveness of partners’ participation,
the work-progress and respect of deadlines, the modalities according to which work is being carried out and additional solutions/ products, the respect of all quality indicators, with special attention on software developed and on-line learning units, the effectiveness and impact of activities involving end-users groups in testing and using the products, the quality and reach of dissemination exploitation activities.
2. The process and resources used and their efficiency to reach application objectives
the impact and contribution of TOOLS outputs to teaching/learning the 8 languages (LWULs, neighbour, regional, minority, mainstream)
the CLIL benefits for the teachers from the 3 target sectors- Erasmus, Grundtvig and Leonardo, in participating countries
the critical factors at the origin of the success and/or delays of the project activities, added-value results and sustainability.

8. IMPLEMENTATION APPROACH AND ACTIVITIES
Sunday, April 21, 2019
ALB Conseil - TOOLS Project
IMPLEMENTATION APPROACH AND ACTIVITIES
Two-fold approach:
ON-GOING MONITORING AND EVALUATIONS
ACTIVITIES: Participation to project meetings, quantitative and qualitative tools/questionnaires and interviews to analyse and record progress against key indicators, taking in account at what extent they match on the project own internal evaluation and quality management strategy.
REFLECTIVE REPORTS: Summative and/or formative evaluations of the overall progress of the project, partners' contributions in relation to end project achievements, innovative aspects, range/means of communication and dissemination activities, external impact, exploitation and sustainability.

9. MONITORING AND EVALUATION MODEL -TOOLS
Sunday, April 21, 2019
ALB Conseil - TOOLS Project
MONITORING AND EVALUATION MODEL -TOOLS
CLIL in ERASMUS, LEONARDO, GRUNDTVIG
Teachers and Student testing / using TOOLS in 8 target languages
Teachers of other LWULs, VOLL in ALL 4 EDU sectors
Quality Control Group
All TOOLS members
Users Participants to pilots, Partners Peer reviews
EACEA KA2 monitoring / control
Complement Internal QA, Objective evaluation of
Quality of Work-process and results
Added-Value of Content and learning
Quality Plan, monitoring
QA Activities, Reports, Deliverables
Quality indicators & Success criteria
Exploitation QA
INTERNAL EVALUATION
EXTERNAL EVALUATION
EXTERNAL STAKEHOLDER USERS
INTERNAL STAKE HOLDERS

10. ‘TOOLS’ EXTERNAL EVALUATION OVERVIEW
Sunday, April 21, 2019
ALB Conseil - TOOLS Project
‘TOOLS’ EXTERNAL EVALUATION OVERVIEW
EVALUATION SCOPE: Results (assessment in line with Lifelong Programme framework) and the education learning processes and relations between the actors involved (partners /users /stakeholders and results).
KEY AIMS: Measure, forecast, control, support; highlight value /changes.
JUDGEMENT: Factual and quality judgements (based on the EACEA criteria and/or judgement), and value judgments based on project success indicators and other criteria during implementation.
METHODS: Quantitative AND Qualitative Evaluation methods
ACTION-PLAN: pre-established (in line with TOOLS plan work-and LLP/EACEA assessment criteria) and participatory during the whole duration of 2,5 years.
Questions , feedback about the proposed strategy, activities and areas by the external evaluation?

11. Sunday, April 21, 2019
ALB Conseil - TOOLS Project
Questions , feedback about the proposed strategy, activities and areas by the external evaluation?
This discussion can be linked with the internal evaluation and Quality Plan

12. Internal evaluation: TOOLS quality indicators
Sunday, April 21, 2019
ALB Conseil - TOOLS Project
Internal evaluation: TOOLS quality indicators
The application set-up for each WP clear quality indicators and ambitious quantitative indicators
Need to review and agree on the key indicators

13. DISCUSSIONS and AGREEMENT?
Sunday, April 21, 2019
ALB Conseil - TOOLS Project
DISCUSSIONS and AGREEMENT?
Internal evaluation
Main aspects for Kick-off Meeting
External evaluation
Complementarity
Who is in charge in each partner organisation
WPs internal indicators of performance
Reports
Frequency
Etc.
Collaboration
Data gathering
Frequency
Means: s, skype, participation to meetings
Reports
Recommendations

14. WE THANK YOU ! Angelica L. Bucur Lucia Leporati ALB CONSEIL Manager
Sunday, April 21, 2019
ALB Conseil - TOOLS Project
WE THANK YOU !
Angelica L. Bucur
ALB CONSEIL Manager
Lucia Leporati
Consultant
ALB CONSEIL PARIS - EVALUATION SERVICES FOR EU PROJECTS