1. 1 Quality Assurance in Education and Training Magda Kirsch QTEL.NET International Conference Glasgow 15-16 November 2007

2. 2 QTEL.NET International Conference- 15-16 November 2007 References 'Copenhagen Process‘, Quality Assurance in VET, Technical Working Group Progress Report, December 2004 http://www.oph.fi/info/qa/vet/auditory/Oliveira%20Reis%2010.2.05, %20Opening%20Ceremony%20Copenhagen%20process%20and%20Eur opean%20co-operation.pdf,http://www.oph.fi/info/qa/vet/auditory/Oliveira%20Reis%2010.2.05, %20Opening%20Ceremony%20Copenhagen%20process%20and%20Eur opean%20co-operation.pdf Lise-Lotte Ravnmark, ‘An European Guide on Self-assessment for VET providers’, September 2003. How good is our school, (2007), the journey to excellence, part 3, HMIe, improving Scottish education http://www.hmie.gov.uk/documents/publication/hgiosjte3.pdf Effective School Self-Evaluation (ESSE) website http://sici.org.uk/ESSE/ Checklists for external assessment in Flanders and the Netherlands (NVAO)

3. 3 QTEL.NET International Conference- 15-16 November 2007 Rationale QA in VET is a key instrument in supporting the transition to a knowledge- based society –by increasing the effectiveness of VET ‘outcomes’ in terms of improving the match between demand and supply, employability, and access to training. QA assurance is crucial in: –increasing mutual trust in training provision in EU –supporting mobility and lifelong LLL

4. 4 QTEL.NET International Conference- 15-16 November 2007 Rationale – Why QA If Europe is to achieve its aspiration to be the most dynamic and knowledge- based economy in the world (Lisbon Strategy), then European education will need to demonstrate that it takes the quality of its programmes and awards seriously and is willing to put into place the means of assuring and demonstrating that quality.

5. 5 QTEL.NET International Conference- 15-16 November 2007 Common Quality Assurance Framework (CQAF) Developed as an outcome of cooperative work on Quality Assurance between the Commission, the MS, social partners, the EEA – EFTA and the candidate countries Designed to support the development and, where necessary, the reform of quality systems in VET, in MS. Provides a common reference system and concrete reference tools to help MS and participating countries to improve, monitor and evaluate their own policies and practices in the field of QA.

6. 6 QTEL.NET International Conference- 15-16 November 2007 Common Quality Assurance Framework (CQAF) The CQAF consists of four features: –a model, to approach quality assurance and development of VET provision at the appropriate levels in Member States; –a method for assessment and review: the emphasis has been given to self assessment, combined with external evaluation; –a monitoring system: to be identified as appropriate at national or regional level, and possibly combined with voluntary peer review at European level; –a measurement tool: a set of reference indicators aiming at facilitating Member States to monitor and evaluate their own systems at national, regional or local levels.

7. 7 QTEL.NET International Conference- 15-16 November 2007 1. Common Quality Assurance Framework (CQAF ) the model

8. 8 QTEL.NET International Conference- 15-16 November 2007 PURPOSE & PLAN Leadership Aims & values Strategy & planning Partnerships Finance & Resources FEEDBACK & PROCEDURES for CHANGE Planning & implementing for improvements External verification PDCA Quality Assurance Cycle IMPLEMENTATION Management of teachers, trainers and other staff Process management teaching & learning ASSESSMENT & EVALUATION Teaching & Training results Labourmarket & societal results Financial results

9. 9 QTEL.NET International Conference- 15-16 November 2007 QA -A three-step approach Internal quality control, which results in self-evaluation External visitation /audit, which results in a public report Accreditation

10. 10 QTEL.NET International Conference- 15-16 November 2007 Step 1: Internal QA To find out whether the institution has the resources necessary to achieve its stated goals and objectives and whether it achieves its stated mission To provide the institution with an opportunity to evaluate its overall quality and its curriculum To pinpoint the strengths and weaknesses of the institution

11. 11 QTEL.NET International Conference- 15-16 November 2007 Global aim of the self- evaluation (report) To find out how effective the school is in achieving continuous improvement through evaluating the quality of its own provision and taking action to build on its own strengths and address its weaknesses

12. 12 QTEL.NET International Conference- 15-16 November 2007 What do we need for an effective self-evaluation? Have access to statistical data comparing the performance with similar schools, Have access to an agreed set of quality indicators to guide the self-evaluation process, Consult and communicate with stakeholders, Have devolved responsibility for the use of financial and physical resources, Have responsibility for review and development of school staff Produce development or improvement plan.

13. 13 QTEL.NET International Conference- 15-16 November 2007 Step 2: External QA Quality assurance can be undertaken by external agencies/authorities for a number of purposes, including: –safeguarding of national educational standards –accreditation of programmes and/or institutions; –user protection; –public provision of independently-verified information (quantitative and qualitative) about programmes or institutions;/centres –improvement and enhancement of quality.

14. 14 QTEL.NET International Conference- 15-16 November 2007 Elements of external QA Step 1: Criteria and procedures Step 2: Commissions in vocational/subject areas Step 3: Preparation of the on-site visit/audit Step 4: The on-site visit Step 5. The review report (assessment)

15. 15 QTEL.NET International Conference- 15-16 November 2007 HOW do we carry out external QA ? Step 1: Criteria and procedures Key stakeholders should be involved in order to ensure clarity of purpose and transparency of procedures Decisions should be based on explicit published criteria that are applied consistently.

16. 16 QTEL.NET International Conference- 15-16 November 2007 HOW do we carry out external QA ? Step 2: Set up expert commissions Expert commissions with specialists from the sectors concerned are set up to conduct an external assessment Expert commission members should be representative in their field, so that decisions taken are acceptable for all parties concerned.

17. 17 QTEL.NET International Conference- 15-16 November 2007 HOW do we carry out external QA ? Step 3: Preparation of the on-site visit Is there a need for the qualification or course that will be licensed or accredited. ? Investigating number of key documents such as: –1) existence of mission, vision of the school –2) a number of data such as teacher qualifications, success rate of the pupils or trainees –3) quality of examinations –4) programme brochures, etc.

18. 18 QTEL.NET International Conference- 15-16 November 2007 HOW do we carry out external QA ? Step 3: Preparation of the on-site visit –Work with checklists –Checklists contain a number of quality indicators that are organised into key areas; –Each quality indicator has a number of themes referring to observable areas of activity e.g. ensuring equality and fairness –Key areas can differ according to the country

19. 19 QTEL.NET International Conference- 15-16 November 2007 HOW do we carry out external QA ? Step 3: Preparation of the on-site visit Key areas for NVAO Aims and objectives Curriculum Staff Financial resources QA Attainment

20. 20 QTEL.NET International Conference- 15-16 November 2007 HOW do we carry out external QA ? Step 3: Preparation of the on-site visit Each key area contains a number of Quality indicators - NVAO Curriculum –Relation between curriculum and aims and objectives –Professional requirements in an international perspective –Coherence –Study time –Methodology and study materials –Evaluation –Project work

21. 21 QTEL.NET International Conference- 15-16 November 2007 HOW do we carry out external QA ? Step 3: Preparation of the on-site visit key areas – HMI –Curriculum –Attainment –Learning and Teaching –Support for pupils –Ethos –Resources –Management, leadership and QA

22. 22 QTEL.NET International Conference- 15-16 November 2007 HOW do we carry out external QA ? Step 3: Preparation of the on-site visit Each key area contains a number of Quality indicators – HMI e.g. Ethos Climate and relationship Expectations and promoting achievement Equality and fairness Partnerships with parents, the school board and the community

23. 23 QTEL.NET International Conference- 15-16 November 2007 HOW do we carry out external QA ? Step 3: Preparation of the on-site visit How can quality indicators be used? –Can help us assess the extent to which the school aims are being met –Aims can be evaluated against four or more levels of performance Very good: major strengths Good: strengths outweigh weaknesses Fair: some important weaknesses Unsatisfactory: major weaknesses

24. 24 QTEL.NET International Conference- 15-16 November 2007 HOW do we carry out external QA ? Step 4: The on-site visit The main purpose of the on-site visit is to check whether the data provided are accurate and whether the buildings and facilities meet the existing requirements. Therefore: – Interviews with director, teachers, trainees, admin. staff, employers, parents, students, management board etc. –Visit of the facilities, inspection of materials and equipment.

25. 25 QTEL.NET International Conference- 15-16 November 2007 HOW do we carry out external QA ? Step 5. The review report –A secretary should be appointed to write the report. He/she needs to prepare the draft report of the external review meeting(s) and the on-site visit. –External review not in view of accreditation: Main part: advice to enhance quality –External review in view of accreditation: Main part: motivation of scores given

26. 26 QTEL.NET International Conference- 15-16 November 2007 HOW do we carry out external QA ? Step 5. The review report – Based on a framework with key areas and quality indicators – Score is given for each of the quality indicators after a consensus meeting with all panel members – Each score should be thoroughly motivated

27. 27 QTEL.NET International Conference- 15-16 November 2007 HOW do we carry out external QA ? Step 5. The review report Each quality indicator will be illustrated against the levels used above (evidence and motivation) e.g. Student support Score: good (strengths outweigh weaknesses) –There is a systematic and structured approach as far as student support is concerned –Each student gets counselling from a tutor –Teachers are readily available for all students with questions or problems –However, there is not enough differentiation as far as the approach of mature and other students is concerned

28. 28 QTEL.NET International Conference- 15-16 November 2007 HOW do we carry out external QA ? Step 5. The review report – The report is sent to the school or centre that has been assessed; – The school or centre can make comments on the reports and have all inaccuracies removed; – After all comments have been discussed the final version of the report is sent to the school.

29. 29 QTEL.NET International Conference- 15-16 November 2007 Step 3. Accreditation –Procedures should be clear –Results of accreditation procedures should be open

30. 30 QTEL.NET International Conference- 15-16 November 2007 Step 4: Follow-up Quality assurance processes should have a predetermined follow-up procedure which is implemented consistently. External quality assurance of institutions and/or centres should be undertaken on a cyclical basis.

31. 31 QTEL.NET International Conference- 15-16 November 2007 Trends in QA in VET –A number of countries such has Ireland, Romania, Italy, Denmark, Sweden, Flanders, the UK etc. have introduced self-assessment. –In certain countries like Denmark the focus is entirely on self-assessment and there is hardly any external assessment or audit in place. –This is probably due to the important involvement of Social partners in the VET system and on a ‘ mutual trust ’ consensus.

32. 32 QTEL.NET International Conference- 15-16 November 2007 Trends in QA Guidance instead of control Assessment as control is often seen as a judgement on individuals; Assessment as a decision-making aid is more dynamic and transparent and is seen as having a training function. It relates to collective actions and results which must be constantly improved and made more relevant.

33. 33 QTEL.NET International Conference- 15-16 November 2007 Trends in QA in VET Stakeholder involvement in QA procedures –In Sweden specific initiatives to increase quality in VET, for improving cooperation between the school and the local industry. –In Flanders experts from the sectors concerned are involved in as well self-evaluation as external evaluation of Professional Bachelor Programmes –In France development of qualifications in VET as well as QA is largely sector-based –IN BG procedures for evaluation are agreed with social partners and ministries, and accepted at the NAVET`s Executive Council –In many countries students/parents are involved, not only in internal but also external evaluation of schools, colleges.

34. 34 QTEL.NET International Conference- 15-16 November 2007 Trends in QA in VET Benchmarking –Achievements of schools, colleges, universities are set against against national data on learner achievements and other core performance indicators –Therefore several countries developed measurement tools (success measures) which can be applied to evaluate performance. The common set of measures includes a.o.: learner success rates; employer engagement; staff qualifications; learner satisfaction etc.

35. 35 QTEL.NET International Conference- 15-16 November 2007 Trends in QA in VET International peer reviews International panels to carry out QA –Exchange of experience and mutual learning should remain the rationale underpinning the European cooperation for enhancing the quality of VET in the European Union and to increase its transparency.

36. 36 QTEL.NET International Conference- 15-16 November 2007 Trends in QA in VET Transparency –Policy guidelines to stimulate transparency of methods, criteria, assessment results and accountability are developed in several countries; –In a number of countries Inspection results are published for all providers and are used along with other published data on learner achievements to inform the choices made by prospective learners.

37. 37 QTEL.NET International Conference- 15-16 November 2007 Thank you for your attention and good luck with your QA!