Relating language examinations to the Common European Framework of Reference for Languages (CEFR) Waldemar Martyniuk Waldemar Martyniuk Language Policy Division Council of Europe Strasbourg, France

Council of Europe set up 1949 by 10 member states in February 2006, the Council of Europe has 46 member states based in Strasbourg, France political international Organisation main bodies: Committee of Ministers, Parliamentary Assembly, Congress of Local and Regional Authorities of Europe and European Court of Human Rights. Guiding principles: democracy, human rights and rule of law.

Council of Europe Approach Standards - Diversity

Council of Europe Language Education Policies aim to promote: plurilingualism plurilingualism linguistic diversity linguistic diversity mutual understanding mutual understanding democratic citizenship democratic citizenship social cohesion social cohesion

Guiding principles for language learning Language learning is for all Language learning is for all Language learning is for the learner Language learning is for the learner Language learning is for intercultural communication Language learning is for intercultural communication Language learning is for life Language learning is for life

Guiding principles for language teaching Language teaching is co-ordinated Language teaching is co-ordinated Language teaching is coherent and transparent Language teaching is coherent and transparent Language learning and teaching are dynamic lifelong processes Language learning and teaching are dynamic lifelong processes

A plurilingual person has: a repertoire of languages and language varieties a repertoire of languages and language varieties competences of different kinds and levels within the repertoire competences of different kinds and levels within the repertoire

Plurilingual education promotes: An awareness of why and how one learns the language one has chosen An awareness of why and how one learns the language one has chosen An awareness of and the ability to use transferable skills in language learning An awareness of and the ability to use transferable skills in language learning A respect for the plurilingualism of others and the value of languages and varieties irrespective of their perceived status in society A respect for the plurilingualism of others and the value of languages and varieties irrespective of their perceived status in society

Plurilingual education promotes: A respect for the cultures embodied in languages and the cultural identities of others A respect for the cultures embodied in languages and the cultural identities of others An ability to perceive and mediate the relationships which exist among languages and cultures An ability to perceive and mediate the relationships which exist among languages and cultures A global integrated approach to language education in the curriculum A global integrated approach to language education in the curriculum

Recommendations to the member states: Encourage all Europeans to achieve a degree of communicative ability in a number of languages Encourage all Europeans to achieve a degree of communicative ability in a number of languages Diversify the languages on offer and set objectives appropriate to each language Diversify the languages on offer and set objectives appropriate to each language Encourage teaching programmes at all levels that use flexible approach and give them appropriate recognition in national qualification systems Encourage teaching programmes at all levels that use flexible approach and give them appropriate recognition in national qualification systems

Recommendations to the member states: Encourage the use of foreign languages in the teaching of non-linguistic subjects Encourage the use of foreign languages in the teaching of non-linguistic subjects Support the application of communication and information technologies to disseminate teaching and learning materials for all European national and regional languages Support the application of communication and information technologies to disseminate teaching and learning materials for all European national and regional languages

Recommendations to the member states: Support the development of links and exchanges with institutions and persons at all levels of education to offer the possibility of authentic experience of the language and culture of others Support the development of links and exchanges with institutions and persons at all levels of education to offer the possibility of authentic experience of the language and culture of others Facilitate lifelong learning of languages Facilitate lifelong learning of languages

Institutions Language Policy Division (LPD) Language Policy Division (LPD) Strasbourg, France Strasbourg, France European Centre for Modern Languages (ECML) Graz, Austria

Language Policy Division Development of policies to promote linguistic diversity and plurilingualism Development of policies to promote linguistic diversity and plurilingualism Assistance to member states with policy analysis and planning Assistance to member states with policy analysis and planning Common reference instruments for European standards and quality Common reference instruments for European standards and quality Language education policy Language education policy

European Centre for Modern Languages Support for the implementation of language education policies Support for the implementation of language education policies Promotion of innovative approaches to the learning and teaching of languages Promotion of innovative approaches to the learning and teaching of languages Professionalism and professional status of language educators Professionalism and professional status of language educators

LPD: Policy instruments and initiatives Common European Framework of Reference for Languages (CEFR) Common European Framework of Reference for Languages (CEFR) European Language Portfolio (ELP) European Language Portfolio (ELP) Reference Level Descriptions for national/regional languages Reference Level Descriptions for national/regional languages Manual for Relating Language Examinations to the CEFR Manual for Relating Language Examinations to the CEFR Guide for the Development of Language Education Policies Guide for the Development of Language Education Policies Language Education Policy Profiles Language Education Policy Profiles

Language Policies for Democratic Citizenship and Social Cohesion (2006-9) (new initiatives) A framework of reference for language(s) of school education A framework of reference for language(s) of school education Language testing and migration – a framework of reference Language testing and migration – a framework of reference Curriculum reference framework for Romani Curriculum reference framework for Romani

The CEFR Published 2001 Published language versions (February 2006) 25 language versions (February 2006) A Descriptive Scheme A Descriptive Scheme Common Reference Levels Common Reference Levels Illustrative scales of descriptors Illustrative scales of descriptors

Impact of the CEFR on language education At learner/citizen level – the ELP At learner/citizen level – the ELP At institutional level At institutional level At national level At national level At European level At European level At international level At international level

Survey on the use of the CEFR 111 respondents from 37 European states, Egypt and Mexico. 111 respondents from 37 European states, Egypt and Mexico. They represent the view of the following types of institutions: Higher education (39); Central authority (29); Teacher training centre (18); Teacher education/Teacher college (18); Examination provider (16); Language school/centre (14); Adult education (12); Other: Further education, publisher, primary or secondary school, cultural agency/centre (28) They represent the view of the following types of institutions: Higher education (39); Central authority (29); Teacher training centre (18); Teacher education/Teacher college (18); Examination provider (16); Language school/centre (14); Adult education (12); Other: Further education, publisher, primary or secondary school, cultural agency/centre (28)

Survey results The CEFR is rather widely known in the responding institutions (3,16 on a 0-4 scale) and it is quite widely used (2,24 on a 0-4 scale) The CEFR is rather widely known in the responding institutions (3,16 on a 0-4 scale) and it is quite widely used (2,24 on a 0-4 scale) It is used mostly by teachers, teacher trainers, test developers, and pedagogic material writers It is used mostly by teachers, teacher trainers, test developers, and pedagogic material writers It is used mostly in the domains of teacher training (pre-service and in-service), language testing/assessment, language curriculum development, textbook/material production, and communication with stakeholders (learners, parents, teachers, staff, clients, etc.) It is used mostly in the domains of teacher training (pre-service and in-service), language testing/assessment, language curriculum development, textbook/material production, and communication with stakeholders (learners, parents, teachers, staff, clients, etc.)

Survey results The clearly best known/most frequently used parts of the CEFR are the common reference levels of language proficiency - the scales of illustrative descriptors, the global scale and the self assessment grid The clearly best known/most frequently used parts of the CEFR are the common reference levels of language proficiency - the scales of illustrative descriptors, the global scale and the self assessment grid The usefulness of the CEFR has been rated at 2,44 on a 0-3 scale The usefulness of the CEFR has been rated at 2,44 on a 0-3 scale The CEFR proved to be most useful in the domains of testing/assessment/certification (2,70 on a 0-3 scale) and curriculum/syllabus development (2,66 on a 0-3 scale) The CEFR proved to be most useful in the domains of testing/assessment/certification (2,70 on a 0-3 scale) and curriculum/syllabus development (2,66 on a 0-3 scale) Institutionally, the CEFR proved to be most useful for examination providers (2,88 on a 0-3 scale) Institutionally, the CEFR proved to be most useful for examination providers (2,88 on a 0-3 scale)

The Manual Project - timetable Drafted by an authoring group in 2003 Drafted by an authoring group in 2003 Piloting the Draft: Piloting the Draft: Revising and publishing: 2008 Revising and publishing: 2008

Suggested Procedures Familiarisation with the CEFR Familiarisation with the CEFR Linking on the basis of specification of examination content Linking on the basis of specification of examination content Linking by specification and standardisation of judgements Linking by specification and standardisation of judgements Empirical validation: checking that exam results relate to CEFR levels as intended Empirical validation: checking that exam results relate to CEFR levels as intended Possible option: indirect linking via a Reference Level Description (like the „Profile Deutsch”) Possible option: indirect linking via a Reference Level Description (like the „Profile Deutsch”)

Reference Materials Reference Supplement Reference Supplement Videos and DVDs with calibrated sample spoken performances Videos and DVDs with calibrated sample spoken performances CD-ROMs with calibrated sample items and tasks for reading and listening CD-ROMs with calibrated sample items and tasks for reading and listening calibrated sample written tasks and performances (web page based) calibrated sample written tasks and performances (web page based) Grids for the Analysis of test items, tasks and performances Grids for the Analysis of test items, tasks and performances

Piloting the Manual Participants Participants 40 institutions 40 institutions 12 languages 12 languages 20 countries 20 countries

The Linking Exercise Practical objectives Practical objectives Relating existing examinations to the CEFR Relating existing examinations to the CEFR Analysis and revision of existing examinations / specifications Analysis and revision of existing examinations / specifications Adjusting the level of difficulty of items/exams “to bring them into line with CEFR level descriptors” Adjusting the level of difficulty of items/exams “to bring them into line with CEFR level descriptors”

The Linking Exercise Practical objectives Practical objectives Developing new examinations based on the CEFR and the Manual Developing new examinations based on the CEFR and the Manual Collecting performance samples and testing items and calibrating them to the CEFR Collecting performance samples and testing items and calibrating them to the CEFR Establishing common standards for a set of existing examinations Establishing common standards for a set of existing examinations

Piloting the Manual – first conclusions The concept of a CEFR toolkit The concept of a CEFR toolkit Benchmarking seminars Benchmarking seminars Validation of claims Validation of claims Quality of language testing Quality of language testing attaining standards attaining standards sustaining diversity sustaining diversity

References Further information and reference available at: Further information and reference available at: