1. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 1 The Role of the Common European Framework John H.A.L. de Jong EALTA Conference, Kranjska Gora, Slovenia, May 14 – 16, 2004

2. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 2

3. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 3

4. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 4 Language The Common Framework Application Overview

5. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 5 Bilingual Brain

6. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 6 Speaking: From intention to articulation Levelt, 1989

7. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 7

8. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 8 language structures declarative discourse declarative discourse language structures Decoding Spoken turn social Encoding Decoding Spoken turn Person APerson B

9. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 9 Facility in Spoken Language is not Oral Proficiency Language Language use Culture World knowledge –OP includes the ability to use the language in socially appropriate ways in a range of complex functions over a variety of topics. –High OP implies adequate FSL –High FSL does not guarantee high OP OP(L2)  f (OP(L1), FSL(L2), Culture(C2), Social (S2)) FSL(L2)  OP(L2) - OP(L1)

10. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 10 Some Basic Issues in the CEF Some of which are often forgotten

11. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 11 Types of Scores Continuousinfinite number of values (1, 2, 3, etc. But also 1.2, 1.53 1.6 etc.) Discrete fixed set of fixed values 0 1 3 4 2

12. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 12 Defining stages on a continuous scale Stage 1 Stage 2 Stage 1 Stage 2 Stage 3 Stage 4 Defining stages on a continuum is arbitrary. We can define different numbers of stages, according to our liking. As long as we clearly define where a one stages ends and another begins. Note: There is always a stage below the lowest defined stage Below Stage 1

13. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 13 Quality Quantity Potential Language Development: follows two basic dimensions

14. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 14 Dimensionality The quantity development is in fact multidimensional and quality can develop along each of the dimensions

15. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 15 the number of... domains (school, work, home, etc.) functions (ask, command, inquire, etc.) notions (north, south, table, mother, cat, eat, drink, etc.) situations (meeting, e-mail, telephone, etc.) locations (market, school, police station, etc.) topics (weather, study, holidays, etc.) roles (listener in audience, participant in discussion, etc)... that a language user can deal with. QUANTITY refers to:

16. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 16 Language use is effective leading to degree of precision 1) in understanding what is meant 2) in expressing one’s meaning leading to communication with least possible effort Language use is efficient the degree to which QUALITY refers to:

17. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 17 Combining Q & Q Note: there is always a level below the lowest defined level A combination of Quantity & Quality: Breakthrough Waystage Threshold Vantage Operational Proficiency Mastery defines Levels of Language Proficiency Below Breakthrough Independent Co-operative Basic functions Fully independent Fluent and Spontaneous No pr blem Note: It is essential to understand that in the CEF Mastery does NOT mean ‘native speaker’ level. Surely an illiterate native speaker is not even at Breakthrough for reading. And there are big differences among native speakers with respect to the other skills as well.C1B2 B1 A A1 I’m afraid you are below A1

18. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 18 Simplify A1 A2 B1 B2 C1 C2 Below A1 A more simple system of level names (universal: no translation problems)

19. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 19 Broad & Narrow A B C A1.1 A1.2 A2.2 A2.1 B1.1 B1.2 B2.1 B2.2 C1.1 C1.2 C2.1 C2.2 A1 A2 B1 B2 C1 C2 Below A1 Now we can make level distinctions and ones

20. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 20 Oral scale C2 C2 Conveys finer shades of meaning precisely and naturally. C1 C1 Shows fluent, spontaneous expression in clear, well-structured speech. B2 B2 Relates information and points of view clearly and without noticeable strain. B1 B1 Relates comprehensibly main points he/she wants to make on familiar matters A2 A2 Relates basic information on, e.g. work, background, family, free time etc.. A1 A1 Makes simple statements on personal details and very familiar topics. EXAMPLE: Oral Interaction Scale Note: a person at the border between level A2 and B1 will probably be able to do 80% of the tasks at level A2, but can also do 50% of the tasks at level B2, etc, because the underlying scale is continuous and the CEF descriptors have been IRT scaled. 1%5% 15% 15%.50%80%95% 80%

21. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 21 Communicative Strategies Communicative Language Competencies Hierarchy of Scales To illustrate the hierarchical principle one branch of the hierarchy is worked out in detail to the right  ReceptionProduction Understanding Note: this branch of the hierarchy is part of the quantity dimension: how much a language learner can do a native speaker Conversation Informal Discussion Formal Discussion Obtaining Goods and Services Interviewing & being interviewed SpokenWritten InteractionMediation Overall language Proficiency Global Scale And at each further node in the hierarchy the CEF offers descriptive scales. Communicative Activities The Global scale is for Overall Language Proficiency.

22. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 22 The Quality Hierarchy Communicative Strategies General Linguistic Vocabulary Range Grammatical Accuracy Phonological Control Vocabulary Control Orthographic Control LinguisticSociolinguisticPragmatic Communicative Language Competencies Communicative Activities Overall language Proficiency Global Scale The same Global Scale analyzed here into the constituent elements of the quality dimension: how well a learner can do. And here too there is a scale at each node.

23. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 23 Automatic speaking tests Integrated “listen then speak” items Real-time processing of spoken linguistic forms Overall score and diagnostic subscores Sentence Mastery Vocabulary Fluency Pronunciation

24. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 24 What is tested? Facility in spoken English: ability to track what is said, extract meaning in real time, formulate responses produce responses at a conversational pace, that are relevant and intelligible

25. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 25 Conversational Skill hear utterance get lexical items extract structure build semantic field contextualize infer demand (if any) articulate response construct response select lexical items build phrase structure select register decide on response (After: Levelt, 1989)

26. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 26 Native & Non-Native Overall CDF

27. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 27 Experiment: Score interpretation on CEF scale PhonePass SET 10 ~ CEF

28. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 28 C2 B1 B2 C1 A2 English and Spanish on CEF scale

29. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 29 Figure 1: Estimates for CEF level Cut-off from North (2000) and from responses to open-ended tasks in Spanish and English English and Spanish vs. CEF scale

30. L ANGUAGE T ESTING S ERVICES John H.A.L. de Jong 2004 30 is So what is the role of the CEF? Transparency - Tool for communication Comparability - Common yardstick Progress - Stimulates discussion