1. Basing writing tests for young learners to the CEFR Cecilie Carlsen & Eli Moe, University of Bergen, Norway Second Annual Conference of EALTA, Voss, Norway 2nd –5th June 2005

2. Background National tests in English, writing and reading, for Norwegian school children (4th, 7th, 10th and 11th grades) Employer: The Norwegian Ministry of Education and Research Test developers: University of Bergen Framework: CEFR Aim: Reporting and pedagogical

3. Focus of presentation 12-13 years old pupils’ writing (7th grade) Adjusting the CEFR-scales to young learners Pilot study: Validation of our benchmarks/ levels Questions for discussion

4. WHY adjust? define relevant domains/ themes define relevant tasks, functions, can dos and text-types * ** the test should function within a pedagogical setting

5. HOW to adjust? develop adapted candos for young learners develop analytic rating scale for teachers/raters select benchmarks use in-between levels (A1/A2, A2/B1, B1/B2)

6. Pilotstudy: Validation of benchmarks Benchmarks: 22 Raters: 5 Raters’ profile Raters’ task

7. Results (I) All raters & all scores

8. Results (II) Aggregated rating vs Bergen (1)

9. Results (III) Aggregated rating vs Bergen (2)

10. Results (IV) Distribution of scores across levels

11. Agreement? %Cum % Same assessment 40.9 (9) Minor diff. (within level) 22.7 (5) 63.6 Major diff. (different level) 27.3 (6) ** 90.9 One whole level difference 9.1 (2) 100.0

12. Pilotstudy: Main findings 40% (63%) of the assessments made by the external group were the same as the Bergen group The Bergen estimates are consistently higher than those of the external raters The Bergen group uses more levels than most of the external raters. Short texts is a source of disagreement. Very strong pupils is a source of disagreement (B1 or B2?). There is a considerable variation among the external raters.

13. Questions for discussion Do our adaptations of the CEFR-descriptors simplify the levels? Is it possible to adjust the CEFR-scales to young learners without changing the levels?? Is it possible to place a 12 year old L2-learner at a level above B1? How to proceed?