Barbara E. Hanna and Alicia Toohey School of Languages and Cultures University of Queensland (Faire) repenser les contrôles d'une section de langues modernes: le CRCRL à l'Université du Queensland Revitalising assessment across a School of Modern Languages: The Common European Framework of Reference for Languages at the University of Queensland

Readability of courses and progression Student experience Grade distributions Clarity of outcomes Employability Authentic assessment Criterion-referenced assessment BA reset UQ Student Strategy School strategies

School of Languages and Cultures (>2000 enrolments x year) School of Languages and Cultures PROFILE CHINESE FRENCH GERMAN INDONESIAN JAPANESE KOREAN RUSSIAN SPANISH APPLIED LINGUISTICS School of Languages and Cultures (>2000 enrolments x year) TRANSLATION AND INTERPRETING LINGUISTICS Linguistics and Translation and Interpreting, while no less important disciplines in the School, are not directly involved in this project.

Students taking French were also studying 9/10/18

Cross-disciplinary research team School of Languages and Cultures the Institute for Teaching and Learning Innovation Research expertise in: Second Language Acquisition, discourse analysis, language pedagogy and curriculum design, language assessment and intercultural communication, higher education curriculum design, professional learning for higher education staff

Presentation The project The process What the findings tell us about the French section Charting a course for the future 9/10/18

(UQ Teaching Innovation Grant) Revitalizing language assessment: Sustainable change through international benchmarking and knowledge transfer (UQ Teaching Innovation Grant) Research Project Team Dr Kayoko Hashimoto Lecturer Japanese Associate Professor Marisa Cordella Chief Investigator Spanish Dr Juliana de Nooy Senior Lecturer French Dr Adriana Diaz Lecturer Spanish Dr Deanne Gannaway Lecturer, ITaLI Higher Education Dr Barbara Hanna Senior Lecturer French Ms Monica Waung Research Assistant Dr Noriko Iwashita Senior Lecturer Applied Linguistics Dr Paul Moore Lecturer Applied Linguistics Ms Jasmine Miller Research Assistant Ms Alicia Toohey Research Assistant Dr Anna Mikhaylova Lecturer Russian

UQ Student Strategy, Green Paper, 2015, p. 14 Project rationale Assessment and feedback are the least satisfying features of higher education, yet assessment is one of the most significant influences on students’ experience of higher education. UQ Student Strategy, Green Paper, 2015, p. 14 Language students’ satisfaction on courses and teaching experiences is rated the highest at UQ, however assessment and feedback are frequently flagged as an area in need of attention.

CEFR self-assessment grid (pre 2017) https://www.britishcouncil.mk/sites/default/files/self_assessment_grid.pdf ANOTHER OPTION VIDEOS: https://www.youtube.com/watch?v=KquYBc11FP0

What the project was An attempt to discover, describe and revitalize approaches to assessment in the school. recalibrate assessment practices across the School, to align with an internationally recognized standard, the CEFR. go beyond alignment in terms of level, and to use the CEFR to interrogate assessment practices in the school and ask whether assessment items actually allow students to demonstrate the competencies at those levels. give students a transferable way of describing their language level to employers and overseas institutions. provide a consistent student experience regardless of language. 9/10/18

What the project was not A claim that that the CEFR captures all the desirable learning outcomes of a university language course. An imposition of a single set of proficiency standards and a roadmap to get there. The imposition of a single assessment regime. An attempt to use the CEFR as a list of assessment items. A language levels framework is a series of language descriptions arranged sequentially to indicate an expected order of language progression or development over time. A framework is NOT a test. (O’Loughlin, 2009)

Process Information gathering and conversation starters Mapping the assessment regimes in all language acquisition subjects against the CEFR self-assessment grid in terms of level and weighting of skills Student questionnaires (introductory subjects) Student focus groups (introductory subjects) Teacher questionnaires Teacher focus groups Knowledge transfer Four staff workshops* on assessment. (*Paid participation for sessional staff) Mail out of Mapping results to all Discipline co-ordinators 9/10/18

FREN1020 Message on a Chat Forum 10% ( Example of the integrated assessment of linguistic accuracy and complexity) 9/10/18

Responses to mapping in the French programme A closer interrogation of the authenticity of some tasks (particularly re Spoken Interaction v Spoken Production) Greater coherence in level within and across subjects Each assessment task is now described in terms of the CEFR in the subject outline Reflection on the apparent disappearance of opportunities to demonstrate receptive skills e.g. listening and reading 9/10/18

CEFR mentioned in task descriptions Individual written activity (In-class test) Type: Tutorial Exercise Learning Objectives Assessed: 2, 5, 6 Due Date:           27 Aug 18 - 29 Aug 18     Week 6, first class Weight: 10% Task Description: In the first hour of the first class of Week 6 each student will read a short message and write a reply (110 words plus or minus 10%) in French.  The mark for a message which does not reach the minimum required length will be reduced proportionally and material beyond the maximum length will not be marked. This message will relate to Unité 7, which is about healthy living. This task allows you to demonstrate competence at low A2 level on the CEFR scales for Written Production and Reading Comprehension.  Permitted materials: Dictionary; verb tables. Students will also be provided either with clean photocopies of selected relevant pages from the textbook and exercise books or of a summary of useful structures studied. Not permitted: electronic resources, notes, students' own copies of the textbook and workbook. Each class will respond to a different prompt message. This is an in-class test : if you need to sit it at a time other than that scheduled for your class, you will need to apply for a deferred examination through SI-net. See "Deferred Exams" in Section 5.3 of this ECP for details. 9/10/18

Student questionnaire Pilot French, Russian and Spanish 2nd semester introductory October 2017 N=167 Main Chinese, French, German, Indonesian, Japanese, Korean, Russian, Spanish 1st semester introductory May 2018 N=429 (Questions drew upon Normand-Marconnet & Lo Bianco, 2015)

French students in the survey Highest scores for: (1) Do you know what the Common European Framework of Reference for Languages is Yes 69.47% (2) Do you know that the CEFR uses six levels to describe the proficiency of language learners? Yes 89.4% (3) French students were also the most likely to report having heard about the CEFR in class 85% of French (FREN1010) students (n=95) either agreed or strongly agreed that linking the assessment in their language course to the CEFR is or could be helpful As a language learner 78% In professional life after university 77% 9/10/18

International enrolments in introductory language subjects 26% of all enrolments in introductory language courses are international students 27% of enrolments in French introductory courses are international students ☞ We have a substantial cohort of students who are already internationally mobile. 9/10/18

One participant in the student focus groups commented that: They could see the CEFR as being useful for a big-picture overview rather than at the level of individual task preparation. (FG#7 debriefing notes, September 2018)

Charting a course for the future Better signposting from academic staff (re level; purpose of tasks; differences between subjects; progression across the curriculum) Equipping students with tools to signpost for themselves where their destination is and what it might take to arrive 9/10/18