1. Online CEF-Based Assessment of Oral Proficiency for Intercultural Professional Communication Lutgart Dusar Prof.dr Lut Baten dr. Jan Van Maele 2-12-2010www.cefcult.eu

2. 1. CEFcult, the sequel With CEFcult we want to contribute to ongoing development & validation of CEFR (scaling language proficiency) and INCA (intercultural competence assessment) frameworks a new e-learning culture bringing together self- learning, the worlds of work and education mainstreaming e-Portfolio-based assessment 2-12-2010 Www.Cefcult.Eu

3. 2. From CEFR (‘93-96) to WebCEF (‘06-09) WebCEF established a social learning community for self, peer and expert assessment in which both productive and interactive performance in a foreign language is video recorded and online displayed to be assessed by CEFR-scales. http://www.webcef.eu/ 2-12-2010www.cefcult.eu

4. … to CEFcult (2009 – 2011) + Intercultural Communicative Competence Assessment: Allowing a set of communication tasks (scenarios) through which foreign language learners can assess their speaking skills for intercultural professional purposes Map the road to a tested and viable model for online collaborative assessment among HEi, enterprises and various partnership brokers. 2-12-2010www.cefcult.eu

5. e-Portfolio (Europass: Language, Mobility) Video sample Scenario Scenario Database Social media (LinkedIn) Export to: Self assess Self assess Peer assess Peer assess Expert assess Expert assess Select scenario Add profile Create sample Follow scenario Follow scenario Feed- back Invitation to: Add scenario 2-12-2010www.cefcult.eu CEFcult User Groups (Alumni,..)

6. 3. Assessing ‘Intercultural Competence’ The model: INCA (EU 2001-04) criterion based assessment model, linking Byram’s framework to learning needs of (engineering) industry Scaling: to what extent INCA criteria including value systems, are developed and put into action: Full – Intermediate – Basic – Respect for otherness – Ambiguity tolerance – Knowledge discovery – Empathy – Behavioral flexibility – Communicative Awareness Warning “INCA tasks and grid are tentative “ 2-12-2010www.cefcult.eu

7. - INCA based online scenarios 2-12-2010www.cefcult.eu Simulations designed in a way to guarantee fairness, reliability and accuracy Work-sample approach: a selected real-world task transformed into assessment task Challenge: lack of definition in INCA descriptors

8. - Different values, different ratings ‘I am honest’: – Considered a quality in low context cultures: ‘we say what we mean, we mean what we say’ – Considered ‘blunt’ in high context cultures where a meaning is stated in an indirect way What matters: cultural (self) awareness: O bserve D escribe I nterpret S uspend your judgment, look for and verify other perspectives before E valuating (S.Ting Toomey) 2-12-2010www.cefcult.eu

9. - Pilot Leuven ‘Screening interview’1 st experiences (October, November, December 2010) Positives Descriptors are written generically, can be used with many tasks Corresponding rating criteria determined through collaboration of language assessment specialists and (lay) stakeholders from the workplace Expert raters from the world of work tend to apply scales consistently, resulting in more reliable measurement. Critical Summative work-sample testing mainly indicates a degree of overall communication competency, much less in detail Performances may meet INCA criteria in two or more categories, making it difficult to select the one best description Peer reviews (student community) do not provide specific feedback to test takers about the strengths and weaknesses of their performance. 2-12-2010www.cefcult.eu

10. - Task specific descriptor writing & review cycle Peer assessment Task descriptor acceptable, understood? Flag (What’s the problem?) Comment explanation recommendation Task specific Yes No Internal review

11. 4. A Tested Model Specification Experiment with INCA scales for the most meaningful feedback Outcomes: development of graphic profile, bank of scales Standard Setting Benchmarking Test validity: getting the CEF&INCA rating levels right Validation Writing a living ‘CEFcult Technical manual’ containing recommended usage and flexible implementation guidelines 2-12-2010www.cefcult.eu

12. 5. A Viable Model Outcome: e-Portfolio based assessment 1)The emergence of new global actors will make our world more colorful and more polyvalent. There is a need for foreign languages learning, inter-personal & inter-cultural monitoring and development tools in a safe, respectful environment. 2)Existing ICC-assessment tools mostly address criterions and references for individual competence development; not enough the organizational needs and seldom those of development partnerships (N-S). 3)E-Portfolio tools must go mainstream, embedded in the internal learning and monitoring processes of companies, organizations and higher education institutions. 2-12-2010www.cefcult.eu

13. References John De Jong & Glyn Jones, Getting the levels right: deriving item writer guidelines from CEFR descriptors (Pearson, Item Writer Guidance-extract ppt) Peter Lenz and Raphael Berthele, Assessment in Plurilingual and Intercultural Education. Paper Policy Forum ‘The right of learners to quality and equity in education – The role of linguistic and intercultural competences’, Geneva, Switzerland, 2-4 November 2010 Byram … 2-12-2010www.cefcult.eu www.mymahara.com

14. References Peter Lenz and Raphael Berthele, Assessment in Plurilingual and Intercultural Education. Paper Policy Forum ‘The right of learners to quality and equity in education – The role of linguistic and intercultural competences’, Geneva, Switzerland, 2-4 November 2010 Elisabeth Prechtl and Anne Davidson Lund, Assessing and developing intercultural competence. Perspectives from the INCA Project in Handbook of Intercultural Communication, Section 5, ch.22, p.467- 490(2007) eBook ISBN: 978-3-11-019858-4 John De Jong & Glyn Jones, Getting the levels right: deriving item writer guidelines from CEFR descriptors (Pearson, Item Writer Guidance-extract ) www.cefcult.eu - www.incaproject.org - http://www.webcef.eu/ www.cefcult.euwww.incaproject.org http://www.webcef.eu/ 2-12-2010www.cefcult.eu