TASK-BASED ASSESSMENT: INTEGRATING RESEARCH AND PROFESSIONAL DEVELOPMENT Geoff Brindley, Stephen Moore, Helen Slatyer & Steven Ross Macquarie University, Sydney, Australia International Conference on Task-Based Language Teaching Leuven 2005

Background: The Certificates in Spoken and Written English (CSWE)  The Adult Migrant English Program (AMEP)  Competency-based curriculum and assessment framework  Explicitly described learning outcomes at four proficiency levels

CSWE assessment: Validity and reliability issues  Construct validity (cf listening/ reading descriptors: can invisible operations be described?  Comparability of teacher-developed tasks  Consistency of rater behaviour  Generalizability of skills across task types

Research findings  Speaking -Effects of interlocutor behaviour on test scores -Effects of task type on difficulty (cf structured vs - unstructured)  Listening - Effects of task characteristics on scores (cf speed, item format)  Writing -Problems with interpretability of performance criteria -Rater halo effect  Rater inconsistency

Research findings (contd) TASK DIFFICULTY: TEACHER PERCEPTIONS OF FACTORS AFFECTING DIFFICULTY  Lack of fit between task and text  Task type design and cognitive load  Ambiguity in question wording

Research findings (contd) LEARNER PERCEPTIONS OF FACTORS AFFECTING TASK DIFFICULTY  Match between text and question  Ambiguity in question wording  Lexical knowledge

Creation of an assessment resource for teachers Aims  To develop a bank of trialled assessment tasks to be made available on-line  To create a program of professional development in assessment task design

Creation of an assessment resource (contd) Task development process  Tasks sourced from teachers or new tasks developed by NWG  Tasks vetted, modified and piloted by NWG  Tasks moderated by panel of teachers and academics  Tasks trialled nationally

Addressing professional development needs  Professional development program derived from task development workshops  Devolvement of responsibility to teachers and educational institutions  Workshops provide a forum for exchange “It has been extremely useful and rewarding to be involved in the national think-tank” “(one of the most valuable aspects is) the opportunity to meet people round the country”

Future developments Task bank  Improve the quality of tasks  Increase the number of tasks  Extend coverage of LOs to acccommodate curriculum changes

Future developments (contd) Professional development program  Online PD kit: resources for local PD -Choice of 3 workshops 1. Task design issues 2. Task quality evaluation 3. Development of new tasks

Future developments (contd) Professional development program  PD kit includes Powerpoint presentations, assessment materials and readings  Develop workshop formats according to needs  Develop interactive self-access package

Future developments (contd) Research  Refine think aloud protocol analysis  Investigate effect of specific task/item types on perceptions of difficulty

Conclusions Four key principles underpin project:  Partnership between educational institutions, teachers and university research centre  Devolvement of responsibility to institutions & teachers through workshop program  Sustainability of the project through a commitment from stakeholders in government and educational institutions  Ongoing program of collaborative research