1. TBLT 2007 A comprehensive model for speaking and oral interaction: theoretical en practical implications for the development of task-based language assessment instruments Kathelijne Jordens – Sara Gysen – Nele Maddens

2. Introduction Project: "To construct instruments for assessing the development of speaking and oral interaction skills of primary school pupils." Goal: -Signal deficits -Provide tips for extra support -Standardized assessment

3. Introduction 1. literature  theoretical model + 2. official learning goals PARAMETERS  type tasks  test tasks

4. Development: Questions: Which concepts and parameters in the model should be used for characterizing tasks? What are guidelines to ground this decision? How did we take into account the official language learning goals?

5. Selection of parameters Leading principals to construct an assessment instrument: 1. validity 2. reliability 3. efficiency

6. Theoretical model

7. Parameters describing the TASK Language user - Knowledge of the world - Knowledge of the subject Conversation partner - Knowledge of the world - Knowledge of the subject Relation between those two - Familiar or unfamiliar - Overlap frames of reference Conversation - conversation type (monologue, polylogue) - Subject - Formal/informal - Speech act Level of processing (describing, restructuring, evaluating)

8. Quality parameters

9. Quality Parameters Quality: performance - Relevance and completeness of the message (WHAT is said) - Clearness and organization (HOW pupils are speaking) - Fluency - Articulation - Manner of speaking - Variation and extensiveness language use - Correctness language use: word level / sentence level - Interaction skills in mostly monological situations - Interaction skills in dialogues en polylogues - Attitudes - Ability to manage - Reflection skills - Non-verbal communication

10. The official learning goals (OLGs) What and why? 4 language skills Construct, containing ten goals, describing expected speaking skills at the end of primary school Basis for didactics in class Every school in Flanders Task-based formulated Some parameters are set already (link with model) Link to type en test tasks

11. Development: 2 steps 1.Construct type tasks in relation to selected learning goals + investigate which parameters in the model we want to select to describe the type task 2.Fill out the parameters in the theoretical model for every test task and corresponding assessment model

12. Condition for validity Model – bridge (olg) – test task Overlap between the model and the official learning goals?

13. Example of an official learning goal Official learning goal 2.3: Pupils can use the appropriate language register when they report to the teacher on a subject, previously discussed at school. (level = describing) Parameters: Language user: pupil Speech act: to report Conversation partner: teacher Subject: familiar / previously discussed Level: describing

14. Condition for validity: overlap

15. Language user - Knowledge of the world - Knowledge of the subject Conversation partner - Knowledge of the world - Knowledge of the subject Relation between those two - Familiar or unfamiliar - Overlap frames of reference Conversation - conversation type - Subject - Formal/informal - Speech act Level of processing Official learning goal 2.3: Pupils can (level = describing) use the appropriate language register when they report to the teacher on a subject, previously discussed at school. Parameters: Language user: pupil Conversation partner: teacher Subject: familiar / previously discussed Speech act: to report Level: describing

16. The olg’s as a guide: constraint or help? Model – bridge (olg) – test task Constraint:  Only ten  Parameter values set already  Formulated in a rather strict way Help: selection of task based types, often frequented at school.

17. Concrete example Based on the official learning goals, we extract type tasks, and we characterize them with the parameters of the model. EXAMPLE Official learning goal 2.3: Pupils can (level = describing) use the appropriate language register when they report to the teacher on a subject, previously discussed at school. Corresponding type task: Pupils can report to the teacher on a subject, previously discussed at school + PARAMETERS

18. Parameters describing the type task Official learning goals: Language user: pupil Level: describing Speech act: to report Conversation partner: teacher Subject: familiar / previously discussed Model  type task: Interaction: teacher supports Turn taking: long turns

19. Development: 2 steps 1.Construct type tasks in relation to selected learning goals + investigate which parameters in the model we want to select to describe the type task 2.Fill out the parameters in the theoretical model for every test task and corresponding assessment model

20. Concrete example Based on the type tasks, we extract test tasks, and we characterize them with the parameters of the model. EXAMPLE type task: Pupils can report to the teacher on a subject, previously discussed at school + PARAMETERS Test task: Pupils can report to the teacher on a film about scuba diving, the report being based on photographs, extracted from the film.

21. Parameters describing the type task Official learning goals: Level: describing Speech act: to report Conversation partner: teacher Subject: familiar / previously discussed Model  type task: Interaction: teacher supports Turn taking: long turns

22. Scuba diving photographs Headlines task ‘scuba diver’

23. Step 2: type task to test task Parameters for assessment What does he say? Relevance and completeness Clearness Organization How does he say it? Fluency Articulation Manner of speaking Variation and extensiveness Correctness: word/sentence level Interactionskills Extra: Ability to manage Daring to speak =Quality of performance

24. Assessment model

25. Conclusion We think this set of parameters can guaranty sufficient validity because we showed how we carefully constructed our test tasks, selecting and combining parameters on theoretical and empirical basis. Questions remain unanswered, e.g. the role of context 1 versus context 2. Further investigation is needed in this domains.

26. links Model test task TBLT scuba diving Hawaii

27. Thank you! Questions, suggestions, remarks?  kathelijne.jordens@arts.kuleuven.be

28. Validity to represent as direct as possible to the official learning goals and model Step 1: constructs  type tasks: to describe type tasks that represent the official learning goals, using the parameters in the model. To take into account relevant parameters. step 2: we have made type tasks concrete, and kept parameters constant + construct assessment model based on the theoretical model Reduce memory, rehearsal, new subject

29. Efficiency: Step 1: from constructs to type tasks Conversation tasks in little groups if inherent to task Clustering learning goals in tasks Only crucial goals for school success Parameter values: teacher/peer, classroom, etc. Step 2: from type task to test task and assessment Well organized assessment model, no need of writing to much, Checklists Clear instructions for test takers

30. Reliability Standardization : Test taker’s performance: instructions for the test taker in the task itself and in the de manual reliability of the raters: inter- and intra rater reliability in the assessment model Parallel tasks for big classes Clearly described