1. Workshop LSP Assessment Christian Krekeler Konstanz University of Applied Sciences

2. LSP tasks for teaching LSP tasks for assessment1
LSP tasks for teaching LSP tasks for assessment

3. LSP teaching, LSP tasks
Task Renewable energy supply: Make a video of a news item on a catastrophe/problem with a source of renewable energy (water, wind, solar). Work in groups of three or four students.

4. LSP teaching, LSP tasks Complexity Complex tasks
Meaning Focus on a communicative goal
Authenticity Real-world relationship
Cognition Congintively demanding tasks
Skehan 1998; Ellis 2003; Van den Branden 2006

5. LSP teaching, LSP tasks Task Content Language Renewable energy Types
Applications
Efficiency …
Language
Presenting
Pronunciation
Genre
Terminology
Structures

6. LSP tasks and language assessment
Task Renewable energy supply gone wrong: Make a short video of a news item on a catastrophe/problem with a source of renewable energy (water, wind, solar). Work in groups of three or four students.
Why is this a problematic task for assessment purposes?

7. task design: adapting LSP tasks to assessment purposes2
task design: adapting LSP tasks to assessment purposes

8. TBLT and Assessment mismatch LSP TBLT LSP Assessment Complexity
Meaning
However, more often than not, tasks set for the assessment do often not incluce these qualities. They are often very different and there is a mismatch between the course content and the assessment. This should be avoided because assessements communicate to learners what is valued.
Authenticity
Cognition

9. Task-based language assessment
TBLA is a direct performance assessment which uses authentic, complex and cognitively demanding tasks. Such "holistic" assessments are scored with regard to the desired communicative outcome of the task.
My first point is therefore that assessment and couse content should be alingned and that TBL courses should include TBLA which can be defined as:
I will return to the scoring aspect later on in my talk.
Ellis 2003; Wigglesworth 2008

10. TBLT and Assessment LSP TBLT LSP Assessment Complexity Complexity
Meaning
Meaning
If a language course is based on the use of complex, meaningful, authentic and cognitively demanding tasks, this should be reflected in the assessment. This relationship works both ways, as the assessments will also impact the curriculum, causing positive washback.
Authenticity
Authenticity
Cognition
Cognition

11. LSP tasks and language assessment
Content and Language
Tensions
Task Difficulty
Generalizability
From an assessment perspective, it is positive to have a good performance sample from a learner. However, the are also tensions:
content and language: role of background knowledge, content as construct irrelevant variance
task difficulty: complex tasks, task features
generalizability: language use in other situations, rationale of performance testing
scoring: communicative goal
Scoring

12. LSP tasks and language assessment
Assessment Task Renewable energy gone wrong: Make a short video of a news item on a catastrophe/problem with a source of renewable energy (water, wind, solar). Work in groups of three or four students.
Alter/replace this task to make it more suitable for assessment purposes.

13. LSP tasks and language assessment
Alter/replace this task to make it more suitable for assessment purposes.
compare results
describe competences
reduce complexity
control input

14. LSP tasks and language assessment
Assessment Task The dam of a large reservoir used for hydroelectricity is leaking. Write and read a news item on the possible reasons, the efforts of the engineers to stop the leakage and the design of the reservoir.
Input: Use the chart "Hydroelectric dam".
Language …
Time …

15. 3
Scoring Lsp tasks

16. Scoring TBLA quantitative scoring general impression scoring
Focus on meaning and a communicative goal
quantitative scoring
general impression scoring
global/holistic scoring
Before we turn to rating scales, it may be useful to look at four different scoring methods and consider their usefulness for scoring LSP assessment tasks.
systematic/analytic scoring

17. Quantitative scoring Counting errors Error ratios  objective, safe
 not suitable for scoring complex LSP tasks
for the very insecure teacher, who rates exams with the lawyers in mind, knowing that a score based on an error ratio is easily justifiable.

18. General Impression Scoring
No fixed criteria
unsystematic, difficult to legitimize
improve accuracy by comparisons
flexible, open mind

19. Global/holistic scoring Systematic/analytic scoring
Rating scales
Rating scale with global descriptors
Global score for performance
Rating scales
Sub-scales for different features
Separate scores
Weighted overall score

20. Rating scale development
Write descriptors for a rating scale for the task on the reservoir dam.

21. Rating scale development
Specific descriptors – not:
Can understand the main ideas of complex text on both concrete and abstract topics, including technical discussions in his/her field of specialisation. Can interact with a degree of fluency and spontaneity that makes regular interaction with native speakers quite possible without strain for either party. […] CEFR
for the overly confident teacher: I know a good picture when I see one.
At least they are keeping an open mind, wich may be useful because there are a wide range of performances.
It is also a flexible method which can take unforseen strenghts or weaknesses into account.
However, from a measurement perspective, GIS is a no go.

22. Rating scale development
a priori method
rating scale developed by experts
"armchair approach"
a posteriori method
descriptors derived from observations of performance
behavioural rating scales
theory-derived rating scales
Taylor 2011

23. Rating scale development
Write descriptors for a rating scale for the task on the reservoir dam.
specific, based on task
observable
Would you use your descriptors for global/holistic scoring or systematic/analytic scoring?

24. 1 tasks and assessment 2 task design 3 Scoring LSP tasks
LSP Assessment
1 tasks and assessment 2 task design 3 Scoring LSP tasks