Testing testing: developing tests to support our teaching and learning Martin Wedell School of Education University of Leeds UK

Everybody cares about tests if people in a change context (parents, learners, teachers, institutional leaders) see an obvious lack of consistency between the behaviours/practices underlying the proposed changes and those that are perceived to help learners pass high stakes exams, it is the practices that support success in exams that will ‘win’. (Wedell 2009 :25)

What and how we test can influence teaching Washback- the effect of tests on what teachers teach – the parts of the language they emphasise and what they ignore. How they teach it a) they only allow learners to repeat correct sentences from the text book. b) they try to provide opportunities for learners to use what they know more freely ( even if incorrectly).

what and how we test can also influence learning Learners notice What is in the tests, and what aspects of language are emphasised by teachers . How the teachers expect them to learn a)The teacher will be angry if I get something wrong, so I will be very careful never to make any mistakes. b) The teacher encourages us to try and use what we know to talk/write about what we are interested in, even if we sometimes get it wrong.

What and how we test, can influence how teachers and learners view English a) English is just another school subject English is a set of knowledge to be learned and memorised. Knowledge called ‘grammar’ Knowledge called ‘vocabulary’ Knowledge called ‘pronunciation’ Knowledge called ‘spelling’ b) English is a language, like Burmese. The reason for learning the knowledge is to talk /write /read about and listen to what I and other people are interested in.

Testing habits are difficult to change BUT teachers can begin to try to make whatever tests they write / give to their learners as good as possible.

3 features of any ‘good’ test A good test is one for which the answer to’ each of the following questions is ‘YES a) VALIDITY- Does everyone understand what is being tested, and is it being tested appropriately ? Can the teacher explain what a learner who does well in a test does know/can do? b) RELIABILITY -Are the test questions and the way the test is administered fair to all the learners? Are teachers confident that whatever mark is given to each learner is accurate? c) PRACTICALITY -Can the test actually be administered and marked as planned, in the real classroom?

a) Is the language test valid? (1) For a test to be ‘good’- teachers/learners need to be clear about what it is testing. … Grammar- what grammar? Vocabulary.. Which vocabulary? Anything else- What else? If nothing else, then it can only claim to be a test of grammar and vocabulary.. If we claim it is testing anything else we are pretending. For a test to be ‘good’ teachers and learners have to be able to see that it is actually testing what it says it is.

a) Is the language test valid (2) For a test to be ‘good’ , it needs to test whatever it claims to test, through asking learners to ‘do’ things that a normal human being might do. (grammar-vocabulary-reading comprehension?) If we only test if learners can recognise the ‘right’ grammar/vocabulary item (MCQ), then we are only testing PART of what a person would normally do. To be a valid test of what learners know about grammar and vocabulary , it also needs to test if they can use their grammar/vocabulary knowledge in new language settings.

a) Is the language test valid (3) What can a teacher say about what each of these learners knows/can do in English ? a learner who gets a high mark on a grammar and vocabulary test that asks her/him i) to fill gaps in sentences that the s/he has already seen in the textbook? ii) to fill gaps in sentences that s/he has never seen/heard before?

b) Is the language test reliable (1) for a test to be ‘good’ we need to be sure that if the same learners took the same test at a slightly different time, they would get the same marks. To be able to say this depends partly on how we design the test itself and partly on how it is marked.

b) Is the language test reliable (2) TEST DESIGN ‘The more the merrier’- give the learners as many chances as possible to show what they can do. The more separate, different, types of questions about the same/similar grammar or vocabulary , the better. Make sure everybody does exactly the same test. All learners understand what they are supposed to be doing in the same way. Clear instructions No tricks Same time, same space, same (dis) comfort.

b) Is the language test reliable? (3) MARKING The markers need to mark in exactly the same way ( can any test be ‘good’ if markers mark differently?) The more ‘objective’ an answer is, the easier it is to get all markers marking identically.. Even a machine can do it (TOEFL) What does objective mean ? 1 possible correct answer only? A limited number of answers ? (do all the markers agree on these?)

c) Is the test practical? (1) In your context Can the test be administered easily in terms of time and space? without costing too much? Is the test easy to mark? Are the test results easy to interpret?

a)Validity, b) Reliability and c) Practicality, a) If a test has low VALIDITY it may be unclear exactly what forms /skills were actually tested, and so will be difficult to say much about what a high (or low) mark means that a learner knows ( or can do). b) If a test is not RELIABLE it is difficult to be sure if the mark a learner got, is an accurate measure of his/her ability in whatever was tested. c) If a test is not practical, then however ‘good’ it is we will not use it.

High marks on a reliable test with low validity Validity or Reliability – how might their lack affect language teaching and learning? (1) High marks on a reliable test with low validity We know that somebody with a high mark is ‘better’ than someone with a low mark- but we do not know what they are better at. High marks on a valid but unreliable test We seem to know what each learner knows/can do, but we cannot be sure that they REALLY do, because they might score differently if they did it again.

Validity or Reliability – how might their lack affect language teaching and learning? (2) If we decide that reliability is most important and so decide only to use objective test items- we limit ourselves in what we can test… grammar, vocabulary , factual reading and listening comprehension YES,.. writing and speaking NO. Effect on teaching and learning will be the temptation to ignore ‘productive skills’ because they are not tested.

Validity or Reliability – how might their lack affect language teaching and learning? (3) If we emphasise validity , and decide try to test forms and skills in as ‘authentic’ a way as possible Effect on teaching and learning will be to make developing ‘skills’ and using language to ‘communicate’ more important to both teachers and learners, BUT it may mean that tests are more difficult to administer and mark (less practical) , and that learners ( and their parents), complain that the test results are not objective, and so not fair.

Language Tests that support teaching There is no perfect language test. Validity, reliability and practicality are in conflict ( to some degree) in ALL tests. Each teacher needs to decide what the purpose of teaching English is, in their classroom. .. and then needs to try to balance the three aspects of a ‘good’ test , so that the tests they give provide as much support for their teaching goals as possible.

Starting point = what we already do in our local context Adjusting how we test.. to support how we want to teach does not mean tests have to be completely different… Starting with small changes to our MCQs, gap fills and matching exercises… Can we adjust how we design these, to allow learners to show how well they can use what they know, beyond what they are familiar with from the textbook?

(English) Tests are powerful All our education systems /societies have a tendency to ‘worship’ test results Passing tests can become the main purpose of all (English) teaching and learning. Education systems need to encourage /show teachers how to ‘tame’ tests , so that they support the goals of helping learners to begin to use English..