1. ACE TESOL Diploma Program – London Language Institute OBJECTIVES You will understand: 1. How to explain and analyze various methods of assessing advanced grammar students, including discrete and integrated items. 2. How to evaluate the accuracy and effectiveness of sample grammar tests. 3. How to develop advanced grammar test questions/activities. You will be able to: 1. Effectively assess advanced grammar students.

2. ACE TESOL Diploma Program – London Language Institute INTRODUCTION For advanced grammar students, the learning curve has often leveled off and progress can be more difficult to measure than at lower levels. Assessment should therefore take into account the complexity of advanced learners’ knowledge and ability. Although traditional grammar testing tends to focus on mechanical questions, a more accurate picture of advanced students’ proficiency can be obtained through realistic tasks using authentic grammar in context.

3. ACE TESOL Diploma Program – London Language Institute TEST TYPES There are generally four types of tests which can be used to evaluate grammar and other areas:  Placement: Used to place new students in the correct class, and generally test grammar, vocabulary and productive and receptive skills.  Diagnostic: Used to determine learners’ weaknesses, gaps and difficulties to address students’ needs in the classroom.  Progress or Achievement: Used to test students according to their course syllabus. Progress tests are used at various points during the course; achievement tests are used at the end of the course or term. These tests should try to demonstrate what students have learned.  Proficiency: Used to give a general view of language knowledge and ability; often used as criteria to obtain employment, certification or admission to an educational institution. Proficiency tests include commercial tests like TOEIC, TOEFL, CAEL, and IELTS.

4. ACE TESOL Diploma Program – London Language Institute When designing tests, teachers should consider the validity and reliability of the test. Validity means that a test actually tests what is intended. For example, a grammar test question is not valid if it requires students to know vocabulary or factual information that wasn’t taught in class. Reliability means that a test yields consistent results. If the same students take the same test more than once, the results should be essentially the same each time. A reliable test should also have clear instructions and should be scored consistently by different evaluators.

5. ACE TESOL Diploma Program – London Language Institute In The Practice of English Language Teaching (Chapter 23), Harmer discusses the distinction between direct and indirect test questions. A direct item tries to replicate real-life language use, and asks students to demonstrate a specific communicative skill. In contrast, indirect questions use controlled items to assess students’ knowledge. Indirect question types may include gap-filling, choosing the correct form of a verb or other word in sentences or texts, and finding errors in sentences, along with the following:

6. ACE TESOL Diploma Program – London Language Institute TYPES OF INDIRECT TEST QUESTIONS  Multiple-choice: Very easy to score, but “choices” can unfairly distract students. Also, performance may be based more on test-taking strategies than language ability.  Cloze exercises: Every nth (5th-10th) word is randomly deleted from a text, and students have to fill in the gaps. The task is fairly integrative because it tests a variety of grammar and other points. *In non-random cloze tasks the teacher can ensure that the deleted words are ones the students have learned.  Transformation/paraphrasing: Students are asked to re-write sentences using different grammar forms while maintaining the original meaning. Teachers can assess the learners’ knowledge of the item being tested. For example: I’m sorry that I didn’t study for the test. = I wish ____________________________________.  Sentence ordering: These questions are fairly easy to write and test knowledge of syntax and lexical-grammatical items, as students have to put words in order to form a correct sentence.

7. ACE TESOL Diploma Program – London Language Institute DIRECT TEST QUESTIONS With direct questions, validity and reliability can be a challenge. Particularly in speaking or writing tests, teachers should ensure that students don’t require specialized knowledge to answer a question successfully. This is also true with receptive skill (listening and reading) testing, where very technical or specialized texts unfairly test students’ general knowledge and vocabulary. Receptive test questions should also not inadvertently test production by requiring students to perform well in writing or speaking.

8. ACE TESOL Diploma Program – London Language Institute SOME DIRECT QUESTIONS MAY INCLUDE:  Speaking: students are “interviewed” about themselves or asked to role-play a phone conversation to make a doctor’s appointment.  Writing: students write a reply letter to a job advertisement or instructions for a common task.  Reading: inserting sentences into the correct place in a text, choosing best summary of text or matching jumbled headings to paragraphs.  Listening: completing charts with information from text, identifying which speaker says what, following directions on a map. While indirect questions are easier to prepare, generally more objective, and thus yield more reliable scores, direct questions can be better at assessing how students can use language in the ‘real world’.

9. ACE TESOL Diploma Program – London Language Institute ANOTHER DISTINCTION BETWEEN TEST ITEMS Test questions can also be either discrete or integrated. Discrete-item questions test only one thing at a time, for instance by having students choose the correct preposition to complete a sentence. Integrated questions require students to use a variety of language to complete a task, such as writing a story or doing an oral test. *What is the value of using a variety of direct and indirect, discrete and integrated test items to assess students?

10. ACE TESOL Diploma Program – London Language Institute CREATING AND SCORING TESTS When writing grammar tests, the teacher must decide:  The content, (language and skills) to be tested  The types of questions to be included  The time it will take for students to write the test  The number of marks allotted to each question To make sure a test will “work” and yield accurate results, it is very useful to have colleagues or other students try the test. A detailed marking scheme helps to ensure consistency when scoring a test. This is especially important with more subjective, direct and integrated grammar test questions. When testing productive skills, a teacher should prepare an assessment scale with descriptions of performance at each level. Performance can also be analyzed in greater detail, by using a set of criteria designated in a rubric, where each criterion has a specific rating scale.

11. ACE TESOL Diploma Program – London Language Institute FOR EXAMPLE: (an integrated speaking test) CriteriaScore from 1-5 (see below for details) Pronunciation Fluency Vocabulary Grammar Task Completion ScoreDescription 1Student speaks hesitantly in short, uneven bursts 2Student speaks slowly with many pauses 3Student speaks at comfortable speed with many hesitations 4Student speaks at comfortable speed with occasional hesitations 5Student speaks quickly with very few hesitations

12. ACE TESOL Diploma Program – London Language Institute OTHER ISSUES IN TESTING GRAMMAR Some classes are specifically intended to prepare students to write tests like the CAEL or IELTS. Test-preparation and test- taking strategies are the focus of such classes. In other classes, while teachers may think of tests as isolated events, they can in fact be effectively integrated into a course.

13. ACE TESOL Diploma Program – London Language Institute HOW CAN TESTS FURTHER IMPACT LEARNING? Thornbury describes two positive benefits of testing: Progress tests in particular have positive spin-off. This means that tests can be used to help a teacher determine what material requires further review. Test questions that were answered poorly can be used in future classes as a basis for revision. Performance (or direct, or integrated) tests can have a beneficial backwash effect. This means that students will spend at least as much time on communication as on grammar when preparing for these types of tests. Both spin-off and backwash are positive factors associated with testing.

14. ACE TESOL Diploma Program – London Language Institute Complete Task Journal question 1 in class. Task Journal question 2 and 3 can be submitted via email to paula@llinstitute.com (preferred) or printed and handed in. paula@llinstitute.com