1. ACE TESOL Diploma Program – London Language Institute OBJECTIVES You will understand: 1. Concepts in language assessment and testing theory. You will be able to: 1. Use your knowledge of language assessment and testing theory to more effectively develop tests and assessments for your students.

2. ACE TESOL Diploma Program – London Language Institute ASSESSMENT VERSUS TEST The words test and assessment are sometimes used interchangeably when we talk about how to evaluate a student’s language knowledge, ability or performance skill. They are, however, slightly different. A test is “a set of questions, problems, or the like, used as a means of evaluating the abilities, aptitudes, skills, or performance of an individual or group” (www.dictionary.com). Further, a test is limited to a specific given domain, which can be knowledge, ability or performance. Students taking a test are usually forewarned about it and know ahead of time that they are being evaluated.

3. ACE TESOL Diploma Program – London Language Institute An Assessment is a larger process and encompasses much more subject matter, or more technically, a larger domain. Assessment is any means with which we determine whether students are learning or not. A test is just one way for teachers to assess students. If students answer a question orally, write in a journal, or complete a project, the teacher is in a position to evaluate their learning. A teacher is literally assessing students all the time. We can divide assessment into informal assessment and formal assessment:  Informal assessments are the evaluations made of students on a daily basis, based upon classroom participation and homework.  Formal assessments are tasks that are deliberately set, with clear guidelines, to assess students at specifically designated times in a course.

4. ACE TESOL Diploma Program – London Language Institute FORMATIVE AND SUMMATIVE ASSESSMENT We can also divide assessment into formative assessment and summative assessment.  Formative assessment is conducted throughout a course to make sure that students are on track with the learning process.  Summative assessment is conducted at the end of a unit or course to determine what students have learned as a result of the whole course of study. While formative assessment is generally informal, it can also be formalized. Summative assessment is generally always formal.

5. ACE TESOL Diploma Program – London Language Institute TESTING There are some key components to the definition of a test that need to be looked at in more detail. The test, first of all, has a method. Each test a teacher designs uses a specific method for determining if whether or not students know the language being tested. Secondly the test measures something. As a teacher, be clear in your mind what it is you are measuring. Are you measuring ability (competence), knowledge or performance? You must also be clear about what domain you are measuring. For example, are you measuring students’ ability to ask questions, to answer them, or both? Finally, you must keep in mind that with a test you are measuring the abilities of individual students; a test must match who test-takers are in terms of ability, background, and experience.

6. ACE TESOL Diploma Program – London Language Institute NORM-REFERENCED AND CRITERION-REFERENCED TESTS Tests can be divided into norm-referenced tests and criterion- referenced tests. In norm-referenced tests, each student’s score is interpreted in relation to the mean score of the whole class, the median score of the whole class, or in terms of a percentile. In criterion-referenced tests, student scores represent the degree to which they have mastered the specific objectives that the test assesses. These test scores are expressed in terms of a percentage, a mark out of a total, or a letter grade.

7. ACE TESOL Diploma Program – London Language Institute OBJECTIVE-BASED TESTS  Diagnostic tests to determine what a student already knows. The results are used either as a guide to designing the course or to place students in an appropriate level at the start of a course.  Progress tests are used to evaluate how successful your teaching has been and how much learning has taken place. They are also used to provide encouragement and positive feedback to students.  Achievement tests are used to evaluate how successful a course has been, to provide students with feedback on what has been learned, and to provide students with a grade at the end of the course.  Proficiency tests are used to provide an objective and standardized measure of a student’s capacity for the domain, in our case for the language. Examples of proficiency tests include TOEFL, TOEIC, IELTS and the Cambridge exams.

8. ACE TESOL Diploma Program – London Language Institute The three main delivery options for testing include the traditional pen and paper type test and computer-based, either via specific software found on a CD ROM, or the Internet. Tests delivered using a computer can either be static (they don’t change) or adaptive (the questions change according to how the student answers the question). There are also many choices when it comes to test tasks or test questions: multiple choice; fill in the blank (cloze); matching; re-ordering; transforming (changing the sentence structure from one form to another); paraphrasing (expressing the idea in different words); long answer; and essay questions. Test tasks or questions can also be divided into discrete questions (testing only one small, specific piece of knowledge) and integrative questions (testing a student’s ability to integrate new knowledge with existing knowledge).

9. ACE TESOL Diploma Program – London Language Institute FINAL CONSIDERATIONS There are two final considerations when creating and administering tests, which are more to do with the psychology of test-taking than the technicality of writing and delivering tests. In order to be effective, tests need to be authentic and motivating. Students need to perceive the tests as a necessary and real endeavour. We also need to factor in the stress that tests can induce in some students: If a student panics when writing a test, the test result will hardly reflect the student’s ability. Thus, we also need to ask ourselves how we can make tests a positive experience for students that will bring out their best, rather than their worst.

10. ACE TESOL Diploma Program – London Language Institute PRINCIPLES OF LANGUAGE ASSESSMENT A test must first of all be practical, meaning the following:  It is not too expensive in terms of development, materials and staffing costs  It can be completed within the time frame allotted for testing  It is easy to administer  It is easy and fast to mark A test must also be reliable. A reliable test is consistent and dependable. Reliability essentially answers this question: If the same student took the same test at a different time, and it was evaluated by a different marker, would the student’s results on the test be the same?

11. ACE TESOL Diploma Program – London Language Institute There are four types of reliability that we need to consider:  Student-Related Reliability: If the student is ill, tired or hungry when writing the test, they would obviously not fare as well as if they were NOT ill. However, there is little the teacher can do to control for this. The teacher CAN however control student anxiety to some extent. The teacher also needs to be wary of whether students can pass the test simply by using test-taking strategies, rather than by actually understanding the material.  Rater Reliability: Rater reliability can be divided into inter-rater reliability (two raters are not scoring the same) and intra-rater reliability (the same rater, because of fatigue, starts introducing inconsistencies into their rating). The former type of reliability can be addressed by having what we call norming sessions. All raters get together and decide how they rate different responses to the questions. The latter type of reliability can be addressed simply by the rater taking a break once in a while and then going back to previously marked tests to ensure there is consistency.  Test Administration Reliability: This is reliability that results from the testing conditions being consistent from one test to another, or within the testing environment for the same test. For example, all students should be able to see and hear equally well. All students should be equally warm or cool, and have equally comfortable chairs and tables.  Test Reliability: This is reliability that results from the test structure itself. This includes the test length (if it is too long, students will get tired) and test questions (are they clear or ambiguous?).

12. ACE TESOL Diploma Program – London Language Institute A test must be valid: Does the test actually test what you want it to? If you want to test listening, are you actually testing reading? There are several different types of test validity:  Content validity indicates that there is a match between the subject matter or content that you want to test and the subject matter or content that is actually in the test. If you want to test students on the history of World War II, then the content of your test should reflect this.  Criterion-related validity is the extent to which the test indicates whether students met the learning objectives. If a test has criterion-related validity, then we can say that a student with an 80% on the test knows 80% of the material that the teacher wanted to test. If a test does NOT have criterion-related validity, we can NOT say that a student with an 80% on the test knows 80% of the material.

13. ACE TESOL Diploma Program – London Language Institute VALIDITY CONTINUED  Construct validity is a slightly more difficult type of validity to attain. If a test has construct validity, then the test is constructed according to the underlying principles of the instruction that students are receiving. For example, you have a language class that you are teaching using a variation of Communicative Language Teaching (CLT). For your test to have construct validity then, your test must also be structured according to the underlying principles of CLT. Communicative language teachers everywhere are guilty of not meeting construct validity with their tests. Does a paper test that requires students to change a sentence from the present simple to the simple past have construct validity if the teacher has been teaching according to communicative language teaching methodology? The answer is no. To have construct validity, the test would have to assess student ability to use the simple past accurately and fluently in a real communicative situation.

14. ACE TESOL Diploma Program – London Language Institute FURTHER TYPES OF VALIDITY  Consequential validity answers the question of whether the outcomes of students taking the test are valid and useful. If students do not improve their language ability as a result of preparing and taking the test, then the test is said to have poor consequential validity. The most analyzed and discussed type of consequential validity is called washback. Washback refers to how the testing affects the teaching and learning process that goes before it. A test has positive washback if the teaching and learning processes improve as a result of the upcoming test. A test has negative washback if the teaching and learning processes weaken as a result of the upcoming test. The well-known phenomena of teaching to the test is a classic example of negative washback.  Face validity answers the question of whether the test actually looks like a fair, relevant, and useful test. If not, then students will actually not take the testing experience seriously, nor will they value the test results.

15. ACE TESOL Diploma Program – London Language Institute AUTHENTICITY Another principle of language assessment is authenticity. Authenticity is the degree to which a test task mimics how students would use the knowledge or skill in the real world. A task that closely matches a real-world task (such as writing an e-mail) is said to have high authenticity. A task that does NOT closely match a real world task (such as changing a sentence from simple present to simple past) is said to have low authenticity. The more authentic a test task, the more motivating and relevant it is.

16. ACE TESOL Diploma Program – London Language Institute Complete Task Journal questions 1 and 2 and submit via email to jenrjones@rogers.com (preferred) or print and hand in. **For Question 1, choose three question formats and create a sample question of each, (i.e. one multiple choice, one matching, and one re-ordering) for a total of three questions.