1. Limitations and Future Directions of Tests

2. Test Interpretation and Use A valid test involves valid interpretation and valid use of the test scores A valid test involves valid interpretation and valid use of the test scores E.g. TOEFL – what was it constructed for? E.g. TOEFL – what was it constructed for? Making high-stakes admission decisions to North American universities Making high-stakes admission decisions to North American universities Can you accurately interpret a score to reflect what it claims it wants to measure? Can you accurately interpret a score to reflect what it claims it wants to measure? Will you be able to use the test scores for the purpose the test was constructed? Will you be able to use the test scores for the purpose the test was constructed? This issue refers to Consequential Validity This issue refers to Consequential Validity

3. Limitation of tests (Bachman) Subjectivity Subjectivity In setting of questions as well as grading In setting of questions as well as grading Underspecification of domain Underspecification of domain Not everything taught or about the subject can be included in a test Not everything taught or about the subject can be included in a test Incompleteness (from students’ perspective) Incompleteness (from students’ perspective) Not everything the student knows can be demonstrated Not everything the student knows can be demonstrated Indirectness which involves all tests Indirectness which involves all tests Imprecision in the grading and the points alloted for each item/question Imprecision in the grading and the points alloted for each item/question

4. Attaining Acceptable Standards (Linn, Baker, Dunbar, 1991) Consequences Consequences Does the test have positive consequences? Does the test have positive consequences? Fairness Fairness Is the ethnic and cultural background of the student taken into consideration? Is the ethnic and cultural background of the student taken into consideration? Transfer and generalisability Transfer and generalisability Does the assessment support accurate generalisations about student ability? Does the assessment support accurate generalisations about student ability? Cognitive and linguistic complexity Cognitive and linguistic complexity Does the test require the student to use complex thinking, problem solving and an appropriate level of linguistic complexity? Does the test require the student to use complex thinking, problem solving and an appropriate level of linguistic complexity?

5. Attaining Acceptable Standards (Linn, Baker, Dunbar, 1991) Content quality Content quality Is the selected content representative of the current understanding of the construct? Is the selected content representative of the current understanding of the construct? Content coverage Content coverage Are the key elements of the curriculum covered? Are the key elements of the curriculum covered? Meaningfulness Meaningfulness Do the students feel that the tasks are realistic and worthwhile? Do the students feel that the tasks are realistic and worthwhile? Cost and efficiency Cost and efficiency Is the information students receive through the test worth the time and money to obtain? Is the information students receive through the test worth the time and money to obtain?

6. Future Directions in Tests and Evaluation

7. Professionalism of teachers “Informed” assessment by teachers “Informed” assessment by teachers Need knowledgeable teachers who “systematically observe and selectively document their students’ performance through multiple methods, across diverse contexts, and over time as students participate in meaningful learning activities” (Wolf, 1993: 519) Need knowledgeable teachers who “systematically observe and selectively document their students’ performance through multiple methods, across diverse contexts, and over time as students participate in meaningful learning activities” (Wolf, 1993: 519)

8. Integration of communicative elements Communicative – authentic Communicative – authentic Authenticity is in implementation NOT design Authenticity is in implementation NOT design Communicative tests involve performance, are authentic and are scored on real-life outcomes (Fuller, 2000) Communicative tests involve performance, are authentic and are scored on real-life outcomes (Fuller, 2000)

9. Computer based assessments E.g. TOEFL – TOEFL CBT; TOEFL iBT E.g. TOEFL – TOEFL CBT; TOEFL iBT Computer adaptive tests Computer adaptive tests Ease of scoring and delivery Ease of scoring and delivery Changes on the part of the test taker? Changes on the part of the test taker? Validity of computer based tests? Validity of computer based tests?

10. Specialised examinations Literature, ESP, EST Literature, ESP, EST Contextualised – for specific purposes Contextualised – for specific purposes Often could be more relevant Often could be more relevant

11. Enhancement of the testing- teaching relationship Social obligation from tests Social obligation from tests Consequential validity – i.e. are the uses of the test score valid? Consequential validity – i.e. are the uses of the test score valid? Tests can be used to drive teaching and learning reform Tests can be used to drive teaching and learning reform

12. Emphasising the student Humanistic and constructivist approaches Humanistic and constructivist approaches Independent and autonomous Independent and autonomous Relevance of student backgrounds Relevance of student backgrounds Feedback and score reporting Feedback and score reporting