Learning About Language Assessment. Albany: Heinle & Heinle Evaluation From: Bailey, Kathleen M. (1998). Learning About Language Assessment. Albany: Heinle & Heinle

Testing Test: a method of measuring a person’s ability or knowledge in a given domain, with an emphasis on the concepts of METHOD and MEASURING. Tests: instruments carefully designed with identifiable scoring rubrics. Used periodically when Ss can offer peak performance

Assessment Ongoing process that encompasses a wider domain. Tests are subsets of assessment

Does ALL teaching involve assessment?

Informal assessment Teacher’s observation – constant Embedded in classroom tasks Not intended to provide recorded results

Formal Assessment Exercises, procedures to check skills and knowledge Based on learning objectives

Principles of Language Assessment PRACTICALITY RELIABILITY VALIDITY AUTHENTICITY WASHBACK

Practicality A PRACTICAL test: within financial limitations, time constraints, ease of administration, scoring and interpretation. Norm-referenced vs Criterion-referenced tests

Norm-referenced tests Purpose: to place the taker in a rank order Mean, median, standard deviation, percentile rank… Predetermined responses Practicality is of utmost importance

Criterion-referenced Tests Designed to give Ss feedback on specific course or lesson objectives. Connected to curriculum T’s time and effort to give feedback Practicality is secondary

Reliability Consistent and dependable Sources: The test itself (test reliability) Its administration The test-taker (Student-related reliability) The scoring (rater reliability)

Validity The degree to which the test actually measures what it is intended to measure. Content validity Face validity Construct validity

Content validity If test actually samples subject matter

Face validity “Does the test, on the face of it, appear from the learner’s perspective to test what it is designed to test?”

Construct validity Does the test actually tap into the theoretical CONSTRUCT as it has been defined? Construct: proficiency, communicative competence.

Authenticity The degree of correspondence, of the characteristics of a given language test to the features of a target language task. Real-world tasks Language is natural It’s contextualized Interesting topics Thematically organized

Washback Feedback provided based on Ss’ performance Diagnoses of strengths and weaknesses Effects of assessment on teaching and learning Washback enhanced by comments Praise strengths Give constructive criticism of weaknesses Give strategic hints on how to improve Strategies BEFORE and AFTER giving a test

Principles of Language Assessment PRACTICALITY RELIABILITY VALIDITY AUTHENTICITY WASHBACK

Kinds of Tests Proficiency Tests Diagnostic Tests Placement Achievement Aptitude

Techniques and Procedures Scoring Open-ended to structured Design formats Written to oral performance modes

The Role of Background Knowledge in the Assessment of Receptive Skills Listening comprehension task

Fill-in, Cloze, C-tests The cat ate the ____________.

Skills involved Syntagmatic competence: syntax Paradigmatic competence: semantics

“Fill-in” Items 1. A fill-in item typically consists of one ______. 2. Syntagmatic competence involves rules of _______ order. 3. Paradigmatic competence involves _______ sets. Criticism: discrete-point approach. One and only one possible answer (good or bad?)

1. A fill-in item typically consists of one BLANK. 2. Syntagmatic competence involves rules of WORD order. 3. Paradigmatic competence involves SEMANTIC sets.

1. A fill-in item typically consists of one SENTENCE. 2. Syntagmatic competence involves rules of SYNTACTIC order. 3. Paradigmatic competence involves LEXICAL sets.

CLOZE TESTS Cloze tests consist of texts of at least one paragraph in length (usually longer). The first sentence is left intact, BUT thereafter every “nth” word is systematically REMOVED / DELETED (every fifth, seventh, or ninth word), SO / AND the students’ task is to write AN / THE appropriate and grammatical word in each BLANK. The ability to do so depends ON the learners’ syntagmatic competence and paradigmatic COMPETENCE. as well as their discourse competence. THERE are usually thirty to fifty blanks IN each cloze passage. Although originally devised AS a test of first language reading COMPETENCE / SKILLS / ABILITY the cloze test is thought to ASSESS the general proficiency (in the written FORM / MODES) of second language learners. The two MOST common scoring procedures are the “exact WORD method” and the “acceptable word method.” ALTHOUGH the latter is more generous to STUDENTS / LEARNERS (it yields higher scores), the two SCORING systems yield similar rankings among the RESULTS / SCORES of the test-takers.

Authentic Assessment Alternative assessment consists of any method of finding out what a student knows or can do that is intended to show growth and inform instruction, and is an alternative to traditional forms of testing, namely, multiple-choice tests (Stiggins 1991). Alternative assessment is by definition criterion-referenced and is typically authentic because it is based on activities that represent classroom and real-life settings (Bailey 1998)

Contrasting Traditional and “Alternative” Assessment One-shot tests ………………… Continuous, longitudinal assessment Indirect tests Direct tests Inauthentic tests Authentic assessment Individual projects Group projects No feedback provided to learners Feedback provided to learners Speeded exams Untimed exams Decontextualized test tasks Contextualized test tasks Norm-referenced score interpretation Criterion-referenced score interpretation Standardized tests Classroom-based tests

Internationally used standardized tests National standardized tests An Example Continuum of Standardized Assessment to Individualized Assessment Internationally used standardized tests National standardized tests Statewide or provincial standardized tests Schoolwide tests Programwide tests Classroom-based tests Individualized assessment

Portfolio Assessment A portfolio is a purposeful collection of student work that exhibits the S’s efforts, progress, and achievements in one or more areas. The collection must include student participation in selecting contents, the criteria for judging merit, and evidence of student self-reflection

Direct vs. Indirect Tests Discrete-Point vs. Integrative Objective vs Direct vs. Indirect Tests Discrete-Point vs. Integrative Objective vs. Subjective Scoring