Assessing Vocabulary and Writing
Assessing Vocabulary
Dimensions of Word Knowledge Accessibility Morphophonology Syntax Semantics Collocation Association Conceptualization Frequency Formality Register
Each dimension represents a continuum No knowledge Complete knowledge
Commonly used vocabulary tests Which dimensions of word knowledge do they usually assess? Do they take into consideration the scalar (continuum) qualities of word knowledge?
Dimensions of Word Knowledge Accessibility Morphophonology Syntax Semantics Collocation Association Conceptualization Frequency Formality Register
Vocabulary Size versus Vocabulary Strength Size: the number of words a learner knows Strength (or Depth): how well the learner knows certain words
Popular tests of vocabulary size Nation’s Vocabulary Levels Test The Eurocentres Vocabulary Size Test TOEFL Peabody
Popular tests of vocabulary size and strength The Vocabulary Knowledge Scale The Depth of Vocabulary Knowledge Test Computer Adaptive Test of Size and Strength
Comprehensive Assessment of Vocabulary Knowledge (Based on samples of learners’ speech and writing) Holistic measures Primary trait measures Rubrics Analytical measures
Comprehensive Analytical Assessment of Vocabulary Knowledge and Use Tokens Types Lexical Density Lexical Sophistication Lexical Frequency Profile Lexical Diversity Lexical Error Indices Lexical Choice Analysis Lexical Style/Lexical Signatures
Assessing Writing
Popular approaches to assessing writing abilities Holistic assessment Primary trait assessment Rubric assessment Analytical assessment Portfolio assessment Showcase portfolios Collections portfolios Assessment portfolios
Which approaches have these advantages? A. Indicates to what extent students are benefiting from instruction. B. Even with minimal rater training can produce high reliability. C. Gives a very direct measure of writing effectiveness, using a simple protocol. D. Provides the most detailed feedback to writers. E. Is the most learner-centered. F. Is best for diagnosing grammatical and mechanical needs. G. Is best for diagnosing discourse-level, content, and organizational needs. H. Is the most practical for proficiency-rating and placement-level purposes. I. Most directly links assessment with instruction. J. Encourages students to reflect on their work, analyze progress, and set improvement goals. K. Encourages students to produce writing that is worth sharing and retaining for review.
Which approaches have these disadvantages? A. Provides the writer with little or no feedback. B. Can make the teacher feel that she is giving up control of the assessment process. C. Provides feedback that may steer the writer’s attention to areas of writing that are least important to the overall effectiveness of her text. D. Relies on teacher-selected criteria that may be either over-representative or under-representative of the components of good writing. E. Requires multiple trained raters in order to achieve adequate reliability. F. Can be extremely time-consuming. G. Requires raters to evaluate writing in a way that real-world readers do not do. H. Relies on several rating criteria that should presumably be weighted differently, but weighting criteria is difficult and subjective. I. Is least likely to lead directly to improvements in writing. J. Provides the least useful assessment information to teachers. K. Provides the least useful assessment information to writers. L. Is the most subjective. M. Provides the most confusing assessment results. N. Is the least pedagogically sound. O. Is the least criterion-referenced.
Principles of Testing Writing Set writing tasks that are properly representative of the tasks that learners should be able to perform. The tasks should truly represent learners’ writing ability. Use as many tasks as you can. Test only writing ability. Restrict options.
Principles of Testing Writing (cont.) The tasks must be scored reliably. Elicit sufficiently long samples. Establish a clear set of criteria for scoring Whether holistic, primary-trait, rubric, or analytic Use 2-4 raters. Train the raters. Monitor raters’ performance.
Some new directions in assessing writing Self evaluation Peer evaluation Picture and film stimuli Writing for authentic purposes Process evaluation and progress profiles
Some realistic day-to-day writing tasks Writing checks Paying bills Taking notes and phone messages Writing reminder notes to oneself Writing a shopping list Writing a schedule/itinerary for oneself Writing letters/e-mail messages to friends and relatives Writing letters to people/companies you do business with -inquiries -complaints -appreciation Filling out forms -credit card applications and the like -medical background forms for healthcare providers -personal background forms for potential employers Writing a classified ad for a newspaper Writing a letter to the editor Writing a curriculum vitae or resume Writing a cover letter for a job application
How do you assess the writing abilities of low-level learners? realistic tasks (tasks they are truly likely to encounter) structured/discrete-point test items