Assessing Writing Writing skill at least at rudimentary levels, is a necessary condition for achieving employment in many walks of life and is simply taken for granted in literate cultures.

Contents Micro-skills vs. macro-skills Types of writing Scoring writing Holistic scoring Analytic scoring Objective scoring

Genres of written language 1. Academic writing 2. job-related writing 3. Personal writing Theses, shopping list, phone messages email, diaries, advertisements, compositions The most common genres that a second language writer might produce, within and beyond the requirements of a curriculum

Micro-skills vs. Macro-skills Phoneme-grapheme correspondences Mechanics such as spelling and punctuation Word order patterns, grammatical systems, & rules Cohesive devices in written discourse Macro-skills: Rhetorical modes, communicative functions & purposes Main/supporting ideas, new/given info., generalization/exemplification Literal & implied meanings Culturally specific references Prewriting devices, paraphrases, synonyms, drafting, feedback, revising, and editing We turn once again to a taxonomy of micro-and macroskills that will assist you in defining the ultimate criterion of an assessment procedure. The earlier microskills apply more appropriately to imitative and intensive types of writing task, while the macroskills are essential for the successful mastery of responsive and extensive writing.

Types of Writing (1) Imitative (micro-skills): Mechanics of writing: fundamental, basic tasks of writing letters, words, punctuation, and short sentences Focus on form. Handwriting letters, words, and punctuation Copying Listening cloze selection task Spelling tasks and detecting phoneme Spelling/ multiple-choice reading-writing spelling tasks Matching phonetic symbols (p. 225) Imitative writing: the rudiments of forming letters, words, and simple sentence. Spelling tasks – disadvantages – students don’t recognize the phonetic alphabet or use it in their daily routine. Advantages – it helps students to perceive the relationship between phonemes and graphemes.

Types of Writing Intensive/controlled (micro-skills): Sentence production: proper vocabulary, collocations, idioms, grammar Still focus on form, with some concern about meaning and context It is called controlled writing and also be thought of as form-focused writing, grammar writing, or simply guided writing. A good deal of writing at this level is display writing as opposed to real writing: students produce language to display their competence in grammar, vocabulary, or sentence formation, and not necessarily to convey meaning for an authentic purpose. The traditional grammar/vocabulary test ahs plenty of display writing in it.

Intensive Writing Assessment Dictation and Dicto-com Grammatical Transformation Tasks (p. 226) Change the tenses in a paragraph Change the statement to yes/no questions Picture-cued tasks Short sentences Picture description Picture sequence description Vocabulary Assessment Tasks Reordering words in a sentence Short-answer and sentence completion tasks

Dictocomp A variation on the traditional dictation procedure T reads while Ss listen carefully Ss write a summary (not word for word as in dictation) Decide scoring criteria (Bailey 151) Rationale: Meaning-oriented Conveying & capturing meaning = key component of communication

Types of Writing (2) Responsive (macro-skills): Paragraph writing: narratives, descriptions, short reports, lab reports, summaries, responses to reading, interpretations of charts/graphs Guided question and answer Paragraph construction tasks Emphasis on context and meaning Extensive (macro-skills): All-purpose essay writing Term paper, research project report, thesis Process of multiple drafts

Scoring Writing Holistic scoring Analytical scoring Objective scoring

Holistic Scoring (1) A single scale is used to describe different levels of writing performance. Not to think about the individual components of the writing skill or to count the number of errors in the writing. The rater reacts to the writing as a whole. A single score is given. E.g., on a scale of 1 ~ 10. Focuses more on communication; overall impression; e.g.,TOEFL’s TWE (Brown 239; Bailey 187)

Holistic Scoring (2) Advantages Fast Higher inter-rater reliability Easily understood by teachers and students alike Widely applicable to writing across different disciplines Emphasizes the writer’s strengths Holistic scoring- TWE 1-6 point, each point on a holistic scale is given a systematic set of descriptors, and the reader-evaluator matches an overall impression with the descriptors to arrive at a score.

Holistic Scoring (3) Disadvantages (Brown 242) One score masks differences across individual composition or the subskills within each score. No diagnostic info. Is available (no washback). The scale may not apply equally well to all genres of writing. Raters need to be extensively trained to use the scale accurately. In general, teachers and test designers lean toward holistic scoring only when it is expedient for administrative purpose.

Training Procedure Norming process: Familiarization with the scale Read several benchmark papers & discussing how they relate to the scale Raters independently read a mixed set of papers and score them. Comparison of scores awarded by all raters Discuss discrepancies Another set of papers is read & scored.

Primary Trait Scores Focuses on “how well students can write within a narrowly defined range of discourse” A four-point scale ranging from zero to 4. The accuracy of the account of the original (summary) The clarity of the steps of the procedure and the final result (lab report) The description of the main features of the graph (the graph description) The expression of the writer’s opinion (response to an article) This type of scoring emphasizes the task at hand and assigns a score based on the effectiveness of the text’s achieving that one goal

Analytic Scoring (1) “Embodies hypotheses about the underlying constructs that comprise a given skill,” or evaluate separately various components Assessing Ss’ performances on a variety of categories E.g., 5 components: Content 13 ~30 points Organization 7 ~ 20 points Vocabulary 7 ~20 points Language use 5 ~25 points Mechanics 2 ~5 points

Analytic Scoring (2) Why called “analytical scoring?” Because scale designers have analyzed the hypothesized components of the writing skill, and it’s these components that make up the categories used in scoring

Analytic Scoring (3) Evaluating separately various components: Points-off, or weighted score, or split grade Scoring profile: based on five components Content Organization Vocabulary Language use Mechanics