Coding and Evaluating Deaf Students' Productive English Gerald P. Berent, Paula M. Brown, & Brenda H. Whitehead National Technical Institute for theDeaf Rochester Institute of Technology Rochester, New York, U.S.A. Presented at the annual convention of Teachers of English to Speakers of Other Languages, St. Louis, MO., USA February-March, 2001

Challenge Develop a coding system for analyzing English language samples produced by: u Deaf and hard-of-hearing students u Students of English as a second language

Types of Samples for Coding n Spoken conversations (transcribed) n Written or typed essays n Individual sentences (exercises) n Videotaped signed English transcriptions

Coding System Users n Speech-language teachers n English teachers n Researchers n Administrators

Applications n Scoring language samples n Assessing English development n Global or primary trait scoring n Developing individualized instruction plans n Maintaining databases of large numbers of students n Monitoring students’ progress over time

Desired Characteristics n Flexible n User-friendly n Comprehensive n Theoretically current

Flexible Usage n With existing language transcription software n As an independent scoring system n To identify successes as well as errors n By teachers, therapists, researchers, administrators

User-Friendly n Codes follow mnemonic principles (e.g., RC = relative clause) n A small, medium, or large set of codes can be used n Coding can focus only on one domain of grammar or discourse n Student assistants can be trained to do partial coding

Comprehensive Covers a full range of structures/processes: u Syntactic categories u Grammatical relations u Morphological properties u Sentence and verb types u Semantic relations u Information structure u Discourse processes u Rhetorical devices u Mechanics and punctuation

Theoretically Current Coding includes: u Traditional categories F Noun phase F Verb phrase u Categories discussed in current linguistic and acquisition research F Complementizer phrase F Inflectional phrase F Determiner phrase

SALT: Systematic Analysis of Language Transcripts Jon F. Miller & Robin S. Chapman Language Analysis Laboratory Waisman Center University of Wisconsin--Madison Software tool for analyzing and interpreting the speech of one or more speakers

SALT Features n Transcription conventions for entering and coding the transcript n Editing tools for checking the transcript for errors n Predefined conventions (e.g., marking bound morphemes and verb tenses) n Ability to define your own codes

SALT Summaries n Transcript summary (percentages, frequencies) n Word and morpheme summary (TTR, MLU, etc.) n Maze summary (reformulations, etc.) n Utterances by length n Word lists u Pronouns (personal, relative, etc.) u Questions u Negatives u Conjunctions u Auxiliaries

Elicitation Techniques n Conversation/Interview n Picture series u Pictures depicting dilemma or complicated scenario u GOAL: To elicit complex structures n Essay topic u Various topics to prompt rhetorical techniques

Picture Series Technique n Student studies pictures n Pictures are removed n Student tells story as if not known to teacher n Production is tape-recorded (or videotaped) n Production is transcribed and entered into SALT Program

Picture Series: Handling conflict n Student studying in dorm room n Friend calls inviting him to go out n Student declines, studies, goes to bed n Friends stop by anyhow, disrupt student’s sleep n Student expresses anger over disruption n Friends finally leave

Coding for Cohesive Ties (CT) n REF = Reference u PRO = Pronominal u COMP = Comparative u DEM = Demonstrative n LEX = Lexical u REP = Repetition u SYN = Synonym u SUP = Superordinate n CON = Conjunction u ADD = Additive u TEMP = Temporal u CAUS = Causative u ADV = Adversative

NTID Writing Test (Albertini et al., 1986) n Write as much as you can about the topic. n You have 30 minutes to write. n TOPIC: u You are in a new place. u What do you like about NTID and the people here? u What don’t you like about NTID and the people here? u Explain.

Sample Coding n There/'s a boy who was study/ing in his[S:CTRefPro] room. n In a moment[E:CTConTemp] there was a flashing on the wall. n It[S:CTRefPro] was a phone call. n RefPro SuccessfulAmbiguous 4135 (85%)6 (15%)

Contact Information Gerald P. Berent, Ph.D. Department of Research Paula M. Brown, Ph.D., Chair Department of Speech and Language Brenda H. Whitehead, M.A., CCC-SP Department of Speech and Language National Technical Institute for the Deaf Rochester Institute of Technology Rochester, New York