1. Coding and Evaluating Deaf Students' Productive English Gerald P. Berent, Paula M. Brown, & Brenda H. Whitehead National Technical Institute for theDeaf Rochester Institute of Technology Rochester, New York, U.S.A. Presented at the annual convention of Teachers of English to Speakers of Other Languages, St. Louis, MO., USA February-March, 2001

2. Challenge Develop a coding system for analyzing English language samples produced by: u Deaf and hard-of-hearing students u Students of English as a second language

3. Types of Samples for Coding n Spoken conversations (transcribed) n Written or typed essays n Individual sentences (exercises) n Videotaped signed English transcriptions

4. Coding System Users n Speech-language teachers n English teachers n Researchers n Administrators

5. Applications n Scoring language samples n Assessing English development n Global or primary trait scoring n Developing individualized instruction plans n Maintaining databases of large numbers of students n Monitoring students’ progress over time

6. Desired Characteristics n Flexible n User-friendly n Comprehensive n Theoretically current

7. Flexible Usage n With existing language transcription software n As an independent scoring system n To identify successes as well as errors n By teachers, therapists, researchers, administrators

8. User-Friendly n Codes follow mnemonic principles (e.g., RC = relative clause) n A small, medium, or large set of codes can be used n Coding can focus only on one domain of grammar or discourse n Student assistants can be trained to do partial coding

9. Comprehensive Covers a full range of structures/processes: u Syntactic categories u Grammatical relations u Morphological properties u Sentence and verb types u Semantic relations u Information structure u Discourse processes u Rhetorical devices u Mechanics and punctuation

10. Theoretically Current Coding includes: u Traditional categories F Noun phase F Verb phrase u Categories discussed in current linguistic and acquisition research F Complementizer phrase F Inflectional phrase F Determiner phrase

11. SALT: Systematic Analysis of Language Transcripts Jon F. Miller & Robin S. Chapman Language Analysis Laboratory Waisman Center University of Wisconsin--Madison Software tool for analyzing and interpreting the speech of one or more speakers

12. SALT Features n Transcription conventions for entering and coding the transcript n Editing tools for checking the transcript for errors n Predefined conventions (e.g., marking bound morphemes and verb tenses) n Ability to define your own codes

13. SALT Summaries n Transcript summary (percentages, frequencies) n Word and morpheme summary (TTR, MLU, etc.) n Maze summary (reformulations, etc.) n Utterances by length n Word lists u Pronouns (personal, relative, etc.) u Questions u Negatives u Conjunctions u Auxiliaries

14. Elicitation Techniques n Conversation/Interview n Picture series u Pictures depicting dilemma or complicated scenario u GOAL: To elicit complex structures n Essay topic u Various topics to prompt rhetorical techniques

15. Picture Series Technique n Student studies pictures n Pictures are removed n Student tells story as if not known to teacher n Production is tape-recorded (or videotaped) n Production is transcribed and entered into SALT Program

16. Picture Series: Handling conflict n Student studying in dorm room n Friend calls inviting him to go out n Student declines, studies, goes to bed n Friends stop by anyhow, disrupt student’s sleep n Student expresses anger over disruption n Friends finally leave

17. Coding for Cohesive Ties (CT) n REF = Reference u PRO = Pronominal u COMP = Comparative u DEM = Demonstrative n LEX = Lexical u REP = Repetition u SYN = Synonym u SUP = Superordinate n CON = Conjunction u ADD = Additive u TEMP = Temporal u CAUS = Causative u ADV = Adversative

18. NTID Writing Test (Albertini et al., 1986) n Write as much as you can about the topic. n You have 30 minutes to write. n TOPIC: u You are in a new place. u What do you like about NTID and the people here? u What don’t you like about NTID and the people here? u Explain.

19. Sample Coding n There/'s a boy who was study/ing in his[S:CTRefPro] room. n In a moment[E:CTConTemp] there was a flashing on the wall. n It[S:CTRefPro] was a phone call. n RefPro SuccessfulAmbiguous 4135 (85%)6 (15%)

20. Contact Information Gerald P. Berent, Ph.D. Department of Research gpbnci@rit.edu Paula M. Brown, Ph.D., Chair Department of Speech and Language pmbnci@rit.edu Brenda H. Whitehead, M.A., CCC-SP Department of Speech and Language bhwncp@rit.edu National Technical Institute for the Deaf Rochester Institute of Technology Rochester, New York 14623-5604