Language, Dialect, and Register: Sociolinguistics and the Estimation of Measurement Error in the Testing of English-Language Learners* Guillermo Solano-Flores American Institutes for Research NCCRESt Conference, Arizona State University in Tempe, AZ November 18, 2004 *Research funded by the National Science Foundation, REC

A warm-up exercise What is the animal that eats shoots and leaves? What is the animal that eats, shoots, and leaves?

Overview The sociolinguistic dimension of ELL testing Key concepts: dialect, register, linguistic misalignment Heterogeneity of populations, score variation, and measurement error Dialect and register: as important as language in ELL testing

ELL testing: Levels of analysis language dialectregister

The concept of dialect Dialects used by different speech communities Language

Dialect in testing: An example How much money does Laura need to pay for two candies that cost 45 cents each? How much money does Laura need to pay for two sweets that cost 45 cents each? How much money does Laura need to pay for two 45-cent candies?

Dialect intersection A  B AB Dialect A: Speech Community A Dialect B: Speech Community B

register The concept of register Registers: Uses of language in different contexts Language

Register in testing: An example Reproduced from TIMSS Population 1 Item Pool. Copyright © 1994 by IEA, The Hague

The interaction of dialect and register Standard English: The length of a dinosaur was reported to have been 80 feet (rounded to the nearest 10 feet). What length other than 80 feet could have been the actual length of this dinosaur?

The interaction of dialect and register Standard Spanish: Se reportó que la longitud de un dinosaurio era 80 pies (redondeada a los 10 pies más cercanos). ¿Cuál longitud además de 80 pies podría haber sido la longitud real de este dinosaurio? Southern California: Se reportó que un dinosaurio midió 80 pies de largo (redondeado a la decena más próxima). Aparte de 80 pies, ¿qué pudo haber sido la medida exacta del dinosaurio?

Dialect and register Miami: Yo di yon dinozò te mezire 80 pye de longè (Rapwoche pi pre 10 pye). Ki lòt longè pase 80 pye ki ta kapab vrè longè dinozò a? Repons:___________ Brooklyn: Yo rapòte longè yon dinozò te 80 pye (longè a awondi nan dizèn ki pi pre a). Apa 80 pye, ki lòt vrè longè dinozò a ta ka genyen? Repons:___________

Linguistic misalignment: Frequency and severity How much money does Laura need to pay for two candies that cost 45 cents each? How much money does Laura need to purchase two sweets at 45 cents a piece? How much money does Laura need to pay for two 45-cent candies?

Linguistic misalignment Student: Non-standard dialect Partially unfamiliar with testing register Content area register shaped by instructional context T  S T S Test: In standard dialect Assumes full command of testing register Content area register based on standards documents

Find instances of misalignment: Sam can purchase his lunch at school. Each day he wants to have juice that costs 50¢, a sandwich that costs 90¢, and fruit that costs 35¢. His mother has only $1.00 bills. What is the least number of $1.00 bills that his mother should give him so he will have enough money to buy lunch for 5 days? NAEP: Mathematics Grade 4, 1996 Public Release

Students’ interpretations* of His mother has only $1.00 bills 1.His mother has only bills of one dollar 2.His mother has only one dollar 3.His mother has only dollar bills *As reflected by students’ read-alouds

Linguistic misalignment probabilistic space Frequency of misalignment Severity of misalignment Linguistically challenging items Linguistically sound items

Generalizability (G) theory in one slide student (s) item (i) dialect (d) si sd id sid,e Main effect Interaction effect Object of measurement Sources of measurement error Error due to unknown sources Facets

Miami students tested in Standard English and Standard Haitian-Creole student (s) 20 rater (r) 0 item (i) 6 language (l) 5 sr 0 si 11 sl 7 ri 0 rl 0 il 1 sri 0 srl 0 sil 39 ril 0 sril,e 11 Percentage of score variation Source

Miami students tested in Miami Haitian- Creole and Standard Haitian-Creole student (s) 22 rater (r) 0 item (i) 5 dialect (d) 0 sr 0 si 23 sd 2 ri 0 rd 0 id 0 sri 0 srd 0 sid 33 rid 0 srid,e 14 Percentage of score variation Source

Approaches to ELL testing Item response theory Scaling Differential item functioning Between-group designs Reference groups Population homogeneity Generalizability theory Measurement error Score dependability Within-group designs No reference groups Population heterogeneity

Conclusions Dialect and register are as important as language in the testing of ELLs Testing and psychometric approaches for ELLs should and can be well grounded on reasonings from sociolinguistics