Making Consistent Decisions About Accommodations for English Language Learners – Research Summit – Texas Comprehensive SEDL Austin, Texas March 16–17, 2009

Linguistic Accommodations on Assessments: Linguistic Modification Edynn Sato, Ph.D. Director, Special Populations for AACC WestEd

3 Content and Language What is the relationship? ContentLanguage ContentLanguage Content

4 What is Linguistic Modification? A theory- and research-based approach to clarifying and simplifying the language of the assessment without simplifying the content of the assessment. Linguistic modification is not simply “good editing” practice and does not result in “easier” test items.

5 What is Linguistic Modification? Linguistic modification involves a process intended to increase English learners’ access to assessment content and subsequently increase the validity of the assessment results. –Access is the minimization or removal of conditions (i.e., sources of construct-irrelevant variance, such as aspects of presentation/format of test information, aspects of response requirements, sociocultural contexts or references that may disadvantage certain students, etc.) that may interfere with students’ ability to meaningfully engage with content or demonstrate their content knowledge and skills. –Appropriate access does not significantly change the targeted construct –Providing students the opportunity to learn, exposing them to grade-level content, or applying broadly the principles of Universal Design do not necessarily address the particular access needs English learners adequately. –Lack of appropriate access has consistently posed a threat to validity.

6 Why Linguistic Modification? Relative to other strategies such as portfolios and native language translations, linguistic modification lends itself more readily to standardization and broader application across language groups.

7 Linguistic Modification: Key Points Addresses the linguistic needs of ELLs Increases the linguistic accessibility of the test item/task Does not change the content/construct tested Does not alter essential grade-level content- related language

8 Assessment of English Learners: General Considerations Critical and interdependent factors include: Population –Characteristics (diverse) –Access needs (linguistic) Content –Language skill vs. content skill (how interrelated are the language and content skills) –Construct –Language demands –Proficiency expectations Context –Assessment vs. instruction –Design/Presentation

9 Linguistic Modification: Considerations Item context Item graphics Item vocabulary/wording Item sentence structure Item format …and, the interactions among these dimensions

10 Linguistic Modification: Strategies Current Literature/research-basedAdditional Item context Familiar to students; no cultural or linguistic bias Sometimes context is added to: remove passive voice construction, past tense and conditionals; break stem into shorter, less complex sentences (create a familiar story line) Helps make abstract/highly-generalized language more concrete Item graphics Familiar to students; no cultural or linguistic bias Symbols, legends, key vocabulary are relevant to the construct and familiar Graphics support understanding Clarify: key/central aspects of content/construct; what the student is expected to do (purposeful, meaningful) Represents the situation in its totality (not misleading) Consistent labeling/naming Excerpt from Sato (2008)

11 Linguistic Modification: Strategies (continued) Current Literature/research-basedAdditional Item Vocabulary/Wording High frequency words Common/familiar words Relevant technical terms; technical terms defined as appropriate No ambiguous words or unnecessary words with multiple meanings No irregularly spelled words Use of formal proper names are relevant and appropriate Naming conventions are consistent Consistent with standards/expectations “Common” and “familiar” is not necessarily appropriate—consider Academic English necessary Repeat key words—providing synonyms within the item may not be helpful Avoid words that are both nouns and verbs (e.g., value, cost, carpet) Avoid hyphenated and compound words Formal proper names should be ones that students are familiar reading Relative pronouns (e.g., which) should have a referent Excerpt from Sato (2008)

12 Study of Linguistic Modification: Overview 4,617 seventh and eighth grade students from 13 schools in California ELs (Spanish), low reading ability non-ELs, high reading ability non-ELs Matched sets of original and linguistically modified versions of mathematics items (number sense, measurement) Random assignment of students

13 Study of Linguistic Modification (continued) Preliminary findings: Created a set of linguistically modified items that did not significantly alter the mathematics constructs assessed –expert judgment, cognitive interviews of EL and non- EL students, analysis of non-EL student performance on linguistically modified item and matched original items, analyses of non-EL student performance on statewide academic achievement tests and on the two versions of items developed for the study

14 Study of Linguistic Modification (continued) Preliminary findings: Positive effect, though not significant, of linguistic modification of test items on the mean scores of EL students and on the mean scores of non-EL students with low-reading ability

15 Framework for understanding language demands on ELLs Language Demands A language demand is categorized as either a Linguistic Skill or an Academic Language Function based on whether the demand is fundamental to the development and use of language or is a contextual application of language. From: Sato, Lagunoff, Worth, Bailey, and Butler (2005); Bailey, Butler, and Sato (2007)

16 Language Demands From Sato (2008, June)

17 Language Complexity—Density LowHigh Length ranges from a word to paragraphs  No/little variation in words and/or phrases in sentences/paragraphs; consistent use of language  Repetition of key words/phrases/sentences reinforces information  Language is used to present critical/central details  No/little abstraction; language reflects more literal/concrete information; illustrative language is used; language is used to define/explain abstract information  Graphics and/or relevant text features reinforce critical information/details Length ranges from a word to paragraphs  Some variation in words and/or phrases in sentences/paragraphs  Repetition of key words/phrases/sentences introduces new or extends information  Language is used to present critical/central details, but non-essential detail also is presented  Some abstraction; language may or may not be used to define/explain abstract information; illustrative language may or may not be used; technical words/phrases are used  Graphics and/or relevant text features may or may not reinforce critical information/details From Sato (2008, June)

18 Language Complexity—Construction SimpleComplex Mostly common/familiar words/phrases; no/few uncommon words/phrases, compound words, gerunds, figurative language, and/or idioms  Language is organized/structured  Mostly simple sentence construction  No/little passive voice  Little variation in tense  Mostly one idea/detail per sentence  Mostly familiar construction (e.g., ’s for possessive; s and es for plural)  Mostly familiar text features (e.g., bulleted lists, bold face) Some common/familiar words/phrases; some uncommon words/phrases, compound words, gerunds, figurative language, and/or idioms  Language may or may not be organized/structured  Varied sentence construction, including complex sentence construction  Some passive voice  Variation in tense  Multiple ideas/details per sentence  Some less familiar/irregular construction  Some less familiar text features (e.g., pronunciation keys, text boxes) From Sato (2008, June)

19 Types of “Expertise” Needed Knowledge of language development and language acquisition Knowledge of English language learners Knowledge of content (English language arts, mathematics, science, social studies) Knowledge of (large-scale) assessment Knowledge of instruction

References Bailey, A. L., Butler, F. A., & Sato E. (2007). Standards-to-standards linkage under Title III: Exploring common language demands in ELD and science standards. Applied Measurement in Education, 20(1), 53–78. Sato, E., Lagunoff, R., Worth, P., Bailey, A. L., & Butler, F. A. (2005). ELD standards linkage and test alignment under Title III: A pilot study of the CELDT and the California ELD and content standards (Final Report to the California Department of Education). San Francisco: WestEd. Sato, E. (2008). Linguistic modification: Part II — A guide to linguistic modification: Increasing English language learner access to academic content. Washington, DC: LEP Partnership. Sato, E. (2008, June). Language as Content and Language of Content. What's the Difference? — Defining the Language Construct: Strategies and Implications. Presentation at the annual conference of the Council of Chief State School Officers National Conference on Student Assessment. Orlando, FL. 20

21 For more information: Edynn Sato, Ph.D. Director, Special Populations Assessment and Accountability Comprehensive Center (AACC) To learn more about the AACC: