1. Jamal Abedi University of California, Davis/CRESST Presented at: The Race to the Top Assessment Program Public & Expert Input Meeting December 2, 2009 Denver, Colorado Assessment of English Language Learners

2. Assessment Results Have Major Impact on ELL Students’ Academic Life more so than on Non-ELL Students Classification (for ELLs) Instruction (assessment before instruction for ELLs) Accountability (multiple accountability requirements for ELLs) Promotion (reclassification) Graduation

3. Assessment Challenges for ELL Students ELL students go through assessment and accountability requirements and challenges twice: Title III, the English language proficiency assessment (AMAO1 and AMAO 2) Title I assessment in reading/language arts, math, and science They are faced with the assessment issues in both areas

4. Focus of this presentation A.General assessment issues that apply to ELL students as well B.Assessment issue specific to ELLs C.Answers to the RTTT questions regarding ELL assessment and my recommendations

5. A. General assessment issues that apply to ELL students (Already being discussed at the earlier RTTT presentations by Baker, Bennett, Braun, Darling Hammond, Gong, Marion and others.) 1. Theory of Action 2. Link between assessment and instruction 3. Using multiple measures 4. Interim & formative assessments 5. Preparation of RFP 6. Use of technology 7. Common content standards 8. Growth measures over time 9. Providing teacher professional development opportunities (We will elaborate on some that are more related for ELLs)

6. 2. Link between assessment and instruction (currently, a disconnect) As Darling-Hammond indicated, “..high-achieving nations used open-ended performance tasks and school-based, curriculum-embedded assessments to give students opportunities to develop and demonstrate higher-order thinking skills.” Instruction should inform development of assessment and assessments should inform instruction It is therefore essential to involve state assessment folks and teachers in the process of test item writing and test development Teachers should be trained and be involved in all different phases of test development process and use for ELL students including item writing, scoring, and interpreting the results The assessment process should help teacher in preparing students for college and career ready standards

7. 3. Using multiple measures Once a year assessments in Title I and Title III, with all their limitations particularly for ELL students, may not produce valid and fair outcomes. There are concerns over reliability and validity of these instruments for ELLs (e.g., for some content areas reliabilities are as low as.50) These assessments include a substantial amount of measurement error which make high-stakes decisions based solely on their outcomes quite risky A series of measures from different tests with different format and different tasks, given at different times, would be needed to make fair decisions about classification and academic performance of ELL students.

8. 4. Interim & formative assessments Summative assessments while quite important for accountability purposes may be too little too late to inform curriculum and instructions They are used mostly for accountability purposes without much instructional values Interim and formative assessments provide teachers with the information needed to help ELL students Outcomes of formative assessments may also help parents of ELL students to identify skills in which their children need help

9. 6. Use of technology As an example of the use of technology in assessment, computerized assessment system can be discussed Computerized assessment system has the flexibility and capability of incorporating many accessibility features for assessments of ELL students Examples are: English and bilingual glossary, read aloud of content- based assessment items and providing extra time Assigning test items with different levels of linguistic complexities to students at different levels of language proficiency Providing opportunities for students to be tested in a language that produces the most valid assessment outcomes

10. 8. Growth measures over time Growth measures are important for ELLs English language proficiency (Title III) lends itself well to growth over time An example of application of the concept of growth model is for the AMAO 2 While it is extremely helpful to think about growth concept and its measurement model, it is also important to think about its limitations such as: issues concerning baseline changes in the measures overtime the nature of the content being assessed establishing a meaningful vertical scale

11. B. Assessment issues specific to ELLs The misconception that ELL students are only quantitatively different from native speakers of English should be discussed 1.Understanding of the two different assessment systems for ELLs, their similarities, differences and interactions 2.Lack of interaction between the two systems, English language proficiency and content assessments 3.Construct-irrelevant sources in measurement of ELL students 4.Applicability of measurement theories in ELL assessments 5.Impact of L1 proficiency on the assessments and instructions for ELL students 6.Classification, reclassification, and misclassification of ELL students Recommendations

12. 1. Understanding of the two different assessment systems for ELLs, their similarities, differences and interactions There are two different assessment systems for ELLs, Title III ELP and Title I content assessments While they involve different tests based on different content standards and different objectives, they should work together. We have made significant progress in the assessment of ELP for ELL students due to the implementation of NCLB, e.g., ACCESS for ELLs, ELDA, LAS LINK, SELP, etc There is a need for substantial work on improving Title I assessments for ELLs Recommendation: Provide support for more improvement in the Title III assessments and support creating more valid assessment system in measuring ELL content knowledge (RTTT).

13. 2. Lack of interaction between ELP and content assessments ELL students must be at a certain level of English proficiency to be able to meaningfully participate in the Title I assessment Only students at the proficiency level 4 or above may be able to participate in Title 1 assessments. However, there is a disconnect between student’s level of ELP and their participation in content-based assessments. Recommendation: Include ELL students in content assessments in English if they are at the proficiency level to meaningfully participate (level 4 or above); otherwise, provide valid alternatives such as native language testing, relevant accommodations, etc.

14. 3. Construct-irrelevant sources in measurement of ELL students There are different sources of construct irrelevant variance affecting ELL students assessment outcomes Unnecessary linguistic complexity of assessment as a source of construct irrelevant variance adds an additional factor (dimension) to the assessments for ELLs Other sources of construct irrelevant variance such as cultural biases also add additional dimensions to the assessment outcomes for ELL students Recommendation: Provide ELL professional training to the test item writers and include teachers and linguistic/cultural experts at the item development process to control for these sources.

15. 4. Applicability of measurement theories in ELL assessments A major assumption underlying classical theory of measurement is uni-dimensionality Construct-irrelevant sources for ELL students introduce complexity into the measurement model for these students While multidimensional assessment can be handled both in classical and IRT models, those dimension should be clearly relevant to the assessment Recommendation Revisit psychometric principles in light of current research on the assessment of ELLs and make appropriate revisions

16. 5. Impact of L1 proficiency on the assessments and instructions for ELL students A major oversight in the assessment of ELL students is the lack of attention to their level of native language proficiency Proficiency in L1 would help in both instruction and assessment of ELLs Native language instruction and assessment could be a great success if students are academically proficient in their native language. Recommendation Include valid and comprehensive measures of ELL students’ level of L1 academic proficiency in all proficiency domains (reading, writing, speaking and listening) and seriously consider the results in planning their curriculum and assessment

17. 6. Classification, reclassification, and misclassification of ELL students If students are not properly classified as ELLs/non-ELLs, instructions, assessments and accommodations for ELL students will not be productive Similarly, if ELL students are not properly re-classified as fluent in English when they reach fulency, they may not benefit from proper instruction and their assessment outcomes may not be valid There are major concerns on misclassification of ELL students as those having learning disabilities. Recommendation Multiple reliable and valid criteria should inform decisions about classification/reclassification for ELL students.

18. C. C. My answers to the RTTT questions regarding ELL assessment and my recommendations Provide recommendations for the development and administration of assessments for each content area that are valid and reliable for English language learners How do you recommend that assessments take into account the variations in English language proficiency of students? How can technology be utilized to make assessments more inclusive to ELL students? what are the relative merits of developing and administering content assessments in native language? What are the technical, logistical, and financial requirements?

19. Question1a: Provide recommendations for the development and administration of assessments for each content area that are valid and reliable for English language learners General recommendations: ELL students should not be treated differently in the content being assessed; otherwise comparability of assessment outcomes may become a major concern However, assessments in the content-based areas (such as math, science and social sciences) should be free of unnecessary linguistic complexities and cultural biases Multiple measures should be implemented in both Title I and Title III that utilize different measurement approaches Provide accommodations that help reduce the effects of construct- irrelevant factors

20. Question 1b: How would you recommend that the assessments take into account the variations in English language proficiency of students in a manner that enables them to demonstrate their knowledge and sills in core academic areas? Provide assessments with varying degree of linguistic complexity Provide appropriate accommodations for ELL students to help them with their common needs of language assistance Provide native language testing opportunities for students who are academically proficient and are instructed in their native language

21. Question 1c: How can technology be utilized to make assessments more inclusive to ELL students? Provide computerized assessments that are capable of offering effective and valid accessibility features including appropriate accommodations Provide web-based tutorials for ELL students for having additional opportunities to learn Provide web-based training for parents to help their students Provide diagnostic information to teachers, parents, and students on the areas need attention.

22. Question 2a: In the context of reflecting student achievement, what are the relative merits of developing and administering content assessments in native language? If students are proficient in their native academic language and have been instructed in native language then native language assessments would be productive Assessments in native language could be less affected by linguistic and cultural biases Language is no longer a source of construct-irrelevant variance If students have not been instructed in their native language, then these assessments would not help in measuring student achievement

23. Question 2b: What are the technical, logistical, and financial requirements? Logistical: Alignment between the language of assessments and language of instructions Issues addressing all of the languages spoken in our schools Issues concerning translations in different languages Financial: Issues with developing, field testing and preparing operational forms for all languages

24. For more information please contact Jamal Abedi at UC Davis/CRESST (530) 754-9150 or jabedi@ucdavis.edu