1. Improving the Validity of Assessment Results for English Language Learners with Disabilities
Laurene Christensen, Ph.D.
Linda Goldstone, M.S.
National Center on Educational Outcomes

2. Overview
IVARED is an Enhanced Assessment grant that includes a consortium of 5 states: Minnesota, Maine, Michigan, Washington, and Arizona
Goals are to improve our understanding of ELLs with disabilities and to support improved decision-making efforts about how they participate in assessments

3. Project Activities Longitudinal look at data
Review of accommodations/technical assistance related to accommodations
Focus groups with educators
Principles and Guidelines for assessment decision-making
Pilot of an online training module

4. What We Learned from Data
Slightly more boys than girls are ELLs with disabilities in each state
Ethnic diversity and home languages varied widely
In general, the contexts of each state were so different, it is not possible to make comparisons across states

5. What We Learned about Participation and Accommodations
There are challenges in including some ELLs with low-incidence disabilities in all domains of an English language proficiency assessment
Braille and sign interpretation can be challenging to the construct being measured in some domains of the ELPA
In general, there is an increase in the mention of ELLs with disabilities in accommodations policies

6. What We Learned from Educator Focus Groups
35 focus groups across five states
239 Educators across disciplines; over 500 waitlisted
Over 2,300 pages of qualitative data

7. Participant job titles

8. Participant demographics

9. Participant demographics

10. Participant’s student grade levels

11. Focus GroupThemes Validity of assessment results
Alignment of assessment and instruction
Assessment decision making
Collaboration
Assessment of leadership and structures

12. Theme #1: Validity of Assessment Results
Bias:
-student background
-culture
Accessibility:
- format
- English language level

13. Theme #2: Alignment of Assessment and Instruction
Student skills vs. grade level standards
-multiple assessments
Teaching practices
-test preparation
-scaffold

14. Theme #3: Assessment decision making
Assessment participation
-state mandates
-Individualized education program (IEP)
Accommodations
-ESL
-implementation

15. Theme #4: Collaboration
Team
-multidisciplinary
-inclusive
Constraints
-resources
-logistics

16. Theme #5: Leadership Policies -explicit for ELLDs Support
-communication
-time
-resources
-training
-success

17. Language or Disability?
High English language needs
Low disability-related needs
High disability-related needs
Low English language needs
From Christensen, Shyyan et al., forthcoming

18. Principles and Guidelines
Focused on ELLs with disabilities
Developed by an expert panel
Second language experts
Special education experts
Assessment experts
Online Delphi process
Started with the topic areas from the regulations for standards and assessments
Included open-ended prompts, and consensus building

20. Principle 1: Content standards are the same for all students.
Guideline 1A: Include individuals with knowledge of content, second language acquisition, and special education on the team that writes standards.
Guideline 1B: Design standards so they are accessible to all students, including ELLs with disabilities.
Guideline 1C. Provide ongoing professional development on implementation of standards for ELLs with disabilities to ensure high quality instruction and assessment.

21. Principle 2: Test and item development include a focus on access to the content, free from bias without changing the construct being measured.
Guideline 2A. Understand the students who participate in the assessment, including ELLs with disabilities. 
Guideline 2B. Involve people with expertise in relevant areas of test and item development.
Guideline 2C. Use Universal Design principles in test and item development.
Guideline 2D. Consider the impact of embedded item features and accommodations on the validity of assessment results.
Guideline 2E. Include ELLs with disabilities in item try-outs and field testing.
Guideline 2F. Conduct committee-based bias reviews for every assessment through continuous, multi-phased procedures.

22. Principle 3: Assessment participation decisions are made on an individual student basis by an informed IEP team.
Guideline 3A. Make participation decisions for individual students rather than for groups of students.
Guideline 3B. Make assessment participations in an informed IEP team representing all instructional experiences of the student, as well as parents and students, when appropriate.
Guideline 3C. Provide the IEP team with training on assessment decision making for ELLs with disabilities.
Guideline 3D. Use written policies that specifically address the assessment of ELLs with disabilities to guide the decision-making process.

23. Principle 4: Accommodations for both English Language Proficiency (ELP) and content assessments are assigned by an IEP team knowledgeable about the individual student’s needs.
Guideline 4A. Provide accommodations for ELLs with disabilities that support their current levels of English proficiency, native language proficiency, and disability-related characteristics.
Guideline 4B. Collect and examine individual student data to determine appropriate accommodations for ELLs with disabilities taking ELP and content assessments.
Guideline 4C. Develop assessment accommodations policies for ELLs with disabilities that account for the need for language-related and disability-related accommodations.
Guideline 4D. Provide decision makers with training on assessment accommodations for ELLs with disabilities.

24. Principle 5. Reporting formats and content support different uses of large-scale assessment data for different audiences.
Guideline 5A. Use disaggregated data for ELLs with disabilities to account for demographic and language proficiency variables.
Guideline 5B. Highlight districts and schools with exceptional performance to identify characteristics that lead to success of ELLs with disabilities.
Guideline 5C. Provide interpretation guidance to educators about ways in which large-scale assessment data can be interpreted and used for educational planning.
Guideline 5D. Provide different score report formats as guides to parents and students.

25. Online Training Module
Currently under development
A series of “mini-modules” on key topics related both to the principles and guidelines as well as issues that arose in the focus groups
Will be piloted this spring

26. Questions?
Contact information:
Laurene Christensen
Linda Goldstone
NCEO nceo.info
IVARED ivared.org