1. National Center on Educational Outcomes
Accessibility and Accommodations Challenges in Technology-Based State Assessments
Martha Thurlow, Sheryl Lazarus, Laurene Christensen, and Vitaliy Shyyan
National Center on Educational Outcomes

2. Topics Technology Challenges and Solutions
Terminology Challenges and Solutions
Accessibility Challenges and Solutions
Opportunities

3. Technology Challenges
Digital divide still exists
Frequent complaints that schools did not have enough computers
Frequent complaints that bandwidth not sufficient for assessment needs
Testing interruptions – occur even when the best technology is available

4. Testing Interruptions
CCSSO (2015) produced a report on testing interruptions: Recommendations for Addressing the Impact of Test Administration Interruptions and Irregularities
Among the things defined as “testing interruptions” were:
Delayed log-in
System delays in loading items
System slows down
Students are unexpectedly logged out
Students lose all progress
Items or stimuli are rendered poorly
Accessibility issues

5. Accessibility Issues*
Resources not available or not working properly
Example: Highlighter tool is not available for all or a portion of the administration time
Accessibility features not working properly
Example: Read-aloud feature is not functioning for a portion of the administration time
*Identified by CCSSO

6. Importance for Students with Disabilities
May affect students with disabilities differently from other students*
Example: Student with a disability might be frustrated and not continue to try when other students are not as frustrated and do continue to try
*Effect may be direct or indirect

7. Framework for Analyzing Effects of Test Interruptions
See:
CCSSO framework provides both a way to analyze the effects of test interruptions and a way to determine appropriate treatment of potentially affected scores.
Step 1 is intended to verify that data are clean and to create the necessary additional data elements to conduct analyses.
Step 2 is intended to determine the prevalence of each type of test interruption for student subgroups (e.g., disability) and educational entities (e.g., school or district). This step can identify subgroups that should be closely examined in subsequent steps.
We won’t go though all the steps here, but recommend that you read the CCSSO document.

8. Terminology Challenges
Many different terms now being used to mean the same thing
Same terms also are used to mean different things
Next slide shows terminology used in six assessment consortia

9. Accessibility Frameworks
Type of Assess-ment
Assessment
For All Participating Students
For Some Students with Educator Input
For Few Students with Documented Needs
General Assess-ment
PARCC
Features for All Students
Accessibility Features Identified in Advance
Accommodations
Smarter Balanced
Universal Tools
Designated Supports
ELP Assess-ment
ELPA21
Universal Features
Designated Features
WIDA ASSETS
Accessibility Tools
Alternate Assess-ment
DLM
Supports Provided within DLM via PNP
Supports Requiring Additional Tools/Materials
NCSC
Optimal Testing Conditions
Accessibility Features
Note the DIAMOND White Paper calling for common terminology across consortia. Also note the CCSSO presession.

10. National Education Technology Plan (2016)
“Future of Assessment” compares traditional and next generation assessments, with the goal of “more flexibility, responsiveness, and contextualization” – accessibility that is universally designed in one of the features of next generation assessments:
When both assistive technologies and assessments effectively interoperate, students are better able to demonstrate what they know and how to apply this knowledge. (NETP, 2016, p. 57)

11. Accessibility Challenges
Inconsistent testing platforms for different assessments
May have been more of a challenge for adults than for students, but still a challenge!

12. Accessibility Challenges
Scrolling issues
An issue for all students mentioned by all consortia
May have a greater impact on students with disabilities and English learners than other students

13. Accessibility Challenges
Accessibility features and accommodations did not work as intended
Not able to turn off universal features (an important designated feature)
Limitation on number of re-reads did not activate

14. Accessibility Challenges
Guidance on how to configure computers and other devices to handle videos not available
Tutorials for students needed to be more “kid friendly”

15. Accessibility Challenges
Use of accessibility features and accommodations was not tracked
Ability to track the USE of accommodations (and other accessibility features) was a huge promise of technology-based assessments
No consortium assessment was able to do this –only could track the accessibility features and accommodations that were SELECTED for a given student

16. Accessibility Challenges
Many concerns related to the use of assistive technology (AT)
Lack of security with “3rd party” AT
Incompatibility with “3rd party” AT
Result is often that students are not allowed to use AT they regularly use during instruction

17. Some Solutions
Develop test security policies and procedures to address AT issues (e.g., erase assessment content; turn off features not allowed on assessment)
Allow the student to use AT that is regularly used during instruction (unless it would compromise test validity.

18. Successfully Using AT
Students need to be familiar with AT they will use on the assessment.
Ensure that students using AT have access to a practice test session that enables them (and their teachers) to see that the AT works as intended in the test environment.
AT compatibility issues should be checked before test day.
Tools and procedures should be developed to make it easy for LEAs to report any compatibility issues. Procedures also are needed that address how reported issues will be addressed.

19. More Solutions
Require vendors to demonstrate their capabilities before signing a contract!
Ensure that practice tests have the same embedded (and non-embedded) accessibility features and accommodations as the actual assessment.
Check out the technology, not just the assessment items.
Involve students in cognitive labs and usability studies.
Include students with disabilities in infrastructure trials prior to testing.

20. Opportunities
Race-to-the-Top Assessment Consortia were a first step (and the source of issues already highlighted)
Innovative Assessment Program is a next opportunity for large-scale assessments – NOW IN COMMENT PERIOD, until May 18, See: 18/pdf/ pdf
Note that the Innovative Assessment Program is part of the Enhanced Assessment Initiative, which provides grants to state departments of education.

21. Innovative Assessment Program
Priority 1: Developing Innovative Assessment Item Types and Design Approaches
develop, evaluate, and implement new, innovative item types for use in summative assessments in reading/language arts, mathematics, or science;
develop new approaches to transform traditional, end-of- year summative assessment forms with many items into a series of modular assessment forms, each with fewer items
for either a or b, provide a dissemination plan such that models and resources can be shared with states across nation

22. Innovative Assessment Program
Priority 2: Improving Assessment Scoring and Score Reporting
develop innovative tools that leverage technology to score assessments;
propose projects, in consultation with …to address needs related to score reporting and improve the utility of information about student performance…
for either a or b, provide a dissemination plan such that models and resources can be shared with states across nation

23. We want to hear from you! Please click on the link in the chatbox and complete the short webinar survey.