National Center on Educational Outcomes Accessibility and Accommodations Challenges in Technology-Based State Assessments Martha Thurlow, Sheryl Lazarus, Laurene Christensen, and Vitaliy Shyyan National Center on Educational Outcomes

Topics Technology Challenges and Solutions Terminology Challenges and Solutions Accessibility Challenges and Solutions Opportunities

Technology Challenges Digital divide still exists Frequent complaints that schools did not have enough computers Frequent complaints that bandwidth not sufficient for assessment needs Testing interruptions – occur even when the best technology is available

Testing Interruptions CCSSO (2015) produced a report on testing interruptions: Recommendations for Addressing the Impact of Test Administration Interruptions and Irregularities Among the things defined as “testing interruptions” were: Delayed log-in System delays in loading items System slows down Students are unexpectedly logged out Students lose all progress Items or stimuli are rendered poorly Accessibility issues

Accessibility Issues* Resources not available or not working properly Example: Highlighter tool is not available for all or a portion of the administration time Accessibility features not working properly Example: Read-aloud feature is not functioning for a portion of the administration time *Identified by CCSSO

Importance for Students with Disabilities May affect students with disabilities differently from other students* Example: Student with a disability might be frustrated and not continue to try when other students are not as frustrated and do continue to try *Effect may be direct or indirect

Framework for Analyzing Effects of Test Interruptions See: http://www.ccsso.org/Documents/Computer%20Based%20Interruptions.pdf CCSSO framework provides both a way to analyze the effects of test interruptions and a way to determine appropriate treatment of potentially affected scores. Step 1 is intended to verify that data are clean and to create the necessary additional data elements to conduct analyses. Step 2 is intended to determine the prevalence of each type of test interruption for student subgroups (e.g., disability) and educational entities (e.g., school or district). This step can identify subgroups that should be closely examined in subsequent steps. We won’t go though all the steps here, but recommend that you read the CCSSO document.

Terminology Challenges Many different terms now being used to mean the same thing Same terms also are used to mean different things Next slide shows terminology used in six assessment consortia

Accessibility Frameworks Type of Assess-ment Assessment For All Participating Students For Some Students with Educator Input For Few Students with Documented Needs General Assess-ment PARCC Features for All Students Accessibility Features Identified in Advance Accommodations Smarter Balanced Universal Tools Designated Supports ELP Assess-ment ELPA21 Universal Features Designated Features WIDA ASSETS Accessibility Tools Alternate Assess-ment DLM Supports Provided within DLM via PNP Supports Requiring Additional Tools/Materials NCSC Optimal Testing Conditions Accessibility Features Note the DIAMOND White Paper calling for common terminology across consortia. Also note the CCSSO presession.

National Education Technology Plan (2016) “Future of Assessment” compares traditional and next generation assessments, with the goal of “more flexibility, responsiveness, and contextualization” – accessibility that is universally designed in one of the features of next generation assessments: When both assistive technologies and assessments effectively interoperate, students are better able to demonstrate what they know and how to apply this knowledge. (NETP, 2016, p. 57)

Accessibility Challenges Inconsistent testing platforms for different assessments May have been more of a challenge for adults than for students, but still a challenge!

Accessibility Challenges Scrolling issues An issue for all students mentioned by all consortia May have a greater impact on students with disabilities and English learners than other students

Accessibility Challenges Accessibility features and accommodations did not work as intended Not able to turn off universal features (an important designated feature) Limitation on number of re-reads did not activate

Accessibility Challenges Guidance on how to configure computers and other devices to handle videos not available Tutorials for students needed to be more “kid friendly”

Accessibility Challenges Use of accessibility features and accommodations was not tracked Ability to track the USE of accommodations (and other accessibility features) was a huge promise of technology-based assessments No consortium assessment was able to do this –only could track the accessibility features and accommodations that were SELECTED for a given student

Accessibility Challenges Many concerns related to the use of assistive technology (AT) Lack of security with “3rd party” AT Incompatibility with “3rd party” AT Result is often that students are not allowed to use AT they regularly use during instruction

Some Solutions Develop test security policies and procedures to address AT issues (e.g., erase assessment content; turn off features not allowed on assessment) Allow the student to use AT that is regularly used during instruction (unless it would compromise test validity.

Successfully Using AT Students need to be familiar with AT they will use on the assessment. Ensure that students using AT have access to a practice test session that enables them (and their teachers) to see that the AT works as intended in the test environment. AT compatibility issues should be checked before test day. Tools and procedures should be developed to make it easy for LEAs to report any compatibility issues. Procedures also are needed that address how reported issues will be addressed.

More Solutions Require vendors to demonstrate their capabilities before signing a contract! Ensure that practice tests have the same embedded (and non-embedded) accessibility features and accommodations as the actual assessment. Check out the technology, not just the assessment items. Involve students in cognitive labs and usability studies. Include students with disabilities in infrastructure trials prior to testing.

Opportunities Race-to-the-Top Assessment Consortia were a first step (and the source of issues already highlighted) Innovative Assessment Program is a next opportunity for large-scale assessments – NOW IN COMMENT PERIOD, until May 18, 2016. See: https://www.gpo.gov/fdsys/pkg/FR-2016-04- 18/pdf/2016-08726.pdf Note that the Innovative Assessment Program is part of the Enhanced Assessment Initiative, which provides grants to state departments of education.

Innovative Assessment Program Priority 1: Developing Innovative Assessment Item Types and Design Approaches develop, evaluate, and implement new, innovative item types for use in summative assessments in reading/language arts, mathematics, or science; develop new approaches to transform traditional, end-of- year summative assessment forms with many items into a series of modular assessment forms, each with fewer items for either a or b, provide a dissemination plan such that models and resources can be shared with states across nation

Innovative Assessment Program Priority 2: Improving Assessment Scoring and Score Reporting develop innovative tools that leverage technology to score assessments; propose projects, in consultation with …to address needs related to score reporting and improve the utility of information about student performance… for either a or b, provide a dissemination plan such that models and resources can be shared with states across nation

We want to hear from you! Please click on the link in the chatbox and complete the short webinar survey.