Planning for the Successful Inclusion of Your Students in Computer-Based Tests NCEO University of Minnesota Martha Thurlow Deb Albus Laurene Christensen Yi-Chen Wu Sheryl Lazarus

Background

Brief Analysis of State Administered CBTs

Search for state CBTs Searched state department of education Web sites late October/November, 2009 Verified in February/March,

Formative/ Diagnostic Regular (with Exit) End of Course Alternate based on Modified Achievement Standards English Language Prof. Test # States # Tests Table 1. Number of States with CBTs Found by Type of Assessment

Table 2. Approaches of States Offering Paper Versions of Regular CBT Any Accomm. Requires Paper Certain Accomm. Requires Paper Paper Version is an Accomm. Other Situations Needing Paper Anyone May Use PaperProhibited TX CT FL OR VA KS OK CT MD MN UT WV IN MD None # States

Table 3. Approaches of States Offering Paper Versions of Computer-based End of Course Tests Any Accomm. Requires Paper Certain Accomm. Requires Paper Paper Version is an Accomm. Other Situations Needing Paper Anyone May Use PaperProhibited None OKLA MS SD GA IN MD MO SC TX VA None # States

Accommodations Considerations for Computer-based Assessments

Current Accommodations Policies Collecting policy information since 1992 Synthesis report every two years Current data is from 2009 Does not specifically address computer-based assessment Goal is to describe what is, not to determine whether or not states comply with NCLB/ESEA

Analysis Procedures Obtain publicly available information from state websites (all states plus DC) Read the participation and accommodation policies and enter data Verify the data with states Write the report Data is made available through a searchable online tool called the Data Viewer (

Relevant Accommodations AccommodationState Policy Computer or Machine Allowed=33 Allowed with some qualification=13 Prohibited=1 Communication Device Allowed= 21 Allowed with some qualification=8 Prohibited=0 Spell Checker/Assistance Allowed=8 Allowed with some qualification=13 Prohibited=16 Speech/Text Device Allowed=12 Allowed with some qualification=14 Prohibited=1 Assistive Technology Allowed=16 Allowed with some qualification=13

Future Accommodations on Computer-based Platforms Read-aloud Sign interpretation Highlighting Masking/templates Refreshable Braille Calculator Manipulatives Magnification/large print Sound amplification Others?

Test Read Aloud Directions-Computer The following information applies only to students assigned the Tests Read Aloud accommodation. Before beginning a test, ensure the following: ● MP3 files are not blocked or restricted on computers used for testing students with this accommodation. Consult the technology coordinator or network administrator, if necessary. ● Headphones are available to students. Students may use personal headphones or ear buds. ● The mute setting is turned off. ● The volume is set to a reasonable level so that other students cannot hear the audio. Although directions for playing the audio and controlling the volume appear on the instructions page at the beginning of each test session, it is important to make sure students understand these tools before testing. Use the information below to prepare students for testing.

Test Read Aloud Directions-Computer (continued) Using the Audio Tool: ● Play All, Stop, and Play/Pause buttons are available using the audio tool at the top of the screen (figure 4.2). To hear everything showing on the screen including directions, the test item(s), and answer options, click Play All. The Play/Pause button will alternate between Play and Pause when clicked. To pause, click Pause. To resume audio, click Play. To stop, click Stop. ● The Volume control bar is also located at the top of the screen. To raise the volume, slide the Volume bar to the right. To lower the volume, slide the Volume bar to the left. EOC Test Administration Manual, April/May ● Audio icons appear to the left of each part of a test item (figure 4.2). Click on the icon next to each part of the item to hear the audio file for that part.

Considerations for Technology-Based Testing Accommodations 1.Do technology-based assessments mean that students will no longer need accommodations? 2.What is the difference between accommodations, embedded features, and good testing practices? 3.Who makes decisions about what is used during technology-based assessments, and when are those decisions made? 4.What training is needed for teachers and students? 5.What should be tracked and documented within the technology-based platform?

Do technology-based assessments mean that students will no longer need accommodations?

What is the difference between accommodations, embedded features, and good testing practices?

Definitions Accommodations are changes in materials and procedures designed to give access to students who need them, and to produce test scores with greater validity for those students. Embedded Features are interactive tools that are part of the test platform and used to customize the assessment forindividual test takers. Good Testing Practices are testing practices and procedures that should be available to all.

Who makes decisions about what is used during technology-based assessments, and when are those decisions made?

Technology-based accommodations require thinking through logistics!

What training is needed for teachers and students?

What should be tracked and documented within the technology-based platform?

The Preliminary Results for OAKS 23

Background information Oregon Assessment of Knowledge and Skills (OAKS) Students can choose they want to take paper or online version Every student in the state will have up to three opportunities to take the test One of the highest score will be marked as the “Best score” Document accommodation “Yes vs. No” 24

Research Questions How many students with disabilities take a paper version of the OAKS? Did students with disabilities have less opportunity to take online test compared to students without disabilities? In which month did students with disabilities generally take OAKS online and how does this compared to students without disabilities? How many students with disabilities used accommodations for OAKS online? What are the performance patterns of students with disabilities for OAKS online? 25

How many students with disabilities take a paper version of the OAKS? Very few students with disabilities took the paper version None of the students took the paper version more than once 26

Did students with disabilities have less opportunity to take online test? Reading No. SPED students have more opportunities to take the test than Non-SPED student Math 27

In which month did students generally take OAKS online? One opportunity Results are similar across content area SPED: more spread out from Nov. to April Non-SPED : Feb-April 28

In which month did students generally take OAKS online? SPED 1 st : Jan & Feb 2 nd : April & May Non-SPED 1 st : November & Jan 2 nd : April & May 29 Two Opportunities

In which month did students generally take OAKS online? SPED 1 st : Nov., Dec, & Jan. 2 nd : March & April 3 rd : May Non-SPED 1 st : Oct. &Nov. 2 nd : March & April 3 rd : May 30 Three Opportunities

How many students with disabilities used accommodations for OAKS online? Reading Less than 20% of SPED received accommodations across different opportunities in Reading. Math Less than 19% of SPED received accommodations across different opportunities in Math. 31

What are the performance patterns of students with disabilities for OAKS online? Reading Most of students took the test once. For students took the test twice, more than 65% of them got the best score on the second attempt [Non-SPED: 26/(14+26); SPED:25/38]. 32

What are the performance patterns of students with disabilities for OAKS online? Math Most of students took the test once. For students took the test twice, more than 69% of them got the best score on the second attempt [Non-SPED: 30/41; SPED:27/39]. 33

Conclusions SWD have more opportunity to take online test SWD (one opportunity): were more spread out across months. Less than 20% of students with disabilities used accommodations for OAKS online. The performance patterns of students with disabilities for OAKS online is similar to students without disabilities.

Questions? Comments?

Presenter Contact Information: Martha Deb Laurene Yi-Chen Sheryl