Presentation is loading. Please wait.

Presentation is loading. Please wait.

Universal Access to Assessments. Project Overview Four Implementing States: New Hampshire, Vermont, Rhode Island, and Maine Eight Partner States: Connecticut,

Similar presentations


Presentation on theme: "Universal Access to Assessments. Project Overview Four Implementing States: New Hampshire, Vermont, Rhode Island, and Maine Eight Partner States: Connecticut,"— Presentation transcript:

1 Universal Access to Assessments

2 Project Overview Four Implementing States: New Hampshire, Vermont, Rhode Island, and Maine Eight Partner States: Connecticut, Maryland, Montana, Iowa, Florida, North Carolina, South Carolina, and Georgia Goal: Examine the feasibility, effect, and capacity to deliver state achievement tests using a computer-based test delivery system specifically designed to provide universal access to test content

3 Operational Test Plan Spring 2009: Grade 11 Science Spring 2010: Grade 4, 8, & 11 Science Fall 2010: Grade 11 Reading & Mathematics

4 Operational Test Components - 2009 Notify Schools of NimbleTools Option - Email, Training Sessions, WebEX, Mailers, Website Orientation & Practice Tests - Mailed & Web Download Registration & Tool Assignment Secure Client Installation Disk Mailed Test Administration Manual Call Center

5 Tool Assignment

6

7 Characteristics of Read Aloud Students ComputerHuman IEP 477 (.79) 368 (.84) Non-IEP 125 (.21) 68 (.16) Total 602 436 Note: Six students were marked as NT and Human. These students were excluded from analyses.

8 Regression Analysis BSEBetaSig. Math.24.03.29<.001 Reading.32.02.39<.001 Computer2.1.47.12<.001 Paper Read Aloud and Computer Read Aloud

9 Survey Responses Overall Very Positive Feedback 92% of test administrators agreed or strongly agreed that it was easy to prepare computers for testing. 96% of test administrators agreed or strongly agreed that they would like their students to use NimbleTools in the future. 92% of test administrators agreed or strongly agreed that NimbleTools helped students appropriately access the test content. “The read aloud accommodation really allowed each student to work independently. They didn't need to wait for me to read to them and could work at their own pace.” “Students who utilized the computer based accommodations seemed to be less stressed or exhausted than those who took the paper and pencil method.” Suggestions for the Future “Allow open end responses to be typed.” “I had a student with a broken arm last minute – it would have helped to have been able to allow him to use NimbleTools.”

10 Operational Test Components - 2010 Notify Schools of NT Option - Low Vision RA - Email, Training Sessions, Mailers, Website Teacher & Student Orientation & Practice Tests with Tactile booklets - Mailed & Web Download Registration & Tool Assignment Secure Client Installation Disk Mailed/Web download Test Administration Manual Call Center

11 2010 Participation Grade 4Grade 8Grade 11Total Number of Students 6296066531888 Number of Schools 835052185

12 Low Vision Scripting

13 Tactile Overlays

14 Scripting Research Should all elements in a table, graph, or diagram be read to students in the order they are presented in the item or should these elements only be available to be read to students when clicked on? Should chemical equations be read to students using chemistry symbols (e.g. Na) or words (e.g. sodium)? When reading table elements to a student, should units of measure be included in the description?

15 Scripting Research Participating students were randomly assigned to one of four 21 item test forms. Each test form contained the same 21 items in the same order, but with variations on the scripting rules applied. After each set of items, students were asked two survey questions about their preference in the way the items were read aloud. Analyses will look at differences in performance and preference between scripting rules.

16

17 8 of 16 preferred automatic reading of all information in tables 6 of 16 preferred no automatic reading of information in tables 2 of 16 had no preference

18 Preferred - facilitated item comprehension –“It didn’t say the stuff on the chart. I know I can read it, but what if I didn’t know what Celsius was or how to pronounce it. It is helpful to read” –“I like how it did the table – it went over it so the grams and stuff so you don’t have to figure it out” Not Preferred – redundancy was distracting –“It doesn’t need to read it to me because I can read it to myself. If I had wanted it to read it to me I would click on it” –“I think it would quicker doing it this way (not read - #6) it would use up more of the time than reading stuff we don’t need to read”

19 12 of 16 students preferred formulas voiced as words 2 of 16 students preferred formulas voiced as letters 2 students had no opinion

20 Formulas as words facilitated comprehension of items and formulas –“It just confuses me – NA2 + CH3” –“I didn’t like how #15 was read. It was a lot easier to me so I know what the heck they are talking about” –“I like it and it explains the equation. If I was a real smart chemist I would know all these formulas, but I think reading the formulas is better”

21 2 of 16 students preferred no reading of units of measure 6 of 16 students preferred reading of units of measure 8 of 16 students had no opinion (the majority of these students did not recognize any differences in scripting)

22 “Annoying” factor (from the mouths of babes!) – proficient computer users used the word “annoyed” frequently when sitting through tutorials or items that took a perceived long time for scripting Pacing – some students preferred faster pace than test was scripted, some preferred slower pace. With current technology, natural reading cannot be adjusted for pace Every student preferred Nimble Assessment to previously taken assessments


Download ppt "Universal Access to Assessments. Project Overview Four Implementing States: New Hampshire, Vermont, Rhode Island, and Maine Eight Partner States: Connecticut,"

Similar presentations


Ads by Google