Partnership for Accessible Reading Assessment Partnership for Accessible Reading Assessment (PARA) Research Martha Thurlow National Center on Educational Outcomes University of Minnesota

Partnership for Accessible Reading Assessment Research Program Assumptions Assumption #1: We do not know everything about what goes into accessible reading assessment yet. Assumption #2: Preliminary research must inform design of accessible assessment. Assumption #3: Both preliminary and experimental research will inform development of Principles and Guidelines for future assessments.

Partnership for Accessible Reading Assessment Three Major Research Question Areas What characteristics of current assessment practices hinder accessibility? What characteristics of students require more accessible assessment of reading? What characteristics would accessible assessment have?

Partnership for Accessible Reading Assessment Two Research Phases Exploration Accessible Assessment Characteristics Student Characteristics Current Assessment Characteristics Study Phases Confirmation

Partnership for Accessible Reading Assessment Sample of Studies Exploration Accessible Assessment Characteristics Student Characteristics Current Assessment Characteristics Study Phases Confirmation State Documents Analysis – Identifying what is assessed in reading relative to state standards (via test specifications), analyzing accommodation policies for reading assessments, and examining current test forms for characteristics that promote and hinder accessibility. State Documents

Partnership for Accessible Reading Assessment Mean Percent of Items by Construct (across states)

Partnership for Accessible Reading Assessment Sample of Studies Exploration Accessible Assessment Characteristics Student Characteristics Current Assessment Characteristics Study Phases Confirmation Literature Scans+ – Literature and Web site reviews, as well as input and feedback from organizations and experts to describe disabilities, general approaches to reading instruction, and assessment of reading for those disabilities that affect reading. Literature+

Partnership for Accessible Reading Assessment Sample of Studies Exploration Accessible Assessment Characteristics Student Characteristics Current Assessment Characteristics Study Phases Confirmation Expert Review – Using a variety of procedures (focus group, MACB, Delphi), select accessible assessment practices based on likelihood of effectiveness (improve performance), validity (retain reading construct integrity), feasibility (practical to implement) Expert Review

Partnership for Accessible Reading Assessment Sample of Studies Exploration Accessible Assessment Characteristics Student Characteristics Current Assessment Characteristics Study Phases Confirmation Pictures Study – Pictures or other interest items (decorative graphics, icons, etc.) that are not needed to answer test items are randomly assigned to control and experimental groups to confirm which practices differentially suppress performance. Pictures Study

Partnership for Accessible Reading Assessment Sample of Studies Exploration Accessible Assessment Characteristics Student Characteristics Current Assessment Characteristics Study Phases Confirmation Nomination + Scrutiny Study – Information gathered from exploration phase used to solicit teacher nominations of students who are poorly assessed under current assessment practices. Students will be observed, individual assessment data examined, and think aloud procedures used to confirm the need for accessible assessment. Nomination + Scrutiny

Partnership for Accessible Reading Assessment Reasons For Inadequate Measurement of Students Reading Knowledge and Skills 1.Limited reading fluency obscures comprehension capabilities. 2.Difficulty with sophisticated comprehension obscures basic reading skills. 3.Has strengths that typical tests do not cover. 4.Responds poorly to standardized testing circumstances or materials.

Partnership for Accessible Reading Assessment Sample of Studies Exploration Accessible Assessment Characteristics Student Characteristics Current Assessment Characteristics Study Phases Confirmation Differential Boost Study – Conduct experiments with students (those identified from Nomination Study as inaccurately assessed by current state tests) using random assignment to experimental and control groups to confirm which practices (e.g., no interest graphics, motivation) differentially enhance (boost) performance. Differential Boost Study