Do we have enough evidence on the validity of accommodations to justify the reporting of accommodated assessments? Jamal Abedi University of California, Davis National Center for Research on Evaluation, Standards and Student Testing (CRESST)

Do we have enough evidence on the validity of accommodations to justify the reporting of accommodated assessments?  Different forms of accommodations have been used for ELLs and SDs nationally (Abedi, Kim-Boscardin, & Lardon, 2000; Rivera, Stansfield, Scialdone, & Sharkey, 2000; Thurlow, & Bolt, 2001)  However, not enough validity data are available  Existing research on some forms of accommodations may not be conclusive

Research Findings Braille version of the test may be more difficult for some items than other items, i.e., items with diagrams, special symbols (Bennett, Rock, & Kaplan, 1987; Bennett, Rock, & Novatkoski, 1989; Coleman, 1990) Computerized assessment: Some studies suggest that this accommodation increased performance of students (Russell & Haney, 1997; Russell, 1999; Russell & Plati, 2001) Other studies did not find computerized assessment to be effective (MacArthur & Graham, 1987) Or even not as effective as traditional assessments (Hollenbeck, Tindal, Stieber & Harniss, 1999; Watkins & Kush, 1988; Varnhagan & Gerber, 1984)

Research Findings (continues…)  Dictate Response to a Scribe has been shown to have impact on the performance of students with disabilities (Fuchs, Eaton, Hamlett, & Kams, 2000; MacArthur & Graham, 1987)  Tippets & Michaels (1997) found this accommodation in combination with other accommodations such as read aloud and extended time to help students with disabilities. However, there are concerns over the validity of this accommodation.  Koretz (1997) found this accommodation to help students with learning disabilities

Research Findings (continues…)  Extended Time is one of the most commonly used accommodation for students with different types of disabilities  Some studies found extended time to be an effective accommodation for students with disabilities particularly for learning disability (Chiu and Pearson, 1999; Gallina, 1989)  However, other studies did not show extended time to be effective for students with disabilities (Munger & Loyd, 1991; Fuchs et al., 2000; Marquart, 2000)  Studies on the effect of extended time on language arts did not find this accommodation to be effective (Munger & Loyd, 1991; Fuchs et al., 2000)  The issue is that extended time may affect performance of both SD and non-SD students, therefore, makes the validity of this accommodation suspect

Research Findings (continues…)  Large Print: Bennett, Rock, and Kaplan (1987) revealed that using this accommodation for visual impaired students does not affect the construct under measurement  Other studies suggest that extra time may be needed with this accommodation (Wright & Wendler, 1994)  Large print has also been used for students with learning disabilities  Most studies show no impact of this accommodation for students with learning disabilities; one study, however, showed some impact for students with learning disabilities (Perez, 1980)

Research Findings (continues…)  Mark Answer in Test Booklet: Research found no significant difference between those tested under this accommodation and those using separate answer sheets (Rogers, 1983; Tindal, Heath, Hollenbeck, Almond, & Harniss, 1998)  Read Aloud Test Items Studies found this accommodation to be valid in math assessments (Tindal et al., 1998)  There are concerns over the use of this accommodation in reading and listening comprehension tests (see for example, Phillips, 1994; Burns, 1998) that may affect the validity of assessment (Meloy, Deville, & Frisbie, 2000; Bielinski, Thurlow, Ysseldyke, Freidebach, & Freidebach, 2001)

Research Findings (continues…)  Test Breaks: Some studies found that students scored significantly higher than those under standard testing conditions (DiCerbo, Stanley, Roberts, & Blanchard, 2001)  Middle and low-ability readers benefited more from this accommodation than high-ability readers  However, another study (Walz, Albus, Thompson, & Thurlow, 2000) found that students with disabilities did not benefit from a multiple-day test administration while students without disabilities did benefit

Research Findings (continues…)  Providing an English Dictionary (Abedi, Courtney, & Leon, 2003; Abedi, Lord, Kim, & Miyoshi, 2000) and extra time (Abedi, Lord, Hofstetter, & Baker, 2000; Hafner, 2001; Thurlow, 2001) affects performance of all students (see also, Maihoff, 2002; Thurlow & Liu, 2001)  Translation of assessment tools into students’ native language may not produce desirable results and may even provide invalid assessment results if the language of instruction and assessment is not aligned (Abedi, Lord, Hofstetter, & Baker, 2000)

Conclusion  In spite of the concerns expressed by researchers over the validity, effectiveness and feasibility of some forms of accommodations, these accommodations are used frequently by states and districts across the nation  That is, decisions on the use and type of accommodations for English language learners and students with disabilities do not seem to have been influenced by research findings  Results of accommodation studies should be presented clearly so they can be useful for the accommodation decision makers

For more information contact: Jamal Abedi: (530)