1. Qualitative Meta-Analysis/Data Synthesis of Recent Research on Validity of Test Accommodations for Students with Disabilities Mohamed Dirir Connecticut Dept. of Education Presented at the 2012 National Conference on Student Assessment Minneapolis, MN

2. Background Purpose of CT EAG Project: Conduct studies of the validity of score interpretations from selected test accommodations used for students with disabilities Five states volunteered for the studies: Connecticut (lead state), Kentucky, Michigan, and Nevada Work was coordinated by CCSSO Deliverables were: A guidebook of the findings Extensive database of the studies Comprehensive technical report Lessons learned document

3. Background (contd.) The compiled information can be used by states as references or validity evidence to support the use of certain accommodations The outcomes of the CTEAG project then led to a meta analysis of studies on test accommodation The results of the meta analysis will be presented later in the session

4. Design and Data Samples of examinee in each state were two groups: Students with disabilities (SWD) and students without disabilities (SWOD) Two parallel spilt-half forms were developed for each study A 2X2 counterbalanced design in which disability group (SWD or SWOD) was crossed with testing administration condition (accommodated or regular)

5. Connecticut # 1Connecticut # 2 NevadaMichiganKentucky Grade studied 75764 AccommodationText Reader Reading Enhanced directionsRead aloud TestReadingMathematics Language ArtsReading 2x2 design followed? Alternate Design Adequate Sample SUMMARY OF CTEAG STUDIES

6. Research Questions Validity of test scores and their interpretations for SWD provided with accommodations Do test accommodations affect item and test properties? Do test accommodation affect test construct? Do accommodations affect test scores for SWD more than test scores for SWOD?

7. Results In four of the five studies, there were no differences in classical item statistics and reliability indices between the two test administration conditions Exploratory and confirmatory factor analysis revealed that test accommodations do not alter test dimensionality and structure Using Repeated Measures ANOVA, significant main effect (accommodation) was found in 3 of the 5 studies. Significant interaction effect (Disability status X Testing condition) was found 3 of the 5 studies.

8. Example: Means for Kentucky

9. Accommodation EffectsConnecticut # 1Connecticut # 2NevadaMichiganKentucky Item statistics not changed NA Change Test dimensionality not changed Test structure not changed Significantly increased test scores Borderline Significant (P=0.0523) Significantly increased test scores for SWD more than SWOD (Disability X Accommodation) Summary of CTEAG Results

10. Recommendations Additional studies such as the CTEAG should be done to look at validity for other accommodations, grades, content areas. More states should do these types of validity studies using this research design. The database developed for CTEAG could be expanded as data and results from similar studies become available.