Qualitative Meta-Analysis/Data Synthesis of Recent Research on Validity of Test Accommodations for Students with Disabilities Mohamed Dirir Connecticut.

Slides:

Advertisements

Similar presentations

Critical Reading Strategies: Overview of Research Process

Advertisements

Designing Accessible Reading Assessments National Accessible Reading Assessment Projects General Advisory Committee December 7, 2007 Overview of DARA Project.

Listening. Learning. Leading. Using Factor Analysis to Investigate the Impact of Accommodations on the Scores of Students with Disabilities on English-Language.

Fairness in Testing: Introduction Suzanne Lane University of Pittsburgh Member, Management Committee for the JC on Revision of the 1999 Testing Standards.

The Journey Toward Accessible Assessments Karen Barton CTB/McGraw-Hill Validity & Accommodations:

Evaluation of the Iowa Algebra Aptitude Test Terri Martin Doug Glasshoff Mini-project 1 June 17, 2002.

Designing Accessible Reading Assessments Reading Aloud Tests of Reading Review of Research from the Designing Accessible Reading Assessments Projects Cara.

Issues of Technical Adequacy in Measuring Student Growth for Educator Effectiveness Stanley Rabinowitz, Ph.D. Director, Assessment & Standards Development.

Greenville Technical College Assessing and Developing Student Computing Technology Skills September 19, 2012 Dr. Lenna Young, Mark Krawczyk, and Mary Locke.

LEP Updates: Testing Accommodations & Alternate ACCESS Carrie Perkis Division of Accountability Services September 12, 2012.

Michigan Merit Exam – To Be or Not to Be? OEAA 2006 Conference Sessions 44 and 50.

PARCC Accommodation: Text-to-Speech, Screen Reader Version, ASL Video, Human Reader/Human Signer For the ELA/Literacy Assessment December 2014.

Work Session 1 Locating Baseline Data Office of Research, Evaluation and Policy Studies Ximena D. Burgin, Ed.D. November 30 th, 2010.

Large Scale Assessment Conference June 22, 2004 Sue Rigney U.S. Department of Education Assessments Shall Provide for… Participation of all students Reasonable.

BASIC TERMINOLOGY n Test n Measurement n Evaluation.

Meeting NCLB Act: Students with Disabilities Who Are Caught in the Gap Martha Thurlow Ross Moen Jane Minnema National Center on Educational Outcomes

Georgia Modification Research Study Spring 2006 Sharron Hunt Melissa Fincher.

The State of the State TOTOM Conference September 10, 2010 Jim Leigh Office of Assessment and Information Services Oregon Department of Education.

N C E O National Center on Educational Outcomes Helping Students Select and Use Test Accommodations Martha L. Thurlow Sandy Thompson University of Minnesota.

Designing Accessible Reading Assessments Research on Making Large Scale Assessments More Accessible for Students with Disabilities Institute of Education.

1 The New York State Education Department New York State’s Student Reporting and Accountability System.

Chapter 4 Evaluating and Creating Interactive and Content- Based Assessment.

The University of Central Florida Cocoa Campus

Psychometric Issues in the Use of Testing Accommodations Chapter 4 David Goh.

Foundations of Recruitment and Selection I: Reliability and Validity

SESSION 2 EVALUATION INSTRUMENT FOR ASSESSMENT OF PROGRAMME ACCREDITATION.

Accommodations in Oregon Oregon Department of Education Fall Conference 2009 Staff and Panel Presentation Dianna Carrizales ODE Mike Boyles Pam Prosise.

Alternate Assessment Attainment Task Part 2.1 Click here to download the Administration Guide Required for completion of this training Overview/Attainment.

Cara Cahalan-Laitusis Operational Data or Experimental Design? A Variety of Approaches to Examining the Validity of Test Accommodations.

Evaluating a Research Report

CCSSO Criteria for High-Quality Assessments Technical Issues and Practical Application of Assessment Quality Criteria.

Karen Stein Understanding, Selecting and Documenting Accommodations.

Assessing Students with Special Needs Alternate Assessments.

CALIFORNIA DEPARTMENT OF EDUCATION Tom Torlakson, State Superintendent of Public Instruction Enhanced Assessment Grant: English Language Proficiency Assessment.

Issues in Selecting Assessments for Measuring Outcomes for Young Children Issues in Selecting Assessments for Measuring Outcomes for Young Children Dale.

Illustration of a Validity Argument for Two Alternate Assessment Approaches Presentation at the OSEP Project Directors’ Conference Steve Ferrara American.

Martha Thurlow Laurene Christensen Courtney Foster April 22, :15-2:15 MONITORING ACCOMMODATIONS FOR INSTRUCTION AND ASSESSMENT.

Modifying Achievement Test Items: A Theory-Guided & Data-Based Approach Stephen N. Elliott Learning Sciences Institute and Department of Special Education.

Advanced Research Methods Unit 3 Reliability and Validity.

Enhancing the Technical Quality of the North Carolina Testing Program: An Overview of Current Research Studies Nadine McBride, NCDPI Melinda Taylor, NCDPI.

CAROLE GALLAGHER, PHD. CCSSO NATIONAL CONFERENCE ON STUDENT ASSESSMENT JUNE 26, 2015 Reporting Assessment Results in Times of Change:

Regional Assessment Network (RAN) Update Chun-Wu Li, Ph.D. Assessment and Accountability Services Riverside County Office of Education November 22, 2013.

Connecticut SDE Bureau of Student Assessment CTEAG DISSEMINATION WORKSHOP WITH ASES AND TILSA SCASS GROUPS Wednesday, June 23, 2010.

Establishing the Validity of Test Accommodations for Students with Disabilities: A Collaborative of State-based Research CTEAG Project Summary of Accomplishments.

JS Mrunalini Lecturer RAKMHSU Data Collection Considerations: Validity, Reliability, Generalizability, and Ethics.

Assessment: Reliability & Validity. Reliability Refers to the repeatability of a given testing instrument The extent to which a student would be expected.

PARCC Accessibility Features and Accommodations Manual Training for Parents Presented on November 20, 2014 Presented by the: Office of Special Education.

Measuring and Improving Child and Family Outcomes Conference New Orleans, Sept 19-21, 2011 Using the Child Outcomes Measurement System (COMS) Self-Assessment.

EGR 335 Technical Communications for Engineers and Computer Scientists Coordinator: Leo Finkelstein, Jr. Course Review.

Jamal Abedi, UCLA/CRESST Major psychometric issues Research design issues How to address these issues Universal Design for Assessment: Theoretical Foundation.

Report on the NCSEAM Part C Family Survey Batya Elbaum, Ph.D. National Center for Special Education Accountability Monitoring February 2005.

So What is Going to be Happening with State Assessment for Students with Disabilities for 2007/2008? Peggy Dutcher Fall 2007 Assessment and Accountability.

Math Performance Tasks: Scoring & Feedback Smarter Balanced Professional Development for Washington High-need Schools University of Washington Tacoma Maria.

1 National Center on Educational Outcomes (NCEO) Martha L. Thurlow Accommodations for Students with Disabilities Decision Making Considerations for Validity.

WIDA ACCESS Testing Information Session & Community Literacy Resources Parents as Educational Partners Tuesday, January 13, 2015 Jonathan Hudgens- WIDA.

Qualitative Meta-Analysis and Data Synthesis of Recent Research on Test Accommodations for Students with Disabilities: Summary and Implications John F.

Principles of Language Assessment

Accessibility Features and Accommodations

Connecticut Enhanced Assessment Grant

Proctoring the PARCC Assessment

Test Design & Construction

Tests and Measurements: Reliability

Reliability & Validity

AXIS critical Appraisal of cross sectional Studies

Instructional Practices in the Early Grades that Foster Language & Comprehension Development Timothy Shanahan University of Illinois at Chicago

Michigan’s Lessons and Uses of the CTEAG

Using outcomes data for program improvement

Evaluating MAT 7- Metropolitan Achievement Test

Examinations Council of Zambia

Qualities of a good data gathering procedures

Presentation transcript:

Qualitative Meta-Analysis/Data Synthesis of Recent Research on Validity of Test Accommodations for Students with Disabilities Mohamed Dirir Connecticut Dept. of Education Presented at the 2012 National Conference on Student Assessment Minneapolis, MN

Background Purpose of CT EAG Project: Conduct studies of the validity of score interpretations from selected test accommodations used for students with disabilities Five states volunteered for the studies: Connecticut (lead state), Kentucky, Michigan, and Nevada Work was coordinated by CCSSO Deliverables were: A guidebook of the findings Extensive database of the studies Comprehensive technical report Lessons learned document

Background (contd.) The compiled information can be used by states as references or validity evidence to support the use of certain accommodations The outcomes of the CTEAG project then led to a meta analysis of studies on test accommodation The results of the meta analysis will be presented later in the session

Design and Data Samples of examinee in each state were two groups: Students with disabilities (SWD) and students without disabilities (SWOD) Two parallel spilt-half forms were developed for each study A 2X2 counterbalanced design in which disability group (SWD or SWOD) was crossed with testing administration condition (accommodated or regular)

Connecticut # 1Connecticut # 2 NevadaMichiganKentucky Grade studied AccommodationText Reader Reading Enhanced directionsRead aloud TestReadingMathematics Language ArtsReading 2x2 design followed? Alternate Design Adequate Sample SUMMARY OF CTEAG STUDIES

Research Questions Validity of test scores and their interpretations for SWD provided with accommodations Do test accommodations affect item and test properties? Do test accommodation affect test construct? Do accommodations affect test scores for SWD more than test scores for SWOD?

Results In four of the five studies, there were no differences in classical item statistics and reliability indices between the two test administration conditions Exploratory and confirmatory factor analysis revealed that test accommodations do not alter test dimensionality and structure Using Repeated Measures ANOVA, significant main effect (accommodation) was found in 3 of the 5 studies. Significant interaction effect (Disability status X Testing condition) was found 3 of the 5 studies.

Example: Means for Kentucky

Accommodation EffectsConnecticut # 1Connecticut # 2NevadaMichiganKentucky Item statistics not changed NA Change Test dimensionality not changed Test structure not changed Significantly increased test scores Borderline Significant (P=0.0523) Significantly increased test scores for SWD more than SWOD (Disability X Accommodation) Summary of CTEAG Results

Recommendations Additional studies such as the CTEAG should be done to look at validity for other accommodations, grades, content areas. More states should do these types of validity studies using this research design. The database developed for CTEAG could be expanded as data and results from similar studies become available.