Psychometric Issues in the Use of Testing Accommodations Chapter 4 David Goh

Testing Accommodations and Psychometric Soundness Federal and state laws require that nonbiased procedures be used in assessment But does using testing accommodations make the technical aspects of the tests questionable? –Reliability: accuracy –Validity: effectiveness, consistency and stability of scores

Standardization Standardization implies uniformity of procedures in administering a test and scoring the results Poses a problem for diverse learners who exhibit different physical, sensory, linguistic, cultural and or psychological qualities from the general population In theory all accommodations compromise standardization, so issues of validity and reliability need to be examined

Reliability Refers to consistency and stability High reliability produces consistent results Low reliability produces inconsistent results Ex- ELLs using a bilingual interpreter may make different interpretations Accommodations and modifications may increase measurement errors and decrease reliability Methods of examining reliability include comparing scores of the same test takers on different administration, with different sets of times, or with different scores or examiners

Validity Refers to effectiveness- is the test testing what it intended If used right- accommodations can increase validity If used inappositely, accommodations may nullify the results Types: content, response process, internal structure, other variables, and consequences of testing

Effect of Accommodations Limited evidence on the effect of test accommodations on reliability and validity Studies have shown that testing accommodations on SAT and GRE did not significantly effect reliability and validity –However, findings are not generalizable –But, some accommodations have been found to increase student performance- ex- extended time

Flagging Test Scores On large scale test a very controversial issue –Reveals examinee has a disability; prompts invalid inferences about score and prejudice; identifies and violates person with disabilities privacy that is protected under the law If comparability can be demonstrated between standard and non-standard administration, there is no need to flag –However, if test results are not comparable, then testing accommodations need to be noted and explained