Tests and Measurements Intersession 2006

Test Bias Ethnic groups Why individual differences? Environment? Biology? Differential validity Language base of IQ tests Focus of criticism Individual items, classes of items Differential item functioning (DIF) analysis

Test Bias Criterion-related sources of bias Differential prediction of scores? Regression revisited One regression equation vs. two Equal slopes = equal predictive evidence

Prediction Bias Test score Criterion score Test score Criterion score Group A Group B Test score Criterion score

Testing Minority Group Members Outlaw tests? Develop alternative strategies Basic assumption Ignorance vs. stupidity The Chitling Test Theory: Hodos and Campbell (1969) African American specific body of information (mid 1960’s culture) Validity:

Black Intelligence Test of Cultural Homogeneity (BITCH) Williams et al.: scientific racism Survival Quotient (SQ) BITCH 100 vocabulary words relevant to African-American culture AA children > white children Reliability: AA > white Validity: poor Children unfairly assigned to EMR

System of Multicultural Pluralistic Assessment (SOMPA) Different bases of knowledge All groups have same average potential 3 approaches: Medical Social-system Pluralistic Validity

Solutions Unqualified Individualism Quotas Qualified Individualism Models of Fairness Regression Constant ratio Cole/Carlington Quota

Solutions New Interpretations of Data Differences in problem-solving approaches by group Differential process theory (Goldman) Different types of intelligence Different Criteria Usually standardized tests Opportunity to learn vs. ability to learn Grades Academic performance vs. career performance

Changing Society Social and economic inequality reflected in tests Aptitude? Criterion prediction Test or society?

Future of Testing –Professional Issues Theoretical Concerns Reliability based on stability Trait question: Stable characteristics that hold despite environment Trait? Environment? Both? Intelligence: adaptability Index of competence Underlying theory

Future of Testing –Professional Issues Adequacy of Tests How are tests used? Validity documentation should be improved for all tests Selection bias Prediction Accuracy: actuarial vs. clinical prediction

Moral Issues Human rights Right not to be tested (informed consent) Right to know results Confidentiality Labeling Stigma Access to help Medical label = not responsible

Moral Issues Invasion of privacy Limitations of tests Is it wrong to know about a person? Misuse of information Limits of confidentiality Divided loyalties IP Adverse decision Responsible test use

Social Issues Dehumanization Computer interpretation Usefulness of tests Better predictions Risk-benefit analysis Access to service Who refers? Who pays? Physicians