Classroom Assessment: Bias Characteristics of a test that offend or unfairly penalize examinees because of their membership in particular groups or populations: Religion. socioeconomic status. race, ethnicity or gender. Geography. Bias can result from offensiveness when the content of the test (or items) offend certain classes of individuals. E.g., when the items or tasks reflect stereotypes. When menial tasks are depicted with women, while more important tasks are depicted with men. Such bias compromises the validity inferences based on the assessment. Bias can result from unfair penalization when particular classes of students’ performance on the assessment suffers because the content (or tasks required) in the assessment is unfamiliar or foreign to them (the students). E.g., Higher socioeconomic students may hold an advantage when an assessment relies to heavily on experiences generally reserved for those families who can afford the experience. Vacations to historic sites. Access to cable TV. Unfair penalization results only when it is not the students poor accomplishment that leads to poorer performance.

Further Considerations of Bias and Validity Two kinds of bias (from Howe, 1995) Predictive and Criterion Predictive bias: External: Concerned with the question, “How well does the test predict the performance it is design to measure?” Internal: Concerned with how well items in the test predict overall performance on the test.

More on bias and validity Criterion bias: Across criteria bias: When the performance measured by the test is overly emphasized relative to other (often desirable) criteria. Within criterion bias: When the performance being measured by the test is (partially) differentially defined by the test itself.

Fairness is Required in All Assessments Assessments need to be fair to students from all ethnic and socioeconomic backgrounds. Is the language in word problems interpreted by students of all backgrounds? Does the assessment accommodate students with disabilities? Is the assessment free of artifacts that perpetuate racial, ethnic, or gender stereotypes?

Poor Performance  Bias Just because different classes of individuals perform differently on an assessment does not mean that the assessment is biased against one or more of the groups. All assessments are biased against ignorance. Poor performance can indicate poor achievement.

Strategies for Eliminating Bias in Assessments Judgmental Approaches. Review panels. Judgements regarding specific items and overall test content. Differential Item Functioning (DIF) procedures. Being sensitive to the possibility of bias in classroom assessments. Judgmental Approaches Bias review panels. Bias review panels are routinely employed by crafters of high stakes assessments (such as the EOGs and EOCs). These panels are also routinely employed by commercial test developers. Item and test content review. See page 73 in Popham for a typical instruction to judges to review individual items. See page 74 in Popham for a typical instruction to judges to review the overall content of an assessment. Differential Item Functioning A set of complex statistical procedures. Looks at the relationships between performance on items (or overall assessment) and class membership. DIF does not necessarily imply bias. Research has shown that on math assessments, woman perform better than males on easy items, while men perform better than woman on difficult items. Being sensitive to bias Recognize that other groups of individuals may not see the world the way you do. They may not learn, study, behave, the way you do. Particularly in performance and portfolio assessments.

End