How biased is your assessment? Searching for realistic solutions to student concerns about fairness… Moira Mitchell Equality and Diversity Manager

The “Attainment Gap” Student Perceptions Anonymous Marking Unconscious Bias Other issues related to assessment

The Attainment Gap

NUS Race for Equality Report 2011 “When asked to speculate why minority ethnic students were less satisfied about their course and did not achieve as well as their peers, respondents consistently highlighted problems with the curriculum, academic environment, teaching quality, assessment, and academic support. At the heart of this discussion was the importance of equal treatment from teachers and tutors. Respondents expressed how essential it is to receive fair and balanced teaching, assessment and support – treatment that many stated was currently lacking, to some extent, at their current institution.”

Anonymous Marking “a process designed to eliminate both conscious and unconscious bias on the part of the examiners; to protect them from accusations of bias and discrimination; and to reassure students that the process is fair and impartial” University of Bradford “to protect candidates against bias, conscious or otherwise, on the part of examiners” Durham University

Unconscious Bias

Daniel Kahneman Thinking, Fast and Slow System 1: fast, automatic, frequent, subconscious System 2:slow, effortful, infrequent, conscious

Systems 1 and 2

System 1 Heuristics: Experience based techniques for problem solving and discovery “Where exhaustive methods are impractical, heuristic methods speed up the process of finding a solution via mental shortcuts to ease the cognitive load of decision making” (Wikipedia) Examples: rule of thumb, educated guess, intuitive judgement, stereotypin g

Unconscious Bias “Jumping to conclusions is efficient if the conclusions are likely to be correct and the costs of an occasional mistake acceptable, and if the jump saves much time and effort. Jumping to conclusions is risky when the situation is unfamiliar, the stakes are high, and there is no time to collect more information.”

Social categorisation theory Interacting with other people we categorise them for efficient decision making (System 1). Stereotype A formulaic and often over-simplified conception of a person or group, usually based on limited knowledge. In-groups and Out-groups Our “in-group” members have a shared identity, characteristic or interest to us. Bias Control The extent to which people are motivated to control their biases varies according to the individual and circumstances.

Techniques for Reducing Unconscious Bias 1. Discounting commonly held stereotypes. 2. Using context to influence implicit responses. 3. Changing the way an out-group member is evaluated and categorised. 4. Using contact to change the level of threat evoked by an out-group. 5. Using motivation to change responses to an out-group. 6. Encouraging people to take responsibility for their implicit bias.

Academic Standards

Sue Bloxham (drawing on many others…) Descriptions of external examining occur within a techno-rationalist discourse, assuming knowledge is “monolithic, static, universal” and that staff are acculturated into “guild knowledge”. There is an assumption that a “gold standard” existed pre-massification and that standards are independent of those who create or guard them. This sits in contrast to a socio-cultural perspective which emphasises the “situatedness of practice” and a social constructivist approach which emphasises the “constructedness of knowledge”. Educational research from different traditions emphasises the interpretive, personalised and subjective nature of standards.

“In cases of uncertainty or complexity, expert judgement utilises a number of simplifying cues (heuristics) because a more rational and detailed judgement is too difficult or time consuming. Such heuristics allow for systematic biases to creep in, of which the expert is unaware and may lead to assessors predicting performance in advance of completing reading. For example, recent research has identified that knowledge of the student group, group stereotyping and surface features or work can be sources of bias.” (External Examining: fit for purpose? Bloxham, S. and Price, M.)

Fairness Fairness in multicultural assessment systems Gordon Stobart (2007) We know that the form of assessment can differentially affect results for different groups. “There is no cultural neutrality in assessment or in the selection of what is to be assessed.” Assessments tend towards standardising knowledge, thus encouraging assimilation. There is a tension between diversity and issues of interpretation and comparability.

Fairness in Assessment “The best defence against inequitable assessment is openness. Openness about design, constructs and scoring, will bring out into the open the values and biases of the test design process, offer an opportunity for debate about cultural and social influences, and open up the relationship between assessor and learner. These developments are possible, but they do require political will” (Gipps quoted in Stobart).

Stobart’s Questions: 1.What are the nature and requirements of the assessment system itself (practicalities) 2.How does the assessment content reflect experiences of different groups? 3.How do the assessment methods meet the diversity of the candidates? 4.How is the performance of different groups measured and how is this fed back into the system?

Questions for further discussion: 1.Do you have concerns about bias in the assessment of your programme(s)? 2.If so, is there evidence for the concerns (e.g. differential outcomes) 3.Do you think greater awareness of unconscious bias would help? 4.In light of what has been discussed today, what are your thoughts on anonymous marking?