1. Evaluating implicit bias training
Dr Jules Holroyd, Dr Joseph Sweetman

2. Evaluation: relative to aims
How well did the session achieve what we intended it to?

3. Putative aims -tick boxes -change individual behaviour
-devolve institutional responsibility for dealing with bias/diversity/inclusion to individuals
-get people to implement specific strategies to change their own biases
-get people to implement specific strategies to change their own behaviour
-function better as an organisation (in terms of recruitment/profit/non-diversity related goals)
-get people to implement specific strategies to change the way their institution does things
-raise awareness about bias
-reduce implicit biases
-change group behaviour
-reduce explicit biases or stereotypes
-reduce discrimination
-equip people to notice when others are being biased
-improve workplace climate
-improve inclusion

4. Some of these are bad aims (alone)
-tick boxes
-devolve institutional responsibility

5. Some of these are infeasible aims (on the basis only of engaging in the training session):
-reduce implicit bias [more is needed to do this! Cf. Devine et al 2012, but see Forscher et al 2017]
-function better as an institution [issues about using this as a motivating aim?]  

6. Where we have modest evidence and reasons for optimism:
-reduce explicit biases or stereotypes
-equip people to notice when others are being biased

7. Where we have modest evidence and reasons for optimism:
-get people to implement specific strategies to change their own biases
-get people to implement specific strategies to change their own behaviour
(Devine et al 2012; Forscher et al 2017) 

8. Re: Changing biases and behaviour
NB: changing bias does not seem to be what mediates changes in behaviour (Forscher et al 2017b)

9. Some of these aims are such that that we haven’t really begun to look at the relationship between implicit bias training and these aims:
-get people to implement specific strategies to change the way their institution does things
[requires further follow up – what changes resulted from those changes?]

10. Some of these aims are such that that we haven’t really begun to look at the relationship between implicit bias training and these aims:
-change group behaviour
-reduce discrimination
-improve workplace climate
-improve inclusion

11. What evaluative tools we use will depend on what our aims are:
-Implicit measures?
-Self-report measures?
-Actual outcomes?

12. Issues: -who to ask about outcomes.
People who participated in the session?
People who work in the institution?
Service users? (Cf Hausman 2014).
Hausman: participants reported better awareness of bias. Service users – patients at Veterans Hospital – reported less positive interactions overall with those who had participated in training than those who had not.

13. Issues:
-trade-offs: is it a win if any individual implements strategies?
If some people stereotype more but others implement strategies to change behaviour, is that an acceptable trade-off?
Hausman: participants reported better awareness of bias. Service users – patients at Veterans Hospital – reported less positive interactions overall with those who had participated in training than those who had not.

14. In sum:
We need to think a bit more about what bias training should aim for (and possible tensions between different aims?];
And then about how to evaluate whether those aims have been met.
Hausman: participants reported better awareness of bias. Service users – patients at Veterans Hospital – reported less positive interactions overall with those who had participated in training than those who had not.