What is evidence – and how can I use it in my EBI What is evidence – and how can I use it in my EBI? Plenary session: “Making a difference” residential school (B839) November 2014

Aims of this session What do we mean when we talk about evidence? Why is evidence important? What kinds of evidence might I use in my EBI? How might I critically reflect on and evaluate the quality and scale of the evidence I use? An opportunity to work with others in the resi school beyond your own tutor group Have some fun (with evidence?? Yes!) Start with buzz groups after introducing the aims. Ask all the people in the room to talk to their neighbour for a couple of minutes about what they consider to be evidence in an EBI. Ensure everyone working in couples. No more than 2 minutes. At the end, ask 2 or 3 people to share their ideas with the whole group. Then move on. Don’t take too long on the feedback, but it is a useful way to ensure the session is interactive early on.

Buzz group Talk with your neighbour, in pairs, for 2 minutes: What is evidence in an EBI – how might we define it? Introduce with a sense of energy. Only give them 2 minutes. Don’t go round the pairs, just let them get on with it.

Evidence: a definition “the available facts, circumstances etc supporting or otherwise a belief, proposition…. Or indicating whether or not a thing is true or valid” Concise Oxford Dictionary Both facts and circumstances (so might include context etc) Evidence helps to support OR NOT – so may not be supportive – the aim is to test beliefs, propositions against evidence. Truth – often contested but perhaps might say whether the evidence is consistent with the theory or proposition.

Small group activity 10 minutes Why is evidence important in an EBI? Why is evidence important in management more generally? List as many reasons as your group can think of, with illustrative examples. At the end of the activity time, the plenary leader should encourage plenary discussion by asking some of the tables to give their reasons why evidence is important. You could ask another tutor to write these up on a flip chart. Collect as many as you can. Encourage later groups to only give additional reaons not repeat existing ones. You don’t need to get feedback from all the groups – the main thing is to get some reasons out into the room for discussion. Once you have a number of reasons, you might start trying to group the reasons in themes. You can present the next slide as a summary of the disussion though the groups may have produced other reasons which you should also acknowledge.

The importance of evidence Supports and justifies your argument Raises questions about whether your argument is accurate (e.g. where the evidence does not fit your argument) Encourages you to improve your thinking about or use of theory or practice Enables you to go beyond assertion (“this is what I think”) into persuasion (“I can reinforce my argument with data”) Demonstrates that you have engaged with course material

Collecting evidence is not…… Cherry-picking the “best” evidence that fits your ideas. Presenting illustrations and examples which support your ideas and ignoring the counter-examples Being drawn to vivid cases which support your argument Thinking that one or two examples “will do”

Collecting evidence is…… Systematic Give serious thought to how you are collecting evidence, that you are drawing on the full range of possible evidence, that you think about your role and values in collecting and interpreting data Sceptical Subject your evidence to disconfirmation not just confirmation; be critical of your work; think about other possible explanations of your evidence Ethical Collect your data in ways which are fair to other people and do not create harm to others. (Robson, 2002: Real world research) Don’t spend too long explaining this slide as the following exercise asks people to explore these areas in more detail.

Small group activity 15 minutes Tables 1 & 2: What does systematic evidence collection look like in an EBI? How would you know it is systematic? Tables 3 & 4: What does a sceptical stance to your EBI look like? What can you do in practical ways to be sceptical? Tables 5 & 6: How can you behave in ethical ways in collecting and using evidence in your EBI? Note: you may need to remind people of the difference between sceptical and cynical (as not everyone is conscious of the distinction)

Types of evidence (1) Primary data: data that you collect yourself (e.g. you conduct some interviews or you design and use a questionnaire) Secondary data: data that has been collected by others but which you use to analyse your improvement initiative (e.g. you analyse sales figures, or annual appraisal scores, company reports, government figures) Which type of evidence are you going to use? Both? One or the other? Why? Be clear about how it will help your EBI? What will it add to your argument?

Types of evidence (2) Quantitative: Numbers, or data which is expressed in numerical form (e.g. sales figures, rating scales on a questionnaire, counting how many interviewees mention a particular practice) Qualitative Data which is about meanings, interpretations and perspectives (e.g. how people feel about working for company X; the varied emotions people talk about in relation to a new work practice; their questions about your EBI) After introducing this slide, get the group to work in groups of 3-4 to talk about what types of evidence they are using or considering using in their EBI. Encourage them to help each other with ideas and suggestions, and to critique the quality of the evidence being sought.

Small group activity 15 minutes. Discuss: Are you clear about whether you are using primary or secondary data, and mainly numbers or words in your EBI? On what basis can you justify your choice (what are the advantages/disadvantages of your choice?) Are there any themes about the choice of types of data on your table?

How much evidence do I need for my EBI? A very good question! There is no fixed answer to this. But remember you are not doing a PhD – you should collect enough evidence to ensure your argument is plausible and interesting. But your evidence is likely to be indicative rather than definitive. Think about what would make strong evidence and collect as much of it as you can in the time available (but allow time for analysing it as well). Indicative – may need to explain. That it points towards a particular conclusion but you can’t be fully sure because for example, your sample is too small, or the improvement has not had time to bed in, or you have only tried out the improvement with volunteers and havent yet tried it more widely,…… Indicative means the results look promising but there would need to be more work to be sure. Perhaps use the example of the man who is now walking after a severed spinal cord – medics are very interested in this evidence of spinal cord repair but are aware it is only one patient so far… will it work for others? With different variations on that injury?

Stay critical about your evidence All evidence is flawed in some way or another. What are the weaknesses as well as the strengths of your data collection and analysis? Have you surveyed enough people? What about the non-responders? Have people in interviews told you what they think you want to hear? Has fear in the workplace affected how honest people are prepared to be? What about the evidence which did NOT fit your improvement initiative? Use the discussion part of your EMA to write up some reflections on the quality and scope of your evidence. Good researchers are critical of their own work.

Evidence and reflection Your EMA asks you to critically reflect on your EBI. So use ideas, reflection, speculation, and so on to think about your EBI. But be careful to be clear and explicit about where you have evidence to support your argument and where you are using reflection to expand on or critique your initiative. Both are important – but they are different types of understanding of your EBI.

Aims of this session What do we mean when we talk about evidence? Why is evidence important? What kinds of evidence might I use in my EBI? How might I critically reflect on and evaluate the quality and scale of the evidence I use? An opportunity to work with others in the resi school beyond your own tutor group Have some fun (with evidence?? Yes!) Start with buzz groups after introducing the aims. Ask all the people in the room to talk to their neighbour for a couple of minutes about what they consider to be evidence in an EBI. Ensure everyone working in couples. No more than 2 minutes. At the end, ask 2 or 3 people to share their ideas with the whole group. Then move on. Don’t take too long on the feedback, but it is a useful way to ensure the session is interactive early on.

Buzz group Talk with a neighbour for 1 minute each. What has been most valuable for you in this session? What are you going to do as an action for your EBI following this session. Finish the session with some hubbub in the room. And also make sure each student finishes in an active way, thinking about the implications for their EB. Don’t take feedback. End by saying something like, hope that was useful. Here are some final pointers (and go on to final slide, finishing briskly).

Over to you…… Hope the aims have been fulfilled for this session. What else do you need/want to know in order to design and collect evidence for your EBI? Remember to use your research methods textbook to help you with particular data collection techniques. GOOD LUCK!!