1. ‘Evidence’ Searching, Sourcing and Evaluating
What it means …
How to do it …

2. Overview Reading? – Give it up! Sourcing and evaluating evidence
What we are looking for: critical reading
The anatomy of a journal article
Critical evaluation: examples
Review: what will you remember?

3. 1. Reading? – Give it up! Picture from: http://tinyurl.com/jufg263
Ask the group:
Who has had to make quick sense of reports – prior to meetings?
Do they read the whole report?
What do they do instead of reading the whole report?
When they receive reports from another team, a supplier, their direct reports:
Do they read it all?
How do they prioritise?
Do they take as fact everything that is written?
Why?
What persuades them of the validity of what is reported?
What counts as ‘evidence’?
Differentiate
Picture from:

4. Critical reading
A consultancy has produced a 30+ page report, recommending a redesign of your organisation’s recruitment and selection processes – which will require a substantial investment. The senior team want you to evaluate the report’s content and recommendations. You haven’t the time to do this yourself, so you delegate it to ‘Ali’ a direct report. Ali lacks critical evaluation skills. Provide him with a list of questions that will guide him as he analyses the report.
How can you ensure that Alil delivers a critical evaluation that is based on evidence, that is relevant to your own organisation?
What questions do you, as his manager, want answers to? And bear in mind that you might want Ali to cross-reference with other data / sources of information.
Get the large group to voice 1-3 questions to start with – and flipchart them. Then ask the group to work in 2’s/3’s – give them 5 mins.
Examples of possible questions:
How much is the investment?
How does this compare with what is spent currently?
What evidence is there that the redesign will deliver better results?
How credible is this evidence?
Is the evidence from a comparable industry / size of organisation?
Is the redesign relevant for your organisation’s culture?
Will it fit with other processes?
What other implications might there be of the redesign? … e.g. after the initial piloting, does it seem that the redesign can be resourced entirely within the organisation – or is the cost of the investment ongoing?
What theoretical base is there for the redesign? Does it make sense psychologically?
How long will it take to run the new process from initial publicising of a new post to confirming the new job holder?
After flipcharting example questions:
“So this is different from reading a novel, or reading to regurgitate what has been said …

5. What do you know about topic?
What questions do you need to answer in order to undertake the task you have been set?
Where are the gaps? (In your knowledge, in what is researched or ‘known’?)
Develop more questions as you go.
End with even more questions.
What are your information-seeking goals for your 1000 word report?

6. 2. Sourcing and evaluating evidence
Define analysis:
Identifying the component parts – detailed examination of the elements of the concept / topic

7. Searching for robust evidence
Choose the ingredients for your final creation carefully. Quality ingredients, well put together.
Actually, your time with any source is never wasted:
Even discarded articles can provide the basis for future growth and development of your knowledge of what is available
The references are a good start point for deeper research
The methodology might give you ideas of what can be done in practice.

8. Evaluating what you read
Task
Assumptions
methodology
Aim of text
Credibility
Relevance
Evidence
Key concepts and theories
Argument
Ask the group to produce a checklist for what to evaluate when reading research:
Research question (aim, purpose - what it is all about) – and relevance for your own task
Definitions of key concepts
Based on previous work – using theory well
Sample
Generalisability, especially how generalisable are the findings / conclusions to the student’s own situation?
Evidence / data: Statistical data? What is the source? Is response rate sufficient? Life story/personal accounts/perceptions? What claims are authors making from their interpretation of these data? Are they valid? Strengths of the article
Weaknesses / limitations
Analyse: take the reading apart – understand the component parts
Compare and contrast what is said within different sections of the report – and compare and contrast with other sources
Evaluate – assess – understand the value, the strengths and weaknesses
Summarise main points and concepts
Consider the evidence – weigh it
Consider the validity of the writing, i.e. is it tackling what it claims to be tackling?
Consider the reliability, i.e. are the claims based on ‘good’ evidence
Consider the sample that the idea has already been tested on – is it relevant to your own situation?

9. Textbooks
Summary of main recognised theories
Wider coverage
Simple explanations
Some available online
Journal articles
Original research or theory
Useful summaries of the literature
For now – you don’t need to read the methods sections
Can access online
Websites / Tedtalks / YouTube
Not usually based on research
Use websites to access management reports and other resources

10. Professional journals
Hierarchy of Sources
Research
Government reports
Professional journals
Newspaper articles
Anecdotal evidence
Evidence-based
Refereed
Scientific sampling and methodology

11. Literature Search Write down your keywords
Use ‘advanced search’ – Google Scholar or some other search engine
Skim titles only.
Add or delete words from your search
Skim abstracts – keep a note of what sounds useful for your current purpose
Reduce the number you have to a manageable list – then move on to skimming the actual article
Depending on time – and on how much the librarian has covered – maybe do an initial literature search for thefirst assignment.

12. 3. What we are looking for: critical reading
Define analysis:
Identifying the component parts – detailed examination of the elements of the concept / topic

13. Evidence of ‘critical reading’ 1. ‘Knowledge and Understanding’
Evidence that a wide range of relevant high-quality literature has been accessed, e.g.
Original research work accessed whenever possible. BUT – if you include the original in your ref list, your reader will expect evidence you have read it!
Contemporary - with exception of seminal work.
Claims are substantiated (e.g. with a reference).
Literature/reference material is appropriate and clearly linked to the assignment topic.

14. Evidence of ‘critical reading’ 2. ‘Analysis and Evaluation’
Evidence of critical analysis e.g.
Identifying and challenging assumptions.
An awareness of the importance of context in creating meaning/when analysing a topic.
Critical evaluation of the strengths and weaknesses of cited research/literature
Always look at the Generic marking criteria

15. Use sources to help you – at every level
Picture from:

16. Use sources to help you – at every level
For a quick overview, background
To understand, explain and evidence;
An idea or theory
A way of practically applying research or theory
How someone else has analysed a problem / situation
Ways of bringing together different ideas or theories (particularly look for 'literature reviews' or 'reviews')
Ways of evaluating a theory, research study and/or work by a particular author

17. 4. The Anatomy of a Journal Article

18. The anatomy of a journal article
(Note: they may not always have these headings)
Abstract
Introduction
Development of argument/hypotheses
Method
Results
Discussion
Limitations
Conclusion / Implications

19. Step One: Skim for an overview
Skim the title and abstract
Skim the conclusion
Skim titles of the sections
Look at any diagrams
Write down questions – what do you want to know more about?
Handout: Leicht-Deobald, U., Busch, T., Schank, C., Weibel, A., Schafheitle, S., Wildhaber, I. and Kasper, G. (2019) The Challenges of Algorithm-Based HR Decision-Making for Personal Integrity., Journal of Business Ethics, pp.1-16.
Handout an example article, chosen to help the students with their first assignment.  Maybe one of the following:
Husser, J., Gautier, L., André, J.M. and Lespinet-Najib, V. (2014) ‘Linking purchasing to ethical decision-making: An empirical investigation’, Journal of Business Ethics, 123(2), pp
Leicht-Deobald, U., Busch, T., Schank, C. et al. J Bus Ethics (2019) 
No more than 5 minutes – maybe only 3! – and then flipchart questions

20. Step Two: Scan for answers
Scan only to find answers to your questions
Allow a maximum of 5 minutes for the group to find answers to as many of the flipcharted questions that they can manage.
Discuss.

21. 5. Critical Evaluation: Examples
Compare the two examples. Explain the differences.
Provide two pieces of student writing.
Students should identify:
The differences between description and analysis
Flipchart the groups’ comments

22. 5. Review: What will you remember?