1. impact in relation to REF2014
Richard Thorpe
27th April 2016

2. The Challenge
The impact agenda (and REF) is a response to demand for public value and for research to drive wealth, innovation and social value
It is not going to go away
To maximize impact a strategy is required in advance of research activity
Identifying, estimating and achieving impact is not a core skill of the research community – these skills take time to acquire
It is critical to take advice when learning the ‘rules of this new game’
RICHARD
This slide is adapted from Professor James Goodwin’s presentation to the university
Our focus on this as a business school and through these seminars is more fundamental and broader than the REF agenda - it relates to how we see our research, create and support the right research environment and think strategically about how we focus our activity. This does not mean that “pure” research is not valued and indeed some researchers might see that as being where their expertise lies. But management and business is inherently an applied area and we want to be at the forefront of demonstrating tangible benefits of our research outside HEI.
A significant group of people within the business school (not just the case authors but including them) have learnt a considerable amount about impact over the past two and half years. We only really started working on our impact cases as a school from July 2011 and that was when the University started to take the issue seriously as well. Before that it was not on the radar. In terms of the expertise developed around the school we want to share those experiences and make sure that going forward we plan from the start and collect appropriate evidence as we go along.

3. Looking forwards: the shape of the university 2014-2020?
Social and economic
impact
Research and
knowledge creation
Education and
student experience
But the dynamic is not simply focused on business schools, there is pressure on Universities also to ensure that the research and knowledge creation undertaken will be distinctive and in what ever terms be world leading, and will address major societal challenges and deliver societal and economic impact. The universities graduates will also need to define the research challenges of the future and deliver the research outputs and impact.
Our education and student experience will be excellent and relevant to the needs of society and the economy and will deliver high quality graduates who are in great demand with high levels of employability, who themselves will deliver social and economic impact in the future.
Graduation and
employability

4. What is Research Impact in relation to REF working methods?
Impact is defined as an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life beyond academia
Impact includes but is not limited to:
The activity, attitude, awareness, behaviour, capacity, opportunity, performance, policy, practice, process or understanding
Of an audience, beneficiary, community, constituency, organisation or individuals
In any geographic location whether locally, regionally, nationally or internationally

5. What is NOT Research Impact?
In the context of REF:
Impacts on research or the advancement of academic knowledge within the higher education sector (whether in the UK or internationally) are excluded
Impact on students, teaching or other activities within the HEI (except CPD activities) are excluded
Media activity, PR and dissemination, public engagement, consultancy – are grey areas
RICHARD

6. What does the literature say?
Category 1: Climate and context
Category 2: Relationships between researchers & knowledge users
Factors that influence research use
Category 3: Research Production
Category 4: Researchers’ roles
Category 5: Knowledge user role
Category 6: Evaluation

7. Some observations about management and business

9. The Present Reality?
If ‘The duty of an intellectual in society is to make a difference’ (Sir Thomas Moore) then what is happening in the field of management and business is nothing short of the management of Indifference (Andrew Pettigrew) Concluding that the social science research community has a long way to go to realise its potential 4

10. Improve the measurement and assessment of research impact
Engagement is important for good research, not its antithesis
In order to pursue impact there may be moves pressure to attach impact ratings to individuals as well as institutions
Universities are recognizing the need to support a more rounded assessment of research quality
REF has encouraged schools to focus in on them selves to maximise the reputational value REF offers and has in the process overlooked the synergies and opportunities that might come from wider institutional involvement on the one hand and how this might enhance their global reputation on the other.

11. Two themes in our recommendations
Incentives
Rebalance incentives for university departments and staff, to better recognise and reward activities that link to impact and engagement
Capabilities
Build capabilities at individual and school level, so that engagement can better take place

12. “How can we change ladders ?

13. The challenge is:
To understand the ‘rules of this game’ and how to manage within them
Learn some new skills
Make some choices?
Measure our progress
“I am not what happened to me, I am what I choose to become.”
– C.G. Jung

14. Impact Models & Frameworks

15. Process Issues: the Knowledge Translation Value Chain
Renewal and Extension of Research Agendas
Theory to Practice thought experiments
Practitioner –oriented research outputs
Output designed to engage practitioners
Theory development
Reporting basic research
Engagement of Users
Widespread dissemination
Services & Support
Directly useable output
Sector Reports
KT0
OP1
KT1
OP2
KT2
OP3
KT3
OP4
KT4
OP5
Code: KT = Knowledge Translation
OP = Output

16. C-TIE Impact Portfolio
Renewal and Extension of Research Agendas
Theory to Practice thought experiments
Practitioner –oriented research outputs
Output designed to engage practitioners
Theory development
Reporting basic research
Engagement of Users
Widespread dissemination
Services & Support
Directly useable output
Sector Reports
KT0
OP1
KT1
OP2
KT2
OP3
KT3
OP4
KT4
OP5
Emerging Technologies
Entrepreneurial Narratives
Information Management
Evolution of Business Knowledge
4* Academic journals
IKC in Regenerative Medicine
Interdisciplinary research with Nanoscientists
Models for accelerating innovation outcomes
Business collaborations (1-1), Balfour Beatty, Connect Yorkshire, Rolls-Royce
Joint papers with practitioners
Seminars for practising managers
SME Knowledge Network
Keyworth Institute
Northern Leadership Academy
AIM Report on Knowledge Generation in SMEs
“Leaderful Communities” book
Action Learning Sets in leadership
Information Management consultancy
Consultancy to NHS
Manufacturing Excellence consultancy
Manufacturing Leadership Masters
Innovation Module on MBA
AIMTech database platform
Textbooks on Research Methodology
Code: KT = Knowledge Translation
OP = Output
Bullet points are examples of c-TIE Activities

18. Cross-activity learning to inform IKC sustainability
What to measure ?
Programme
Evaluation
People
Development
PoC
Projects
Innovation
Management
Research
Cross-activity learning
to inform IKC sustainability
Investments
actions
Outputs
Impact
Outcomes
Regular meetings with core team to
review innovation management within
programme, formal interviews with Exec
members, mid-term evaluation of
programme management
Observation/feedback on work of
project teams, support for business
planning
Medical Technology Innovation Personal
Development Programme for PDRAs, action learning reviews for IKC Core Team
Topic based research projects, production
of briefing notes aimed at innovation
professionals (accelerating innovation,
cultures of innovation)
Impact and engagement
Better programme management
Better projects
Capability building
Faster, more relevant skills and knowledge development
Key
activities
Contribution to Tranche 2 proposal
Leadership-for-innovation programme
Responsible Innovation research project
Contribution to wider university Impact agenda

19. Cross-activity learning to inform IKC sustainability
What to measure ?
Human
capital
People
Development
PoC
Projects
Innovation
Management
Research
Cross-activity learning
to inform IKC sustainability
Investments
actions
Outputs
Impact
Outcomes
Development of individuals through the course of the interaction/involvement
Observation/feedback on work of
project teams, support for business
planning
Medical Technology Innovation Personal
Development Programme for PDRAs, action learning reviews for IKC Core Team
Topic based research projects, production
of briefing notes aimed at innovation
professionals (accelerating innovation,
cultures of innovation)
Impact and engagement
Enhanced contribution to an organisation
Better projects
Capability building
Faster, more relevant skills and knowledge development
Key
activities
Contribution to Tranche 2 proposal
Leadership-for-innovation programme
Responsible Innovation research project
Contribution to wider university Impact agenda

20. Cross-activity learning to inform IKC sustainability
What to measure ?
Human
capital
People
Development
Social
Capital
Innovation
Management
Research
Cross-activity learning
to inform IKC sustainability
Investments
actions
Outputs
Impact
Outcomes
Development of individuals through the course of the interaction/involvement
The development of relations and networks through which knowledge can flow
Medical Technology Innovation Personal
Development Programme for PDRAs, action learning reviews for IKC Core Team
Topic based research projects, production
of briefing notes aimed at innovation
professionals (accelerating innovation,
cultures of innovation)
Impact and engagement
Enhanced contribution to an organisation
Greater exchange of knowledge
Capability building
Faster, more relevant skills and knowledge development
Key
activities
Contribution to Tranche 2 proposal
Leadership-for-innovation programme
Responsible Innovation research project
Contribution to wider university Impact agenda

21. Cross-activity learning to inform IKC sustainability
What to measure ?
Human
capital
Innovation
capacity
Social
Capital
Management
Research
Cross-activity learning
to inform IKC sustainability
Investments
actions
Outputs
Impact
Outcomes
Development of individuals through the course of the interaction/involvement
The development of relations and networks through which knowledge can flow
New ways of thinking and operating
Topic based research projects, production
of briefing notes aimed at innovation
professionals (accelerating innovation,
cultures of innovation)
Impact and engagement
Enhanced contribution to an organisation
Greater exchange of knowledge
Capability building
Faster, more relevant skills and knowledge development
Key
activities
Contribution to Tranche 2 proposal
Leadership-for-innovation programme
Responsible Innovation research project
Contribution to wider university Impact agenda

22. Cross-activity learning to inform IKC sustainability
What to measure ?
Human
capital
Innovation
capacity
Social
Capital
Knowledge
generation
Cross-activity learning
to inform IKC sustainability
Investments
actions
Outputs
Impact
Outcomes
Development of individuals through the course of the interaction/involvement
The development of relations and networks through which knowledge can flow
New ways of thinking and operating
What has come out of the interaction
and engagement that is available
within the public domain
Impact and engagement
Enhanced contribution to an organisation
Greater exchange of knowledge
Translating and disseminating
Faster, more relevant skills and knowledge development
Key
activities
Contribution to Tranche 2 proposal
Leadership-for-innovation programme
Responsible Innovation research project
Contribution to wider university Impact agenda

23. What to measure ? Human capital Social Capital Innovation capacity
Knowledge
generation
A measure that incorporates both academic outputs and societal benefits
Investments
actions
Outputs
Impact
Outcomes
Development of individuals through the course of the interaction/involvement
The development of relations and networks through which knowledge can flow
New ways of thinking and operating
What has come out of the interaction
and engagement that is available
within the public domain
Impact and engagement
Enhanced contribution to an organisation
Greater exchange of knowledge
Capability building
Faster, more relevant skills and knowledge development
Key
activities
Economic Value Added

24. Measuring Research Impact: Key Concepts
Reach
Significance
Evidence
Research underpinning an impact case needs to be highly focused – you can’t/shouldn’t try to include everything and everyone. It’s about having a focused body of practitioner relevant research
You are required to give unequivocal evidence of the quality of your underpinning research (does not have to be based on papers in the REF output section but must be minimum 2*)
You need to make the linkage and show the sequence of events (CHAIN OF EVIDENCE) from the research findings to the alleged impact
Corroboration of Impact – key point is that it must be independent, reliable and accessible
What might look like evidence at face value, when you dig deeper you often find that it isn’t actually being specific enough or does not fit within the requirements in some way or other (e.g. who wrote it, date, content)

25. Climbing the Ladder of Research Utilization
STAGE 6: APPLICATION – research gives rise to applications & extensions by professionals concerned
STAGE 5: INFLUENCE – research results influenced the choice & decisions of practitioners & professionals
STAGE 4: EFFORT – efforts were made to adopt the results of the research by practitioners/professionals
STAGE 3: REFERENCE – work has been cited as a reference in the reports, studies & strategies of action elaborated by practitioners and professionals
STAGE 2: COGNITION – research reports were read and understood by the practitioners and professionals
STAGE 1: TRANSMISSION – research results were transmitted to practitioners and professionals
Stage 1 – if all you are going to do is dissemination this is as far as you will get
Nick Ellison, our reviewer for most of the REF period used this model – for him this is how he defined “Reach”

26. Flows of Knowledge, Expertise and Influence: a method of assessing policy and practice impacts from social science research
Source: Laura Meagher, Catherine Lyall and Sandra Nutley (2008) Research Evaluation 17(3) p

27. Useful Links
vy-advice-note-2-case-study/
izing-the-impacts-of-your-research-a-handbook-for-social- scientists-now-available-to-download-as-a-pdf/
beginner%e2%80%99s-guide-to-the-different-types-of-impact/
and-innovation/