impact in relation to REF2014 Richard Thorpe 27th April 2016

The Challenge The impact agenda (and REF) is a response to demand for public value and for research to drive wealth, innovation and social value It is not going to go away To maximize impact a strategy is required in advance of research activity Identifying, estimating and achieving impact is not a core skill of the research community – these skills take time to acquire It is critical to take advice when learning the ‘rules of this new game’ RICHARD This slide is adapted from Professor James Goodwin’s presentation to the university Our focus on this as a business school and through these seminars is more fundamental and broader than the REF agenda - it relates to how we see our research, create and support the right research environment and think strategically about how we focus our activity. This does not mean that “pure” research is not valued and indeed some researchers might see that as being where their expertise lies. But management and business is inherently an applied area and we want to be at the forefront of demonstrating tangible benefits of our research outside HEI. A significant group of people within the business school (not just the case authors but including them) have learnt a considerable amount about impact over the past two and half years. We only really started working on our impact cases as a school from July 2011 and that was when the University started to take the issue seriously as well. Before that it was not on the radar. In terms of the expertise developed around the school we want to share those experiences and make sure that going forward we plan from the start and collect appropriate evidence as we go along.

Looking forwards: the shape of the university 2014-2020? Social and economic impact Research and knowledge creation Education and student experience But the dynamic is not simply focused on business schools, there is pressure on Universities also to ensure that the research and knowledge creation undertaken will be distinctive and in what ever terms be world leading, and will address major societal challenges and deliver societal and economic impact. The universities graduates will also need to define the research challenges of the future and deliver the research outputs and impact. Our education and student experience will be excellent and relevant to the needs of society and the economy and will deliver high quality graduates who are in great demand with high levels of employability, who themselves will deliver social and economic impact in the future. Graduation and employability

What is Research Impact in relation to REF working methods? Impact is defined as an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life beyond academia Impact includes but is not limited to: The activity, attitude, awareness, behaviour, capacity, opportunity, performance, policy, practice, process or understanding Of an audience, beneficiary, community, constituency, organisation or individuals In any geographic location whether locally, regionally, nationally or internationally

What is NOT Research Impact? In the context of REF: Impacts on research or the advancement of academic knowledge within the higher education sector (whether in the UK or internationally) are excluded Impact on students, teaching or other activities within the HEI (except CPD activities) are excluded Media activity, PR and dissemination, public engagement, consultancy – are grey areas RICHARD

What does the literature say? Category 1: Climate and context Category 2: Relationships between researchers & knowledge users Factors that influence research use Category 3: Research Production Category 4: Researchers’ roles Category 5: Knowledge user role Category 6: Evaluation

Some observations about management and business

The Present Reality? If ‘The duty of an intellectual in society is to make a difference’ (Sir Thomas Moore)........................ then what is happening in the field of management and business is nothing short of the management of Indifference (Andrew Pettigrew) Concluding that the social science research community has a long way to go to realise its potential 4

Improve the measurement and assessment of research impact Engagement is important for good research, not its antithesis In order to pursue impact there may be moves pressure to attach impact ratings to individuals as well as institutions Universities are recognizing the need to support a more rounded assessment of research quality REF has encouraged schools to focus in on them selves to maximise the reputational value REF offers and has in the process overlooked the synergies and opportunities that might come from wider institutional involvement on the one hand and how this might enhance their global reputation on the other.

Two themes in our recommendations Incentives Rebalance incentives for university departments and staff, to better recognise and reward activities that link to impact and engagement Capabilities Build capabilities at individual and school level, so that engagement can better take place

“How can we change ladders ?

The challenge is: To understand the ‘rules of this game’ and how to manage within them Learn some new skills Make some choices? Measure our progress “I am not what happened to me, I am what I choose to become.” – C.G. Jung

Impact Models & Frameworks

Process Issues: the Knowledge Translation Value Chain Renewal and Extension of Research Agendas Theory to Practice thought experiments Practitioner –oriented research outputs Output designed to engage practitioners Theory development Reporting basic research Engagement of Users Widespread dissemination Services & Support Directly useable output Sector Reports KT0 OP1 KT1 OP2 KT2 OP3 KT3 OP4 KT4 OP5 Code: KT = Knowledge Translation OP = Output

C-TIE Impact Portfolio Renewal and Extension of Research Agendas Theory to Practice thought experiments Practitioner –oriented research outputs Output designed to engage practitioners Theory development Reporting basic research Engagement of Users Widespread dissemination Services & Support Directly useable output Sector Reports KT0 OP1 KT1 OP2 KT2 OP3 KT3 OP4 KT4 OP5 Emerging Technologies Entrepreneurial Narratives Information Management Evolution of Business Knowledge 4* Academic journals IKC in Regenerative Medicine Interdisciplinary research with Nanoscientists Models for accelerating innovation outcomes Business collaborations (1-1), Balfour Beatty, Connect Yorkshire, Rolls-Royce Joint papers with practitioners Seminars for practising managers SME Knowledge Network Keyworth Institute Northern Leadership Academy AIM Report on Knowledge Generation in SMEs “Leaderful Communities” book Action Learning Sets in leadership Information Management consultancy Consultancy to NHS Manufacturing Excellence consultancy Manufacturing Leadership Masters Innovation Module on MBA AIMTech database platform Textbooks on Research Methodology Code: KT = Knowledge Translation OP = Output Bullet points are examples of c-TIE Activities

Cross-activity learning to inform IKC sustainability What to measure ? Programme Evaluation People Development PoC Projects Innovation Management Research Cross-activity learning to inform IKC sustainability Investments actions Outputs Impact Outcomes Regular meetings with core team to review innovation management within programme, formal interviews with Exec members, mid-term evaluation of programme management Observation/feedback on work of project teams, support for business planning Medical Technology Innovation Personal Development Programme for PDRAs, action learning reviews for IKC Core Team Topic based research projects, production of briefing notes aimed at innovation professionals (accelerating innovation, cultures of innovation) Impact and engagement Better programme management Better projects Capability building Faster, more relevant skills and knowledge development Key activities Contribution to Tranche 2 proposal Leadership-for-innovation programme Responsible Innovation research project Contribution to wider university Impact agenda

Cross-activity learning to inform IKC sustainability What to measure ? Human capital People Development PoC Projects Innovation Management Research Cross-activity learning to inform IKC sustainability Investments actions Outputs Impact Outcomes Development of individuals through the course of the interaction/involvement Observation/feedback on work of project teams, support for business planning Medical Technology Innovation Personal Development Programme for PDRAs, action learning reviews for IKC Core Team Topic based research projects, production of briefing notes aimed at innovation professionals (accelerating innovation, cultures of innovation) Impact and engagement Enhanced contribution to an organisation Better projects Capability building Faster, more relevant skills and knowledge development Key activities Contribution to Tranche 2 proposal Leadership-for-innovation programme Responsible Innovation research project Contribution to wider university Impact agenda

Cross-activity learning to inform IKC sustainability What to measure ? Human capital People Development Social Capital Innovation Management Research Cross-activity learning to inform IKC sustainability Investments actions Outputs Impact Outcomes Development of individuals through the course of the interaction/involvement The development of relations and networks through which knowledge can flow Medical Technology Innovation Personal Development Programme for PDRAs, action learning reviews for IKC Core Team Topic based research projects, production of briefing notes aimed at innovation professionals (accelerating innovation, cultures of innovation) Impact and engagement Enhanced contribution to an organisation Greater exchange of knowledge Capability building Faster, more relevant skills and knowledge development Key activities Contribution to Tranche 2 proposal Leadership-for-innovation programme Responsible Innovation research project Contribution to wider university Impact agenda

Cross-activity learning to inform IKC sustainability What to measure ? Human capital Innovation capacity Social Capital Management Research Cross-activity learning to inform IKC sustainability Investments actions Outputs Impact Outcomes Development of individuals through the course of the interaction/involvement The development of relations and networks through which knowledge can flow New ways of thinking and operating Topic based research projects, production of briefing notes aimed at innovation professionals (accelerating innovation, cultures of innovation) Impact and engagement Enhanced contribution to an organisation Greater exchange of knowledge Capability building Faster, more relevant skills and knowledge development Key activities Contribution to Tranche 2 proposal Leadership-for-innovation programme Responsible Innovation research project Contribution to wider university Impact agenda

Cross-activity learning to inform IKC sustainability What to measure ? Human capital Innovation capacity Social Capital Knowledge generation Cross-activity learning to inform IKC sustainability Investments actions Outputs Impact Outcomes Development of individuals through the course of the interaction/involvement The development of relations and networks through which knowledge can flow New ways of thinking and operating What has come out of the interaction and engagement that is available within the public domain Impact and engagement Enhanced contribution to an organisation Greater exchange of knowledge Translating and disseminating Faster, more relevant skills and knowledge development Key activities Contribution to Tranche 2 proposal Leadership-for-innovation programme Responsible Innovation research project Contribution to wider university Impact agenda

What to measure ? Human capital Social Capital Innovation capacity Knowledge generation A measure that incorporates both academic outputs and societal benefits Investments actions Outputs Impact Outcomes Development of individuals through the course of the interaction/involvement The development of relations and networks through which knowledge can flow New ways of thinking and operating What has come out of the interaction and engagement that is available within the public domain Impact and engagement Enhanced contribution to an organisation Greater exchange of knowledge Capability building Faster, more relevant skills and knowledge development Key activities Economic Value Added

Measuring Research Impact: Key Concepts Reach Significance Evidence Research underpinning an impact case needs to be highly focused – you can’t/shouldn’t try to include everything and everyone. It’s about having a focused body of practitioner relevant research You are required to give unequivocal evidence of the quality of your underpinning research (does not have to be based on papers in the REF output section but must be minimum 2*) You need to make the linkage and show the sequence of events (CHAIN OF EVIDENCE) from the research findings to the alleged impact Corroboration of Impact – key point is that it must be independent, reliable and accessible What might look like evidence at face value, when you dig deeper you often find that it isn’t actually being specific enough or does not fit within the requirements in some way or other (e.g. who wrote it, date, content)

Climbing the Ladder of Research Utilization STAGE 6: APPLICATION – research gives rise to applications & extensions by professionals concerned STAGE 5: INFLUENCE – research results influenced the choice & decisions of practitioners & professionals STAGE 4: EFFORT – efforts were made to adopt the results of the research by practitioners/professionals STAGE 3: REFERENCE – work has been cited as a reference in the reports, studies & strategies of action elaborated by practitioners and professionals STAGE 2: COGNITION – research reports were read and understood by the practitioners and professionals STAGE 1: TRANSMISSION – research results were transmitted to practitioners and professionals Stage 1 – if all you are going to do is dissemination this is as far as you will get Nick Ellison, our reviewer for most of the REF period used this model – for him this is how he defined “Reach”

Flows of Knowledge, Expertise and Influence: a method of assessing policy and practice impacts from social science research Source: Laura Meagher, Catherine Lyall and Sandra Nutley (2008) Research Evaluation 17(3) p163-173

Useful Links http://blogs.lse.ac.uk/impactofsocialsciences/2012/10/24/dunlea vy-advice-note-2-case-study/ http://blogs.lse.ac.uk/impactofsocialsciences/2011/04/14/maxim izing-the-impacts-of-your-research-a-handbook-for-social- scientists-now-available-to-download-as-a-pdf/ http://blogs.lse.ac.uk/impactofsocialsciences/2011/05/24/a- beginner%e2%80%99s-guide-to-the-different-types-of-impact/ http://lubswww.leeds.ac.uk/restricted/staff-intranet/research- and-innovation/