1. Tracking the external impacts of academic research in long-term ways and for HEFCE purposes Patrick Dunleavy London School of Economics and Political Science See our ‘Impact of Social Sciences’ blog at: http://blogs.lse.ac.uk/impactofsocialsciences/

2. Structure of this talk 1. Defining external research impacts and exploring how they operate 2. Capturing useful evidence of external impacts (starting now for the longer term) 3. HEFCE’s maximalist view of external “impacts”, and its case study method

3. Defining research impacts in an evidence-based way A research impact is: “a recorded or otherwise auditable occasion of influence from university research upon another actor or organization”. a. Academic impacts from research are influences upon other actors in academia or universities, e.g. as measured by citations. b. External impacts are influences on actors outside higher education, that is, in business, government or civil society, e.g. as measured by references in the trade press or in government documents, or by coverage in mass media.

4. Primary impacts – observable occasions of influence AcademicworkAcademicwork Academic impacts e.g. citations External impacts e.g. influence counts Academic/ university outputs Economic/ public policy/ civil society outputs Academic work and external impacts: a simple view

5. Primary impacts – observable occasions of influence AcademicworkAcademicwork Academic impacts e.g. citations External impacts e.g. influence counts Dynamic knowledge inventory Academic/ university outputs Economic/ public policy/ civil society outputs Academic work and external impacts: allowing for time lags and cumulation/ delay effects

6. Media, cultural and civil society systems Economic and business systems Public policy systems Discovery Application Single discipline processes Wider society A naïve (science-based) view of how an academic discipline achieves external impacts

7. Media, cultural and civil society systems Economic and business systems Public policy systems Discovery Integration Application Renewal Single discipline processes Impacts interface Wider society How key forms of scholarship within each academic discipline begin to achieve external impacts

8. Media, cultural and civil society systems Economic and business systems Public policy systems Discovery Integration Application Renewal Single discipline processes Joined-up scholarship Impacts interface Wider society Academic Service Bridging University - local integration How cross-disciplinary influences add to and mediate external impacts

9. Media, cultural and civil society systems Economic and business systems Public policy systems Discovery Integration Application Renewal Think tanks Specialist Media Professions Entrepreneurs Consultants Policy communities NGOs Single discipline processes Joined-up scholarship Impacts interface Wider society Academic Service Bridging University - local integration Media Corporations How the impacts interface shapes external impacts

10. 2. Capturing useful evidence of external impacts (starting now for the longer term)

11. Six metrics of academic/university involvements with external impact External funding linkages, especially business- to-sciences/ technology departments Media and specialist media presence Government website presence Academic service in government Reported ‘knowledge transfer’ activities Academics’ perceptions of impacts

12. 2007-082008-09Change% Change Collaborative research (£000s)697,030731,73434,7045 Contract research Total number of contracts27,05128,1111,0604 Total value of contracts (£000s)834,627937,373102,74611 Consultancy Total number of contracts64,29264,025-2670 Total income (£000s)334,768331,541-3,227 of which, number with SMEs 22,80220,596-2,206-11 number with large business10,49910,360-139 Patents Number of new patent applications1,8982,0971999 Number of patents granted in year5906536310 Intellectual property income Total revenues (£000s)66,271124,36858,09747 Total costs (£000s)21,00327,7946,79124 Spin-off companies Number created2,2232,289663 Estimated external investment received (£000s)89,497154,45164,95442 Universities’ key interactions with business in the UK

13. Proportion of research found in UK national press from different disciplines (2008)

14. Proportion of research found on UK government websites from different disciplines

15. Academic Service: University staff involved in UK central government quasi-government agencies Source: Griffiths, 2010

16. Knowledge transfer practice % academics involved 2009 % academics involved 2008 type of academic activity Attending conferences8756general Informal advice to business 5735application External lectures6534application Networks6732integration Joint publication4626application Advisory boards3822service Student projects/placements 3320renewal External visits19application Formed/run consultancy1418application Contract research3718application Undertaken consultancy4317application Academics’ reporting of ‘knowledge transfer’ activities - top 11 activities

17. Knowledge transfer practice % academics involved 2009 % academics involved 2008 type of academic activity Been involved in consortia 3517application Joint research4917application Post-course placementsna14renewal Prototyping and testing10naapplication Patenting712application Licensed research510application Standards forum3110application Spin out companies47discovery application Enterprise education64renewal / application External secondment103application Academics’ reporting of ‘knowledge transfer’ activities – next 10 activities

18. Social science and humanities academics’ perceptions of external impacts (2008)

19. Eight longer-term steps towards universities boosting their external impacts Clean up ‘multiple branding’ problems Reappraise events programmes Develop an impacts file for all individual academics, and then for departments Pull together databases and services to give improved ‘customer relationship management’ of research clients – e.g. bring in CRM systems Start (and fill up) an online depository For all research on the closed-web, publish a useful open-web version (see session D) Improve professional communication – starting with multi-author blogs (see session D) Work better in networks (see Panel 2)

20. 3. HEFCE’s expansive view of external impacts, and case study method

21. Primary impacts – observable occasions of influence AcademicworkAcademicwork Academic impacts e.g. citations External impacts e.g. influence counts Dynamic knowledge inventory Academic/ university outputs Economic/ public policy/ civil society outputs PRIMARY EXTERNAL IMPACTS OF RESEARCH

22. Primary impacts – observable occasions of influence Changes in organizational activities or outputs AcademicworkAcademicwork Academic impacts e.g. citations External impacts e.g. influence counts Dynamic knowledge inventory Economic/ public policy/ civil society outputs Multiple other factors WHAT MORE DOES HEFCE NEED? – 1. EVIDENCE OF CHANGED OUTPUTS DEMONSTRATED CETERIS PARIBUS, CONTROLLING FOR ALL OTHER INFLUENCES

23. Primary impacts – observable occasions of influence Changes in organizational activities or outputs Changes in societal outcomes AcademicworkAcademicwork Academic impacts e.g. citations External impacts e.g. influence counts Dynamic knowledge inventory Economic/ public policy/ civil society outputs Economic/ public policy/ civil society outputs Multiple other factors WHAT MORE DOES HEFCE NEED? – 2. EVIDENCE OF CHANGED OUTCOMES AGAIN CETERIS PARIBUS, CONTROLLING FOR ALL OTHER INFLUENCES

24. Primary impacts – observable occasions of influence Changes in organizational activities or outputs Evaluations of outcome changes – against societal values Changes in societal outcomes AcademicworkAcademicwork Academic impacts e.g. citations External impacts e.g. influence counts Dynamic knowledge inventory Economic/ public policy/ civil society outputs Economic/ public policy/ civil society outputs Multiple other factors + ? - WHAT MORE DOES HEFCE NEED? – 3. PROOF OF POSITIVE SOCIAL BENEFITS

25. HEFCE’S SCORING BANDS FOR IMPACT CASE STUDIES’ ‘SIGNIFICANCE’/ VALUE AND ‘REACH’/RELEVANCE

26. lowhighmedium low medium high Reach (or relevance) Significance (or value) CHARTING ALL HEFCE’S CRITERIA

27. lowhighmedium low medium high 4* exceptional 3* excellent Reach (or relevance) Significance (or value) CHARTING ALL HEFCE’S CRITERIA

28. lowhighmedium low medium high 4* exceptional 3* excellent 2* very good good 1* Reach (or relevance) Significance (or value) CHARTING ALL HEFCE’S CRITERIA

29. lowhighmedium low medium high U unclassed 4* exceptional 3* excellent 2* very good good 1* Reach (or relevance) Significance (or value) CHARTING ALL HEFCE’S CRITERIA Additional criteria: ▪ Impact occurring in the last 15 years ▪ Quality of Underlying Research ▪ Distinctiveness of Research

30. Short-run steps to prepare for HEFCE case studies Develop a large long list of possible case studies Collate grants, publications, contacts and testimonials evidence from (senior) academics on all potential cases, especially for 2005-11 For ‘probable’ cases - firm up outside evidence of outputs, activity and outcome changes; - push back pre-history to 1997 if needed Enlarge single-person cases wherever feasible Strengthen ‘sub-group’ case studies, by adding members (if feasible, without diluting ‘quality’), and re-branding for greater coherence by 2013 Monitor potential cases closely from now to 2013, and discuss with uninvolved senior case-writers

31. Thank you for listening See our ‘Impact of Social Sciences’ blog at: http://blogs.lse.ac.uk/impactofsocialsciences/ From there, download the free Handbook: Maximizing the Impacts of Social Science Research