1. Research Excellence Framework REF 2014

2. Summary Introduction REF the nitty gritty Aston’s Code of Practice PURE EAS submission strategy What you can do

3. Introduction

4. REF has replaced RAE (last RAE 2008) Similar format i.e. panels (36 not 67) assessing units of assessment (UOA) Results will be 4*, 3*, 2*, 1* or unclassified! Results will inform HEFCE research funding from 2015 Results will also inform league tables, recruitment etc. Aston wants to be a research led university...

5. REF the nitty gritty

6. REF – Assessment ELEMENTWEIGHTINGASSESSMENT CRITERIA OUTPUTS65%Rigour, originality and significance IMPACT20%Reach and significance ENVIRONMENT15%Vitality 3 elements of assessment... Results published as a profile RAE 2008 58.73 FTE (6 th highest out of 52) 4* 3* 2* 1* U/C 10 25 50 15 0

7. REF Outputs (65%) Outputs period from 1 st January 2008 – 31 st December 2013 Up to four items of research output per submitted member of staff 1 Published papers in peer-reviewed journals Conference papers and reports (highlighted by CSI panel) Books, book chapters and research monographs New materials, devices, products and processes Patents Software, computer code and algorithms Standards documents Technical reports including confidential reports A 100 word summary per output describing its significance Outputs & statements to be entered into PURE (more later) 1 Panels publish a profile for the UoA not individuals

8. REF – Outputs (65%) Double weighting of outputs – is permitted but we don’t envisage any of these - if anyone is thinking of this please talk to Andy asap We need to be wary of multiple outputs with “significant material in common” which run the risk of being counted as just one output An output may (but only in highly exceptional circumstances) be listed twice in a submission (i.e. against more than one author) An output can readily be listed twice in separate UoA submissions For our subpanels (11, 13 & 15) the order of authors is NOT important and author position or contribution DOES NOT require justification 300 word summary (in addition to 100 word summary) if an output is non-text or practice-based or a review article – anyone putting forward outputs of this type please talk to Andy / Ian Nabney asap. Don’t necessarily need 4 outputs...

9. REF Outputs (65%) - < 4 outputs Early Career Researchers (ECR) – first academic appointment where you can lead a research project Date at which the individual first met the definition of an ECR Number of outputs may be reduced without penalty by up to: On or before 31 July 20090 Between 1 August 2009 and 31 July 20101 Between 1 August 2010 and 31 July 20112 After 1 August 20113

10. Early Career Researchers (ECR) Part time working Total months absent between 1 January 2008 and 31 st October 2013 due to working part-time, secondment or career break: For part-time staff this equates to contracted hours for the following average FTE over the period 1 Jan 2008 to 31 Oct 2013: Number of outputs may be reduced without penalty by up to: 0 – 11.990.829 – 10 12 – 27.990.601 – 0.8291 28 – 45.990.343 – 0.6002 46 or more0.342 or less3 REF Outputs (65%) - < 4 outputs

11. Early Career Researchers (ECR) Part time working Prolonged period(s) of illness – more complex (accompanied by a written narrative) but again based on Total months absent between 1 January 2008 and 31 st October 2013 due to working part-time, secondment or career break: For part-time staff this equates to contracted hours for the following average FTE over the period 1 Jan 2008 to 31 Oct 2013: Number of outputs may be reduced without penalty by up to: 0 – 11.990.829 – 10 12 – 27.990.601 – 0.8291 28 – 45.990.343 – 0.6002 46 or more0.342 or less3 REF Outputs (65%) - < 4 outputs

12. Early Career Researchers (ECR) Part time working Prolonged period(s) of illness Statutory maternity leave taken substantially during the assessment period regardless of length / paternity leave > 4 months each period of maternity / adoption / paternity leave results in a reduction of one required output please let Susan Doughty know if this applies to you REF Outputs (65%) - < 4 outputs

13. In assessing outputs EE and Gen Eng subpanels, 13 and 15, will NOT use citation data to assess outputs In assessing outputs CSI subpanel 11 will use Citation data Google Scholar in addition to Scopus data provided by REF team thus hard to define 4*, 3* etc. (see p 46 REF 01.2012 guide) 4* ‘agenda setting’, ‘leading research’, ‘major policy changes’ etc... 3* ‘important contribution’, ‘important knowledge’, ‘significant changes in policy’ etc... 2* ‘useful knowledge’, ‘incremental advances’, ‘influence on policy’ REF Outputs (65%) – sub-panel differences

14. REF – Impact (20%) Impact is defined as an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia Impact will be assessed in terms of its ‘reach and significance’ Two elements of assessment impact template impact statements Assessment framework and guidance on submissions REF 02.2011, July 2011

15. REF – Impact (20%) Impact template one per UoA document to inform on Context – main non-academic users, impact type, etc. the unit’s approach to maximising impact during the period 2008-2013 strategy and plans for supporting impact the relationship between the unit’s approach to impact and the submitted case studies (recognising that individual case studies may not relate directly to the approach)

16. REF – Impact (20%) Impact statements – case studies The impact in the case study should occur during the period 1 January 2008 – 31 July 2013 Underpinned by research that took place in the period 1 January 1993 – 31 July 2013 Underpinning research must be of at least 2* quality Number of case studies – depends on number of staff Number of Category A staff submitted (FTE) Required number of case studies Up to 14.992 15 – 24.993 25 – 34.994

17. REF – Impact (20%) Impact statements – those of you drafting a case study complete a form describing each case study chain of evidence and evidence must be supplied refer to HEFCE guidelines and remember impacts on research or the advancement of academic knowledge within the higher education sector (whether in the UK or internationally) are excluded. impacts on students, teaching or other activities within the submitting HEI are excluded. Panel criteria and working methods REF 01.2012

18. REF – Environment (15%) Two elements of assessment various metrics doctoral degrees awarded research income – many sources but must be spent research income in kind – beam time etc. environment template (7 or 8 pages) one per UoA Overview Research strategy People, including Staffing strategy and staff development Research students Income, infrastructure and facilities Collaboration and contribution to the discipline

19. Aston’s Code of Practice

20. REF 2014 Code of Practice Code of Practice is a requirement of the REF and will need to be approved by HEFCE Principles include transparency, consistency, accountability and inclusivity It requires all decision-makers to attend Equality and Diversity training It is currently being drafted by the Research Executive and the draft will be made available to staff for comment prior to the adoption and submission of the final draft

21. Code of Practice The Code sets out:  The decision-making process  The composition of decision-making groups  Guidance on selection and submission  How the process will be communicated to staff  Detail of Individual Staff Circumstances, how this will be applied and and how to disclose  Appeals-how to do this  The timetable

22. Individual Staff Circumstances – how it will be applied The proposed University selection criteria for inclusion in the REF are 4 publications at least 3 of which are 3* or above. This will require modifying for staff who wish to claim reductions in outputs according to ECR status, or other clearly defined circumstances (including maternity leave) or complex circumstances as follows. The proposition is: 3 outputs: of which 2 must be 3* or above. 2 Outputs: both must be 3* or above. 1 Output: must be 3* or above.

23. PURE

24. Will be used to Compile all sections of the REF2014 return for each UoA Facilitate the review and decision making process for each UoA Report on progress on REF preparations to Senior Management Submit the institution’s final REF return to Hefce

25. EAS submission strategy

26. EAS Strategy REF has 4 main panels A - D Engineering will fit into main panel B EAS will make 3 submissions (Units of Assessment, UoAs) Sub-panel 11 – Computer Science and Informatics (CSI) Sub-panel 13 – Electronic Engineering, Metallurgy & Materials (EE) Sub-panel 15 – General Engineering (Gen Eng) Each sub-panel has slightly different assessment criteria An EAS REF intranet page has been set up please see www1.aston.ac.uk/eas/staff-intranet/research/ref/

27. University REF Organisation University REF Team Martin Griffin Sally Puzey (RSO) Judy White (PURE) Research Executive Committee Martin Griffin Sally Puzey Heads of School Associate Deans Research and Andy The Vice Chancellor School Teams

28. REF Strategy Group Robert BerryAndy Sutherland Alison HodgePaul Knobbs Mike WrightSusan Doughty Philip DaviesJohn Leigh EAS Strategy – organisational structure EE Working Group Sergei Turitsyn (lead) Dave Webb Lin Zhang Kate Sugden CSI Working Group Ian Nabney (lead) David Lowe Radu Calinescu Gen Eng Working Group Andy Sutherland (lead) Philip Davies Brian Tighe Paul Topham Greg Swadener

29. Oversee submission of their UoA Read and rate paper outputs Liaise with authors to establish optimal outputs selected Read and assess 100 word summaries of outputs Liaise with authors to establish optimal wording of summary statements Discuss publication pipeline with academic staff Meet with external assessors/advisors Ensure specific REF documents completed Impact Template Impact Statements Environment Template EAS Strategy – role of the Working Groups In close liaison with REF Strategy Group

30. EAS Strategy – inclusion of staff In the last RAE (2008) the University and School policy was to include as many people as possible For the REF (2014) we have a goal for each submission to have a majority of outputs being at the Internationally Excellent (3*) or World Leading (4*) level. (With the balance made up with 2*) We will submit as many staff as we can that will allow us to achieve this goal Exclusion from the REF submission will not be viewed by the School, or the University, as career limiting. The university will submit an HR-driven ‘Code of Practice’ to the REF team regarding Aston’s REF inclusion policy.

31. EAS Strategy - mock exercise 3 submissions assessed by an external expert(s) Prof Yorick Wilks (Emeritus – Leeds) Prof Polina Bayvel (UCL) Prof Nigel Slater (Cambridge) Michael Kenward Reports received over summer 2011 – overall reasonably positive... General Engineering Outputs are almost all journal articles (which is what I would expect) and a fair number are in highly respected journals. Notes of explanation that accompany the outputs are rather weak

32. Impact It is very pleasing to see that Tony and Brian are continuing their world leading research activities...I look at these sections with great admiration and no small envy The Bioenergy statement worries me much more. Again the achievements are very impressive but I fear they will be almost entirely discounted as “influence on scientific intellectual knowledge and academia Environment The Strategy section addresses organisation and processes very well, but it is rather thin on Vision and plans for the Future EAS Strategy - mock exercise

33. 30 th Jan 2012 HEFCE published final sub-panel guidelines 20 th May 2012 Another mock exercise (!) but on impact only 31 st July 2012 University to submit Code of Practice Sept/Oct 2012 Internal audit of outputs Dec 2012 Survey of submission intentions 1 st Jan 2008 – 31 st Dec 2013 Publication period 1 st Aug 2008 – 31 st July 2013 Finance and Student data (HESA data) 1 st Jan 2008 – 31 st July 2013 Period impact must arise during 1 st Jan 1993 – 31 st July 2013 Impact can be based on research undertaken at Aston in this period 31 st Oct 2013 Census date for staff 29 th Nov 2013 Submission closing date Dec 2014 RESULTS PUBLISHED!!! EAS Strategy – REF timeline key dates OR

34. EAS Strategy - portfolio of upcoming activities Feb 2012School communication events Feb 2012 – May 20 th 2012Prepare Impact documents for mock impact exercise Feb 2012 – Assessment of outputs and 100 word statements Feb 2012 – Mentoring on publication pipelines Feb 2012 – Mar 2012PURE training events and launch May 20 th 2012Mock Impact Deadline May 2012 – Oct 2012Draft environment statements and refine impact documents in line with assessors comments

35. What you can do

36. REF – what you can do Write up results for publication – target high end journals and for CSI personnel - high end conferences money for open access journals – contact Philip Davies mentoring available for those struggling to get 4 outputs get your papers cited if you can... 3*/4* journals hard to assess – impact factor alone not enough but... >15% for 3* (data from Web of Knowledge) > 5% for 4*

37. REF – what you can do – journal assessment

39. Please see Journals Ranking document on the intranet www1.aston.ac.uk/eas/staff-intranet/research/ref/

40. REF – what you can do Write up results for publication – target high end journals and for CSI personnel - high end conferences money for open access journals – contact Philip Davies mentoring available for those struggling to get 4 outputs get your papers cited if you can... 3*/4* journals hard to assess – impact factor alone not enough but... >15% for 3* (data from Web of Knowledge) > 5% for 4* Conferences – more difficult - base on published acceptance rates?? Attend PURE training session (please) Update your content in PURE – publications, conferences and other activities (from March 2012). Instructions and guidance notes available from http://www1.aston.ac.uk/staff/rso/pure/http://www1.aston.ac.uk/staff/rso/pure/

41. REF – what can you do Select up to 6 of your top papers (papers/conferences for CSI) and enter 100 word summaries into PURE (from March 2012). This will be audited in Sept/Oct 2012. 100 word summaries - not the abstract nor mention citations but should highlight why the work is significant, has led to further developments etc. Examples of good and bad practice will be posted on the intranet Review (and update) your PURE entries regularly Update personal web page(s) pages regularly Any suggestions re impact / environment - please contact Andy a.j.sutherland@aston.ac.uk x3425a.j.sutherland@aston.ac.uk John j.leigh@aston.ac.uk x3746j.leigh@aston.ac.uk Paul p.r.knobbs@aston.ac.uk x4150p.r.knobbs@aston.ac.uk Susan s.m.doughty@aston.ac.uk x3665 but only until May...s.m.doughty@aston.ac.uk

42. REF – what can you do Any questions?? It’s tastic!!