1. REF2014
Pauline Muya
Jo Lakey

2. Contents What is the REF?
Elements of Assessment: Outputs, Environment, Impact
REF Timetable
Staff eligibility
Code of Practice
Staff circumstances

3. What is the REF?
Part of the system of funding HEIs Funding formula requires a measure of research quality Previously called Research Assessment Exercise (1986 – 2008), now Research Excellence Framework (2014 onwards) Quality judgements by discipline-based panels of expert peer reviewers Each unit of assessment awarded quality profile (0 to 4*) Results translated into funding formula multiplier

4. Selectivity and Performance
Funding is increasingly selective
% of QR funds to 26 HEIs
% of QR funds to 23 HEIs, 27% QR funds to ‘big four’ HEIs
– 30% to big four
Selective funding drives sector performance
“in the period since the first RAE in 1986, the effectiveness and productivity of the UK research base has significantly increased”
The Role of Selectivity and the Characteristics of Excellence, Evidence Ltd, 2000.
Big four – Oxford, Cambridge, UCL and Imperial. Science helps! Also biggest for RG&C income

5. Quality Profiles
By percentage of research activity in the submission judged to reach quality standard
Sub-Panel x
Cat A FTE Staff submitted for assessment 4* 3* 2* 1*
Unclassified
University A 50 15 25 40 5
University B 20 45 10
Sub-panel = subject

6. Overall quality profile: education20 4* 10 15 30 25
u/c 1* 2* 3*
Quality Level
% of Research Activity
10%
70%
20%
Research Environment
Esteem Indicators
The overall quality profile comprises the aggregate of the weighted profiles produced for outputs, research environment and esteem indicators – rounded to nearest 5%
Research Outputs 4* 3* 2* 1*
u/c 15 25 35 10 4* 3* 2* 1*
u/c 25 15 20 4* 3* 2* 1*
u/c 30 20 10 6

7. Funding the RAE2008
Funding Councils take profiles and devise a funding formula (each FC uses a different formula and it can change each year)
Formula is adjusted to fit overall funds available
Further performance improvement across the sector
Prioritisation of STEM subjects
Further concentration of resources – islands of excellence
volume changes
relative quality changes

8. IOE Funding from RAE2008
FTE entered 35% 4*, 30% 3*, 25% 2*, 10% 1* Funding allocated 3:1 in favour of 4*:3*, no funds for 2* and lower Funding in 2012/13 of £7,854,108

9. Research Excellence Framework
Selection of outputs from members of “research active” staff Research environment: grant funding, PGR students, research degrees awarded New element of Impact Profiles will be 65% outputs, 15% environment, 20% impact

10. REF Timetable Intention to submit: December 2012 Submit: November 2013
Results: December 2014
Funding decision: March 2015
Funding: August 2015
Census date October 2013
Outputs 1 January 2008 to 31 December 2013
Environment and Impact 1 January 2008 to 31 July 2013
Data August 2008 to 31 July 2013

11. Elements of Assessment
Research outputs 65%
Originality, significance and rigour
Environment 15%
Sustainability and vitality
Impact 20%
Significance and reach

12. Outputs
The sub-panels will assess the quality of submitted research outputs in terms of their ‘originality, significance and rigour’, with reference to international research quality standards
All forms of outputs: articles, books, chapters, exhibitions, performances, compositions, media, reports, patents, software, etc.
Outputs can be double-weighted if they consist of a substantial amount of work, but a reserve publication can also be submitted in case the panel don’t accept it as a double output.
The deadline for publication of outputs is December 31st That means that the output must be in the public domain by that date. As submission is 29th November, it means that you mare allowed to submit details of outputs which you know will be available by 31st December.

13. Outputs

14. Environment Criteria:
Vitality will be understood as the extent to which a unit provides an encouraging environment for research, has an effective strategy, is engaged with the national and international research and user communities, and is able to attract excellent postgraduate and postdoctoral researchers
Sustainability will be assessed by considering leadership, vision for the future and investment in people and infrastructure and, where appropriate for the subject area, the extent to which activity is supported by a portfolio of research funding

15. Environment: elements
Data relating to research income and doctoral degrees awarded
Overview – 0%, Panel C gives equal weighting to all other sections
Environment template gives us seven pages to talk about how the institute supports and develops researchers and research. Institutional, faculty and departmental level.

16. Impact Two criteria for impact:
how significant or transformative the impacts have been
their reach (how widely the impacts have been felt)
For the REF, HEFCE have defined two main criteria for assessing impact: how significant or transformative the impacts have been and their reach or how widely the impacts have been felt. To understand how these can be assessed, think of two examples of impact. The first details the development of a drug which completely cures a disease. But the number of people suffering from the disease is relatively small. So whilst the significance is very high, the reach is limited. The second details the development of a drug which helps everybody suffering from minor aches and pains feel slightly better slightly faster than competitor drugs. For this example, the significance is relatively low, whilst the reach is potentially enormous. For the purposes of the REF assessment we would expect both of these cases to receive a similar rating.

17. Impact: general guidance
Impact will count for 20% of overall profile
Research conducted at institution
Underpinning research has to be of high quality (deemed ≥2* or originating from a peer-reviewed competitive project)
2 case-studies up to FTE, 1 more every 10 FTE
Research between January 1993 and December 2013
Impact occurring between January 2008 and July 2013
Impact statement to describe approach to impact and future strategies (20%)
Assessors from user communities
With impact making up 20% of an institution’s overall profile, it’s obviously an important part of an institution’s preparations for the REF. Imagine you’re an institution that receives £10m a year from QR funding. Impact is worth £2m and your case studies are worth £1.6m. That’s a lot of 4* publications!
The research underpinning your case studies has to have been conducted at the submitting institution. Whilst academics take their outputs with them when they move institutions, impact stays with the institution where the research was carried out. There’s a lot of debate about whether it’s a good idea to submit case-studies based on research by staff who’ve left the institution - if the member of staff left under less-than-ideal circumstances for example, would they co-operate with the preparation of a case study? Can one be prepared without their input? Likewise, an institution with a new member of staff whose previous work has had impact would not be able to submit that case study unless further research had been carried out that generated additional impact – all within the REF period, of course!
Research needs to be 2* or above – internationally recognised for its originality, significance and rigour. Or originating from a peer-reviewed competitive project.
Each unit of assessment submits 2 case studies for up to FTE and one additional case study for every 10 FTE thereafter. This could mean some difficult decisions, if a UoA with 24 staff suddenly adds a couple of members of staff in the lead-up to the REF, that will mean an additional case study will be needed. And at this stage it’s unlikely that the new members of staff will be able to submit a case study (unless they are a former member of staff returning to the institution, which does happen). If an additional, strong case study isn’t available, what does the UoA do?
Research needs to have appeared in the public domain between 1 January 1993 and 31 December 2013 – so outputs which are definitely in press and will be available after the submission deadline are allowed. Some UoAs (i.e. Architecture) allow a longer period for the research to recognise the long time lag between research and impact in that area. Impact has to have occurred between 1 January 2008 and 31 July So impact can have occurred before the research is published. This recognises that not all impact stories are linear. There can be occurrences when impact occurs during the research period, for example a project carried out in a local hospital which identified gaps in service and the NHS trust funded additional services whilst the research was still going on.
The case studies count for 80% of the impact assessment, and we also submit a template which describes what the institution has been doing to support and encourage impact and what it plans to do in the future, drawing on the submitted case studies and the lessons learned from how their impact was developed.
It’s important to note that there will be representatives from user communities for each UoA on the assessment panels. Some of these will be recruited to the panels following the Survey of Submission Intentions in December, which will identify the areas of impact that each Institution plans to submit, allowing the panels to recruit the appropriate numbers of assessors with the appropriate expertise. These were invaluable in the Impact Pilot in and enabled the panels to make distinctions between similar examples of impact through their knowledge of the specific circumstances.

18. Staff eligibility Contract of at least 0.2 FTE
All staff on ‘teaching & research’ or ‘research only’ contracts as per HESA definitions
Independent researchers – usually Grade 8 and above, but some Grade 7 staff will be eligible (i.e. John Adams Fellows)
Not eligible if employed to carry out another individual’s research programme rather than as independent researchers in their own right
Usually 4 outputs for each staff member submitted
You are deemed to have started your career as an independent researcher when you have a contract of 0.2FTe or above which includes a primary employment function of undertaking “research” or “teaching and research” with any HEI and undertook independent research, leading or acting as principal investigator or equivalent on a research grant or significant piece of research work. It’s not enough to be named on research outputs. Research assistants are ineligible. (paras of Guidance on submissions)

19. Code of Practice
Institutional practices that ensure the REF submission is transparent, inclusive and consistent
Staff circumstances survey – December 2012
Standard circumstances (maternity/early career/part-time)
Complex circumstances (case by case)
Inclusion threshold – not yet decided

20. Clearly defined circumstances: ECR
Date at which the individual first met the REF definition of an early career researcher:
Number of outputs may be reduced by up to:
On or before 31 July 2009
Between 1 August 2009 and 31 July 2010 inclusive 1
Between 1 August 2010 and 31 July 2011 inclusive 2
On or after 1 August 2011 3

21. Clearly defined circumstances: part-time working, secondment, career break
Total months absent between 1 January 2008 and 31 October 2013 due to working part-time, secondment or career break:
Number of outputs may be reduced by up to: 1 2
46 or more 3

22. Clearly defined circumstances: maternity, paternity or adoption leave
Individuals may reduce the number of outputs by one, for each discrete period of: Statutory maternity leave or statutory adoption leave taken substantially during the period 1 January 2008 to 31 October 2013, regardless of the length of the leave Additional paternity or adoption leave lasting for four months or more, taken substantially during the period 1 January 2008 to 31 October 2013.

23. Complex circumstances
Circumstances which have impacted on an individual’s ability to work productively throughout the assessment period Confidential submission via the staff survey Panel will make decisions about the number of outputs each member of staff needs to submit Panel will consist of HR, RCS and academic representatives Appeals panel

24. IOE Preparations
Internal REF exercise Reviews over Spring term with feedback April 2013 Decisions about threshold for inclusion made by Director, advised by Pro-Director, Research and Development and theme leaders Combined with results of staff survey to determine which staff will be submitted

25. Any questions? Institute of Education University of London
20 Bedford Way
London WC1H 0AL
Tel +44 (0)
Fax +44 (0)
Web ioe.ac.uk