DUAL SUPPORT DUEL FOR SUPPORT Professor Sir Gareth Roberts University of Oxford

Purpose of Research Assessment Allows funders to assess the quality of research arising from investment of public money. Enables the academic sector to assess its success and inform its future strategy. Introduces an incentive to individuals and institutions to improve research performance. Can inform a funding model.

INTERNATIONAL RESEARCH EVALUATION Intensity of Research Evaluation HIGH NETHERLANDS UNITED KINGDOM LOW UNITED STATES OF AMERICA AUSTRALIA Influence on Funding Decision

Evaluation mechanisms disturb the system they purport to evaluate

BENEFITS RESULTING FROM THE INTRODUCTION OF AN RAE Natural research competitiveness is channelled into a pervasive driver of excellence thus improving the Quality and Quantity of Research conducted. A much stronger Research Culture with a greater emphasis on recruiting and retaining high calibre researchers. More active management of staff research.

Proportion of UK Entries in Top 1% of World’s Most Cited Papers

HONG KONG Research Assessment Exercise “Undoubtedly the previous RAEs have helped to encourage the development of a robust and vibrant research culture and to drive a sustained improvement in research activities in terms of both quality and quality of output.” University Grants Committee RAE 2006

LIKELY EQUIVALENCE BETWEEN UK 2008 RAE AND RQF MODEL An Expert-Review based assessment process with a strong international and end-user dimension. A limited number (12 or 15) of Expert Panels whose judgements will be moderated by a special Panel comprising all the Expert Panel Chairs. Researchers expected to have produced 4 Research Outputs over a 6 year assessment period, with protection for those in special categories e.g. early career researchers. Flexibility to include submission by a group of researchers.

LIKELY EQUIVALENCE BETWEEN UK 2008 RAE AND RQF MODEL Expert Panels to have specialist assessors at the sub-discipline level and specific arrangements for cross-referral of multidisciplinary research. Research Quality to be assessed against a 5 point grading scale and reported via a graded profile. No rating of individuals. Block Grant funding to be allocated to institutions based on volume count of eligible staff and assessment scores. Universities not required to select all their eligible staff for inclusion.

AUSTRALIAN MINISTER FOR EDUCATION, SCIENCE AND TRAINING “I believe that a Research Quality Framework is vital for Australia. We need it to lift our overall level of research quality and to shift our focus towards research which really does have an impact on day to day life.” June 2006

RESEARCH AND INNOVATION RESEARCH MONEY KNOWLEDGE INNOVATION KNOWLEDGE MONEY

Henry Kissinger “Professors are persons whose job it is to solve the problems of life which they themselves have tried to avoid by becoming professors”

UK University Research Evaluation | | | | | | Time 1986 1989 1992 1996 2001 2008 Estimated Benefit ~ 1% of Total Distributed Funds Approximate Cost

DfES / Funding Council Consultation Paper We cannot expect metrics to reproduce in full the granularity of peer review conclusions. The quality measures generated by the models are not very highly correlated with individual RAE unit of assessment rating.

A POSSIBLE FUTURE APPROACH A much lighter touch system should be introduced based on ‘FACULTY’ sized units and based on discipline-specific baskets of metrics. Expert Panels should be established for ALL disciplines to monitor movements in performance indicators based largely on metrics approved by the academic communities and whose appropriateness will have been confirmed by the exercise run in parallel with the 2008 RAE. Funding Councils should use financial allocations resulting from 2008 RAE as the baseline for future awards. These would be moderated by relative changes over time in a university faculty’s relevant metrics.

BEYOND 2008 REVIEW International Benchmark Reviews for Top 10% or 20% of UK Research Groups. Rewards for International Partnerships. Assessment of Long Term Impact/Knowledge Transfer Performance of Institutions. Emphasis on ‘Weeding Out’ of Handle-Turning Research. Alternative Research Assessment for Less Research Intensive Universities.