1. Assessment of Law Faculties in the UK
Nigel Duncan
Professor of Legal Education

2. Who is assessing UK Law Faculties?
Student Fees
Formal Informal
Professional Regulators
eg Complete University Guide
Research Excellence Framework
Law Faculty
Employers
Quality Assurance Agency
Students and Parents
Teaching Excellence Framework
National Student Survey

3. Research Excellence Framework
Discipline-specific.
Last ran in Probably next in 2021.
Recently reviewed by the Stern Report.
Findings inform the allocation of central research funding.
Institutions prepare a submission addressing:
Outputs – 65%
Impact – 20%
Research environment – 15%

4. REF: Outputs
Each research-active academic should produce at least four refereed publications at the requisite standard.
4* Quality that is world-leading in terms of originality, significance and rigour.
3* Quality that is internationally excellent in terms of originality, significance and rigour but which falls short of the highest standards of excellence.
2* Quality that is recognised internationally in terms of originality, significance and rigour.
1* Quality that is recognised nationally in terms of originality, significance and rigour.

5. REF: Impact
4* Outstanding impacts in terms of their reach and significance.
3* Very considerable impacts in terms of their reach and significance.
2* Considerable impacts in terms of their reach and significance.
1* Recognised but modest impacts in terms of their reach and significance.
Examples from City Law School:
Immigration detention law and policy
Impunity and the Rule of Law: the Safety of Journalists
National and international development and reform of the law of criminal evidence

6. REF: Environment For example:
An environment that is conducive to producing research of:
4* world-leading quality, in terms of its vitality and sustainability.
3*  internationally excellent quality, in terms of its vitality and sustainability.
2*  internationally recognised quality, in terms of its vitality and sustainability
1*  nationally recognised quality, in terms of its vitality and sustainability
For example:
the number of research doctoral degrees awarded
external research income

7. REF: Critique
“the REF has become a monster, a Minotaur that must be appeased by bloody sacrifices.” Peter Scott,  Professor of higher education studies, Institute of Education,
The Guardian, 4 Nov 2013.
Jean-Etiennes Ramey
£246 million

8. Teaching Excellence Framework (TEF)
The Government says it has introduced TEF as a way of:
Better informing students’ choices about what & where to study
Raising esteem for teaching
Recognising and rewarding excellent teaching
Better meeting the needs of employers, industry, business & the professions
Staged introduction:
Year 1 (2017) All institutions graded: ‘Meets expectations’.
Year 2 (2018) Institutions graded on basis of a series of factors – institution-level only.
Thereafter: subject-level assessment to be introduced.

9. TEF: Assessment: Metrics plus a Narrative Submission
The Metrics
Teaching on my course (NSS Q 1-4 )
Assessment and Feedback (NSS Q5-9 )
Academic Support (NSS Q10-12)
Non-Continuation (HESA & ILR Data )
Employment or Further Study (DLHE 6 months p.g.)
Highly skilled employment or further study (DLHE)
Data is benchmarked to allow meaningful comparisons between institutions by taking into account the different mix of students at each provider.
Results that are significantly different from benchmark are highlighted.
2 percentage points above benchmark = one flag
3+ percentage points above benchmark = a double flag Similarly, if below benchmark (one or two negative flags)

10. The Narrative Submission
Purpose: to enable the provider to:
Add additional context e.g. mission
Support or explain performance against the metrics
Put forward evidence against the assessment criteria
Further explore performance for specific student groups based on split metrics.
Assessment Criteria:
Teaching Quality
Learning Environment
Student Outcomes & Learning Gain
TEF Descriptors (bronze, silver, gold)

11. Narrative Submission (2)
Narrative must demonstrate the impact & effectiveness of teaching on student experience and the outcomes they achieve.
Wherever possible, impact should be demonstrated empirically.
No prior knowledge or external evidence can be taken into account.
Teaching & learning excellence must be demonstrated across an institution’s entire provision (no pockets of excellence).
Evidence should normally be from the last 3 years. (Not future promises)

12. TEF: Critique “Deeply-flawed and impractical”
Prof Sir Keith Burnett, VC University of Sheffield
“The TEF will not serve students, but an ill-defined set of employers”
Dr. Joshua Forstenzer, VC Fellow, U. Sheffield
“the TEF … risks increasing the bureaucracy of modern university life; and the management culture which is displacing an older and more noble idea of the university. We want academics to teach in the spirit of the community of which they are the citizens and makers, not to teach as they are directed to teach by bureaucrats.”
Emran Mian, Director, Social Market Foundation
“A Trojan Horse for raising fees”
Gordon Marsden MP
Domenico Tiepolo

13. National Student Survey
Final year undergraduate only.
27 Questions – on a six-point Likert scale
The teaching on my course
Learning opportunities
Assessment and feedback
Academic support
Organisation and management
Learning resources
Learning community
Student voice
Overall satisfaction
Also: Higher Education Academy:
Postgraduate Taught Experience Survey;
Postgraduate Research Experience Survey

14. Assessment and Measurement
Increasing tendency to demand metrics to assess the quality of a faculty’s work.
Only valuing what is measurable
Metrics in tension, ie
High score in the REF diverts funds from teaching and learning
Therefore funding research is in tension with financial sustainability
Makes it difficult to adopt a holistic approach to managing a faculty
Do we have the capacity to learn to measure what we value?

15. Odysseus and Polyphemus
Measurement is inevitable. Can we use it to achieve our goals and find ways of avoiding its damaging consequences?
Jacob Jordaens

16. Acknowledgement, links, references
My thanks to Fiona Cownie, Pro Vice Chancellor (Education & Student Experience) Keele University, for content on TEF.
Stern Report:
Derek Sayer: Rank Hypocrisies: the Insult of the REF. London, SAGE, 2015
Stefan Collini, Who are the spongers now?, London Review of Books, 2016, Vol 38, No 2, p.33-37
Edwards, Marc A., and Siddhartha Roy  Academic Research in the 21st Century: Maintaining Scientific Integrity in a Climate of Perverse Incentives and Hypercompetition. Environmental Engineering Science 34(1):51-61.