1. UCL experience of the TEF and the subject pilot
UCL ARENA CENTRE FOR RESEARCH-BASED EDUCATION
UCL experience of the TEF and the subject pilot

2. Origin in 2016 white paper setting out reforms to HE
Origin in 2016 white paper setting out reforms to HE. Base in fees and student satisfaction

3. Government policy Drivers
UCL ARENA CENTRE FOR RESEARCH-BASED EDUCATION
The context….
Government policy
Students at the Heart of the System 2011
Tory manifesto 2015
Higher Education and Research Act 2017
Drivers
Dominant focus on research (REF)
How to prove UK has excellent teaching?
Student/media criticism of HE teaching (contact hours, VfM, grade inflation)
Todays wonkHE (16th Sept)
Nothing too specific on the timing of responses to existing policy agendas, but Williamson did flag the Augar review and Shirley Pearce’s independent review of the Teaching Excellence Framework (TEF). Of the Augar review he paid the tribute that it would “make a big difference to our thinking”, a form of words that perhaps signals a degree of polite distancing from the specifics of the recommendations of the review. A response to the TEF review was promised “shortly”. It’s noticeable that teaching quality – or student experience more generally – didn’t really feature substantially in either speech.

4. Purpose Better informed student choice Raise esteem of teaching
UCL ARENA CENTRE FOR RESEARCH-BASED EDUCATION
Purpose
Better informed student choice
Raise esteem of teaching
Recognise and reward excellent teaching
Better meet the needs of employers, business, industry and the professions

8. How TEF2 (2017) Worked
Inputs
Outcomes
Eligibility

12. Rating GOLD SILVER BRONZE TOTAL 295
TEF 2 results
Rating
Number of Providers
GOLD 59
SILVER
116
BRONZE 56
PROVISIONAL 64
TOTAL
295

13. Official “Lessons Learned” document 2017
Process fair and transparent
Clear and robust findings on basis of metrics, submissions and guidance
Balance of evidence between metrics and submissions broadly right
No significant biases
No changes to overall structure or methodology
Areas identified to strengthen accountability and fully recognize excellence

14. 10 Conclusions by Chris Husbands (my list)
Focus on things surveys say matter to students: routes to work, retention, assessment, teaching quality
Changing student focus from inputs, processes and outputs, to outcomes
Metrics matter, but must be benchmarked: generate hypotheses and anchor initial judgements
Core of TEF is strategic clarity – relationship between institutional policies and practices, and students outcomes
Best accounts did not describe initiatives but systematically demonstrated difference made

15. 10 Conclusions by Chris Husbands (my list)
Initiatives carried conviction when part of a coherent strategy for improvement
Technology tools matter less than how the tools are used
Genuine student involvement stood out: an embedded culture of engagement at every level
Too many institutions used context as an excuse rather than analysing and responding to the challenges it offered
Excellence is distributed across the sector and matches its diversity

16. What do students think of TEF

17. The Subject pilot 2019 – confidentiality!
50 institutions
Very short time period – data released Dec 2018 and submission Feb 2019
34 subject groupings (problematic)
The Submission included:
1. A 15-page provider-level narrative submission 
2. A single, two-page summary statement that will accompany each subject submission, setting out the institutional context 
3. A five-page narrative submission for each of the subject areas taught at UCL.  (There are possible 34 total subject areas in total). 

18. A bit about the process Drafters nominated by deans for each subject
We used annual QA process to collect draft narratives but matches to TEF subject groupings not straightforward
Sharepoint area for each subject with all the data – relating to core metrics and contextual data, initial hypothesis and suggestions for areas to address in the narrative
Writing day with Arena TFs available to help interpret the data and support
First drafts reviewed by Arena TFs and internal panels
2nd drafts
Polish by VP Education Comms team
Institutional 15 pager

19. Results and lessons learned
Internally positive – external enemy!
Useful for highlighting areas for development and increasing understanding of the framework and the importance of evaluating the effectiveness of initiatives and interventions
Strengthened networks around L&T and also cross-team working (VP Education, Academic development, Academic Services, Careers, SU, EDI, Digital Education)
However….
Outcomes were not consistent. Different panels appeared to evaluate the evidence differently and sometimes the feedback was contradictory.

20. Further Reading
Beach D (2017) Going for Gold: Lessons from the TEF provider submissions. HEPI report 99
Department for Education (2017) Teaching Excellence Framework: Subject-level pilot specification
Department for Education (2017) Teaching Excellence Framework: Lessons Learned Summary policy document
HEFCE TEF
UUK (2017) Review of Teaching Excellence Framework Year2: Process, results and Next Steps
Wonkhe.com
Forthcoming …. Dame Shirley Pearce