1. Quality and the learning environment: what can our evidence tell us?
Jessica Lichtenstein
Head of Quality Assurance

2. Current QA framework
Our QA Framework shows how we monitor and check to make sure the standards are being met.
However, one area of concern is the GOOD PRACTICE, and I’ll come on to that in a moment.

3. How does evidence inform our QA?
Set standards for educational environments and implementation of curricula & assessment
Use intelligence to screen medical education providers and check standards are met
Identify risks and issues and prioritise them
Make interventions and take targeted action locally (GMC or other regulators)
Use intelligence to monitor impact of interventions and actions
Identify risks and issues and prioritise them

4. NTS results and learning environments
There is a report like this for every training post in the UK
That’s over 5,000 reports
So how do we decide where to focus our attention?
and 225,000 indicator outcomes

5. Learning environments – what’s currently bad?
To find out where the worst places are right now, we look for the training posts with…
the most red flags
This is converted into a proportion to allow fair comparison with different posts
We look at low means as well as flags to double check the worst performing places are picked up
the lowest mean scores

6. Learning environments – what’s been bad for a while?
To find the places with ongoing problems we look at training posts with…
the most triple reds

7. Learning environments – what might be bad in the future?
To find the places where things might get worse in the future we look at training posts with…
the most big drops in mean score
This allows us to detect movement in within the ‘white’ flags which can otherwise hide negative changes

8. Triage - rankings
These measures are then all converted to rankings and given a percentile score:
Current red ranking: 89
Low score ranking: 90
Triple red ranking: 97
Change in score ranking: 00
We currently refer any training post with a score of 100 in any of these measures to Visits and Monitoring for further scrutiny

9. What does this look like?

10. What do we do with this?
We discussed organisations on this list with HEE/deaneries and compared evidence
We agreed status of each organisation e.g. :
local quality team to investigate
issues confirm evidence so escalate to enhanced monitoring
circumstances mean that results don’t warrant further investigation
Monitor appropriately

11. Future development
Currently using only NTS post specialty by site reports- we think this tells us the most about a singular learning environment
Future plans:
Add trainer survey and progression report outputs
Develop views of programme-level results
Positive results & significant improvements

12. Questions??