1. The Big Picture – curricula, the Gold Guide and the assessment system
Training the Trainer/Assessor
Workplace-based assessment
East Midlands Deanery
Wednesday 28 January 2009
The Big Picture – curricula, the Gold Guide and the assessment system
Joanne Brinklow
Head of Educational Standards

2. Aims Postgraduate Medical Education and Training Board (PMETB)
Curricula
Assessment systems
Quality Framework
Gold Guide
Annual Review of Competence Progression (ARCP)

3. PMETB standards (published July 2008)

4. Generic standards for training
Standards for curricula and assessment systems
Standards for deaneries
Available on PMETB website:

5. Standards for curricula and assessment systems
17 standards to meet by 2010 (15 September 2009 submission date!)
Met standards 2, 4 and 11 so far
Most remaining standards at least partially met
Annual submission of minor amendments to curricula and assessment systems

6. Examples of the standards
The purpose of the curriculum must be stated, including linkages to previous and subsequent stages of the trainees’ training and education (1)
The overall purpose of the assessment system must be documented and in the public domain (2)
The choice of assessment method(s) should be appropriate to the content and purpose of that element of the curriculum (8)

7. Curricula:
Chemical Pathology:
Histopathology:
Medical Microbiology and Virology:

9. College assessment system approved by PMETB
Workplace-based assessment (including MSF)
Year 1 assessment
FRCPath Part 1 (test of knowledge)
FRCPath Part 2 (test of skills and applied knowledge)
Feedback on regular basis (WBA)
Educational supervisor’s report
ARCP (Annual Review of Competency Progression)

10. The College assessment system: Miller’s pyramid
WBA – MSF, CbD
Does
Action
WBA e.g. Mini-CEX, DOPS
Shows
FRCPath Part 2 Practical
Performance
Knows how
Competence
Knows
FRCPath Part 1
Knowledge

11. Value of multiple assessments
Key is several modes of assessment – triangulation
A single exam system (FRCPath) does not provide adequate coverage of clinical practice
WBA is also not sufficient in itself

12. Blueprints
Assessments must systematically sample the entire content, appropriate to the stage of training, with reference to the common and important clinical problems that the trainee will encounter in the workplace and to the wider base of knowledge, skills and attitudes demonstrated through behaviours that doctors require (Standard 4)

14. PMETB’s Quality Framework
The purpose of the Quality Framework is to:
Provide public and professional reassurance about the standards and quality of postgraduate medical education in the UK, through a robust, rigorous set of processes;
Reflect fully the principles of good regulation, demonstrate value for money and be fit for purpose;

15. Enable improvement and enhancement of the quality of postgraduate medical education; and
Ensure specialty focus is maintained at local and national level by working with the Academy of Medical Royal Colleges (AoMRC), colleges/faculties and postgraduate deaneries.

16. QA/QM/QC of postgraduate medical education and training
Local education
providers
Quality Management
Quality Assurance
Quality Control
Postgraduate
deaneries
PMETB
Colleges/faculties/
associations
QA/QM/QC of postgraduate medical education and training

17. SHARED EVIDENCE includes minimum data set
PMETB QF
RESPONSES TO CONCERNS
VISITS to Deaneries
SURVEYS
STANDARDS
Includes approval
and action planning
SHARED EVIDENCE includes minimum data set

18. Guide to Specialist Registrar Training
The Orange Book
1998

20. The Gold Guide Replaced the ‘Orange Guide’ for training
First published June 2007
Re-published June 2008
Available on MMC website:
Sets out ARCP process

21. What is the ARCP?
The Annual Review of Competence Progression is “a formal process which uses the evidence gathered by the trainee, relating to his/her progress in the training programme” (Gold Guide, page 64)
Replaces RITA for StRs

22. College input to ARCP
Pathology specific guidance and forms and specialty specific educational supervisors structured report
Amended ARCP form to incorporate information about End of Stage Review
All available on College website
To be facilitated in due course by new web-based system

23. Annual Review of Competence Progression
Educational
Appraisal
Annual Review of Competence Progression
Outcome
Annual Planning
1. Learning agreement:
aims & intended learning outcomes
based on specialty curriculum
2. Advice on portfolio
3. Regular feedback (2 way)
4. Personal Development Plan
5. Trainer’s structured report
6. Workplace based (NHS) appraisal
Educational supervisor and/or TPD meet with trainee to
review competence outcome with trainee
plan next part of training
Based on a paper from PMETB’s Workplace Assessment Group (2005) and Gold Guide
Workplace based (NHS) appraisal
ARCP
A. Evidence
1. Assessment of performance e.g.
workplace based assessments and observational methods
E.g. mini-CEX, CBD, DOPS, ECE
examinations
structured report
2. Assessment of experience, e.g.
portfolio/log book
audit
research
critical incidents
B. Annual Competence Review
Appropriately constituted panel considers evidence
outcome
Review of evidence of progress
Outcome of review
Educational Planning

24. Features of the ARCP Deanery-led process
“The trainee should not normally attend the panel…unless there may be an unsatisfactory outcome” (Gold Guide, pg. 66/67)
Additional or remedial training following an ARCP outcome limited to one year (exceptionally two) within the total training programme duration

25. Educational Supervisors structured report
Role of the educational supervisor defined in the curricula
Must complete a structured report for their trainees’ ARCP
The structured report summarises the trainee’s portfolio
Trainee’s responsibility to provide the documentation to the supervisor

26. What should the report include? The report should include:
a summary of the WPBA outcomes
outcomes from the MSF (if relevant) and Year 1 Assessment
Logbook/portfolio
summary of audit, research, publications, teaching, management, presentations, courses etc.

27. ARCP outcomes (9 in total)
1. Achieving progress
2. Development of specific competences required
3. Inadequate progress
4. Released from training programme
5. Incomplete evidence
6. Recommended for completion

28. End of Stage Review (ESR)
The College monitors the trainee’s progress through both the year and stage of training.
ARCP outcome panel must consider whether the trainee has completed the requirements for the stage of training and record this on the form as appropriate.
If the ESR is not in phase with the ARCP, satisfactory completion of the stage of training can be agreed between ARCPs but must be documented.

29. Observations from the RCPath (so far)
Only StRs should undertake ARCPs
All information on the outcome form must be completed in full
Only tick one outcome box
Ensure that the trainee, deanery and College all receive a copy of the outcome form
Use/refer to College guidance and forms

30. The Big Picture – curricula, the Gold Guide and the assessment system
Training the Trainer/Assessor
Workplace-based assessment
East Midlands Deanery
Wednesday 28 January 2009
The Big Picture – curricula, the Gold Guide and the assessment system
Joanne Brinklow
Head of Educational Standards