1. Curriculum 2019: Programme of assessment
Mr Rehan Khan, Assessment Advisor (Workplace Based)
Welcome to an online presentation about the programme of assessment for the new O&G curriculum. My name is Rehan Khan and I’m the RCOG Assessment Adviser for workplace based assessments. This presentation will provide an overview of the new assessment model, highlights what is staying the same and what will be different, and talks in detail about how assessments will be used to support doctors progressing through the training structure. There is also a section which outlines the concept of global judgements and how these will be applied in a clinical setting.

2. rcog.org.uk/
Key points
The new curriculum will be an integrated framework of exams, assessments in the workplace and judgements made about a learner during their approved programme of training.
Current O&G ‘assessment at work’ methods have been fully reviewed as part of the curriculum development.
Assessment tools, which are well established in O&G training, will be both formative and summative and have been selected on the basis of their fitness for purpose and their familiarity to trainees and trainers.
The revised assessment tools will put the trainee at the heart of the assessment process, with a much stronger emphasis on the trainee as a reflective practitioner.
Assessors will make accountable, professional judgements on whether trainees have made sufficient progress throughout the training year to progress to the next year/level of training.
The successful completion of each stage of training will be dependent on achieving the expected level in all Capabilities in Practice (CiPs) and procedural skills, with the programme of assessment being used to monitor and determine progress throughout the programme.
The new curriculum will be an integrated framework of exams, assessments in the workplace and judgements made about a learner during their approved programme of training.
We’ve fully reviewed current O&G ‘assessment at work’ methods as part of curriculum development and aassessment tools, which are well established in O&G training, will be both formative and summative and have been selected on the basis of their fitness for purpose and their familiarity to trainees and trainers.
The revised assessment tools will put the trainee at the heart of the assessment process, with a much stronger emphasis on the trainee as a reflective practitioner.
Assessors will make accountable, professional judgements on whether trainees have made sufficient progress throughout the training year to go on to the next year and the successful completion of each stage of training will be dependent on achieving the expected level in all Capabilities in Practice (CiPs) and procedural skills, with the programme of assessment being used to monitor and determine progress throughout the programme.

3. Purpose of the programme of assessment
rcog.org.uk/
Purpose of the programme of assessment
To clearly communicate the expected levels of performance and ensure these are met on an annual basis and at other critical progression points and to demonstrate satisfactory completion of training as required by the curriculum.
The purpose of the programme of assessment is to:
Assess trainees’ actual performance in the workplace.
Encourage the development of the trainee as an adult responsible for their own learning.
Enhance learning by providing formative assessment, enabling trainees to receive immediate feedback, understand their own performance and identify areas for development.
Drive learning and enhance the training process by making it clear what is required of trainees and motivating them to ensure they receive suitable training and experience.
Demonstrate trainees have acquired the GPCs and meet the requirements of good medical practice.
Ensure that trainees possess the essential underlying knowledge required for the specialty.
Provide robust, summative evidence that trainees are meeting the curriculum standards during the training programme.
Inform the ARCP, identifying any requirements for targeted or additional training where necessary and facilitating decisions regarding progression through the training programme.
Identify trainees who should be advised to consider changes of career direction.

4. What has changed? rcog.org.uk/ Main changes
The successful completion of each stage of training will be dependent on achieving the expected level in all CiPs and procedural skills
Education Supervisors (ES) will now be required to make a global judgement as to whether trainees should progress to the next year of training
The Education Supervisors Report (ESR) has been redesigned to support the global judgement process
With multi-source feedback, a trainee will now be able to self-assess using a modified TO1 form (SO)
A reflective element has been added to workplace-based assessment tools
A new assessment, the NOTSS tool, will be used to rate and give feedback on non-technical skills.
While the fundamental training structure and waypoints remain the same in the new curriculum the main changes to the assessment structure are highlighted here:
Firstly, the successful completion of each stage of training will depend on achieving the expected level in all CiPs and procedural skills
Education Supervisors (ES) will now be required to make a global judgement as to whether trainees should progress to the next year of training and the Education Supervisors Report (ESR) has been redesigned to support this process
With multi-source feedback, a trainee will now be able to self-assess using a modified TO1 form called an SO form
A reflective element has been added to workplace-based assessment tools
And finally, a new assessment, the NOTSS tool, will be used to rate and give feedback on non-technical skills.

5. rcog.org.uk/
Curriculum 2019 assessments
Assessment type
Assessment method
Changes introduced
Examinations
Part 1 MRCOG – basic and applied sciences
Part 2 MRCOG – clinical knowledge
Part 3 MRCOG – clinical assessment
Summative
Examinations and waypoints are the same as in the old curriculum, but the exam syllabus has been remapped to the new knowledge areas
Workplace-based assessment – Assessment of Performance encounters
Objective Structured Assessment of Technical Skills OSATS
No changes introduced
Workplace-based assessment – Supervised Learning Event encounters
Case-based discussion CbD
Mini-CEX MCX
Non-Technical Skills for Surgeons NOTSS
Formative
Assessments have been revised to include an explicit reflective element
New NOTSS assessment tool
Workplace-based assessment – not encounter-based
Team observation TO1 & TO2
Self observation SO
Reflective practice TP
TO1 form revised to include an explicit reflective element
Trainees will now be able to self-assess using a modified TO1 form (SO)
The programme of assessment comprises the use of a number of individual assessment tools including:
the MRCOG examination
summative and formative workplace-based assessments.
A range of assessments are needed to generate the necessary evidence required for global judgements to be made about satisfactory performance, progression in, and completion of, training. And all assessments, including those conducted in the workplace, are linked to the relevant learning outcomes stated in the curriculum.
This table outlines the different types of assessments that will be used to assess the progression of trainees through the new Curriculum. It also gives an overview of any changes which have been introduced as a result of the new Curriculum. Each of these changes will be looked at in more detail later on in this presentation.
The number and range of assessments will ensure a reliable assessment of the training relevant to stage of training and curriculum coverage.
Those assessment tools which are not identified individually as summative will contribute to summative judgements about a trainee’s progress as part of the programme of assessment.
With the new SO form, a comparison of self rating with team rating is possible, and this opens out the opportunity for reflective learning, analogous to the foundation TAB.

6. MRCOG examination: Part 1, Part 2 and Part 3 Clinical Assessment
rcog.org.uk/
MRCOG examination: Part 1, Part 2 and Part 3 Clinical Assessment
Part 1 MRCOG:
covers basic and applied sciences relevant to the clinical practice of O&G
summative assessment of CiP 6.
Part 2 MRCOG:
covers the knowledge required during clinical practice
summative assessment of CiPs 9-12 and elements of CiP 2.
Part 3 MRCOG:
covers the application of knowledge, clinical competencies and attitudes in clinical practice
summative assessment of CiP 1 and CiP 8 (to show the application of medical knowledge, clinical skills and professional values in the provision of high-quality and patient-centred care and effectiveness as a teacher) and CiP 13 and CiP 14 (to assess skills as a champion of healthcare in all groups within society and a promoter of health and disease prevention).
The Part 1 MRCOG, which covers the basic and applied sciences relevant to the clinical practice of obstetrics and gynaecology, is focussed on the summative assessment of CiP 6. This is because by achieving Part 1 the candidate/trainee will have demonstrated an active participation in acquiring the fundamental scientific knowledge that underpins the development of clinical expertise. They will demonstrate an early indication of helping self to develop beyond the experiential learning within the clinical environment.
The Part 2 MRCOG, which covers the knowledge required during clinical practice, is used for the summative assessment of CiP 9, CiP 10, CiP 11 and CiP 12, with elements of CiP 2 to demonstrate developing clinical expertise. It assesses competence in recognising, assessing and managing emergency and non-emergency cases in gynaecology, early pregnancy and obstetrics.
The Part 3 MRCOG, which assess the application of knowledge, clinical competencies and attitudes in clinical practice, builds on this knowledge platform and summatively assesses CiPs 1 and 8 to show the application of medical knowledge, clinical skills and professional values in the provision of high-quality patient-centred care plus effectiveness as a teacher. It also covers elements of CiPs 13 and 14 to summatively assess skills as a champion of healthcare in all groups within society and a promoter of health and disease prevention.

7. Progression through the training programme
rcog.org.uk/
Progression through the training programme
Progression through the programme will continue to be determined by the ARCP process.
The assessment requirements for each indicative year of training are summarised in the table opposite.
The successful completion of each stage of training will be dependent on achieving the expected level in all CiPs and procedural skills.
Decisions on progression fundamentally rely on the professional judgement of the ES.
Towards the end of the training year, trainees will assess their own progression for each CiP and record this in the Training ePortfolio, signposting to the evidence that supports their rating.
Progression through the programme will continue to be determined by the ARCP process. The assessment requirements for each indicative year of training are summarised in the table.
Successful completion of each stage of training will depend on achieving the expected level in all CiPs and procedural skills.
Decisions on progression fundamentally rely on the professional judgement of the ES.
Towards the end of the training year, trainees will assess their own progression for each CiP and record this in the Training ePortfolio, signposting to the evidence that supports their rating.

8. Matrix of progression rcog.org.uk/
This has been updated so that it is relevant for the new curriculum.
The matrix provides guidance on all aspects of training, both clinical and non-clinical, for each year of training.
It sets out the requirements for a satisfactory ARCP outcome at the end of each training year and critical progression points.
The new matrix of progression is available here.
To ensure a consistent approach to assessment and progression through training, the RCOG has updated the matrix of educational progression so that it is relevant for the new curriculum.
The new matrix provides guidance on all aspects of training, clinical and non-clinical, for each training year.
It is essentially an ARCP decision aid setting out the requirements for a satisfactory ARCP outcome at the end of each training year with critical progression points.

9. rcog.org.uk/
Assessment of CiPs
Detailed guidance on the assessment of CiPs will be provided on Training ePortfolio. The guidance will cover:
what the CiP is about
statement of expectations
the kind of evidence that might be relevant to the CiP
when the CiP can be signed off.
The CiP is the fundamental basis of global judgement.
Trainees will be assessed throughout the training programme, allowing them to continually gather evidence of learning and to provide formative feedback.
Clinical Supervisors and others contributing to assessments will provide formative feedback to the trainee on their performance throughout the training year.
The ES will carry out an annual global judgement, and satisfactory sign off will indicate that there are no concerns and that the doctor in training meets or exceeds the minimum level of performance expected for completion of that stage of O&G training
The method of assessing CiPs varies depending on whether they are generic and non-clinical or a clinical CiP.
Detailed guidance on the assessment of CiPs will be provided on the Training ePortfolio. The guidance will cover:
what each CiP is about
statement of expectations
the kind of evidence that might be relevant and when it can be signed off.
The CiP is the fundamental basis of global judgement.
Assessment involves looking across a range of key skills and evidence to make a judgement about a trainee’s suitability to take on particular responsibilities or tasks appropriate to their stage. It also involves the trainee providing a self-assessment of performance.
Trainees will be assessed throughout the training programme, letting them continually gather evidence of learning and allowing the use of formative feedback.
Clinical Supervisors and others contributing to assessments will be giving the formative feedback to the trainees throughout the training year.
And annually, the ES, will do a global judgement. Satisfactory sign off will indicate that there are no concerns and that the doctor in training is meeting or exceeding the minimum level of performance expected for completion of that stage of training before the trainee can progress to the next assessment point.
The method of assessing CiPs varies depending on whether they are generic and non-clinical or clinical.
Evidence to support the global rating for the CiP will be derived from workplace-based assessments and other evidence, e.g. TO2.

10. Assessment of generic and non-clinical specialty CiPs
rcog.org.uk/
Assessment of generic and non-clinical specialty CiPs
With these CiPs trainees are required to make a self- assessment as to whether they meet expectations for the stage of training, linking to appropriate evidence in the ePortfolio.
ES will indicate on the ESR whether they feel a trainee is meeting expectations or not.
If trainees are not meeting expectations, an explanation will need to be given by the ES.
Trainees will need to meet expectations for the stage of training as a minimum to be judged satisfactory to progress.
Detailed guidance will be available on the Training ePortfolio.
With regards to generic CiPs trainees are required to make a self-assessment as to whether they meet the expectations for each stage of training, linking to evidence on the ePortfolio.
Supervisors can then indicate on the ESR whether the trainee is meeting expectations or not using this anchor statement:
Overall, I consider that the trainee is meeting or exceeding expectations for this stage of training.
The trainee not meeting expectations will have to receive an explanation for this from the supervisor.
Trainees will need to meet the expectation for the stage of training as a minimum to be judged satisfactory to progress.
And the expectations for each stage will be specified in detailed guidance via the Training ePortfolio.
Overall, I consider that the trainee is meeting or exceeding expectations for this stage of training.

11. Clinical specialty CiPs
rcog.org.uk/
Clinical specialty CiPs
Level
Descriptor 1
Entrusted to observe 2
Entrusted to act under direct supervision: (within sight of the supervisor) 3
Entrusted to act under indirect supervision: (supervisor immediately available on site if needed to provide direct supervision) 4
Entrusted to act independently with support (supervisor not required to be immediately available on site, but there is provision for advice or to attend if required) 5
Entrusted to act independently
With clinical specialty CiPs, the trainee will make a self- assessment to consider whether they meet expectations for the year of training, using the five supervision levels listed opposite and linking to relevant evidence in the ePortfolio.
The ES will then indicate whether they feel the trainee is meeting expectations or not by assigning one of the five supervision levels.
Trainees will need to meet expectations for the year of training as a minimum to be judged satisfactory to progress.
With clinical specialty CiPs, the trainee will make a self-assessment to see if they meet expectations for the year, using the five supervision levels that you can see on this slide linked to evidence in the ePortfolio.
The ES can then indicate whether the trainee is meeting expectations or not by also assigning one of the five supervision levels.
Trainees will need to meet expectations for the year of training as a minimum to be judged satisfactory to progress.
This table shows the five levels based on an entrustability scale. This is a behaviourally anchored ordinal scale based on progression to competence and it reflects judgments that have clinical meaning for assessors.
Overall, I consider that the trainee’s performance meets 1-5 of the clinical entrustability scale.
Overall, I consider that the trainee is meeting or exceeding expectations for this year of training.

12. Intermediate training Capabilities in practice
rcog.org.uk/
Clinical specialty CiPs
Basic training
Intermediate training
Advanced training
CCT
Capabilities in practice
ST1
ST2
CRITICAL PROGRESSION POINT
ST3
ST4
ST5
ST6
ST7
The doctor is competent in recognising, assessing and managing emergencies in gynaecology and early pregnancy. 1 2 3 4 5
The doctor is competent in recognising, assessing and managing emergencies in obstetrics.
The doctor is competent in recognising, assessing and managing non-emergency gynaecology and early pregnancy.
The doctor is competent in recognising, assessing and managing non-emergency obstetrics.
This table shows the level of supervision expected for the clinical CiPs as trainees progress through the training programme.

13. Assessment of procedural skills
rcog.org.uk/
Assessment of procedural skills
doctors must demonstrate that they are capable of unsupervised practice in all clinical specialty CiPs and that they meet the minimum requirements for each of the generic CiPs.
Trainees should ideally receive training in procedural skills in a simulated setting before performing these procedures clinically, but this is not mandatory.
Assessment of procedural skills will be made using the OSATS tool.
When a trainee has been signed off as being able to perform a procedure independently, they are not required to have any further assessment (OSATS) of that procedure, unless they or their Educational Supervisor think that this is required (in line with the standard professional conduct).
In order to complete training and be recommended to the GMC for the award of CCT and entry onto the specialist register, the doctor must demonstrate that they are capable of unsupervised practice in all clinical specialty CiPs and that they meet the minimum requirements for each of the generic CiPs.
Trainees should ideally receive training in procedural skills in a simulated setting before performing these procedures clinically, but this isn’t mandatory.
Assessment of procedural skills will be made using the OSATS tool. When a trainee has been signed off as able to perform a procedure independently, they are not required to have any further assessment using OSATS of that procedure, unless they or their Educational Supervisor think that it is required in line with standard professional conduct.

14. The global judgement process: principles
rcog.org.uk/
The global judgement process: principles
The programme of assessment in the new curriculum relies on the ES making a professional judgment as to whether trainees have met the learning outcomes and expected levels of performance set out in the approved curriculum.
This is not a new concept, and actually something which many doctors are already doing daily on an informal basis. The new curriculum seeks to formalise the process.
ES will make accountable, professional judgements on whether progress has been made
The ES will then offer a global judgement as to whether the trainee should progress to the next year of training.
The programme of assessment in the new curriculum relies on the ES making a professional judgment as to whether trainees have met the learning outcomes and expected levels of performance set out in the approved curriculum.
This might sound like a new concept but it’s actually something which many doctors are already doing daily on an informal basis. The new curriculum seeks to formalise the process.
ES will make accountable, professional judgements on whether progress has been made according to the trainee’s self-assessment for each CiP.
They can then offer a global judgement as to whether the trainee should progress to the next year of training.

15. The global judgement process: principles (cont’d)
rcog.org.uk/
The global judgement process: principles (cont’d)
Each CiP will be globally judged against the expectations for the particular stage of training.
Detailed guidance on the expectations for each CiP will be provided on the Training ePortfolio.
There will be a difference between the global judgement of generic and non-clinical specialty CiPs, and clinical specialty CiPs.
The global judgement decision will be recorded on the ESR, which has been redesigned to support the new Curriculum.
Trainees will be given the opportunity to agree or disagree with the recommendations of the ES.
Educational Supervisors report (ESR)
The Educational Supervisors records annually a longitudinal, global report of a trainee’s progress over the full range of CiPs based on a range of assessments, including exams and observations in practice or reflection on behaviour by those who have appropriate expertise and experience. The ESR can incorporate commentary or reports from observations, such as from supervisors, or formative assessments demonstrating progress over time.
The supervisor records annually a longitudinal, global report of a trainee’s progress over the full range of CiPs based on a range of assessments, including exams, observations in practice, reflections on behaviour by those with appropriate experience and expertise. The Educational Supervisors Report or ESR incorporates commentary or reports from these observations, plus formative assessments demonstrating progress over time.
Each CiP will be globally judged against expectations for each year of training. Detailed guidance on the expectations for each CiP will be provided on the Training ePortfolio.
There will be a difference between the global judgement of generic and non-clinical specialty CiPs versus clinical specialty CiPs. This is because of the need to allow the ES to make a decision about how much supervision a trainee needs before they reach the designated level of supervision for each CiP.
The global judgement decision will be recorded on the ESR, which has been redesigned to support the Curriculum.
Trainees will get the opportunity to agree or disagree with the recommendations.

16. The role of evidence in the global judgement process
rcog.org.uk/
The role of evidence in the global judgement process
Evidence is a crucial concept in the new curriculum
Trainees will collect evidence to support their self-assessment,
ES will use evidence to reach a global judgement.
Summative assessment
The MRCOG examination: Part 1, Part 2 and Part 3 Clinical Assessment
Objective Structured Assessment of Technical Skills (OSATS)
Formative assessment
Case-Based Discussions (CbD)
Mini-Clinical Evaluation Exercise (mini-CEX)
OSATS
Team Observation (TO1), TO2 and Self-observation (SO)
Non-Technical Skills for Surgeons (NOTSS) 
Supervisor report
Educational Supervisor Report (ESR)
Evidence is a crucial concept in the new curriculum, and as well as methods listed below, can include other sources, such as Personal Development Plan, quality improvement project or a procedure log.
The trainee collects evidence to support their self-assessment, and the supervisor uses it to reach a global judgement.
Examples of summative assessment include the MRCOG exam and OSATS.
Examples of a formative assessment might include Case-Based Discussions, mini-CEX, formative OSATS, TO1 and SO forms feeding in to a TO2 form and the NOTSS tool.
More information and guidance for trainees and assessors will be available through the ePortfolio.

17. The global judgement process: how it will work
rcog.org.uk/
The global judgement process: how it will work
Trainee uploads evidence on to the Training ePortfolio throughout the training year.
At any stage, trainees can assess their progress for each CiP signposting to the evidence that supports this.
Once they feel they have enough evidence to complete the CiP, they add a self-assessment rating of their performance, stating that it is:
not meeting expectations
meeting expectations OR
exceeding expectations.
Before the ARCP, the ES will generate the ESR, which pulls together a summary of the trainees progress within each CiP.
The ES will review the evidence in the ePortfolio including workplace-based assessments, the TO2 and the trainee’s self-assessment and record their global judgement of the trainee’s performance in the ESR with commentary.
The ESR will make a recommendation to the ARCP panel as to whether the trainee has met the defined levels of achievement for the CiPs and acquired the procedural competence required for that year of training.
The ARCP panel will then make the final decision on whether the trainee can be signed off and progress to the next year or level of training.
So, to explain how global judgement will work, firstly the trainee would upload evidence on to their ePortfolio throughout the training year. And at any stage, trainees can assess their progress against each CiP signposting the evidence which supports this.
Once they think they’ve got enough evidence to complete a CiP, they add a self- assessment rating of their performance, and they should assess themselves as:
not meeting expectations
meeting expectations OR
exceeding expectations.
Before the ARCP, the ES generates a report, which pulls together a summary of trainees progress within each CiP.
They can then review the evidence in the ePortfolio including workplace-based assessments, the TO2 and the trainee’s self-assessment and the supervisor can then record their global judgement of the trainee’s performance with commentary.
They can make a recommendation to the ARCP panel as to whether the trainee has met the defined levels of achievement for the CiPs and acquired the procedural competence required for each year of training.
The panel can then make the final decision on whether the trainee can be signed off and go on to the next year of training.

18. Workplace-based Assessments and reflection
rcog.org.uk/
Workplace-based Assessments and reflection
In line with Foundation training, all formative workplace-based assessments in the new curriculum will include a reflective element:
TO1
Mini-CEX
OSATS
CBD
NOTSS.
Ensures reflection and feedback are an integral component of the assessment process.
Every clinical encounter will provide an opportunity for reflection and feedback, which should take place as soon as possible after events to maximise the benefit for the trainee.
The redesigned formative assessment forms have been piloted nationally and a box on Trainee’s reflection has been added to each of these forms on the Training ePortfolio.
The new Curriculum has a stronger focus on the learner as a reflective practitioner. The added use of reflection showcases both the new generic professional capabilities mandated by the GMC, as well as our College’s own aspirations to develop a lifelong reflective doctor.
Reflection and feedback must be an integral component to any workplace-based assessments. Every clinical encounter can provide a unique opportunity for reflection and feedback and this process should occur frequently – and as soon as possible after any event to maximise benefit for the trainee. We want feedback to be of high quality. We want it to include an action plan for future development for the trainee. And we want both trainees and trainers to recognise and respect cultural difference when giving and receiving feedback.
In line with Foundation training, all formative workplace-based assessments in the new curriculum will include a reflective element. These assessment tools with include:
TO1
Mini-CEX
OSATS
CBD
NOTSS.
This ensures that reflection and feedback are an integral component of the assessment process.
Our redesigned formative assessment forms have been piloted nationally and a box on Trainee’s reflection has been added to each of these forms on the ePortfolio.

19. Communication and teamwork
rcog.org.uk/
Non-Technical Skills for Surgeons (NOTSS) system
Situation
awareness
Decision making
Communication and teamwork
Leadership
NOTSS
Provides feedback on non-technical skills.
It enables behaviour in theatre to be observed and rated in a structured manner.
A pilot version of the framework in use since and the tool will be rolled out more widely with the introduction of the new curriculum.
The NOTSS system provides a framework and common terminology for rating and giving feedback on non-technical skills. Used in conjunction with medical knowledge and clinical skills, NOTSS is a tool to observe and rate behaviour in theatre or on the labour ward in a structured manner. This enables clear and transparent assessment of training needs. NOTSS describes the main observable non-technical skills associated with good surgical or obstetric practice, under the following headings:
Situation awareness
Decision making
Communication and teamwork
Leadership.
The RCOG has piloted NOTSS systems for use on the labour ward and in the gynae operating theatre. We’ve removed the rating system so that we can focus on providing constructive timely feedback. The system includes only those behaviours which are directly observable or which can be inferred via communication. NOTSS covers a wide range of non-technical skills in as few categories as possible.