Workplace-Based Assessment Making it More Effective Jonathan Beard Professor of Surgical Education University of Sheffield, UK.

Slides:



Advertisements
Similar presentations
Educational Supervision & Find Your Way Around in the E-portfolio Dr Jane Mamelok RCGP WPBA Clinical Lead.
Advertisements

Workplace assessment Dr. Kieran Walsh, Editor, BMJ Learning. 2.
Work-Based Learning (WBL) Coordination The Amended Model WORKLIFE COOPERATION PLAN Workplace analysis Workplace database Framework agreement Workplace.
Joint Royal Colleges of Physicians Training Board (JRCPTB) Changes to specialty trainee assessment and review August 2014 Guidance for Supervisors.
School of Surgery Induction Day ISCP Session. Overview ISCP aims and benefits Roles and responsibilities ISCP website Learning Agreements Syllabus Assessment.
Performing an educational supervisor report. Step by Step guide By Dr Kim Emerson January 2013.
Training the Assessor 19 October 2007 Putting pathology into the context of the new framework Joanne Brinklow Training and Educational Standards Manager.
Introduction to the eportfolio and the nMRCGP HEKSS, KSS Deanery GP Specialty School 2013 Dr Susan Bodgener Associate Dean for Assessment.
General Practice Introduction to the eportfolio and the MRCGP KSS Deanery 2014 Dr Susan Bodgener Associate Dean for Assessment KSS Deanery.
Faculty Development WORKPLACE- BASED ASSESSMENT.
Workplace-based Assessment. Overview Types of assessment Assessment for learning Assessment of learning Purpose of WBA Benefits of WBA Miller’s Pyramid.
The 2009 Learning Portfolio and Assessment Framework for DFT.
Ian Whitehead TPD ARCP.  Structured postgraduate medical training is dependent on :  having curricula which clearly set out the standards and competences.
Introduction to Workplace- based Assessments Dr Hani Zakhour Director of Training and Educational Standards The Royal College of Pathologists 19 October.
Workplace-based Assessment (WBA) – Beyond the numbers
Promoting Excellence in Family Medicine nMRCGP Workplace-based Assessment March 2007.
Clinical Examination and Procedural Skills (CEPS) The Introduction of Integrated DOPS The assessment of psychomotor skills in WPBA for the MRCGP examination.
. Workplace-Based Assessment: a true reflection of competency ? Trevor Gibbs.
Assessment of Clinical Competence in Health Professionals Education
Workplace-based Assessment
Workplace-Based Assessment Case-Based Discussion (CBD) These slides have been prepared to facilitate discussion on the use of the CBD. The suggested practical.
Workplace-Based Assessment Clinical Evaluation Exercise (CEX) These slides have been prepared to facilitate discussion on the use of the CEX. The.
Aims, principles and benefits
ASSESSMENT& EVALUATION Assessment is an integral part of teaching. Observation is your key assessment tool in the primary and junior grades.
6 Month ES Reviews Yer What???
Training for Public Health Trainers.  Prof. John Collins’ report ‘Foundation for Excellence’ highlighted many positive aspects of the Curriculum but.
MMC – workplace based assessments Dr Lisa Joels Postgraduate Organiser Singleton Hospital 6 th August 08.
Bob Woodwards SAC Chair, Oral and Maxillofacial Surgery.
The Royal College of Pathologists – the overall assessment system Dr. Trevor Gray Director of Examinations and Assessment.
Belling the Cat – Exams!!! Fedir Abi. Fees Yet to be confirmed. Its not free, you will have to pay.
Understanding Meaning and Importance of Competency Based Assessment
Modernising Medical Careers for GPs Education Supervision and Review of Progression.
GP Workplace Based Assessment
WBAs: the Northern Ireland School of Surgery Experience Richard Mayes 1 Robert Gilliland 1, 2 Jeffrey Campbell 1, 2 Helen Holscher 3 Department of Surgery,
Guide to the ESR By Carol and Barry. Why is the ESR important? An Educational Supervisors Review (ESR) is conducted every six calendar months for all.
Dr. Sarbjit Saini TPD – Sandwell VTS. What we will be covering When to complete ESR Creating a review period What is contained in an ESR Collecting.
General Practice Introduction to the eportfolio and the MRCGP HEKSS 2015 Dr Susan Bodgener Associate Dean for Assessment, HEKSS.
Postgraduate Diploma in General Pharmacy Practice Induction & Refresher session for Educational and Practice Supervisors Online presentation 5: Summative.
Planning your three years Dr Morooj Mohammad GP/ Commissioning Fellow.
Assessment tools MiniCEX, DOPS AH Mehrparvar,MD Occupational Medicine department Yazd University of Medical Sciences.
Workplace based assessment for the nMRCGP. nMRCGP Integrated assessment package comprising:  Applied knowledge test (AKT)  Clinical skills assessment.
GIM Requirements and PYA process
December 13 th, 2007 Winnie Wade Director of Education RCP Education Department How assessments contribute to successful training in Geriatric Medicine.
Planning the next 3 years Dr Nisha Ehamparanathan GP / Clinical Skills Lecturer.
Specialist Associate CESR Evaluation Day
FP Curriculum 2012: summary of key changes that will impact the FP e-portfolio Intended audience: The educational faculty (e.g. supervisors, FTPD/Ts, FSDs,
Update on Supervised Learning Events In Foundation Sonia Panchal Clinical Fellow HEE / Academy of Medical Royal Colleges David Kessel Chair AoMRC Foundation.
TRAINING UPDATE- CSAC Brindha Dhandapani. Aim to cover  Guidance on SLEs for CCH level 3 training  General paediatric competencies for CCH level 3 
Introduction to the eportfolio and the MRCGP HEEKSS 2015 Dr Susan Bodgener Associate Dean of Assessment, HEEKSS.
GMC Approval of trainers in the UK Enid Rowland and Patricia Le Rolland.
How to make the most of wpba. (workshop) Andrew Whitehouse Jan
Making assessment meaningful and useful…….. Andrew Whitehouse January 25th 2011.
Introduction to the eportfolio and the MRCGP On behalf of HEE KSS GP School.
National Dental Core training curriculum and assessment framework Dental Core Trainee training slides September 2016.
Miss Alexis Sudlow, ST3 General Surgery
National Dental Core training curriculum and assessment framework Dental Core Trainee training slides September 2016.
ePortfolio and curriculum
The Big Picture – curricula, the Gold Guide and the assessment system
(Clinical Examination and Procedural Skills)
FACULTY OF PHARMACEUTICAL MEDICINE
The Future of Workplace Based Assessment
Clinical Assessment Dr. H
Trainee induction guidance
Work Place Based Assessment
By Claire, Sally and Barry
nMRCGP Assessments on the e portfolio
Changes in Training and Assessment
ASSESSMENTs DURING TRAINING.
ePortfolio developments for IMT
Presentation transcript:

Workplace-Based Assessment Making it More Effective Jonathan Beard Professor of Surgical Education University of Sheffield, UK

Objectives Check knowledge of Workplace Based Assessment (WBA) Understand the purpose of assessment Review some of the problems of WBA Discuss possible solutions Any other objectives?

Workplace Based Assessment Workplace Based Assessment “Assessment of day-to-day working practices undertaken in the working environment” WPBA now an integral part of medical training (competence-based curricula) Integrates teaching, learning, assessment & feedback Designed to assess skills rather than knowledge Competence best assessed in the workplace as closest to actual performance Primary aim is to aid learning

WBA Tools Mini-CEX DOPS Specialty tools –e.g. PBA (ISCP), ACAT (RCP) CBD Assessment of Teaching or Audit MSF: –Only tool that does not require rater training Opportunistic Planned Once per placement

What is the Point of Scoring WBA? It depends on the purpose of the assessment

Purpose of Assessment? To aid learning through constructive feedback: –Assessment for Learning (formative) –Done frequently To check knowledge or skill has been learned: –Assessment of Learning (summative) –Done infrequently

Time for Teaching and Learning How many hours of teaching and learning does a trainee get per week?

Learning to Drive Driving Lesson Assessment for Learning Driving Test Assessment of Learning

The Danger of Confusing the Purpose of WBA Doctors regard all assessments as tests Therefore they practice informally before requesting an assessment, choose easy tasks and make the request after the event Assessors feel pressured to give a good score More attention paid to score than feedback Primary purpose of WBA as an aid to learning is lost

Changing the Name Supervised Learning Events (SLEs) done frequently to aid progress –trainee chooses the topics and timing –No score but feedback must be documented Assessments of Performance (AoPs) done less frequently to check that key competencies have been achieved –Identified by the specialty curriculum –‘Satisfactory’ or ‘development required’

Placement Two Placement Three Level of Competence Training Rotation Frequency and Timing of WBA Placement One Supervised Learning Event Assessment of Performance Able to perform the task

Changing the Language Similar format for many existing assessments Norm-based rating scale: 1-6 where 4 = ‘meets expectations for completion of stage of training’ Standard not defined but can usually be found ‘somewhere’ in the syllabus

What does ‘Meets Expectations for Completion of a Stage of Training’ Mean? To the Trainee? To the Assessor? To the ES/TPD? To the Employer? To the Patient?

Evaluation of WBA in Surgery DOPS –1370 trainees (ST1-4), 6240 DOPS (m=3.4) –Most domains rated 4 or 5 (m=4.8) –No construct validity and no reliability Similar findings by RCP for CEX and CBD PBA (Procedure-Based Assessment) –Criterion-referenced rating scale –85 trainees (ST1-7), 749 PBAs –Good internal and construct validity –High reliability (G>0.8 for 3 assessments)

Redesign of WBA Tools Criterion-referenced with clear performance anchors (e.g. could perform the task without help) Use descriptors not scores (e.g. able to do the task) Emphasis on feedback (e.g. make documentation of feedback mandatory) Clearer guidance notes for trainees (e.g. timing and frequency) and assessors (e.g. seniority and training)

Revised Surgical DOPS (ISCP) Verbal feedback and documentation of feedback is mandatory

Revised Ratings for CBD and Mini-CEX (RCP)

Assessment Methods Can the same assessment method be used for both purposes? Yes - but only if its use on each occasion is agreed and documented And - feedback is documented if it is an SLE (remove all ratings?) And - reliability is high enough for an AoP (well trained assessors)

Example of Implementation for Surgical Specialties 40 SLEs per year (1 per week) 3 AoPs (PBAs rated at level 4) required for every index procedure before CCT Logbook will ask whether an SLE or AoP was done SLE feedback must documented by trainee and validated by assessor AoPs can only be done by CS or ES

Summary Postgraduate medical curricula will need to be amended in light of the GMC discussion document No sudden change required – direction of travel – but need to be considered when reviewing curricula Trainers and trainees need an update Don’t mention the ‘A’ word Ask the question: driving lesson or test?

Further Reading and Information Further reading: –Impact of workplace based assessment on doctors’ education and performance: a systematic review. Miller & Archer, BMJ 2010 –Changing the Name and the Language. Crossley and Jolly. Medical Education 2012 More information: – – – uk.org/Learning_and_assessment_in_the_clinical_environment.pdf_ pdf uk.org/Learning_and_assessment_in_the_clinical_environment.pdf_ pdfhttp:// uk.org/Learning_and_assessment_in_the_clinical_environment.pdf_ pdf

Some Questions Do we need separately designed forms for SLEs and AoPs? How many hours of teaching and learning should a trainee have per week? Should MSF be designated as an AoP? Should AoPs only be done if there is no better way of testing a skill/behaviour? How do we communicate the new terminology and philosophy to trainers and trainees?