Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr David James, Dr Caroline Fertleman London School of Paediatrics Trainees Committee

Outline Implementation of two new work-based assessments nationally Why How Results of a survey assessing impact of new WBA’s Further developments

Introduction The usefulness of an assessment is a product of its reliability, validity, cost-effectiveness, acceptability and educational impact Traditional approaches to maximise the reliability or reproducibility of assessments can have a negative educational impact on the learner by reducing the opportunity for meaningful developmental feedback.

CbD

SAIL

Methods Trainee led group designed new formative work based placed assessments Focus on personalised feedback Launched by RCPCH in September 2013 London deanery trainees and college tutors surveyed June 2014

New Safeguarding CbD

New DOC

Survey Design Assessment subgroup of the trainees committee designed survey on experience of new assessments Had the new assessments improved educational experience for both trainees and tutors? Sent to London School of Paediatric trainees and to college tutors June after ARCP Responses from 161 trainees, evenly distributed throughout training years 35 tutor responses

Survey Results - CbD Majority of trainees completed one Safeguarding CbDs minutes to complete 69% were completed with the trainer present

Survey Results - CbD Compared with your previous experience of the old CbD, do you feel that the safeguarding CbD better allows the opportunity to: – Assess your safeguarding knowledge, application and clinical skills? – stimulate your learning more? – create clear and achievable learning objectives?

Survey Results – CbD p = <0.05 Trainees

Survey results - CbD Tutors p = <0.05

Survey Results - CbD What are the barriers to completing a Safeguarding CbD?

Survey Results - CbD Describe what you like and dislike about the safeguarding CbD. LikeDislike Like specific safeguarding CbDAssessments are still a tick box exercise Better structureOnly as good as the assessor Opportunity to learn and get feedback The need for reflection Encourages trainers to make comment Finding time with a consultant Only suitable for certain types of placement

Survey Results - DOC Majority had performed 5 DOCs, taking 5-10 minutes each Only 49% performed in person with the trainer

Survey Results - DOC Compared with your previous experience of SAIL, did you feel that performing the DOC better allowed you the opportunity to assess: – Written correspondence? – Stimulate learning? – Create clear and achievable learning objectives?

Survey Results - DOC Trainees

Survey Results – DOC Tutors p = <0.05

Survey results - DOC What are the barriers to completing a DOC?

Survey Results - DOC Describe what you like and dislike about the DOC? LikeDislike Better StructureDifficulty in finding a consultant Better opportunity to learn and get feedback Less relevant for more senior trainees Online better than paper for eportfolio purposes Still not broad enough Straightforward

Conclusions New Safeguarding CbD format favoured by trainers and trainees New DOC favoured by trainers but not trainees Trainees felt their learning was not stimulated more by the new assessments It is a step in the right direction

The future Need to change the culture of work based placed assessments in paediatrics – Possibly having more assessments An app to allow them to truly be supervised learning events done at the time of assessment Tailoring of assessments to GRID trainees and more senior trainees

Any questions?