1. Do the new formative work based placed assessments improve educational impact? Dr Hermione Race, Dr David James, Dr Caroline Fertleman London School of Paediatrics Trainees Committee

2. Outline Implementation of two new work-based assessments nationally Why How Results of a survey assessing impact of new WBA’s Further developments

3. Introduction The usefulness of an assessment is a product of its reliability, validity, cost-effectiveness, acceptability and educational impact Traditional approaches to maximise the reliability or reproducibility of assessments can have a negative educational impact on the learner by reducing the opportunity for meaningful developmental feedback.

4. CbD

5. SAIL

6. Methods Trainee led group designed new formative work based placed assessments Focus on personalised feedback Launched by RCPCH in September 2013 London deanery trainees and college tutors surveyed June 2014

7. New Safeguarding CbD

8. New DOC

9. Survey Design Assessment subgroup of the trainees committee designed survey on experience of new assessments Had the new assessments improved educational experience for both trainees and tutors? Sent to London School of Paediatric trainees and to college tutors June after ARCP Responses from 161 trainees, evenly distributed throughout training years 35 tutor responses

10. Survey Results - CbD Majority of trainees completed one Safeguarding CbDs 15- 20 minutes to complete 69% were completed with the trainer present

11. Survey Results - CbD Compared with your previous experience of the old CbD, do you feel that the safeguarding CbD better allows the opportunity to: – Assess your safeguarding knowledge, application and clinical skills? – stimulate your learning more? – create clear and achievable learning objectives?

12. Survey Results – CbD p = <0.05 Trainees

13. Survey results - CbD Tutors p = <0.05

14. Survey Results - CbD What are the barriers to completing a Safeguarding CbD?

15. Survey Results - CbD Describe what you like and dislike about the safeguarding CbD. LikeDislike Like specific safeguarding CbDAssessments are still a tick box exercise Better structureOnly as good as the assessor Opportunity to learn and get feedback The need for reflection Encourages trainers to make comment Finding time with a consultant Only suitable for certain types of placement

16. Survey Results - DOC Majority had performed 5 DOCs, taking 5-10 minutes each Only 49% performed in person with the trainer

17. Survey Results - DOC Compared with your previous experience of SAIL, did you feel that performing the DOC better allowed you the opportunity to assess: – Written correspondence? – Stimulate learning? – Create clear and achievable learning objectives?

18. Survey Results - DOC Trainees

19. Survey Results – DOC Tutors p = <0.05

20. Survey results - DOC What are the barriers to completing a DOC?

21. Survey Results - DOC Describe what you like and dislike about the DOC? LikeDislike Better StructureDifficulty in finding a consultant Better opportunity to learn and get feedback Less relevant for more senior trainees Online better than paper for eportfolio purposes Still not broad enough Straightforward

22. Conclusions New Safeguarding CbD format favoured by trainers and trainees New DOC favoured by trainers but not trainees Trainees felt their learning was not stimulated more by the new assessments It is a step in the right direction

23. The future Need to change the culture of work based placed assessments in paediatrics – Possibly having more assessments An app to allow them to truly be supervised learning events done at the time of assessment Tailoring of assessments to GRID trainees and more senior trainees

24. Any questions?