1. A pilot e-Portfolio for postgraduate medical trainees Alex Haig NHS Education for Scotland E-Portfolio 2005 28 th October Cambridge

2. NHS Education for Scotland Special health board “exists to advance the knowledge and develop the skills of all NHS staff” Doctors, Dentists, Allied Health Professions, Midwives, Nurses, Pharmacists, Clinical Psychologists, Audiologists, Biomedical Scientists, etc.

3. Educational Background Paper portfolios pre-existed Postgraduate medical training is experiencing fundamental change Shift to generic competency based curriculum for all newly graduated (Foundation) doctors (MMC) Assessment concentrating on the evaluation of clinical skills within the workplace

4. Pilot Background NHS Time Resources Geography Content

5. “The greater the difficulty the more glory in surmounting it. Skilful pilots gain their reputation from storms and tempests.” Epicurus 341 BC - 270 BC

6. Pilot’s Aim “support the educational development of trainee doctors and facilitate a culture of life- long learning”

7. Pilot’s Objectives  To evaluate the feasibility of using an enhanced electronic environment for trainee doctors’ portfolios  To ensure that the e-portfolio has potential for transferability to other healthcare professions  To support NES corporate objectives, such as promoting life-long learning, providing educational infrastructure, e-learning and improve patient care

8. Partners NHS Lanarkshire Medical Education University of Edinburgh Bromley Hospitals NES – regional offices, Training and Development Unit and Central IT

9. Design Accessibility –Dynamic data-driven web interface –Any web-enabled workstation Interface –Simple –Intuitive Adaptability –Living documentation system –All users can recommend adaptations and changes –Potential e-learning environment

10. Technology Sequel Server database Multiple-user roles ASP pages Some XML Planned for phased adoption of interoperability standards

11. Content Assessment – all tools and documentation included Review – ed sups can review trainee details, submissions and progress at any time Reflective Practice – aids easy and convenient storage Lifelong Record – can aid future revalidation Analysis – GMC and Deaneries can access for audit and educational evaluation

12. Efficiency Save time for supervisors allowing continuous updating and communication Features include automatically generated reminders to complete assessments, attend tutorials, meet supervisors Tutors instantly see completed work and which supervisors have met trainees and signed off learning plans Time for shift in medium

13. DOTS NHS Lanarkshire Doctors Online Training System Initial aim: deliver induction material online Both DOTS and e-Portfolio appeared in August 2005

14. DOTS and the E-Portfolio A common gateway –Users accessing either can be directed sideways –Record of training via DOTS can be held on portfolio Shared database –Reduced duplication of effort Unique identification allows user tracking –Automated notification of supervisor where necessary A DOTS hosted messaging system –Trainer contact with trainee –Prompting of trainee and trainer

17. Pilot Groups 1.Foundation One 2.GPST (GP vocational training)

18. Pilot Populations Foundation 1 East1516% North135100% South East176100% West4411% 44 GPSTs from South East N=414

19. Foundation One Bridge between medical school and GP or specialist training Several placements in different specialities and healthcare settings Standards set by GMC August 2005 start Doctors must demonstrate their abilities/competence against set standards

20. “The eleventh commandment – Thou shalt not be found out – is the only one that is virtually impossible to keep these days.” Berta Buxton

21. Foundation One Trainee Tutor (Trainee has one) Educational Supervisor (Trainee has 3x2=6 until GMC registration) Administration

22. Assessment Presented evidence Multi-source feedback Supervisor’s report Workplace assessment 1 x Significant event analysis Completed educational log 3 x Personal development plan Post 1 (self + 4 raters) Post 3 (self + 4 raters) 12 competencies + global rating 1 completed for each post Collated by Foundation Tutor Continuous List of F1 competencies Initiated by trainee. Signed by most appropriate assessor

23. Multi-source feedback Trainee requests 4 colleagues to complete Trainee gives name / GMC number and web address Colleague logs in as ‘guest’ and enters trainee details Random checks by administrator

24. “And since you know you cannot see yourself, so well as by reflection, I, your glass, will modestly discover to yourself, that of yourself which you yet know not of.” Cassius, Julius Caesar, I ii

25. What’s different about e- Portfolio for GPST? Multiple workplace environments –Communication tool linking GP trainer, educational supervisor and trainee (messaging, RITA, appraisal) Monitoring progress –Trainee led –Reflective practice –Information –Leading to finishing Summative assessment in Year 3 Formative assessment –Use as a tool for monitoring progression over a period of time e.g. confidence checklists, appraisal

46. Just a time-saver? Automatic collation of multi-source feedback forms Completion of supervisor’s report Foundation Tutor can access portfolio at any time to check progress Trainee chooses which log entries are public. Increases accuracy – forced choices; no doctors’ signatures… Potential for automatic collation of post assessment questionnaire feedback

47. Evaluation Assess the utility from the users’ perspective (not an evaluation of intrinsic educational value of e-portfolios) I.Usage statistics II.On-line survey III.Focus groups/semi-structured interviews Ongoing feedback from all levels/users Initial evaluation report by end November 2005

49. “Inventions reached their limit long ago, and I see no hope for further development.” Julius Frontinus 1st century A.D.

50. The Future Possible extension further into medicine Other professions –Dentistry –Clinical Psychology –Allied Health Bridge under/post graduate education Innovation and development