1. Portfolios – The 5 Ws David Taylor School of Medical Education University of Liverpool

2. The five Ws Who Who What What Where Where When When hoW hoW

3. Why ? There are many ways of assessing knowledge and skills There are many ways of assessing knowledge and skills But we need some way of recording activities But we need some way of recording activities And developing and demonstrating appropriate attitudes And developing and demonstrating appropriate attitudes

4. Who Our course is five years long, and is followed by two years in foundation posts before the Doctor is fully registered and allowed to practice independently. Our course is five years long, and is followed by two years in foundation posts before the Doctor is fully registered and allowed to practice independently.

5. Who (2) We currently use portfolios in years 1,3 & 5 of the undergraduate course F1, F2 We currently use portfolios in years 1,3 & 5 of the undergraduate course F1, F2 And for consultants, academics and anyone seeking promotion And for consultants, academics and anyone seeking promotion 12345 F1F1F1F1 F2F2F2F2…

6. What is a portfolio It is a collection of documents It is a collection of documents Supported by a commentary Supported by a commentary It can be electronic It can be electronic more convenient in the University context more convenient in the University context It can be on paper It can be on paper More convenient in the clinical context More convenient in the clinical context

7. What documents? It depends on the reason for the portfolio! It depends on the reason for the portfolio! Diaries Diaries Log-books Log-books Case notes Case notes Examination results Examination results Longer discursive pieces Longer discursive pieces

8. Dewey (1933) Reflection is… Reflection is… A purposeful form of thought provoked in learners when they recognize that their understanding is incomplete. A purposeful form of thought provoked in learners when they recognize that their understanding is incomplete.

9. In order to reflect 3 things are needed 3 things are needed something to reflect on something to reflect on reflection time & modelling reflection time & modelling motivation motivation Albanese,2006,Medical Education 40;288- 290 Albanese,2006,Medical Education 40;288- 290

10. What sort of commentary? Technical Technical What and how of practice What and how of practice Facts and theories Facts and theories Audit Audit Practical Practical How decisions are made How decisions are made Articulate concerns Articulate concerns Resolve professional dilemmas Resolve professional dilemmas Critical Critical Social and political context and constraints Social and political context and constraints

11. What is included The portfolios all include examination results and the student commentary on them. The portfolios all include examination results and the student commentary on them. In the first year they also include reflection on settling in and adjusting to University life. In the first year they also include reflection on settling in and adjusting to University life. In the third year they include reflections on professional attitudes students have experienced or observed. In the third year they include reflections on professional attitudes students have experienced or observed.

16. What is included in later years In fifth and subsequent years they also include reflection on particular cases/treatment regimes In fifth and subsequent years they also include reflection on particular cases/treatment regimes In F1/F2 and beyond they will include reflection upon critical incidents In F1/F2 and beyond they will include reflection upon critical incidents

17. Final year PETA

18. Final year PETA Bottom half

19. When Right from the start and often Right from the start and often Habit helps Habit helps Novelty is important Novelty is important So we focus on different things each year So we focus on different things each year Settling in, clinical experiences, ethical behaviour, evidence base to medical practice, audit, future plans, critical incidents Settling in, clinical experiences, ethical behaviour, evidence base to medical practice, audit, future plans, critical incidents It has to be relevant It has to be relevant In the clinical arena it will focus on cases determined by the agreed learning outcomes In the clinical arena it will focus on cases determined by the agreed learning outcomes

20. Where Wherever the student works Wherever the student works In the first/second year this is often at the computer, so we use electronic portfolios In the first/second year this is often at the computer, so we use electronic portfolios In the UK the hospital computers won’t “talk” to the internet, so where a VPN can’t be set up we use…. In the UK the hospital computers won’t “talk” to the internet, so where a VPN can’t be set up we use…. Paper, and interviews, formal and informal, with the educational supervisors Paper, and interviews, formal and informal, with the educational supervisors

21. How How is it assessed? How is it assessed? Through discussion with a supervisor Through discussion with a supervisor The whole thing? The whole thing? The reflective component? The reflective component? Parts chosen by the student? Parts chosen by the student? Parts chosen by the supervisor? Parts chosen by the supervisor?

22. Marks? Best practice currently is to use criteria rather than a “checklist” Best practice currently is to use criteria rather than a “checklist” The criteria need aggregating in domains The criteria need aggregating in domains KSA KSA Burch, Seggie and Gary 2006 SAMJ 96;430- 433 Burch, Seggie and Gary 2006 SAMJ 96;430- 433 The number of raters needed for validity depends on the criteria and the domains The number of raters needed for validity depends on the criteria and the domains Roberts et al.,2006 Med Educ 40;363-370 Roberts et al.,2006 Med Educ 40;363-370

23. How (2) It is the main form of assessment for our final year students. It is the main form of assessment for our final year students. They sit “finals” at the end of their fourth year. They sit “finals” at the end of their fourth year. Their final year is a clinical apprenticeship, assessed through the portfolios. Their final year is a clinical apprenticeship, assessed through the portfolios. 5 different assessors (3 meetings each - plus referees in the case of difficulty/expected difficulty) 5 different assessors (3 meetings each - plus referees in the case of difficulty/expected difficulty) Mid-Year portfolio review (all borderline/fails and sample of others) Mid-Year portfolio review (all borderline/fails and sample of others) End of year review of borderline/fails End of year review of borderline/fails

24. Benefits Allows the student to demonstrate insight and understanding Allows the student to demonstrate insight and understanding Allows student and staff to observe development and progress Allows student and staff to observe development and progress Models the “real world” Models the “real world” Goes one step beyond the “competencies” model – in developing professionalism Goes one step beyond the “competencies” model – in developing professionalism Provides a focus (and evidence) for the discussion between student and doctor Provides a focus (and evidence) for the discussion between student and doctor

25. F1

26. Staff

27. David Taylor’s response..