has online self-assessment helped medical students’ engagement with clinico-pathological conferences (CPCs)?

karen mattick – lecturer in clinical education and academic lead in pathology rob marshall - histopathologist karen mattick – lecturer in clinical education and academic lead in pathology rob marshall - histopathologist & senior clinical teacher sally holden - eLearning support manager

overview  background: PMS  CPCs: what is the problem?  feedback and results so far  discussion of next steps and ideas from you

Year 1 & 2 Years 1 to 4 Year 5 Distributed institution

EMILY…..Peninsula Medical School’s “M” “L” “E”

Pathology PMS  embedded throughout the curriculum  PBL based Years 1 & 2  University based, Community Placements, Life sciences, Special Study Units  Years 3/4/5: Hospital based with academic study days  CPCs in Years 3 and 4  online resources: e.g PathCAL, in-house developments, specific session supporting materials

Year 3 & 4: What is a CPC?  hour long expert led tutorial where pathology is integrated with clinical care i.e. real clinical cases  1 a week delivered to all students on a “Pathway”  = 27 a year at all 3 localities  at least 1 pathologist from sub-specialty and a radiologist

 original plan for students to collaborate on delivery  first cohort’s experiences: staff and students’  what was changed for next cohort  how is that going? CPC session reality check

 feedback from first year of CPCs  staff felt students not as well prepared as they could be  often devoting large part of session to covering basics  students not always clear about what they needed to do in preparation for CPC sessions  underlying science and pathology needed revising Student & staff experience?

 prior learning links and formative assessments added  students know what they should already know or revise before the session  and are given opportunities to do so online  staff now feel secure in NOT having to go back and teach at lowest common denominator  v. useful for staff to see what students have covered New design OL support for CPCs

screenshots 27 CPCs within Year 3 pathway course

prior learning resources

is all this working?  data from annual monitoring questionnaire for Years 3 & 4  USAGE (UG students action group for EMILY)  informal feedback: discussion board comments and anecdotal staff and student comments  log file analysis & statistics: web usage and BB’s tracking  Crystal reports: reporting software  focused questionnaires to students and staff

QA data Response rate 61.4% Very Diss Dissatis Neither Satisfied Very Satis Blank Total Average rating Standard D %Complete Access to computing resources E Appropriateness of electronic information sources to your course E Ease with which electronic resources can be accessed G E/Excellent = average of 3.75 or above, G/Good = 3.25/3.75

is all this working?  USAGE (UG students action group for EMILY)  have made changes to improve CPC layout and information about resources based on students’ suggestions  other methods have been delayed since all systems crashed temporarily but wiped out log files  interviews with staff and students limited due to above but interesting comments  some of which may point the way forwards?

staff views  students still not preparing much but those that do perform far better  some students use the online materials but very few the CAA self-assessments  where staff have time to set more tasks and then chase individuals or small groups up on their work everyone gets far more from session  particularly well done when locality sub dean leading the session!  some improvement this year over last but cohorts very different

staff views  if we could put formative questions in AMK format online after CPC session to check understanding (keep this in mind for later)  clinicians would like feedback on how their specialism is “performing”  if could improve CPC sessions could this then impact on AMK performance?

AMK formative assessment  demo of AMK FAdemo of AMK FA  research project running now and results will be published this year (Chris Ricketts and James Oldham)  previous studies show that use of CAA does have a positive effect on student learning  students ARE using these quite widely as AMK is hugely important to them  watch this space!

e.g. of AMK question

student views  too much to do to prepare for CPCs, low on priority list  would have found it useful if staff had reminded them of online tests availability and purpose  (we obviously got our marketing strategy wrong!)  very good experiences when do prepare using online resources and tests “ I have a skeleton in my head to hang things on” “ feel more engaged and able to ask more questions….even better when we know the clinician….some can be quite scary, particularly if I don’t know the answers”

student views  when CPCs and other activities coincide it’s easier to see the relevance since recent reading will have prepared for both  would find review tests after placements and CPCs very useful  to check understanding & re-inforce learning  would be more likely to use these  pre test results could be collated and given to clinicians to allow accurate pitching of delivery level  end of pathway (term) CPC tests would check retained knowledge

next steps?  review tests requests co-incides with staff views on wanting to check what students know in their area (remember the other slide?)  JISC study (Denise Whitelock’s session yesterday)  CAA often helps check what students don’t know  we may need to reverse our approach?  make the tests “compulsory”  convert to AMK format and align with summative assessment areas

 what do you think?

contact details…