1. Formative Assessment Case Studies (FACS) Shekhar Kumta, Jack Cheng, James Ware Chinese University of Hong Kong (Refer to CD for FACS engine and sample workshop case study) Hong Kong International Consortium

2. What Are FACS?  We have designed a user friendly engine for generating and delivering web-based, interactive, clinical scenarios  These case studies allow accommodation for multimedia features & require formative feedback to facilitate student learning.  RCTs have established the effectiveness of FACS  and students prefer using this learning and assessment model.  FACS engine provides a value added option for administering items from IDEAL’s non restricted (formative) bank.

3. Clinicians feel FACS provide a convenient means to generate & present a meaningful clinical context for one’s formative items. FACS enables not only incorporation of, but enhanced use for formative items that are  Vignette-based A type  Vignette-based R- type  Key features based FACS Strength: enhanced authenticity to the assessment

4. Evidence: one can assess student learning by testing only some elements that are essential to the resolution of that problem (i.e. the clinical problem’s key features). Using key features is also an effective way of facilitating student learning (not just the assessment of their learning).  key features based assessments correlate better with problems solving ability and clinical reasoning than patient management problems Option of using Key Features

5. How to Select Case Studies for FACS Identify areas where students frequently encounter difficulties understanding clinical concepts.  the clinical scenario needs to set a context so that application of knowledge becomes the focus for the virtual problem.  ensure steps within the problem enable strategic formative feedback in order to develop & facilitate understanding.

6. Workshop Objectives for FACS  Short Term  Create a FACS  Write or incorporate existing formative items  Incorporate formative feedback  Intermediate  Install the FACS templates provided on your own or your institution’s computers / servers  Long Term  Develop a bank of educationally relevant and validated FACS for sharing among consortium members

7. + Apply to several cases 1.Graduating Level 2.What do interns do? 3.Objectives = assessment 4.Practical issues 5.Consider theoretical issues Generic Problem Objectives + The Problem and Objectives

8. Generic problem ScenarioCases Clinical detailAction List 1Feedback 1 Explanation 1Action List 2Feedback 2 etc The Problem and Objectives

9. + Chest pain 1.Clinical features 2.Prioritise management 3.Interpret investigations 4.Include discharge etc.. 5.Anatomy of thorax Generic Problem Objectives + Suggest some notes The Problem and Objectives

10. Generic problemScenario Writing the Scenario

11. INCLUDE: 1.Age and gender 2.Site of care 3.Summary of presentation a)Keep generic at this stage b)Onset, duration, severity c)~2 – 6 lines 4.Other (e.g., family history) The Scenario: shall always include common info realism important summary form Writing the Scenario

12. The Scenario: Important, the same stem shall apply to several cases, e.g., pneumothorax and MI etc. 38 year old Chinese male with a twenty year history of heavy smoking is admitted to the A&E Department with sudden onset of chest pain one hour before and is apparently having difficulty in breathing. Writing the Scenario

13. Choices of Cases Generic problemScenarioCases

14. + Case 1Create several related cases: common scenario common cases common objectives + Case 2 Case 3 + Case 4... Choices of Cases

15. + Pneumothorax, tensionThe Cases + Pneumothorax, simple Myocardial infarction + Acute oesophagitis Choices of Cases

16. ScenarioCases Clinical detail Writing Clinical Detail

17. Case 1: Pneumothorax Consider site Usually vital signs Important info: a)Specific clinical findings b)+ special tests Clinical detail Clinical images + Writing Clinical Detail

18. Clinical detail With Added value HR:120bpm BP:80/60 mm Hg RR:30 / min T:37.5 o C Hyper-resonant Rt., absent BS and reduced respiratory movement Rt., Writing Clinical Detail

19. Writing an Action List ScenarioCasesClinical detail Action List 1

20. + Lead-in 1.Brief 2.Likely choices 3.Pick one best response 4.Ensure four or more Include Up to “n” possible actions Writing an Action List

21. + What is your next action? Lead-in Incorrect Correct choice Incorrect + 1.Order chest radiograph 2. Insert a chest drain + 3. Order a chest CT Writing an Action List

22. ScenarioCasesClinical detail Action List 1 Feedback 1 for incorrect choices Writing Feedback

23. + For all incorrect choices 1.Why inappropriate or 2.Trigger further thought 3.Linkages to resources 4.Another image? Feedback Included Writing Feedback

24. + 1. Life threatening situation... For each feedback that is incorrect give a summarised form of info. Will offer re-entry assessment after incorrect choices + 2. Quite incorrect... 3. No place for definitive... + 4. Tests unnecessary... Writing Feedback

25. ScenarioCasesClinical detail Action List 1 Feedback 1 for incorrect choices Explanation 1 for correct choice Explanation for Correct Choice REPEATED SEQUENCE

26. + Summarised 1.Video of procedure 2.Text for action or 3.Result (s) of investigation Explanation Added value Explanation for Correct Choice

27. ScenarioCasesClinical detail Action List 1 Feedback 1 for incorrect choices Explanation 1 for correct choice Action List 2 Writing an Action List

28. + Lead-in 1.Brief 2.Likely choices 3.Pick one best response 4.Ensure four or more Action List Up to “n” possible actions Writing an Action List

29. + What is your next action? Lead-in Partially incorrect Correct choice Incorrect + 1.Order chest radiograph 2. Admit for observation + 3. Discharge after... Writing an Action List

30. ScenarioCasesClinical detail Action List 1 Feedback 1 for incorrect choices Writing Feedback Explanation 1 for correct choice Action List 2 Feedback 2 for incorrect choices

31. + For all incorrect choices 1.Why inappropriate or 2.Trigger further thought 3.Linkages to resources 4.Another image? Feedback Included Writing Feedback

32. + 1. Not an immediate priority.. Don’t forget the further assessment to re-enter at the last action point + 2. Pt. needs monitoring... 3. Too early for discharge... + 4. Review later for need... Writing Feedback

33. ScenarioCasesClinical detail Action List 1 Feedback 1 for incorrect choices Explanation 1 for correct choice Action List 2 Explanation 2 for correct choice (s) Feedback 2 for incorrect choices Explanation for Correct Choice

34. + Summarised 1.Video of procedure 2.Text for action or 3.Result (s) of investigation Conclusion Added value Explanation for Correct Choice

35. + Use relevant clinical info 1.MCQ format 2.+ vignette 3.R type 4.Key features etc... Stem Re-entry Assessment Options +

36. ScenarioCasesClinical detail Explanation 1 for correct choice Action List 2 Explanation for Correct Choice correct choice # failed tries decides if exit Feedback 2 for incorrect choices incorrect choices Action List 1 Assessment

37. CREATING A FACS DEMONSTRATION