INNOVATIVE ASSESSMENT TOOLS PBL AND ASSESSMENT Thorsten SCHÄFER, Andreas BURGER Prof. Dr. med. Thorsten Schäfer ZENTRUM FÜR MEDIZINISCHE LEHRE RUHR-UNIVERSITÄT BOCHUM

INNOVATIVE ASSESSMENT TOOLS IN MEDICAL RUHR-UNIVERSITY BOCHUM ZENTRUM FÜR MEDIZINISCHE LEHRE | 2

WHY INNOVATIONS?  DANGER!  EFFECTIVENESS! ZENTRUM FÜR MEDIZINISCHE LEHRE | 3

MEDICAL EXAMINATION #1  Legalize the civil use of potentially lethal weapons  Knifes [Scalpels]  X-rays  Poisons [Drugs]…  in order to help people  (or at least do not harm) ZENTRUM FÜR MEDIZINISCHE LEHRE | 4

MEDICAL EXAMINATION #2  ASSESSMENT DRIVES LEARNING  Conclusion: Tasks close to reality / practice ZENTRUM FÜR MEDIZINISCHE LEHRE | 5

PROVEN (ANCIENT) TOOLS: ZENTRUM FÜR MEDIZINISCHE LEHRE | 6 ContinuingIntermediateFinal Multiple- Choice XXX (>600 items) Oral (unstructured) & Bed-side XXX

INNOVATIVE TOOLS ZENTRUM FÜR MEDIZINISCHE LEHRE | 7 MEQ TJE SimPat miniCExOSLER OSCE PTM Portfolio 360 Degree CBT cbMCQ LogBook

CASE-BASED TOOLS  Modified Essay Question Tests (MEQ)  An authentic patient‘s history is developed over time (with 10 – 15 questions)  Interdisciplinary questions (basics, diagnostics, therapy, medication, prognosis, etc.)  No way back (new chance after mistake)  Case-based Multiple-Choice Questions  Higher complexity ZENTRUM FÜR MEDIZINISCHE LEHRE | 8

SIMULATED PATIENTS  Trained actors/actresses  Increased objectivity and reliability  Increased comparability  Add a score from the patient‘s perspective  Communication skills, professional behaviour  Used in Objective Structured Clinical Examinations (OSCE) ZENTRUM FÜR MEDIZINISCHE LEHRE | 9

SIMULATED PATIENTS  Trained actors/actresses  Increased objectivity and reliability  Increased comparability  Add a score from the patient‘s perspective  Communication skills, professional behaviour  Used in Objective Structured Clinical Examinations (OSCE) ZENTRUM FÜR MEDIZINISCHE LEHRE | 10

(Fun) OSCE ZENTRUM FÜR MEDIZINISCHE LEHRE | 11 Teacher With Checklist Simulated Patient with Knee Student in Exam

OSCE (Objective Structured Clinical Examination)  Short practical tasks (5 min)  7 – 14 stations  Checklist for scoring  Using simulated patients  Replace unstructured oral exams ZENTRUM FÜR MEDIZINISCHE LEHRE | 12

Andreas Burger Thorsten Schäfer ZENTRUM FÜR MEDIZINISCHE LEHRE RUHR-UNIVERSITÄT BOCHUM

CLOSE TO REALITY  Mini Clinical Examinations (miniCEx)  Small tasks on the ward at different times  Scores collected in a LogBook  360 Degree Examinations  Multiple Scores (on the job) by  The medical teachers  The nurses  The peers  The patients ZENTRUM FÜR MEDIZINISCHE LEHRE | 14

LOW THRESHOLD FEEDBACK  The Progress-Test Medicine (PTM)  200 item Multiple-Choice  5 answers plus „don‘t know“  Graduation level  Applied each semester (12 times per student)  Indicates individual progress  Allows comparisons (with peers, … other faculties) ZENTRUM FÜR MEDIZINISCHE LEHRE | 15

ZENTRUM FÜR MEDIZINISCHE LEHRE | 16 Quelle: AG Progress-Test Charité, 2010

COMPUTER-BASED TESTING  Formative (voluntary, continuous)  Using a learning plattform (Blackboard)  Summative (mandatory)  Work in progress  Security issues ZENTRUM FÜR MEDIZINISCHE LEHRE | 17

TJE and OSLER  Triple Jump Exercise (TJE)  Read case/problem  1. Activate prior knowledge, hypothesize and identify gaps in knowledge  2. define learning goals / objectives, search  3. explain and discuss  Objective structured long examination record (OSLER) using real patient ZENTRUM FÜR MEDIZINISCHE LEHRE | 18

OBJECTIVITY  Define expected solutions and scores beforehand  Attribute tasks to students randomly  e.g. by drawing lots ZENTRUM FÜR MEDIZINISCHE LEHRE | 19

ASSESSMENT AND PROBLEM-BASED LEARNING ZENTRUM FÜR MEDIZINISCHE LEHRE | 20 #2

PROBLEM-BASED LEARNING 1.Read & understand 2.Focus main topics 3.Brainstorming 4.Sort ideas, hypotheses 5.Define learning obj. 6.Learn 7.Teach 8.Give feedback

DEFINITION: PBL Take a problem (a patient, a situation, an observation) as starting point to  Activate prior knowledge  Discover gaps of knowledge  Define learning objectives to close these gaps  Learn  Teach others with your new knowledge ZENTRUM FÜR MEDIZINISCHE LEHRE | 22

PBL versus CBL Problem-based Learning  Be curious, ask, what you want  Be creative and collect all kinds of associations  Learn, what you want  Widen your horizon  Learn to handle similar problems Case-based Learning  Be effective and rule out unlikely ways  Quickly come to the point  Learn/repeat (standard) procedures  Focus your mind  Solve this problem ZENTRUM FÜR MEDIZINISCHE LEHRE | 23

POSITION #1: NO GO  Why not? Problem-based learning  is fun  fosters creativity  encurages brainstorming  opens the mind  creates a cooperative atmosphere  Everything would be destroyed by testing!? ZENTRUM FÜR MEDIZINISCHE LEHRE | 24

ASSESSMENT IN PBL  Test knowledge  Test problem-handling  Test creativity and unconventional thinking  Test team work and management  Test communication skills  Test presentation skills  Feedback: green – yellow – red card ZENTRUM FÜR MEDIZINISCHE LEHRE | 25

FEEDBACK IN PBL  PBL-Tutor  Triple-Jump Exercise  Peers ZENTRUM FÜR MEDIZINISCHE LEHRE | 26

INNOVATIVE ASSESSMENT TOOLS PBL AND ASSESSMENT