Evaluating two feedback mechanisms for MCQ exams David Hope, Kyle Gibson & Helen Cameron August 2013.

Slides:

Advertisements

Similar presentations

Performance Assessment

Advertisements

Moving From Feedback to Feedforward Writing With Integrity 2013 Poynter Institute Erin Goergen, Teri Trede, Floyd Butz IV.

Use of online spaced education to reinforce learning Kitt Shaffer, MD PhD Vice-Chair of Radiology for Education Boston Medical Center.

Feedback Versus Feeding A Randomized Controlled Trial ML Boehler, DA Rogers, CJ Schwind R Mayforth, J Quin, R Williams, G Dunnington Southern Illinois.

Assessing student learning. Diagnostic - The gathering of information at the outset of a course or program of study to provide information for both the.

Foos et al, EASD, Lisbon, 13 September 2011 Comparison of ACCORD trial outcomes with outcomes estimated from modelled and meta- analysis studies Volker.

ROUNDWOOD PARK PE GCSE PE Course assessment (OCR) 50%Practical component - 4 physical activities (used) from minimum 2 areas - Internally assessed and.

‘Enhancing the First Year Experience – A Case Study From Biomedical Sciences’ Paul Hagan Stephen M c Clean University of Ulster.

The importance of Online and Computer aided assessment in E-Learning E.C Makazhe – Computer Science.

Dennis M. Donovan, Ph.D., Michael P. Bogenschutz, M.D., Harold Perl, Ph.D., Alyssa Forcehimes, Ph.D., Bryon Adinoff, M.D., Raul Mandler, M.D., Neal Oden,

MSSTL10-Carlow IT May  Setting the scene  Initial phase of research  Aim of presentation  Profiling at risk students  Predicting failure of.

Portfolios A method of assessment for post-registration health professional education Michelle Green Michelle Green

Tutorials via Social Networking. Samer El-Daher, Lucie Pollard School of Science.

Miguel Martins, Senior Lecturer – Marketing Department eLearning Celebration 22 July 2007 Improving performance on a Marketing module through the use of.

 D Rubosky EDUC 533 – Spring  Creating an instructional foundation based on how individuals acquire knowledge and skills o Supplement lecture.

Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007.

Assessment in Higher Education Linda Carey Centre for Educational Development Queen’s University Belfast.

Developing Satisfying and Effective Feedback for Medical Students David Hope, Avril Dewar, Kyle Gibson, Jamie Davies & Helen Cameron June 2014.

Improving Learning via Tablet-PC-based In-Class Assessment Kimberle Koile, MIT CS and AI Lab David Singer, MIT Brain & Cognitive Sciences Classroom Presenter.

Chiropractic Care A Drug-free, Non-surgical Approach to Health Care.

METHODS Study Population Study Population: 224 students enrolled in a 3-credit hour, undergraduate, clinical pharmacology course in Fall 2005 and Spring.

Ismaila Odogba Department of Geography and Geology The Impact of Collaborative Testing on Student Learning in an Introductory Level Geography Course INTRODUCTION.

Introducing small-group workshops as formative assessment in large first year psychology modules Suzanne Guerin School of Psychology, University College.

SFC Project Re-engineering assessments Weekly multi-choice tests.

EXAMING SOCIAL LIFE Social sciences are disciplines that study human social behavior or institutions and functions of human society. Social interaction.

English and Curriculum for Excellence S3 Course for

Kazakhstan Centres of Excellence Teacher Education Programme Assessment of teachers at Level Two.

Nurse Co-ordinated Rapid Access Chest Pain Clinic

Evaluation of Respondus assessment tool as a means of delivering summative assessments A KU SADRAS Project Student Academic Development Research Associate.

DEPARTMENT OF MARKETING Presented by: Mercedes Douglas, Senior Tutor, Other Reap Team Members:

Felt for signatur(enhet, navn og tittel) Associate Professor Annetine Staff Head of examination committee, member of semester committee Professor Kai-Håkon.

The College Admission Essay Personal Statement This personal statement helps university officials become acquainted with you in ways different from courses,

3/7/2015 WALLER, HYDE, OYEDELE, BERND, SWENSON, MARTINEZ, RAMEN, SCHWINDINGER, TALWAR FORNARI, BAHNER, HENNEMAN Physical Diagnosis Group #2.

Students’ and Faculty’s Perceptions of Assessment at Qassim College of Medicine Abdullah Alghasham - M. Nour-El-Din – Issam Barrimah Acknowledgment: This.

A review of peer assessment tools. The benefits of peer assessment Peer assessment is a powerful teaching technique that provides benefits to learners,

An Audit of OSCE Feedback Across UK Medical Schools Avril Dewar, David Hope and Helen Cameron 1 st September 2014, AMEE.

Improving Academic Feedback - Turning the ship around with the power of 1000 students David Hope

TEACH ON THE RUN! SESSION 3: HELPING LEARNERS THINK! DIAGNOSTIC REASONING Created by Dr. Leslie Ann Sadownik.

Implementation of VCE Psychology 2005 Ann Osman VCAA Curriculum Science Manager.

Adapted from “Best Practices for Student Learning, Assessment in Online Courses”“Best Practices for Student Learning, Assessment in Online Courses”

Cortés J et al. ASCO 2009; Abstract (Poster Discussion)

Nicola Billam, Guinevere Glasfurd-Brown & Martin Sellens CAA Conference 2006 Engaging Students in Formative Assessment: Strategies and Outcomes.

CREATING AN ACTIVE LEARNING ENVIRONMENT Using Inquiry and Primary Sources.

Simulated Patients Improve Medical Student Comfort Level with Breaking Bad News and End of Life Issues Skotti Church, MD Carl J Fichtenbaum, MD, FACP University.

Pathology 430/826 The Molecular Basis of Disease.

Assessment. Summative assessment Summative assessment is the process of evaluating (and grading) the learning of students at a point in time. The University.

Marking and Assessing Andy Wilson Loughborough University.

This work is supported by a National Science Foundation (NSF) collaboration between the Directorates for Education and Human Resources (EHR) and Geosciences.

Information Technology Design & Multimedia. A Comparison of an Innovative Web-based Assessment Tool Utilizing Confidence Measurement to the Traditional.

E-Assessment: Removing the Boundaries of C.A.A. Phil Davies School of Computing University of Glamorgan South Wales UK.

MRCGP The Clinical Skills Assessment January 2013.

The MICRO-HOPE. Microalbuminuria, Cardiovascular and Renal Outcomes in the Heart Outcomes Prevention Evaluation Reference Heart Outcomes Prevention Evaluation.

Innovative Applications of Formative Assessments in Higher Education Beyond Exams Dan Thompson M.S. & Brandy Close M.S. Oklahoma State University Center.

Project VIABLE - Direct Behavior Rating: Evaluating Behaviors with Positive and Negative Definitions Rose Jaffery 1, Albee T. Ongusco 3, Amy M. Briesch.

Non-Medical Prescribing Practice Based Learning and Assessment.

Biomedical Sciences 1: Facilitating the transition to higher education Dr Allison Wroe, Dr Phil Larkman Edinburgh Medical School: Biomedical Sciences Enhancement.

An Alternative Certification Examination “ACE”, to assess the domains of professional practice. Morris M1*, Gillis AE2, Smoothey CO3, Hennessy M1, Conlon.

MRCGP The Clinical Skills Assessment January 2013.

Evidence-based Feedback

Review and evaluation of teaching and assessment of anatomy, physiology and pathophysiology in the Postgraduate pre-registration curriculum John Mears,

College Of Applied Medical science

Chapter 7 The Hierarchy of Evidence

Concepts in Tests and Measurements

David Hope, Avril Dewar and Helen Cameron Background

An Alternative Certification Examination 'ACE' to assess the domains of professional practice.

Charles N. Elliott, Paul A. Story

Curriculum Outline Medicine

The Prediction of National Physical Therapy Examination First Time Pass Rates Using Reading Comprehension and Critical Thinking Skills Tests Latoya green.

INTRODUCTION ENDOCRINE SYSTEM Coordinator Dr. Nabil Amer (M.D., Ph.D.)

Presentation transcript:

Evaluating two feedback mechanisms for MCQ exams David Hope, Kyle Gibson & Helen Cameron August 2013

Background Feedback is seen as an integral part of assessment Most work on feedback focuses on clinical contexts MCQ exams represent much of the assessment in undergraduate medical education (Scouller, 1998) Feedback does not consistently improve performance in clinical contexts (Boehler et al. 2006) Can we deliver satisfying and useful feedback for MCQ exams?

Summary of Work 196 students (completion rate over 95%) sat a 30 minute formative cardiovascular exam OSCA-FM: 50% received detailed, immediate feedback explaining each option for each question but they could not take the feedback away Tagging: 50% received a summary of their performance on a number of domains and they could keep this indefinitely Students sat a further 30 minute formative exam three weeks later

OSCA-FM

Tagging TagScoreAverageMax Cardiovascular Biomedical Sciences Presentation Diagnosis and Management Clinical pharmacology and therapeutics Anatomy999 Physiology Pathology81116 Chest pain323 Congenital Heart Disease81116 Lipids333 Biochemistry ECG999

Performance indicators Students performed similarly on the follow up formative exam – OSCA-FM 73.0 – Tagging 68.8 – Previous cohort 69.5 No significant differences on this or final cardiovascular exam performance after controlling for prior academic ability

Diagnostics Of the 41 students who received a D or failing grade for the whole year the mean mark was in the formative exam (a failing grade) Student performance increased by around one grade between the first formative exam and the cardiovascular summative exam

Student satisfaction Five indicates ‘very satisfied’ Students were relatively satisfied with receiving either form of feedback (3.7) and felt them to be useful (3.6) Combining detailed formative feedback with summative tagging was regarded as extremely useful (4.5) Perceived usefulness did not correlate with performance or performance change

Conclusions Students find detailed feedback to be useful and desirable Such feedback yields no performance gains and perceived performance gains cannot be observed Formative MCQ exams can identify students at risk of failure/underperformance Focusing on how students receive and use feedback rather than how it is delivered may be a useful line of inquiry

References Boehler ML, Rogers DA, Schwind CJ, Mayforth R, Quin J, Williams RG, et al. An investigation of medical student reactions to feedback: a randomised controlled trial. Medical Education. 2006;40: Scouller K. The influence of assessment method on students' learning approaches: Multiple choice question examination versus assignment essay. Higher Education :