Educational Outcomes Service Group: Overview of Year One Lynne Tomasa, PhD May 15, 2003.

Slides:



Advertisements
Similar presentations
Nursing Diagnosis: Definition
Advertisements

Introduction to Competency-Based Residency Education
Standards Definition of standards Types of standards Purposes of standards Characteristics of standards How to write a standard Alexandria University Faculty.
+ The APN role of “Consultation” A Panel Presentation Serena Butler Abi Fitzgerald Renee Latoures.
The Milestone Project For Neurosurgical Resident Coordinators.
Improving ED Bedside Teaching & Resident Evaluation Stanford EM Faculty Development May 21 st, 2003.
AECOM COGME Seminar Implementing the Competencies 360 Degree Evaluations Catherine C. Skae, MD Director, Pediatric Residency Program Children’s Hospital.
NEW COMPETENCY QUESTIONS IN PROGRAM INFORMATION FORM Lois L. Bready, M.D. Associate Dean for GME and DIO UTHSCSA.
GME Lunch n Learn Series Cuc Mai September Common Program Requirements: Competency-based goals and objectives for each assignment at each educational.
1 Why is the Core important? To set high expectations – for all students – for educators To attend to the learning needs of students To break through the.
ELAC SLO RETREAT 2009 Veronica Jaramillo, Ph.D. Mona Panchal Anthony Cadavid ELAC SLO RETREAT 2009.
Continuous Quality Improvement Evidence-Based Medicine In Practice…
Preliminary Feedback from ACGME CLER Site Visit August 19-21, 2014
Chapter 15 Evaluation.
Comparison: Traditional vs. Outcome Project Evaluative Processes Craig McClure, MD Educational Outcomes Service Group University of Arizona December 2004.
GME Jeopardy. Compe 10 cies VISA issues ToolboxOversiteAlphabet Soup
a judgment of what constitutes good or bad Audit a systematic and critical examination to examine or verify.
Competency Based Medical Education and Assessment
360 Degree Evaluation Craig McClure, MD May 15, 2003 Educational Outcomes Service Group.
Developing A Competency-based Curriculum B. Joyce, Ph.D.
Implementation of the MedBiquitous Curriculum Inventory Standardized Vocabulary Robby Reynolds Sr. Director, Medical Education Online Resources, AAMC Terri.
Family Medicine Program By the end of this session, faculty will 1.Understand what is meant by competence and the competence trajectory expected during.
Debriefing in Medical Simulation Manu Madhok, MD, MPH Emergency Department Children’s Hospital and Clinics of Minnesota.
Triple C Competency-based Curriculum: Implications for Family Medicine Residency Programs.
Emory University Department of Gynecology & Obstetrics Morbidity & Mortality Conference - Faculty Moderator - Resident Privileged & Confidential:
Core Competency Assessment in Emergency Medicine from Design to Implementation Christian Arbelaez, MD, MPH Assistant Residency Director Harvard Affiliated.
ACGME OUTCOME PROJECT : THE PROGRAM COORDINATOR’S ROLE Jim Kerwin, MD University of Arizona.
Fundamentals of Assessment Todd L. Green, Ph.D. Associate Professor Pharmacology, Physiology & Toxicology PIES Seminar
Paula Peyrani, MD Medical/Project Director, HIV Program at the 550 Clinic Assistant Director, Research Design and Development Clinical and Translational.
Core Competency Assessment in Emergency Medicine from Design to Implementation National Hispanic Medical Conference Christian Arbelaez, MD, MPH Associate.
Excellence in Clinical Teaching Your Name Here Your Organization.
Module 3. Session DCST Clinical governance
The New ACGME Competencies for Internal Medicine.
“SEE ONE, DO ONE, TEACH ONE” Supervision. Libby Zion Case Issue of work hours galvanized the press and the public and led to subsequent major reforms.
Simulation-Based Assessment Emily M. Hayden, MD, MHPE Associate Director for Curricular Integration Gilbert Program in Medical Simulation Harvard Medical.
The program will start promptly at 2:15 PM For technical assistance please contact Tech Support at or at
Learning Needs and Goal Setting
McGaw’s Overview of the Next Accreditation System (NAS)
Writing Narratives Based on ACGME Competencies. Narratives What Are They?  Written Evaluation of Student Performance Formative  Mid-Course Evaluation.
1. Housekeeping Items June 8 th and 9 th put on calendar for 2 nd round of Iowa Core ***Shenandoah participants*** Module 6 training on March 24 th will.
Outcomes Methods RRC-Internal Medicine Educational Innovations Project: Clinical Quality Improvement and Patient Safety- Deliverables to Healthcare from.
Developing an Assessment System B. Joyce, PhD 2006.
“R.I.M.E.” MODEL – A SYNTHETIC EVALUATION CONCEPT R eporter I nterpreter M anager- E ducator Pangaro LN. A new vocabulary and other innovations for improving.
What is “Competency” in the New Millennium? Shirley Schlessinger, MD, FACP Associate Dean for Graduate Medical Education University of Mississippi Medical.
Comparison: Traditional vs. Outcome Project Educational Paradigms Craig McClure, MD Educational Outcomes Service Group University of Arizona December 2004.
Considerations for Curricular Development & Change Donna Mannello, DC Logan University.
Staff Physician & Resident Physician Toolkit
What is a Planned Curriculum?
- E-Portfolios are being used to support individual learning at multiple levels in a number of different areas of study - Widely used by medical students.
FAIMER Program Evaluation 3: How is it used? Evaluation as Research Stewart Mennin Bill Burdick.
Assessing Learners The Teaching Center Department of Pediatrics UNC School of Medicine The Teaching Center.
Program Evaluation Principles and Applications PAS 2010.
ACGME SIX CORE COMPETENCIES Minimum Program Requirements Language Approved by the ACGME, September 28, 1999 “The residency program must require its residents.
Accreditation Council for Graduate Medical Education Milestones are Coming: A Conversation with the Family Medicine Milestones Committee May 2013.
Patricia Kokotailo, Sarah Pitts, Sheryl Ryan, Karen Soren, Maria Trent
IDENTIFICATION, DIAGNOSIS & REMEDIATION OF THE STRUGGLING LEARNER Jeannette Guerrasio, MD University of Colorado, SOM Chris Knight, MD University of Washington,
Exploring Non-Physician Roles in Competency-Based Resident Education April 1, 2016 Nicole McGuire, Education Coordinator, Union Hospital FMR (Terre Haute,
Click to edit Master subtitle style Competence by Design (CBD) Foundations of Assessment.
Henry M. Sondheimer, MD Association of American Medical Colleges 7 August 2013 A Common Taxonomy of Competency Domains for the Health Professions and Competencies.
Curriculum Development: an Overview of 6 Steps MAJ Heather O’Mara, DO, FAAFP Faculty Development Fellow.
EFFECTIVE DELEGATION AND SUPERVISION
1 Transforming Our Practices Transformed Our Teaching: Meeting ACGME Competencies with New Models of Care Katherine Miller, M.D. John Nagle, MPA U. Of.
Overview of Education in Health Care
FAIMER Assessing Teaching Excellence John Norcini, Ph.D.
Creating Awareness of Psychiatric Nursing. Purpose Increase knowledge of mental/behavioral health triumphs and challenges, increase nursing engagement.
Interpersonal and Communication Skills
EPAs as Curriculum Tools
Precepting Challenging Students
Presentation transcript:

Educational Outcomes Service Group: Overview of Year One Lynne Tomasa, PhD May 15, 2003

Education of Program Directors and Faculty  Program directors were familiar with requirement and toolbox  Programs were mainly utilizing global assessment tools  Programs rarely utilized direct observation and evaluation  Evaluation tools were used mainly for summative feedback

Faculty Buy-In  Program directors did all or most of curriculum development, so group buy-in was minimal  Junior faculty members had limited experience in curriculum design/teaching  Programs with faculty at multiple sites had additional challenges

Curriculum Design  Program coordinators were helpful but had various amounts of expertise in this area  Many program directors were new and junior and had minimal experience in curriculum development

Six ACGME Domains  Overlapped with each other  Forced one to define it according to specialty  Patient care and medical knowledge were easiest to define and understand  Professionalism: “know it when you see it” but behaviors were not clearly documented

Six Domains  Interpersonal and communication skills made sense but often did not incorporate colleagues, staff, and other health professionals  Practice-based learning and improvement and systems-based practice were “fuzzy” and forced one to think outside the “box”  Systems-based practice was least intuitive  Health care professionals were now viewed as part of a team

Paradigm Shift  To Educational Outcomes  From Process and Structure Compares learner’s performance with that of a peer group Fails to provide clear understanding of what resident can or cannot do Cannot determine if benchmarks or performance indicators have been met

Educational Outcomes  Clearly defines curricular objectives or benchmarks to describe competencies  Faculty must be more accountable and possess the skills/competencies themselves  Requires a structure to record, collect, and analyze data  More costly to implement

“Competency” Defined A complex set of behaviors built on the components of knowledge, skills, attitudes, and “competence” as personal ability. A complex but demonstrable integration of numerous related objectives, the latter being discrete measurable behaviors. Carraccio, Wolfsthal, Englander, Ferentz, Martin, Academic Medicine, May 2002

Curricular Design Stepwise Approach 1. Competency identification 2. Determination of competency components and performance levels 3. Competency evaluation 4. Overall assessment of the process Carraccio, et al. 2002

Step 1: Competency Identification  Consensus of individual experts  Group consensus  Task analysis (document all activities over a period of time)  Critical-incident survey (describe observed incidents that reflect good or bad practice)  Behavioral-event interview (star performers describe critical clinical situations and characteristics of a good doctor)  Practitioner surveys

Step 2: Competency Components  Identification of “tasks” that sequentially or in sum, make up the competency  Benchmarks or performance indicators  Must be measurable  In aggregate, determine achievement of the specific competency

Step 2: Performance Levels  Performance levels set threshold for demonstrating competence  Each benchmark must be clearly defined to determine whether competence has been achieved  Must determine the methods by which the competency might be attained

Step 3: Competency Evaluation  Criterion-referenced measures compare performance against a set standard or threshold are the preferred methods  Adult learners  Incorporates a variety of methods and strategies for adult learners

Program Directors’ Concerns  Where’s the evidence? There is little Studies have utilized small numbers  PD time commitment  Faculty buy-in and education

Responses to ACGME Requirements  I wish this goes away  I’ll wait until our specialty board tells us exactly what to do  I/We don’t have the time  I have a large faculty in a variety of sites  This is one more thing we have to do  How do we know that this shift will produce better, more skillful physicians?

Common Themes  Residents were not an active participant in identifying competencies  Residents have either none or minimal knowledge about the goals of the ACGME Outcomes Project  Program directors feeling “overwhelmed” at the enormity of task

Common Themes  Want clearer guidelines: no consensus on what outcomes are desired  On one hand, Program Directors want autonomy and on the other they want more direction  Question of what outcomes are desired

Examples of Desired Outcomes  Improved career satisfaction  Decreased medication errors  Decreased hospital stays  Better balance between personal and professional life  Longer life for patients  Better compliance with preventive health measures

Motivating Factors  RRC visit in the near future  Enthusiastic faculty with interest in competency-based curriculum

Non-Motivating Factors  Implementation of duty-hours takes time away from teaching and curriculum development  RRC visit occurred just prior to July 2002  Lack of support from Director of Medical Education