Improving Resident Education in Quality Improvement and Patient Safety Ramon Cancino, MD STFM 04/30/2011

Objectives Discuss research in residency education in the field of quality improvement and patient safety (QI/PS) Discuss tools available to install an educational program to teach residents about QI/PS Present findings of an IRB-approved study using an online curriculum to teach family medicine residents the principals of QI/PS Discuss methods of how to further evolve this field and establish best practices

Background Quality Improvement (QI) Formal approach to performance analysis and systematic efforts to improve it

History In 1999, the Institute of Medicine (IOM) found as many as 98,000 people die each year from medical errors that occur in hospitals. Medical error: the 5th leading cause of death in the United States. The IOM, in 1999, recommended that performance standards and expectations for heath care organizations should focus greater attention on patient safety (Kohn LT 1999). The IOM created a new awareness that the status quo was no longer acceptable. Crossing the Quality Chasm: A New Health System for the 21st Century (2001) outlined fundamental changes that must be made in order to improve care.

Health Care Must Be: Safe Effective Patient-centered Timely Efficient Equitable

Background

Recent studies into QI/PS curricula have all shown benefit. Motivation Residents involved in QI training strongly believe they gained important skills. (Voss et al. 2008) Recent studies into QI/PS curricula have all shown benefit. (Canal, Torbeck et al. 2007), (Criley, Keiner et al. 2008), (Oyler, Vinci et al. 2008) A web-based interactive educational program led to significant improvement in cardiac examination competence and was better than clinical training alone. (Criley, Keiner et al. 2008) systems thinking and human factors analysis root cause analysis process mapping minimal hospitalist involvement

Question Using an established, widely-used, online curriculum which focuses specifically on QI/PS, this study: Investigated when QI/PS education is beneficial to the family physician in-training. Assessed family medicine resident opinions, views, and experiences in the area of QI/PS. 

Our Aim: Determine whether first year residents have less knowledge of QI/PS than second and third year residents. Determine if training increased resident knowledge of QI/PS Determine whether upper level residents retained knowledge of QI/PS better than first year residents after six months.

Materials Recruitment Letter Pre- and Post-test Questionnaires Institute for Healthcare Improvement Open School website Family Medicine Residents

Materials

Methods Online lessons were chosen based on their relevance to both early and late residency training. Fundamentals of Improvement The Model for Improvement: Your Engine for Change Introduction to Patient Safety Fundamentals of Patient Safety

Methods QI 101: Fundamentals of Improvement Lesson 1: Errors Can Happen Anywhere—and to Anyone Lesson 2: Health Care Today Lesson 3: The Institute of Medicine's Aims for Improvement Lesson 4: How to Get from Here to There: Changing Systems QI 102: The Model for Improvement: Your Engine for Change Lesson 1: An Overview of the Model for Improvement Lesson 2: Setting an Aim Lesson 3: Measuring Lesson 4: Developing Changes Lesson 5: Testing Changes PS 100: Introduction to Patient Safety Lesson 1: Understanding Medical Error and Patient Safety Lesson 2: Understanding Unsafe Acts Lesson 3: A Call to Action — What YOU Can Do PS 101: Fundamentals of Patient Safety Lesson 1: To Err Is Human Lesson 2: Responding to Error Lesson 3: Identifying and Reporting Errors Lesson 4: Error versus Harm Methods

Methods Pre-Test Questionnaires Recruitment letter via electronic mail, which asked for consent to enter this voluntary study. Pre-Test Questionnaires prior training in QI/PS opinion about the area of study opinion in regards to its importance in their current/future careers. Instructions to register with the Institute for Healthcare Improvement Open School in order to access an online curriculum Online curriculum and testing Post-Test Questionnaires

Methods Data was collected by the principal investigators. Data was organized using Microsoft Excel. Analysis was done by Mayo Clinic Biostatistics division.

Results Total = 76 residents Pre-Test Questionnaire: 76 Online Curriculum: 34 Post-Test Questionnaire: 33

Results 6 Institutions Various Community settings Various Institution Type

Pre-Test Results

Pre-Test Results

Pre –Test Results Residents who have had formal training in QI/PS:

Pre-Test Results

Pre-Test Results

Pre-Test Results Important Issue? p = 0.007

Pre-Test Results 86 % believe a formal curriculum in QI/PS belongs in medical education 88% believe a formal curriculum in QI/PS belongs in residency education

Average Testing Results (%) 95.8 QI 101: Fundamentals of Improvement 92.3 QI 102: The Model for Improvement: Your Engine for Change 93.0 PS 100: Introduction to Patient Safety 96.3 PS 101: Fundamentals of Patient Safety

PGY1 PGY2 PGY3 94% 96% 95% 100% Average Scores Fundamentals of Improvement 94% 96% Model for Improvement Introduction to Patient Safety Fundamentals of Patient Safety 95% 100%

Post-Test Results How strongly do you believe that issues of QI/PS are important to patient care?

Post-Test Results

Post –Test Results Level of Difficulty per Training Level p = 0.02

Post –Test Results p = 0.008

Post –Test Results If you were the chair of the QI/PS committee of the department of family practice, which online courses would you require incoming residents to take?

Post –Test Results 85% of residents feel family medicine clinic and hospital staff may benefit from taking an online curriculum in QI/PS

Discussion Residents enter training feeling ill- equipped to handle issues of QI/PS. At the time of questioning, many residents were not yet thoroughly educated on the topic. Residents feel QI/PS education belong in both medical and residency training.

Discussion An online curriculum in QI/PS was generally well-received. Residents gained an appreciation for the importance of this issue. Residents feel all staff members would benefit from this sort of education.

Limitations Number of participants Retention of participants Not double-blind study Testing administration

Future Study 6 months after the start of the residency year, a post-test, which contained questions drawn from the online curriculum, will be administered. This portion of the study will also be optional

Future Study Quality Improvement activities amongst graduates Patient safety-related training Use of technology in resident education

In Summary: Upon entering residency, trainees feel ill- equipped in knowledge of QI/PS Online training and testing modules were well-received Increase in perceived importance of QI/PS All hospital/clinic staff may benefit from this sort of training Increase in this fund of knowledge is best practice for our patients.

Thank You Drs. Pantin, Bachman, Willis. Participating sites: Dr. Brian Arndt at Wisconsin University Dr. Grace Chen Yu at Stanford University Dr. Steve Wolfe at Forbes Family Medicine Residency Program Dr. Janet Albers at Southern Illinois University Dr. Robert Bonacci at Mayo Clinic Rochester Dr. Walter Taylor at Mayo Clinic Jacksonville Ramon S. Cancino, MD Mayo Clinic Florida 4500 San Pablo Jacksonville, FL 32224

Resources Canal, D. F., L. Torbeck, et al. (2007). "Practice-based learning and improvement: a curriculum in continuous quality improvement for surgery residents." Archives of Surgery 142(5): 479-482; discussion 482-473. Criley, J. M., J. Keiner, et al. (2008). "Innovative web-based multimedia curriculum improves cardiac examination competency of residents." Journal of Hospital Medicine 3(2): 124-133. http://www.aafp.org/online/en/home/practicemgt/quality.html Kohn LT, C. J., Donaldson MS, eds (1999). To Err Is Human: Building a Safer Health System. J. M. C. Linda T. Kohn, and Molla S. and Donaldson. Washington, DC. Oyler, J., L. Vinci, et al. (2008). "Teaching internal medicine residents quality improvement techniques using the ABIM's practice improvement modules." Journal of General Internal Medicine 23(7): 927-930. Richardson, W. C. B., Donald M.; et al (2001). Crossing the Quality Chasm: A New Health System for the 21st Century. I. o. M. Committee on Quality of Health Care in America. Washington, DC. Voss, J. D., N. B. May, et al. (2008). "Changing conversations: teaching safety and quality in residency training." Academic Medicine 83(11): 1080-1087.