Crawling Before They Walk: Introducing Quality Improvement to Interns Samuel J. Crutcher, MD University of South Carolina School of Medicine

Objectives On completion of this session, the participants should be able to: list goals of a quality improvement curriculum describe methods to evaluate a resident’s quality improvement knowledge develop an introductory quality improvement curriculum for residents

QI Curriculum Goals Quality Improvement, improvement science, is a field of academic study What should we expect new physicians to know? – What do we want the future’s “old physicians” to know

QI Curriculum Goals At a minimum we want to satisfy requirements related to ACGME competencies programs must define the specific knowledge, skills, and attitudes required and provide educational experiences as needed in order for their residents to demonstrate: – Practice-Based Learning and Improvement that involves investigation and evaluation of their own patient care, appraisal and assimilation of scientific evidence, and improvements in patient care – Systems-Based Practice, as manifested by actions that demonstrate an awareness of and responsiveness to the larger context and system of health care and the ability to effectively call on system resources to provide care that is of optimal value

QI Curriculum Goals IHI proposed eight knowledge domains for the improvement of health care

Customer/beneficiary knowledge Identifying person, persons, or groups for whom health care is provided and assessing their needs and preferences. This domain covers the relationship of the health care provided to those needs and preferences. Health care as process/system The interdependent people (patients, families, eligible populations, and caregivers), procedures, activities, and technologies of health care-giving that come together to meet the need(s) of individuals and communities. Variation and measurement Using measurement to understand the variation of performance in processes and systems of work and to improve the design and redesign of health care. Leading, following and making changes in health care The methods and skills for making change in complex organizations, the general and strategic management of people and the health care work they do (financing, information technology, and daily health care-giving).

Collaboration The knowledge, methods, and skills needed to work effectively in groups, and understand and value the perspectives and responsibilities of others. This domain includes the capacity to foster collaboration in others. Developing new, locally useful knowledge Recognizing the need for new knowledge in personal daily health professional practice. This domain includes the skill to develop new knowledge through empiric testing. Social context and accountability An understanding of the social contexts (local, regional, national, and global) of health care, including health care financing. Professional subject matter The health professional knowledge appropriate for a specific discipline and the ability to apply and connect domains 1–7. This domain includes core competencies published by professional boards, accrediting organizations, and other certifying entities.

QI Curriculum Goals Research Report in Academic Medicine 2003 – performed literature review to attempt to answer this question – combined literature review with expert panel opinion for final recommendations – the core educational goals were based on the IHI knowledge domains adjusted recommendations based on level of training

QI Curriculum Goals Research Report in Academic Medicine 2003 – found didactic and experiential components to be common and productive – for interns they recommended an initial focus on the questions of “who” and “what” as it relates to the patient panel – as the resident progresses in training then as “why” and “how”

QI Curriculum Goals

Literature review shows a couple themes – learning the model for improvement pdsa

The Model for Improvement

QI Curriculum Goals Literature review shows a couple themes – learning the model for improvement – participating in / leading a quality improvement project

QI Curriculum Goals Other specific topics noted – stakeholder analysis – root cause analysis – process mapping – failure mode and effects analysis – resource management – negotiation – leadership

Methods to Assess Knowledge Assessing professional competence – subjective observation – multiple choice test – standardized patient

Methods to Assess Knowledge QI “test” – no publicly available standardized multiple choice QI test that I know of – There is a test at the end of the MUSC online program – There are tests in the IHI Open School program – There is one validated test called Quality Improvement Knowledge Assessment Test (QIKAT)

Methods to Assess Knowledge QIKAT Scenario Three questions – What would be the aim? – What would you measure to assess the situation? – Identify one change that might be worth testing.

Methods to Assess Knowledge Teaching and assessing resident competence in practice based learning and improvement – core learning (QIKAT) – resident self assessed proficiency – resident and project sponsor satisfaction – faculty and resident time investment

Methods to Assess Knowledge Projects – project outcome? that is ultimately what we are trying to accomplish, correct? many changes will fail resident (intern) projects will probably be small is the resident leading or participating or observing – completion of project or PDSA cycle/s?

Methods to Assess Knowledge Curriculum Effectiveness of Teaching Quality Improvement to Clinicians: A systematic Review (JAMA 2007) programs improved QI knowledge more often than outcomes clinical benefits do not occur when learner behavior does not change the use of adult learning principles did not improve educational outcomes

Curriculum Our intern curriculum 2009 – 2010 – conducted during an ambulatory month each intern was assigned to this rotation and there was no month with multiple interns – didactic individual instruction and online – project one project for all interns

Curriculum Time commitment – time commitment from intern 2 hours first week then 1 hour per week dedicated time additional time in month but not “scheduled” – time commitment from faculty one dedicated faculty member for all didactic sessions and to co – lead / be continuity for the project approx 6 hours per month

Curriculum Free internet site to post documents for the project – ZOHO – Google Documents – we did not post documents with sensitive information

Curriculum Didactic – Individual introduction to QI lecture, 1 hour – MUSC online program copy of test results page submitted for verification able to be completed within 2 – 3 hours

Curriculum Project – topic preselected by faculty to fit into current clinic goals – given overview during their orientation with time to brainstorm about background information needed to conduct the project. – gathered necessary background information during July

Curriculum Project – each month the intern on the rotation co-led the project with the faculty sponsor – at the end of the month the intern submitted a summary of the month’s work. This served as the “checkout” to the oncoming intern. – the intern also submitted power point slides showing the month’s PDSA/s

Report generated Report found to be inaccurate Develop automated report to identify active patients with poorly controlled diabetes who need to return for an office visit Rewrite report settings ActPlan DoStudy

Curriculum What we learned Didactic – MUSC website seemed to work well – all interns passed the test – verification was easy with the copy of the test results

Curriculum What we learned Project – Resident participation/exposure varied some months we were able to complete multiple PDSA cycles and others we did not complete a single PDSA – resident vacation – topic of the PDSA – resident interest – quality of the handoff

Curriculum Proposed Changes for – Didactic completion of online training prior to first meeting with faculty sponsor – Project have each intern complete a small personal project (not necessarily healthcare related) to complete at least one PDSA cycle monthly project presentation during residency didactic time. Performed by the oncoming intern.

Resources AAMC – Med Ed Portal ment/mededportal/information/ ment/mededportal/information/ Online programs – IHI Open School IHI Open School – MUSC online program - MUSC online program

Resources QI knowledge Assessment – QIKAT as edited by University of Chicago program QIKAT

Bibliography Aron DC, Headrick LA. Educating physicians prepared to improve care and safety is no accident: it requires a systematic approach. Qual Saf Health Care. 2002;11(2): Boonyasai RT, Windish DM, Chakraborti C, et al. Effectiveness of Teaching Quality Improvement to Clinicians: A Systematic Review. JAMA. 2007;298(9): Canal DF, Torbeck L, Djuricich AM. Practice-Based Learning and Improvement: A Curriculum in Continuous Quality Improvement for Surgery Residents. Arch Surg. 2007;142(5): Henriksen K, Dayton E. Issues in the design of training for quality and safety. Quality and Safety in Health Care. 2006;15(suppl_1):i17-i24. Huntington JT, Dycus P, Hix C, et al. A standardized curriculum to introduce novice health professional students to practice-based learning and improvement: a multi-institutional pilot study. Qual Manag Health Care. 2009;18(3):

Bibliography Ogrinc,G, Headrick LA, Mutha, S, Coleman, M. A Framework for Teaching Medical Students and Residents about Practice-based Learning and Improvement, Synthesized from a Literature Review. Academic Medicine. 2003; 78: Ogrinc G, Headrick LA, Morrison LJ, Foster T. Teaching and Assessing Resident Competence in Practice-based Learning and Improvement. J Gen Intern Med. 2004;19(5p2): Tomolo AM, Lawrence RH, Aron DC. A case study of translating ACGME practice-based learning and improvement requirements into reality: systems quality improvement projects as the key component to a comprehensive curriculum. Qual Saf Health Care. 2009;18(3): Varkey P, Karlapudi SP, Bennet KE. Teaching Quality Improvement: A Collaboration Project Between Medicine and Engineering. American Journal of Medical Quality. 2008;23(4):