East Carolina University-Brody School of Medicine Evaluating Your Quality Improvement Projects: Putting Theory into Practice Kevin Garrett, MSW Lori DeLorenzo, RN, MSN East Carolina University-Brody School of Medicine December 1, 2016 Diane Campbell, MD, MPH, RN Assistant Professor of Gynecology Ryan White Program Administrator East Carolina University-Brody School of Medicine campbelldi@ecu.edu Barry White, MSTS, BSBA Data Manager Division of Infectious Disease Department of Internal Medicine   Ciarra Dortche, MPH Social/Clinical Research Assistant HIV Outreach Coordinator Division of Infectious Diseases Department of Internal Medicine East Carolina University – Brody School of Medicine

Diane Campbell, MD, MPH, RN Assistant Professor of Gynecology Ryan White Program Administrator East Carolina University-Brody School of Medicine campbelldi@ecu.edu Barry White, MSTS, BSBA Data Manager Division of Infectious Disease Department of Internal Medicine   Ciarra Dortche, MPH Social/Clinical Research Assistant HIV Outreach Coordinator Division of Infectious Diseases Department of Internal Medicine East Carolina University – Brody School of Medicine

As you know, today is World AIDS Day As you know, today is World AIDS Day. And it’s a day to unite in the fight against HIV, to honor those who have lost their life and support those living with and affected by HIV. This year’s theme is “Leadership, commitment & Impact”. The next hour and a half sets the stage to have a continuing impact by evaluating the QI projects that are being undertaken to ensure the services are of the highest quality.

Agenda Examine the specific areas of focus for evaluating QI projects Utilize case studies and a QI project evaluation tool to examine QI projects Explore the differences in approach between agency and systems-level QI projects

Do not put us on hold Use chat room Actively participate in the discussion & share your thoughts & ideas Commit to taking a leadership role to make a positive impact.

Type your name and agency in the chat room Raise your hands What Part do you represent: A, B, C, D, F, none Who has formally evaluated a QI project in the past (jot down the persons name so we can call on them)

Two Levels of Assessment During the last call on evaluating QM programs, we talked about evaluation and assessment needing to occur at 2 levels: Overarching Clinical Quality management program—to understand how the overall program is functioning Individual Quality Improvement projects—to understand how the discrete QI projects are working For the purposes of today’s call, we are going to focus exclusively on how you evaluate QI projects.

Why? Why should we evaluate individual QI projects? Allows us to learn from the process—it’s part of our continual quality improvement, applied to the discrete QI projects Provides an opportunity to step back from the project itself and think about what worked well and where we can make some improvement Allows the group to share their own perspectives Building in time to conduct the evaluation provides a valuable window for the team to pause and assess functioning. Structured approach provides an opportunity to understand what worked well and what needs improvement, thereby facilitating planning for the next project.

Quality is Continuous This concept applies to QI projects as well.

When? Check-in regarding the process can happen at any point during the QI project. But at a minimum, the entire process should be reviewed upon conclusion of the QI project. Schedule a final meeting to celebrate the successful completion of the project and review how it unfolded. If you are a Network, Consortia model, TGA or state-based program with subrecipients, you might have different points of review targeted at different levels. One focus maybe at the site of implementation while another maybe at the higher, jurisdiction or network level to explore how the entire system of care has been impacted.

Who? Who should be involved when the QI project is evaluated and assessed? Team Leader Team members, including consumers if they were part of the team Sr. Leader champion, if one was identified If it’s a model with subrecipients, make sure both levels of discussion take place and ensure you have the right people at the table at the appropriate meeting

What? When we think about the discrete QI projects, what are the major elements that we should look at and review?

Quality Improvement Projects: Core Elements for Review Process QI Team Measurement Documentation Sustainability Information Dissemination Ask you review your QI project, it will be helpful to think through each of these elements

Quality Improvement Projects: Core Elements for Review Process How was the project selected? Was a goal clearly articulated? Was the scope of work and timeline realistic? Did you have senior leader support? Was the root cause determined before solutions were identified? How was the project selected? Based on data? 1 person’s area of interest or pet project? Driven by funding agency? Part of a larger QI initiative? Identified as a need by consumers? Was a goal clearly articulated? Was the scope of work and timeline realistic? Did you have senior leader support? Was the root cause determined before solutions were identified?

Quality Improvement Projects (cont.) QI Team Were the right members on the team? Were the team members actively engaged? Did the team members assume the necessary roles and share responsibilities? Were consumers involved? Were subrecipients involved? If your project involves subrecipients, did how you handle communication between sites? Did the team members adequately share the pertinent info with their home site and keep everyone informed? How were competing priorities handled?

Quality Improvement Projects (cont.) Measurement Was an appropriate goal set? Was the correct performance measure used? Were the results tracked for discrete PDSA cycles? Were improvements shown over time? How was this handled if subrecipients are involved? Were the interventions effective? Were there differences across sites? Were improvements shown over time? How was this handled if subrecipients are involved? Were data unblinded? Were the interventions effective? Were there differences across sites?

Quality Improvement Projects (cont.) Documentation Were the change ideas tracked and documented? Was a summary of key change ideas compiled? Sustainability Were the change ideas spread beyond the implementation site? Were the processes institutionalized so the efforts can be sustained over time? Information Dissemination Were the results, successes and lessons learned shared with internal and external stakeholders? Sustainability If subrecipients were involved, did you try out the change ideas in 1 site before expanding to others? Were different strategies implemented at different sites? Where are strategies scaled up before they were really ready? Info Dissemination How were results shared so that the power of peer learning can be maximized? Was everyone kept informed along the way and not just at the beginning and end? Did we share the successes? For example, UNC-Chapel Hill was just showcased in the NQC’s monthly newsletter for their work on STI screening rates. How are your successes shared?

How? What tools are available?

We’ll be using the tool to explore a QI project in just a bit, so if you’d like to download the tool, the link is provided at the bottom of the page. It’s a 3-page tool and set up using a 5 point scale and allows the user to take notes on each of the sections and identify what you would do differently next time around. The 1st page focuses on the process http://nationalqualitycenter.org/files/qi-project-evaluation-tool

Page 2 focuses on the QI team and measurement

Page 3 focuses on documentation, sustainability and information dissemination Other tools may also be available. Do anyone have a different tool available that they would like to share? If so, please send to Kevin or myself.

Putting Theory into Practice Introduce their organization Diane Campbell, MD, MPH, RN Assistant Professor of Gynecology Ryan White Program Administrator East Carolina University-Brody School of Medicine campbelldi@ecu.edu Barry White, MSTS, BSBA Data Manager Division of Infectious Disease Department of Internal Medicine   Ciarra Dortche, MPH Social/Clinical Research Assistant HIV Outreach Coordinator Division of Infectious Diseases Department of Internal Medicine East Carolina University – Brody School of Medicine

East Carolina University Focus of QI project Steps taken Lessons learned Application of learning Advice for others What was the focus of your QI project? What steps did you take to evaluate your project? Walk through the tool to pose the questions to ECU What did you learn? Think about: Your process Your team composition and interaction Measures used Documentation Dissemination of information Ultimate success of reaching goal How will you apply the lessons learned to future projects? What advice would you have for other agencies as they evaluate and assess their Qi projects

Your Turn! Open up the dialogue and ask participants to share their experience in evaluating QI projects. What approach have you used? What tools do you have to share? What worked and what didn’t? What did you learn as you evaluated your QI project? What advice do you have for others? For those agencies that have subrecipients, do you have any specific approaches that you’d like to share?

To listen to the webinar Go to: https://meetny.webex.com/meetny/lsr.php?RCID=2af ce130145d46b4bb372c1b78e64fc3

Diane Campbell, MD, MPH, RN East Carolina University 252-744-5728 Kevin Garrett, MSW Senior Manager, NQC 212-417-4730 kevin.garrett@health.ny.gov Lori DeLorenzo, RN, MSN NQC Coach 540-951-0576 loridelorenzo@comcast.net Once we confirm the Diane Campbell, MD, MPH, RN East Carolina University 252-744-5728 campbelldi@ecu.edu