Module 20 – Day 3 8:00am – 8:15am (30 min) Welcome to Day 3

Agenda – Day 3 8:00 Welcome & Warm-up Activity 8:30 QM and Quality Management Plans 9:30 Break 10:00 Evaluating Training Effectiveness 10:45 Presenting & Facilitating Learning including Training Nightmares, Mishaps & Messes 12:15 Lunch 1:00 Presentation & Role Play with Peers 2:00 Game Plan Going Forward 2:30 Workshop Close, Session Evaluation, Kudos & Celebration 3:00 Adjourn

Fears and Challenges

Quality Management Plans Module 21 – Day 3 8:30 – 9:15am (45 min) Quality Management Plans

Learning Objectives Describe the role of a QM Plan in the overall quality program Describe the elements of a good written QM Plan Analyze a sample QM Plan to determine strengths and areas for improvement

Agenda Definitions of Terms HAB Expectations Elements of a QM Plan 10 Rules Resources Group Exercise

Quality Management in the Context of the Ryan White Program Module 23 – Day 3 9:15am – 9:30am (15 min) Quality Management in the Context of the Ryan White Program

Ryan White Treatment Extension Act of 2009 “The chief elected official/ grantee… shall provide for the establishment of a clinical quality management program to assess the extent to which HIV health services provided to patients under the grant are consistent with the most recent Public Health Service guidelines for the treatment of HIV disease and related opportunistic infection, and as applicable, to develop strategies for ensuring that such services are consistent with the guidelines for improvement in the access to and quality of HIV health services”

Clinical Quality Management HAB released Policy Clarification Notice 15-02 in the fall of 2015. In it, they clarified the legislative requirements for CQM: Recipient is responsible for conducting clinical quality management and ensuring their subrecipients are also Recipients must have a performance measure for each funded service category and two for the service categories with the highest dollar allocation Your Project Officer is the best source of information / clarification on PCN 15-02

Strategic QM Plan (3-5 yrs) Grantee-wide Vision Strategic QM Plan (3-5 yrs) QM Plan (annual) Annual Goals Workplan Execution Annual Evaluation

Definitions of Terms Quality Management Plan: A Quality Management Plan is a written document that outlines the program-wide HIV quality program, including a clear indication of responsibilities and accountability, performance measurement strategies and goals, and elaboration of processes for ongoing evaluation and assessment of the program.

Definitions of Terms Quality Management Program: The term ‘quality management program’ encompasses all grantee-specific quality activities, including the formal organizational quality infrastructure (e.g., committee structures with stakeholders, providers and consumer) and quality improvement-related activities (performance measurement, QI project and QI training activities).

Definitions of Terms Strategic Plan: A strategic plan is a document that describes the long-term (3-5 years) objectives of the QM program with stretch goals that are in line with the overall vision of the organization.

Definitions of Terms Workplan: A workplan or implementation plan describes concrete steps in the implementation of an annual QM plan with a detailed description of responsibilities and timetables and milestones. At times, the workplan is folded into the overall QM plan.

Strategic QM Plan (3-5 yrs) Grantee-wide Vision Strategic QM Plan (3-5 yrs) QM Plan (annual) Annual Goals Workplan Execution Annual Evaluation

HAB QM Plan Expectations Minimum Expectations Establish a quality management plan Establish processes for ensuring that services are provided in accordance with PHS guidelines & standards of care Further details are available at the HRSA HIV/AIDS Bureau website [hab.hrsa.gov]: Part A: hab.hrsa.gov/tools/title1/t1SecVIIChap5.htm Part B: hab.hrsa.gov/tools/title2/t2SecVIIIChap5.htm Part C: http://hab.hrsa.gov/tools/title3/sii_chapter_4.htm

Elements of a Quality Management Plan Quality statement Quality improvement infrastructure Performance measurement Annual quality goals Participation of stakeholders Evaluation

Exercise: Analyze a Sample Plan 20 mins As a table group, choose either Sample Plan A pg 156 or Plan B pg 158. As a group, use the QM Plan review sheet on page 154 to analyze the Plan you chose. Be ready to share the rationale for your rating.

Part 1: Quality Statement What do we want to be? Brief purpose/mission statement describing the end goal of the HIV quality program to which all other activities are directed Assume an ideal world and ask yourselves, "What do we want to be for our patients and our community?“

Part 1: Quality Statement Tips Be brief Be visionary Include internal and external expectations Make references to external legislative requirements on quality management How did your groups rate Plan A quality statement? Plan B?

Part 2: Quality Improvement Infrastructure Tips Not more than 3-5 pages (not every detail is needed) Avoid naming individuals (just job functions) List internal and external stakeholders List linkages How did your groups rate Plan B? Plan A?

Part 3: Performance Measurement How will we assess progress? identify and quantify the critical aspects of care and services provided develop indicators and measure the progress of the QM program

Part 3: Performance Measurement Tips develop quality indicators, keeping in mind three main criteria: Relevance, Measurability and Improvability include the process for reviewing and updating the indicators (who/when/how) include a portfolio of process, outcome and satisfaction measures include strategies how to report and disseminate results and findings How did your groups rate Plan A? Plan B?

Part 4: Annual Quality Goals What are the priorities for the quality program? Quality goals are endpoints or conditions toward which the quality program will direct its efforts and resources Develop annual goals; the following three criteria can be helpful: Frequency: How many patients/clients received and how many did not receive the standard of care/services? Impact: What is the effect on patient health if they do not receive this care/services? Feasibility: Can something be done about this problem with the resources available?

Part 4: Annual Quality Goals Tips pick only a few measurable and realistic goals annually (not more than 5) use a broad range of goals establish thresholds at the beginning of the year for each goal

Part 5: Participation of Stakeholders How will staff, providers, consumer and other stakeholders be involved in the QM program? Engage internal and external stakeholders Communicate information about quality improvement activities Provide opportunities for learning about quality How did your groups rate Plan B? Plan A?

Part 5: Participation of Stakeholders Tips List internal and external stakeholders and their functions/responsibilities Include: Clinical providers Non-clinical providers Consumers Sub-grantees Representatives from agency, such as hospital, network, etc. List proposed training opportunities for staff and providers How did your groups rate Plan A? Plan B?

Part 6: Evaluation How will we evaluate our overall performance as a program? Evaluate infrastructure effectiveness Was the quality committee effective in its efforts to improve the quality of HIV care/services? Does the quality infrastructure require any changes to improve how quality improvement work gets done? Evaluate QI activities Were annual quality goals for quality improvement activities met? How effectively did you meet your goals? Did the implementation of the annual work plan go as planned? Did you meet established milestones? Were stakeholders informed about ongoing quality activities? Were staff and providers trained on QI methodologies and tools? Performance measures Were the measures appropriate to assess the clinical and non-clinical HIV care? Are the results in the expected range of performance?

How did your groups rate Plan B? Plan A? Part 6: Evaluation Tips Detail when and who is performing the evaluation Compare annual QI goals with year-end results Use findings to plan next year’s activities; learn and respond from past performance Routinely use organizational assessment tools How did your groups rate Plan B? Plan A?

The 10 QM Plan Rules Rule 1 - Size doesn’t matter; longer isn’t better Rule 2 - 80% planning, 20% writing (old software programming rule) Rule 3 - Don’t reinvent the wheel; use someone else’s plan to get started Rule 4 - Be inclusive, even it takes a little longer to get a working plan (Make it a plan of many “Parts”) Rule 5 - No plan is complete until it addresses consumer input

The 10 QM Plan Rules (cont.) Rule 6 - The perfect is the enemy of the good (A “perfect” plan was probably written by a consultant and nobody else has a clue what it says) Rule 7- Keep your goals focused (A few visionary annual goals are better than lots of useful ones) Rule 8 - Plans are only as good as their implementation Rule 9 - If you haven’t changed the plan throughout the year, you probably haven’t looked at it Rule 10- If you haven’t looked at the plan in 6 months, bring it to the next QC meeting

Placeholder to insert M22 Evaluating Training effectiveness

Evaluating Training Effectiveness Module 22 – Day 3 10:00 am – 10:45 am (45min) Evaluating Training Effectiveness

Framing Question How and what do you measure?

Kirkpatrick Model REACTION LEARNING BEHAVIOR RESULTS Talk about advantages and disadvantages of each level and difficulties

Bottom Line Question What do you do with the information ? Talk about it in relation to levels

Tools for Effective Presentation and Group Facilitation Module 25 – Day 3 11:30am – 12:30pm (60 min) Tools for Effective Presentation and Group Facilitation

Framing Question “In your experience, what types of things make for a great presentation?” Small group work come up with list – share one idea per table

What They Hear

What We Hear Volume Pitch Pace P a u s e Fillers

What They See

What We See Body Stance/Movement Gestures Facial Expressions Eye Contact Nervousness

Managing Nervousness Have a strategy for questions or exercises Control your breathing Be prepared Move around Practice, practice, practice

Managing Anxiety Plan for it Relaxation techniques Positive visualization Keep perspective Familiarize yourself with the training environment, audience, and content

HOW YOU SHOW IT! (Visual Aides)

They forget 70% of what we tell them. What is the 30% we want them to remember? Example of a picture ? © HOWICK ASSOCIATES, Inc. 2004

“A change in behavior due to experience or continued practice.” LEARNING... “A change in behavior due to experience or continued practice.” Another way to present beside bullets

Adult Learning I Hear I See I Do I Do Frequently and Receive Feedback I Forget I Remember I Understand I Perform Successfully © Howick Associates, Inc. 1997

Riding the Waves of Change MORALE change change change change TIME

Training is a blend of presentation and facilitation Group work is about accomplishing something (the WHAT), that is the leader’s and groups responsibility. How the group works together more effectively is about the process of the group, and that is the domain of the facilitator.

Facilitation—A Brief Review The word facilitate has its origin in the Latin word facilis, which means “easy.” Group work is potentially made easier by the contributions of a facilitator. Group work is about accomplishing something (the WHAT), that is the leader’s and groups responsibility. How the group works together more effectively is about the process of the group, and that is the domain of the facilitator.

Training Nightmares and Mishaps Module 24 - Day 3 9:15am—9:45am(30 min) Training Nightmares and Mishaps

Critical Incident Think of a time when you were training (or you were in a training) and something really went wrong. It was a genuine nightmare. It comes to mind easily and quickly because it was so vivid. Make sure to tell them that they will share the story

Make a few notes What happened, when and where did it happen? Why did it happen and who was involved? What was it about the situation that made it so painful for you? What did you do?

At your tables Pair-share your stories… then we will do a large group debrief Use store is to elicite what to do in these situations… try to get simple rules –KAC

Tips and Simple Rules Tips Pre-flight Prep (Page 175) Simple Rules

Purposes of Communication INFORM CLARIFY ENGAGE INVOLVE GO THROUGH THIS SLIDE AND LINK TO WHAT WE LKEARNED FROM THE NIGHTMARES AND HOW THIS MODEL COULD HELP Awareness Understanding Acceptance Commitment

Interactive Presenting Checking expectations & needs as you go… Parking lot Checking questions * open * closed Application questions Small group / pairs discussion OTHER TOOLS – COULD THEY HAVE HELPED WITH NIGHTMARES?

The Role of the “Traffic Cop” Specific words and phrases useful in directing traffic: Observing Clarifying Focusing Stimulating Balancing Summarizing Go to the book and review pages 40-41. How to use language to help you in your role.

Dealing with Difficult Behaviors

Intervention Strategies High-Level Intervention Medium-Level Intervention Low-Level Intervention Have them read and then ask if any questions??? Advice?? Non-Intervention Prevention

Tips Prevention Intervention (page 194) Prepare Practice intervention comments Solutions to common problems Structure of room Intervention (page 194) Broken Record Gloom and Doom Interrupter Rambler Side conversations

Closing “Don’t put people in learning experiences, put them in doing experiences. Achieving enables people to grow.” Peter Drucker

Lunch

Presentations and Role Play with Peers Module 27 – Day 3 (60 min) Presentations and Role Play with Peers

Objectives Present and facilitate discussion in a safe environment Record development goals based on the role play

What Are We Doing? Form groups of 3; if possible, each person presents on a different topic Go anywhere you like; take TOT Guide with you Identify a time keeper Each presenter has 5 minutes to present and 10 minutes to facilitate a discussion

What Are We Doing? During each presentation, the 2 “learners” select a difficult behavior; the presenter can practice intervention strategies After each presentation, the “learners” provide constructive feedback Make sure to manage time well Be back by 10:50…to document areas for personal improvement

Peer Presentation #1 QI Principles

Success is achieved through meeting the needs of those we serve. Is your facility ready?

Most problems are found in processes, not in people.

Do not reinvent the wheel – Learn from best practices.

Achieve continual improvement through small, incremental changes.

Actions are based upon accurate and measured data.

Infrastructure enhances systematic implementation of improvement activities.

Set Priorities and Communicate clearly

Peer Presentation #2 PDSA Cycle

How can we accelerate change and improvements in HIV programs?

Model for Improvement Improvement is about learning trial and error (scientific method) improvements require change, however not all changes are an improvement Measure your progress only data can tell you whether improvements are made integrate measurement into the daily routine Improvements thru continuous cycles of changes Plan-Do-Study-Act approach changes are initiated on a small scale to test them before implementation

Model for Improvement MFI model for Improvement How many have heard? Elegantly simple model that is useful … 3 questions plus the PDSA cycle Go over 3 questions and plan do study act ; pdsa

What are we trying to accomplish? Model for Improvement What are we trying to accomplish? How will we know that a change is an improvement? What change can we make that will result in improvement? Why this question is important … highly corelated with success of a team ..

that a change is an improvement? Model for Improvement What are we trying to accomplish? How will we know that a change is an improvement? What change can we make that will result in improvement? What is the second question … how will we know? Why is this important ….

What change can we make that will result in improvement? Model for Improvement What are we trying to accomplish? How will we know that a change is an improvement? What change can we make that will result in improvement? So what is the third question? The changes that you make should align with your aim and measures… Where do they come from ? IHI change packages… chanes with a pedigree… have a high degree of belief they willwork .. Have worked .. Ideas in the op doc.. Op Doc.. Each other..

The PDSA Cycle for Learning and Improvement Act Plan Objective Questions and predictions (why) Plan to carry out the cycle (who, what, where, when) What changes are to be made? Next cycle? Study Do Complete the analysis of the data Compare data to predictions Summarize what was learned Carry out the plan Document problems and unexpected observations Begin analysis of the data

Performance Measurement Peer Presentation #3 Performance Measurement

‘How to develop an indicator’

Balance between Performance Measurement and Quality Improvement Activities Although we focused in this tutorial on how to select effective indicators, the ultimate goal is to use performance data results to improve HIV care, balancing performance measurement and quality improvement. Be reminded that measuring the quality of HIV care alone is not quality improvement.

What is a quality indicator? A quality indicator is tool to measure specific aspects of care and services that are optimally linked to better health outcomes while being consistent with current professional knowledge and meeting client needs. In Tutorial 2, we defined quality as “the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge.” An indicator is a way of measuring whether care and services you provide as well as the activities you perform are linked to improved health outcomes for clients. An indicator takes one aspect of HIV care, and provides a way of assessing how often this specific aspect of care is properly provided. By defining what “properly provided” means (this is where the “current professional knowledge” comes in), indicators enable you to learn about the level of quality in the care your HIV program provides.

Dimensions of Quality Technical Quality Provider Perception of Quality of HIV Care Experience Quality Consumer Perception of Quality of HIV Care Leonard Berry, Texas A&M University, IHI conference 2001

What makes a good indicator? Relevance Does the indicator affect a lot of people or programs? Does the indicator have a great impact on the programs or patients/clients in your EMA, State, network or clinic? Measurability Can the indicator realistically and efficiently be measured given finite resources? Clearly, the universe of things that can be measured is vast. How do we begin to select a manageable number of areas to track? There are four main criteria to use in selecting sound indicators. The first two are: Relevance. Are you looking at something that matters to your program? And measurability. Can you actually measure this aspect of care, given the resources you have?

What makes a good indicator? (cont’d.) Accuracy Is the indicator based on accepted guidelines or developed through formal group-decision making methods? Improvability Can the performance rate associated with the indicator realistically be improved given the limitations of your services and population? The next two criteria are: Accuracy. How valid is this indicator? Does it really reflect “current professional knowledge?” Does it build on accepted guidelines for HIV care? If it deals with an aspect of care not yet covered by a guideline, has there been consensus by professionals and peers? Improvability: the ultimate goal is to improve the quality of care. As you select indicators, focus first on those that will help you improve. If you answer "no" to any of these questions, the indicator—while still relevant to patient care—is probably either too difficult to measure or less than critical to patient care. On the other hand, if you answer "yes" to all of the questions, you have most likely found a viable indicator that will give you the most benefit for your measurement resources.

Module 28 Going Forward Reporting

Evaluation, Closing, and Celebration Module 29 – Day 3 Evaluation, Closing, and Celebration 11:20am—12:00n(60 min) Evaluation Day 3

The way the course was delivered today was an effective way for me to learn. Strongly Disagree Agree Strongly Agree

I had sufficient opportunity to participate today. Strongly Disagree Agree Strongly Agree

Materials were useful during the day. Strongly Disagree Agree Strongly Agree

The agenda and content for today were logically organized. Strongly Disagree Agree Strongly Agree

Overall, I was satisfied with the session facilitator(s). Strongly Disagree Agree Strongly Agree

My knowledge and /or skills increased as a result of today. Strongly Disagree Agree Strongly Agree

The workshop had the right balance of lecture and interactive activities. Strongly Disagree Agree Strongly Agree

Overall, I was satisfied with today’s session. Strongly Disagree Agree Strongly Agree

Overall TOT Assessment Please reflect on your 3-day TOT experience to answer the following questions

How ready are you to plan and facilitate a QI workshop? Not Ready Mostly Ready Very Ready

How to you rate the effectiveness: TOT Nomination Process Not Effective Effective Very Effective

How to you rate the effectiveness: TOT Pre-Work Not Effective Effective Very Effective

How to you rate the effectiveness: TOT Session Not Effective Effective Very Effective

Overall, how satisfied were you with the TOT Program experience? Not Satisfied Satisfied Very Satisfied

Overall, how do you rate the TOT Guide? Not Satisfied Satisfied Very Satisfied

Overall, how satisfied were you with the TOT Faculty? Not Satisfied Satisfied Very Satisfied

Last minute things…. Leave your name badges and the remotes on your table Remember to take all your personal belongings

Thank You :-)