1. Putting the Tools to Use: One Hospital’s Experiences Donna Farley, PhD – RAND Ellen Robinson, PT ATC – Harborview Medical Center

2. Format for This Discussion Goals of the discussion Goals of the discussion – Highlight how groups of tools apply at different steps of an improvement process – Offer opportunity for audience questions as each group of tools is discussed Three groups of tools to be addressed Three groups of tools to be addressed – Work with data for the PSIs and IQIs – Diagnose issues and develop strategies – Implement improvement plans 2

3. Structure of the Toolkit Introduction and Roadmap A. Readiness to Change B. Applying QIs to the Hospital Data C. Identifying Priorities for Quality Improvement D. Implementation Methods E. Monitoring Progress and Sustainability of Improvements F. Return-on-Investment Analysis G. Existing Quality Improvement Resources 3

4. Working with PSIs and IQIs Introduction and Roadmap A. Readiness to Change B. Applying QIs to the Hospital Data C. Identifying Priorities for Quality Improvement D. Implementation Methods E. Monitoring Progress and Sustainability of Improvements F. Return-on-Investment Analysis G. Existing Quality Improvement Resources 4

5. Tools for Working With the PSIs and IQIs A.1 Fact sheets on the PSIs and IQIs A.2 Template Powerpoint presentations on the Quality Indicators for Board or staff B.1 Applying PSIs and IQIs to hospital data B.2 Examples of AHRQ software outputs B.3 Spreadsheets and presentations of hospital rates for PSIs and IQIs B.4 Documentation and coding guidance B.5 Assessing hospital rates using trends and benchmarks 5

6. Harborview’s Project Goals Internal Reporting: Internal Reporting: – Utilize the AHRQ software to identify cases of possible preventable harm – Standardize case referral across all teams in the hospital External Reporting: External Reporting: – Understand and validate publicly reported rates of hospital performance 6

7. Readiness for Change – Medical Director - previous director of QI Dept – Leadership support and directive for project – The Board was “on board” – Challenges identified: information dissemination about quality and patient safety to staff at all levels of the organization 7

8. Applying your Data Input data challenges Input data challenges – Format billing system export into a file format that can run through the AHRQ software Output data challenges Output data challenges – Validate rates against external source to ensure capture of all cases – Software versions (currently 4.3) and format (SAS vs. Windows) 8

9. Sharing your Data 9 Surgical Council, Medical Executive Board, Critical Care Council, Hospital Board, Clinical Documentation Specialists, Health Information Management Surgical Council, Medical Executive Board, Critical Care Council, Hospital Board, Clinical Documentation Specialists, Health Information Management – What are the PSIs? Why do we care? – Current performance/UHC ranking – How are we going to review cases and expectations from the medical teams – Possible opportunities for improvement

10. Documentation and Coding Specifications for each PSI and common challenges for “false positives” Specifications for each PSI and common challenges for “false positives” Recognize limitations of administrative data, but also recognize the potential Recognize limitations of administrative data, but also recognize the potential Partnerships with clinical documentation programs and coding department are critical to success of the project Partnerships with clinical documentation programs and coding department are critical to success of the project 10

11. QUESTIONS?

12. Diagnose Issues and Develop Strategies Introduction and Roadmap A.Readiness to Change B.Applying QIs to the Hospital Data C.Identifying Priorities for Quality Improvement D.Implementation Methods E.Monitoring Progress and Sustainability of Improvements F.Return-on-Investment Analysis G.Existing Quality Improvement Resources 12

13. Tools to Assess Readiness, Priorities, Strategies A.3 Getting ready for change self-assessment – Readiness for quality improvement – Readiness to work with the QIs C.1 Prioritization matrix C.2 Example of completed matrix D.1 Improvement methods overview D.2 Project charter D.3 Examples of effective PSI improvements D.4 Best practices for PSI improvements D.5 Gap Analysis F.1 Return-on-investment analysis 13

14. Tools to Assess Readiness, Priorities, Strategies A.3 Getting ready for change self-assessment – Readiness for quality improvement – Readiness to work with the QIs C.1 Prioritization matrix C.2 Example of completed matrix D.1 Improvement methods overview D.2 Project charter D.3 Examples of effective PSI improvements D.4 Best practices for PSI improvements D.5 Gap Analysis F.1 Return-on-investment analysis 14

15. An important decision-support tool An important decision-support tool Considers factors that influence choice of improvement priorities Considers factors that influence choice of improvement priorities – Benchmarks – Costs – Strategic alignment – Regulation – Barriers to implementation Factors Addressed in the Prioritization Matrix 15

16. A useful tool for assessments A useful tool for assessments – Planning phase – estimate potential effects on hospital finances – Post-implementation – estimate actual effects on hospital finances The tool provides instructions for performing an ROI and an example The tool provides instructions for performing an ROI and an example Role of a Return-on- Investment Analysis (ROI) 16

17. Prioritization Matrix 17 Tool allows you to compare to a like group for benchmarking, identify areas that are highest impact, assess barriers.

18. Return on Investment Currently partnering with our Decision Support/Finance teams to identify a meaningful reporting metric Currently partnering with our Decision Support/Finance teams to identify a meaningful reporting metric “Costs” of PSI events vary in the literature so makes it difficult to have a “target” “Costs” of PSI events vary in the literature so makes it difficult to have a “target” 18

19. QUESTIONS?

20. Implement Improvement Plans Introduction and Roadmap A.Readiness to Change B.Applying QIs to the Hospital Data C.Identifying Priorities for Quality Improvement D.Implementation Methods E.Monitoring Progress and Sustainability of Improvements F.Return-on-Investment Analysis G.Existing Quality Improvement Resources 20

21. D.6 Implementation planning D.7 Implementation measurement D.8 Project evaluation and debriefing E.1 Monitoring progress for sustainable improvement F.1 Return-on-investment analysis Tools to Help Make Improvements Happen 21

22. D.6 Implementation planning D.7 Implementation measurement D.8 Project evaluation and debriefing E.1 Monitoring progress for sustainable improvement F.1 Return-on-investment analysis Tools to Help Make Improvements Happen 22

23. For use after completion of an improvement initiative For use after completion of an improvement initiative Focus on sustainability Focus on sustainability Guidance for monitoring system Guidance for monitoring system – A limited set of effective measures – Schedule for regular reporting – Report formats to communicate clearly – Procedures to act on problems found – Periodic assessment of sustainability Monitoring for Sustainable Improvement 23

24. A Project Management “Toolkit” A Project Management “Toolkit” Useful tools for clinicians who may not have as much experience with project management Useful tools for clinicians who may not have as much experience with project management Selected Best Practices Selected Best Practices Assisted with development of “task forces” in our selected PSI areas Assisted with development of “task forces” in our selected PSI areas Kept teams focused and on track during early stages of the implementation Kept teams focused and on track during early stages of the implementation 24

25. 25 Monthly PSI Case Review Monthly PSI Case Review Monthly Data Feed AHRQ QI Analysis Coding or Documentation issue? Documentation Coding Review Update coding Agree? (Wrong code or exclusion criteria code missing) Real Event? Service Review No Event No Coding Issue No QI Concerns QI Concerns

26. Monitoring and Sustainability 26 Case analysis and Tracking of outcomes

27. Ongoing Reporting 27 Web-based reporting on Harborview Intranet

28. Lessons Learned Lessons Learned Validate, Validate, Validate Validate, Validate, Validate Understand details of the specifications and be able to apply to your population Understand details of the specifications and be able to apply to your population Leadership backing for project importance Leadership backing for project importance Presentations to clinical staff should begin with real case examples Presentations to clinical staff should begin with real case examples Coding lead liaison is critical Coding lead liaison is critical 28

29. Harborview MC Outcomes Standardized Case Review - 2011 Standardized Case Review - 2011 PSI 3,6,7,9,11,12,15 PSI 3,6,7,9,11,12,15 – 45% no quality concerns – 18% teams identified possible QI system opportunities – 25% related to documentation/coding – 12% “flawed metric” PSI 11 flagged in a planned two stage surgery PSI 11 flagged in a planned two stage surgery PSI 9 flag related to intra-operative bleeding PSI 9 flag related to intra-operative bleeding 29

30. Hospital Challenges AHRQ Software is one tool to assist with identification of improvement opportunities AHRQ Software is one tool to assist with identification of improvement opportunities As Health Care IT becomes more sophisticated, hospitals have more data As Health Care IT becomes more sophisticated, hospitals have more data Challenge ourselves to be creative and identify clinical systems to provide additional sources for events Challenge ourselves to be creative and identify clinical systems to provide additional sources for events How do we find the “false negatives?” How do we find the “false negatives?” 30

31. VTE Events from Exams vs. PSI 12 Jan to Dec 2011: HAC - VTE Events Jan to Dec 2011: HAC - VTE Events 70% also identified by AHRQ PSI 12 70% also identified by AHRQ PSI 12 30% flagged by diagnostic systems 30% flagged by diagnostic systems – Not identified in administrative data (not coded, not in top 24 diagnosis, or “POA” = Y) – Did not have an operative procedure, so not included in the denominator for PSI 12 Without our internal clinical event search tool, these cases would be missed QI opportunities. 31

32. Allowed our hospital to translate from rate- based tracking to one that provides an opportunity for real changes for patients Allowed our hospital to translate from rate- based tracking to one that provides an opportunity for real changes for patients Hospitals can use the QIs to analyze “gaps” in current clinical care Hospitals can use the QIs to analyze “gaps” in current clinical care Toolkit can assist with “what to do” about areas of opportunity you identify Toolkit can assist with “what to do” about areas of opportunity you identify Download the toolkit at : http://www.ahrq.gov/qual/qitoolkit. http://www.ahrq.gov/qual/qitoolkit 32 AHRQ QI Toolkit

33. QUESTIONS?

34. Getting More Information Where can I find the Toolkit? http://www.ahrq.gov/qual/qitoolkit Where can I find the Toolkit? http://www.ahrq.gov/qual/qitoolkit http://www.ahrq.gov/qual/qitoolkit Can other people hear this presentation later? Yes, a video of this Webinar will be available on the Toolkit page. Can other people hear this presentation later? Yes, a video of this Webinar will be available on the Toolkit page. Will I be able to learn more about the Toolkit? Yes, audio interviews about specific tools will be added to the Toolkit page. Will I be able to learn more about the Toolkit? Yes, audio interviews about specific tools will be added to the Toolkit page.