1. Neal D. Kohatsu, MD, MPH Medical Director May 17, 2012 Evaluation and Quality Improvement

2. 2 What is Evaluation? “The systematic investigation of the merit, worth, or significance of an ‘object’.” Michael Scriven “…the systematic assessment of the operation and/or outcomes of a program or policy, compared to a set of explicit or implicit standards…” Carol Weiss “A systematic way to determine the “value” of a program, program components, or activity.” cited by Ann Powers

3. 3 “Research seeks to prove, evaluation seeks to improve…” M.Q. Patton

4. CDC Evaluation Framework 4

5. 5 Steps in Program Evaluation Step 1: Engage Stakeholders Step 2: Describe the Program Step 3: Focus the Evaluation Design Step 4: Gather Credible Evidence Step 5: Justify Conclusion Step 6: Ensure Use and Share Lessons Learned MMWR; Vol. 48; 9/17/99

6. 6 The Four Standards Utility: Who needs the information and what information do they need? Feasibility: How much money, time, and effort can we put into this? Propriety: What steps need to be taken for the evaluation to be ethical? Accuracy: What design will lead to accurate information? MMWR; Vol 48; 9/17/99

7. Quality Improvement 7

8. Shewart/Deming Cycle 8

9. Rapid-Cycle QI Accelerates system change Smaller change cycles and samples Greater focus on improvement (vs. measurement) 9

10. Approach to Rapid Cycle QI Determine specific aim Define measurement Specify intervention 10

11. Conclusions Evaluation and QI are related. Must adapt to a system context. Foster a learning environment. Encourage collaboration. Focus on outcomes. 11