Performance Monitoring : Thoughts, Lessons, and Other Practical Considerations

slide 2 Objectives  Identify key concepts  Discuss applications  Address questions and concerns

slide 3 Key Concepts  Performance monitoring  What you do  How well you do it  Do you accomplish something  Process, quality, capacity, outcomes  The window  Baselines and standards  Risk or case mix adjustment

slide 4 Performance Monitoring  Part of much larger cycle of program design and implementation  Performance - this is about definitions  Inputs, outputs, or the relationship between inputs and outputs?  Monitoring - This is about data collection and analysis  Important with respect to investment - are you getting something back?

slide 5 The Framework

slide 6 Process of care  Referral, intake, and assessment  Service planning, link to interventions  Reassessment, follow-up, case closure

slide 7 Quality of Care  Human resources  Physical plant and equipment  Practice protocols - evidence base  Supervision  Consumer feedback  Agency management around practice model fidelity

slide 8 Capacity  Enough trained workers  Enough office space  Enough funding  Enough information  Enough is defined by the relationship between process, quality, and outcomes

slide 9 Outcomes  Depends on the program and intervention  Well-being  Safety  Family provides stable nurturing  Education  Health  Behavioral health

slide 10 Process, Quality, and Outcomes  Highly interdependent  Quality depends on a process  Process is different than quality  Quality without outcomes is ‘inefficient’  Agencies invest in process, quality, and capacity

slide 11 The Window  Performance happens in time  Improvement is change in performance over time  Sampling in time is difficult but critical

slide 12 Clinical Experience in Time: (Each line represents the start and end of service within the window)

slide 13 Sampling  Inception  Process vs child  How much time do you have to observe the process?

slide 14 Baselines and Standards  Baseline is a measure taken prior to intervention  Standards of practice and performance  The usual as in standard practice  Fidelity or compliance  Standards are better suited to process and quality; baselines are better suited to outcomes

slide 15 Risk or Case Mix Adjustment  An important question when facing variation in performance: Is the variation a function or performance or the result of client differences  Children/families have different outcomes for reasons that are intrinsic to them  Baseline mortality rates differ by age  Adjustment for case mix refers to taking the intrinsic differences into account somehow when measuring outcomes

slide 16 Case Mix Adjustment Applied  Case mix adjustment makes more sense for outcomes, less so for process and quality  Process/quality standards apply to all children, given the process standard applies in the first instance (differential diagnosis)  Baselines for outcomes should be adjusted  Standards don’t work as well for outcomes because of the random component.

slide 17 Comments, Questions, Concerns Thank you!