1. Camden Coalition of Healthcare Providers Evaluating

2. Camden Coalition of Healthcare Providers Agenda: 1.Why is evaluating difficult? 2.Developing an evaluation framework 3.The evaluation timeline

3. Camden Coalition of Healthcare Providers §1 Why is evaluating difficult?

4. Camden Coalition of Healthcare Providers having data

5. Camden Coalition of Healthcare Providers patients w/ complexities = complex intervention

6. Camden Coalition of Healthcare Providers finding an appropriate comparison group ~ ~

7. Camden Coalition of Healthcare Providers regression to the mean

8. Camden Coalition of Healthcare Providers regression to the mean

9. Camden Coalition of Healthcare Providers 12 1 2 3 4 5 6 7 8 9 10 11 it takes time!

10. Camden Coalition of Healthcare Providers §2 Developing an evaluation framework

11. Camden Coalition of Healthcare Providers

14. Process Outcomes

15. Camden Coalition of Healthcare Providers Customized Data System Solutions

16. Camden Coalition of Healthcare Providers

17. Dashboards

18. Camden Coalition of Healthcare Providers Actions Taken Weekly scorecard to measure success and failure Daily operations dashboard to identify opportunities Weekly root cause deep dives by management team Problem The stated goal of our intervention was to see our patients once a week in person. Too many failures were occurring and our relationship with patients was diluting Results Reduced median days since last engagement Dramatically reduced process variability & outliers WEEK 1WEEK 10 168 DAYS 38 DAYS Days Since Last In-Person Engagement

19. Camden Coalition of Healthcare Providers TimeNMinimumMaximumMeanStandard Deviation Pre1862102.91.43 Post1860111.31.77 Number of 6x month readmissions by enrollment date

20. Camden Coalition of Healthcare Providers §3 Choosing an appropriate evaluation timeline

21. Camden Coalition of Healthcare Providers Planning & Data Analysis Piloting and Early Evaluation More Robust Evaluation (through Randomization) Scaling Program Timeline

22. Camden Coalition of Healthcare Providers 80% Medical Studies

23. Camden Coalition of Healthcare Providers 20% Healthcare Delivery Studies

24. Camden Coalition of Healthcare Providers Key Issues Sample Size Number of treatment arms Randomizing before or after consent? Data (what’s collected administratively? -available for the control group? When? Timing – not too early and not too late Time, expertise, and funding to do it right Why? Clear, credible results on causal effects Helpful in attracting sustainable funding and scaling a program

25. Camden Coalition of Healthcare Providers Standard of Care Key Outcomes: reduced re-hospitalizations and ED visits in 12 month period following discharge CCHP’s Care Management RCTCCHP’s Care Management RCT n = 138 / 800