1. Putting CQI Into Action Starting Early Together Allegheny County, PA (Pittsburgh) Sarah Thurston, Quality Assurance Team

2. Brief Background Allegheny County has received three system of care grants since 1998 –Through the grants, have served 0-25 yr olds in nine neighborhoods –First two grants have graduated from funding and have transitioned to other models (i.e. billing) Starting Early Together –Current grant is Starting Early Together, early childhood focus serving 0-6 yr olds

3. 1998 Grant 2001 Grant CQI plan built around values-based service delivery process 2005 Grant SET Pursued High Fidelity Wraparound service delivery model 2009 Discontinued credentialing for HiFi

5. Intermediate Outcomes (Years 2-4) Partner Community Level  Community Network Teams are fully developed and active  Program staff are skilled and well-trained  Children from diverse settings and racial/ethnic backgrounds are referred to and enrolled in local programs  Local programs enroll 25 children/year  Community Network Teams identify and develop local services/supports/resources  Service planning efforts engage all key partners and result in sustainable plans  Local programs develop and begin implementing sustainability plans  Participation in evaluation activities is strong Practice/System Level  Social marketing plan is implemented  Provider trainings are well-attended  Use of evidence-based practices increases  Interagency transition plans are in place  SOCI staff provide regular updates to key representatives from the PA Office of Child Development and Commission on Children and Families  SOCI staff actively participate in local early childhood collaborative meetings  SOCI staff actively participate in federal system of care training/TA opportunities  SOCI staff actively support local system integration initiatives IMPACT (Years 5&6) Partner Community Level  Caregiver/child/family relationships are strengthened and functioning improves  Family/consumer satisfaction with the system of care is high  Children/families receive appropriate and effective services, treatments and supports  Disparities in access to mental health services decline among racial minorities and children in foster care  Families are empowered and support each other  Local infrastructure for SOCI program is fully developed and self-sustaining Practice/System Level  Public awareness of early childhood mental health issues and effective mental health promotion strategies increase  Provider capacity to identify and treat/address early childhood mental health issues in an effective and culturally-competent manner increase  Family/provider awareness of formal/informal services and how to access them increase  Service/system integration and transition practices improve  Federal/state/local partnerships are developed MEAR CART & National Outcomes Study: YSS-F, MSSC National & Local Outcomes Study: CGSQ, PSI National Outcomes Study: MSSC, CCSP Local MIS Tracking of committee memberships & activities Family & provider focus groups Informal agency self-report

6. “Stuck” in the middle with you… CQI Process

7. So what next?

8. Intermediate Outcomes (Years 2-4) Partner Community Level  Community Network Teams are fully developed and active  Program staff are skilled and well-trained  Children from diverse settings and racial/ethnic backgrounds are referred to and enrolled in local programs  Local programs enroll 25 children/year  Community Network Teams identify and develop local services/supports/resources  Service planning efforts engage all key partners and result in sustainable plans  Local programs develop and begin implementing sustainability plans  Participation in evaluation activities is strong Practice/System Level  Social marketing plan is implemented  Provider trainings are well-attended  Use of evidence-based practices increases  Interagency transition plans are in place  SOCI staff provide regular updates to key representatives from the PA Office of Child Development and Commission on Children and Families  SOCI staff actively participate in local early childhood collaborative meetings  SOCI staff actively participate in federal system of care training/TA opportunities  SOCI staff actively support local system integration initiatives We already regularly view & discuss data from the local MIS How do we convert more referrals to enrollments?

9. Build new partnerships or utilize existing ones to accomplish data collection – you don’t have to do it all yourselves! Local academic institutions Agencies/providers Community groups

10. So we called in our university partner… And we built a plan.

11. ACTION: Tweaking data collection MIS DATA: # of Referrals, # of Enrollments PHASE 1: Focus groups and interviews RESULTS: Problems with communication ACTION: Spokesperson training MIS DATA: # of Referrals, # of Enrollments PHASE 2: Phone interviews RESULTS: Problems with # of contacts Shared with admin staff Shared with Team Leader staff Ongoing PHASE 2: Phone interviews with MIS data And so on, and so on…

12. Are staff implementing the process appropriately? Are we reaching families where they are?

13. SET Stoplight Report SET Referral Report

14. Build in regular and consistent opportunities for data sharing, feedback from stakeholder groups, and creation of action plans. Community Evaluation Team Meeting Operations Meeting Advisory Committee Meeting

15. REFERRAL REPORT

16. STOPLIGHT REPORT

17. Team Leaders use reports as a staff supervision tool Administrators use reports as program monitoring and systems discussion tool

18. IMPACT (Years 5&6) Partner Community Level  Caregiver/child/family relationships are strengthened and functioning improves  Family/consumer satisfaction with the system of care is high  Children/families receive appropriate and effective services, treatments and supports  Disparities in access to mental health services decline among racial minorities and children in foster care  Families are empowered and support each other  Local infrastructure for SOCI program is fully developed and self-sustaining Practice/System Level  Public awareness of early childhood mental health issues and effective mental health promotion strategies increase  Provider capacity to identify and treat/address early childhood mental health issues in an effective and culturally-competent manner increase  Family/provider awareness of formal/informal services and how to access them increase  Service/system integration and transition practices improve  Federal/state/local partnerships are developed CART & National Outcomes Study: YSS-F, MSSC Based on National Outcomes data, families were reporting a high level of satisfaction with services. But were the study tools capturing enough about family/SET staff interactions? Are families satisfied with SET staff??

19. So we partnered with an existing Allegheny County project of NAMI (National Alliance on Mental Illness) … And we built a plan. CART Consumer Action Response Team

20. Phone interviews with enrolled & recently graduated families RESULTS: Confusion about ‘graduation’ ACTION 1: Training for families about graduation Shared with admin & office staff And so on, and so on… Develop survey with CART staff ACTION 2: Each office wrote a plan of correction for any other issues Revise survey with CART staff JULY 2010: Phone interviews with enrolled & recently graduated families

21. Referrals to Enrollments? Referral Details? Service Delivery? Service Satisfaction? What is the SET experience?

22. as often as they will come. Short term = more energy, more time Long term = more efficient, less time Bring as many people to the table as possible…

23. Make your efforts serve multiple purposes… be the “Swiss Army Knife” of CQI / Evaluation!