1. Organizational Learning for System Improvement

2. The Philadelphia Department of Human Services mission is to provide and promote safety, permanency, and well-being for children and youth at risk of abuse, neglect and delinquency

4. Division of PMA started in 2009 Mission: The division of Performance Management and Accountability will support system improvement by monitoring and evaluating the efficiency, effectiveness, and availability of our services both internal and external Five Pillars: DIM, QI, PREP, PBC, and Research

5. Case File Review Quality Visitation Review Quality Service Review Child Fatality Review ChildStat

6. The Quality Service Review (QSR) is a practice improvement approach designed to assess current outcomes and system performance by gathering information directly from families, children and service team members

7. Mid 1990s: The first QSR protocol developed Other states that use the QSR process Alabama Utah Wisconsin Fall of 2009: Pennsylvania and Philadelphia management observe Utahs QSR process January 2010: PAs OCYF adopts the QSR process and selects Philadelphia to participate in the first QSR pilot process June 2010: Philadelphia begins internal QSRs

8. Occurs six times a year Small sample of randomly selected cases Stratified by service area Congregate Care Children with Special Medical Needs Adoption General Foster Care (Foster and Kinship Care) Treatment Foster Care In-Home Services Array Older Youth (SIL/TLP)

9. Review teams Brief review of the case file Initial interview with DHS worker and supervisor Interviews with stakeholders and families Focus child/youth and biological family Foster parents and group home parents Private provider workers Attorneys Mental health, educational, and medical personnel Focus Groups

10. Traditional Review Were visits timely? Is the childs IEP in the file? Was there a referral to a mental health service? Quality Service Review Are children currently safe? Is the child progressing well academically? Is the child functioning well emotionally?

11. Traditional Review Is there a current plan in the file? Did the parents sign the plan? Is there a written assessment? Quality Service Review Is the plan likely to lead to permanence? Do the parents feel that the plan is their own? Does the assessment accurately identify underlying needs?

12. Pennsylvania QSR Protocol 9 Child/Youth and Family Indicators 11 Practice Performance Indicators Scoring Process 6 – Optimal 5 – Substantial Acceptable Range 4 – Fair 3 – Marginal 2 – Poor Unacceptable Range 1 – Adverse

13. Case specific feedback Final interview with DHS worker and supervisor Written Case Story Debriefing Aggregate system-level feedback Findings Present Meeting Findings Present Report

14. Aggregate scores from all 12 cases Comparison of acceptable and unacceptable scores with all QSR cases Content analysis Case Story recommendations Focus Groups Trends and recommendations

15. Demographic information Indicator ratings Child/Family Practice Performance Comparison with overall QSR ratings Case Stories Recommendations

16. All QSR reviewers DHS management Private provider representatives Leaders from the courts Child advocate and parent attorney offices Representatives from the mental health, educational, and medical systems

18. Safety Physical health Engagement of child/youth and substitute caregiver

19. Increase the use of Family Finding and Family Group Decision Making to support family engagement and culturally appropriate practice Use of specific teamings at key case intervals (e.g. Permanency Action Teamings) to improve planning, permanency, and teaming indicators

20. Suggested improvements Submitted to the Commissioner and her cabinet for approval Assigned to a responsible person Quality Improvement tracking system Used to track recommendations from all QI reviews Recommendations entered into a database Monthly updates recorded to track individual implementation Quarterly reports published to track overall implementation

21. This process allows the child, family and team members to be seen and heard rather than just read about. This allows for a better understanding of actual service delivery and needs. -DHS Worker I like that the QSR takes more than just the case record into account. By speaking to all parties a person is able to get a better understanding of the case. - DHS Supervisor

22. I like being heard and feeling that providers are recognized for the service that is being provided. I appreciated the reviewers coming out to our facility making it convenient for me to participate. The reviewers did a great job making me comfortable to share my honest opinions. – Provider Worker I think this process of review is very important to make improvements and give better service. –Medical Personnel

23. Interested in observing the QSR process? Contact Brian Clapier (Brian.Clapier@phila.gov) Pennsylvania QSR Protocol http://www.pacwcbt.pitt.edu/Resources/PA%20Q SR%20Protocol%20Version%201%200.pdf http://www.pacwcbt.pitt.edu/Resources/PA%20Q SR%20Protocol%20Version%201%200.pdf

24. QUESTIONS Contact Information: Susan.Kinnevy@phila.gov Brian.Clapier@phila.gov Allison.Thompson@phila.gov Cynthia.Brown@phila.gov