1. CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley Applying Data for System Improvement: Probation Agency Staff Daniel Webster, MSW PhD Center for Social Services Research University of California, Berkeley UC Davis Resource Center for Family-Focused Practice Project Code 123FAM260 Sacramento, CA January 2013 The Performance Indicators Project at CSSR is supported by the California Department of Social Services and the Stuart Foundation Slides in this presentation from: Emily Putnam-Hornstein, PhD Data 101: Numbers, Graphs and More Numbers http://cssr.berkeley.edu/cwscmsreports/presentations/

2. CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley

3. Key Concepts: Longitudinal Administrative Data Analysis

4. CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley The Current Placement System* (Highly Simplified) Child In Child Out A bunch of stuff happens * adapted from Lyle, G. L., & Barker, M.A. (1998) Patterns & Spells: New approaches to conceptualizing children’s out of home placement experiences. Chicago: American Evaluation Association Annual Conference

5. CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley Data Skepticism

6. CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley Entry Cohorts Exit Cohorts Point in Time Data 3 Key Data Samples

7. CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley January 1, 2012January 1, 2013 How long do children stay in foster care? July 1, 2012 Child 1 Child 2 Child 3 Child 4 Child 5 Child 6 Child 7 Child 8 Child 9 Child 10

8. CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley The Cycle of Experiences Counterbalanced Indicators of System Performance Permanency Through Reunification, Adoption, or Guardianship Shorter Lengths Of Stay Stability Of Care Rate of Referrals/ Substantiated Referrals Home-Based Services vs. Out-of-Home Care Maintain Positive Attachments To Family, Friends, and Neighbors Use of Least Restrictive Form of Care Source: Usher, C.L., Wildfire, J.B., Gogan, H.C. & Brown, E.L. (2002). Measuring Outcomes in Child Welfare. Chapel Hill: Jordan Institute for Families, Reentry to Care

9. CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley 20112012 Jan. 1 Mar. 1 Dec. 31 Jul. 7 Jan. 1Dec. 31 Tracking an Entry Cohort for 1 Year

10. CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley What the !#@* is a Rolling Year ? 20102011 20122013 Jan. 1 Dec. 31Jan. 1 Last day of data prior to cut-off Sept. 30, 2012 ENTRY COHORT Oct. 1, 2010 – Sept. 30, 2011 Dec. 31 Last day of data prior to cut-off Dec. 31, 2012 Last day of data prior to cut-off March 31, 2013 ENTRY COHORT Jan. 1, 2011 – Dec. 31, 2011 April 1, 2011-March 31, 2012

11. CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley POP QUIZ !!

12. CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley Measures of Central Tendency Mean: the average value for a range of data Median: the value of the middle item when the data are arranged from smallest to largest Mode: the value that occurs most frequently within the data 12 4 15 63 7 9 4 174 4 7 9 12 15 17 63 7 = 9.7 = 9

13. CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley Exercise: Means & Medians

14. Computing a Percent Answers.com Dictionary: Rate A measure of a part with respect to a whole; a proportion: the mortality rate; a foster care entry rate. Raw Numbers (counts) # Reunified w/in 12m # Reunified (total) = 290 = 440 What Percentage of Children who were reunified in 2011 went home within 12 months of entering care?

15. Computing a Rate per 1,000 Answers.com Dictionary: Rate A measure of a part with respect to a whole; a proportion: the mortality rate; a foster care entry rate. Raw Numbers (counts) # Entered Care # Child Population = 1,333 = 363,376 What was the foster care entry rate in 2011? (i.e., how many children entered care out of all possible children?) Scales for a meaningful interpretation…

16. CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley Percent Change Time Period 1Time Period 2 10 children11 children

17. CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley Percent Change Time Period 1Time Period 2 10%12% % %

18. CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley Exercise: Percent Change Calculation Baseline Referral Rate (time period 1): Comparison Referral Rate (time period 2): Percent Change: 50.748.3-4.7% 12.010.8-10% Minor Differences due to Rounding…

19. CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley Disaggregation One of the most powerful ways to work with data… Disaggregation involves dismantling or separating out groups within a population to better understand the dynamics Useful for identifying critical issues that were previously undetected Aggregate Permanency Outcomes Race/Ethnicity Age County Placement Type

20. CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley 2000 July-December First Entries California (Child Welfare) Percent Exited to Permanency 72 Months From Entry 85%

21. CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley 2000 First Entries California (Child Welfare) Percent Exited to Permanency 72 Months From Entry 88% 79%

22. CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley 2000 First Entries California (Child Welfare) Percent Exited to Permanency 72 Months From Entry by Relative vs. Non-Relative Placement =94% =84% =75%

23. CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley Safety S1.1 S2.1 Permanency Composite 1: Reunification Composite 2: Adoption Composite 3: Long-Term Composite 4: Placement C1.1 C1.2 C1.3 C1.4 C2.1 C2.2 C2.3 C2.4 C2.5 C3.1 C3.2 C3.3 C4.1 C4.2 C4.3

24. CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley Probation Outcomes System Summary

25. CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley

26. CSSR.BERKELEY.EDU/UCB_CHILDWELFARE Needell, B., Webster, D., Armijo, M., Lee, S., Dawson, W., Magruder, J., Exel, M., Cuccaro-Alamin, S., Putnam-Hornstein, E., Williams, D., Yee, H., Hightower, L., Lou, C., Peng, C., King, B.,& Henry, C. (2013). Child Welfare Services Reports for California. Retrieved 1/25/2013, from University of California at Berkeley Center for Social Services Research website. URL: