1. California’s Child Welfare Outcomes & Accountability System: Using Performance Measures to Encourage Improvement
Barbara Needell, MSW, PhD
Center for Social Services Research
University of California at Berkeley
The Performance Indicators Project is a collaboration of the
California Department of Social Services and the University of California at Berkeley,
and is supported by the
California Department of Social Services and the Stuart Foundation 1

2. Child Welfare is a System
Rate of Referrals/
Substantiated Referrals
Home-Based
Services vs.
Out of Home
Care
Reentry to Care
Permanency
Through
Reunification,
Adoption, or
Guardianship
Counterbalanced
Indicators of
System
Performance
Use of Least
Restrictive
Form of Care
Rule #1 applies to your Toolkit everybit as much as it does our System in CA—even the best measures can be misleading if they are not used as a part of the entire array of measures that capture performance from beginning to end.
Example: reunification rates and their interdependence with both entry rates and reentry rates
Length
of Stay
Positive Attachments
to Family,
Friends, and
Neighbors
Stability
of Care
SOURCE: Usher, C.L., Wildfire, J.B., Gogan, H.C. & Brown, E.L. (2002). Measuring Outcomes in Child Welfare.
Chapel Hill:  Jordan Institute for Families,

3. Background…
California’s Child Welfare System Improvement and Accountability Act
Became law (AB 636) in 2001
Went into effect in January 2004 with publication of first quarterly report
Began with county self assessments and System Improvement Plans (SIPS) that identified key challenges and strengths, based on public data
Currently includes all federal measures (17 in CFSR2), augmented by state measures that capture important aspects of performance
Participation rates (referrals, substantiations, entries, in care)
Sibling placements
Key process measures (e.g., child visits, time to investigation)
Least restrictive placements
Measures added over time
We may not like the CFSR2 measures, but they are current mandates with potential penalties attached.

4. CDSS / CSSR Collaboration
Longstanding Interagency Agreement
Funding from CDSS* and Stuart Foundation
Quarterly Data Reports for CA and counties
Dynamic excel documents with data, charts, etc.
Data Publicly Available: cssr.berkeley.edu/ucb_childwelfare
Site organized around federal measures (with extensions)
Full data refresh quarterly
Ad hoc data tabulations (filtering capacity)
Composite viewer/composite planner
* A portion of CDSS funding comes from AOC for work especially relevant to the courts

5. *Updated with Q4_08 Data

6. Example… Measure C1.3: Reunification within 12 months (entry cohort)
Federal Base
Of all children entering foster care for the first time in the 6-month period who remained in foster care for 8 days or longer, what percent were discharged from foster care to reunification in less than 12 months from the date of latest removal from home?
Site Extensions
Agency, Days in Care, Time Period Views, Episode Count
Age, Ethnicity, Gender, Placement Type, Removal Reason
Exit Status at 3m, 6m, 12m…120m
Subgroup filtering, Count/Percent, Excel Export
Strength based approach to CFSR2 measures—we use them but then enhance/expand

9. *Dates available for Federal (and all) reports despite the fact that this measure was added with CFSR2

10. *Federal Report

11. *STSG Report

12. *Exit status dates available for 2007 entry cohort

13. **Exit status dates available for 1998 entry cohort

14. **Filtering options

15. *STSG Report (%)

16. **Shows count view

17. January 2004-July 2009 California CWS Outcomes System: AB636 Measures, % IMPROVEMENT (+) or (–) indicates direction of desired change
Updated with Q4_08 Data
We believe that publicly available data has contributed to our improvement
Decline in Performance Improvement in Performance

18. January 2004-July 2009 California CWS Outcomes System: Federal Measures, % IMPROVEMENT (+) or (–) indicates direction of desired change
110.4%
Updated with Q4_08 Data
Decline in Performance Improvement in Performance 18

19. Federal Standard/Goal
July 2009 California CWS Outcomes System: Performance Relative to Federal Standard/Goal
Federal Standard/Goal
100%
Updated with Q4_08 Data 19

20. Public Data: Putting it All Out There
PROS:
Greater performance accountability
Community awareness and involvement, encourages public-private partnerships
Ability to track improvement over time, identify areas where programmatic adjustments are needed
- County/County and County/State collaboration
CONS:
Potential for misuse, misinterpretation, and misrepresentation
Available to those with agendas or looking to create a sensational headline
Misunderstood data can lead to the wrong policy decisions
“Torture numbers, and they’ll confess to anything”
Gregg Easterbrook
The Pros and Cons of Data made public:
Pros
Greater performance accountability
(Numbers are there for all to see)
Community awareness and involvement, encourages public-private partnerships
(Funders can be shown “results”)
Allows state and counties to track improvements over time and to identify areas where programmatic adjustments are needed
(Counties can see the big performance picture)
County/County and County/State collaboration
(County A is doing well in X by doing Z. Maybe that would work in our county as well…)
Cons:
Potential for misuse, misinterpretation, and misrepresentation
(e.g., inappropriate county comparisons, treating small insignificant differences as meaningful, taking statistics out of context)
Available to those with agendas or looking to create a sensational headline
(Accurate statistics can be used to bolster inaccurate arguments (or create attention-grabbing headlines))
“Torture numbers, and they’ll confess to anything” (Gregg Easterbrook) 20

21. Child Welfare County Data Profiles: Court Data Reports
Customized excel reports for judges designed by AOC and CSSR staff
Data restricted to court dependent children when possible
County/State and County side by side table and graph comparisons
% change between timeframes
Presents data from child welfare and court sources (filings)
Provides links to source reports on CWS/CMS reports website to allow users to further explore data
Most data is publicly available—arranged in a way that will be most useful for the courts
Ability to “drill down” to subsets may be necessary to understand need and target resources

22. CSSR.BERKELEY.EDU/UCB_CHILDWELFARE
Barbara Needell
CSSR.BERKELEY.EDU/UCB_CHILDWELFARE
Needell, B., Webster, D., Armijo, M., Lee, S., Dawson, W., Magruder, J., Exel, M., Glasser, T., Williams, D., Zimmerman, K., Simon, V., Putnam-Hornstein, E., Frerer, K., Cuccaro-Alamin, S., Winn, A., Lou, C., & Peng, C. (2009). Child Welfare Services Reports for California. Retrieved April 1, 2009, from University of California at Berkeley Center for Social Services Research website. URL: <
Presentation Developed by Emily Putnam-Hornstein and Christine Wei-Mien Lou 22