CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley Safety and Permanence in Child Welfare Second Canadian Roundtable on Child Welfare Outcomes October 8-9, 2009 Montreal, Canada Barbara Needell, MSW, PhD Center for Social Services Research University of California at Berkeley The Performance Indicators Project is a collaboration of the California Department of Social Services and the University of California at Berkeley, and is supported by the California Department of Social Services and the Stuart Foundation

CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley counterbalanced indicators of system performance permanency through reunification, adoption, or guardianship length of stay stability of care rate of referrals/ substantiated referrals home-based services vs. out of home care positive attachments to family, friends, and neighbors use of least restrictive form of care Source: Usher, C.L., Wildfire, J.B., Gogan, H.C. & Brown, E.L. (2002). Measuring Outcomes in Child Welfare. Chapel Hill: Jordan Institute for Families, reentry to care what can we measure?

CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley three ways to measure entry cohorts exit cohorts point in time data

CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley the view matters… How long do children stay in foster care? January 1, 2008December 31, 2008July 1, 2008 Source: Aron Shlonsky, University of Toronto (formerly at CSSR)

CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley California Example: Age of Children in Foster Care (2008 entries, July 1 st 2008 caseload, 2008 exits)

CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley public data: putting it all out there pros: greater performance accountability community awareness and involvement, encourages public-private partnerships ability to track improvement over time, identify areas where programmatic adjustments are needed Province/Province and Province/National collaboration cons: o potential for misuse, misinterpretation, and misrepresentation o available to those with agendas or looking to create a sensational headline o misunderstood data can lead to the wrong policy decisions o “Torture numbers, and they’ll confess to anything” Gregg Easterbrook

CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley The California Experience University/Agency collaboration Publicly available reports since 1994, online since 2000 Nationally mandated measures (CFSR) State mandated measures (California Outcomes and Accountability System—AB636 law since 2001) Enhancements and additional measures Dynamic, user defined drill down and breakout functionality All tables refreshed quarterly Data over time, for California and each of the 58 counties Presentations, tools, etc.

CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley how’s it working for us? All those “cons” WILL happen State, county, and UCB are able to respond quickly and thoroughly to data abuse/number torturing Publicly available data for child welfare has become business as usual Most outcome measures are improving over time

CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley January 2004-July 2009 California CWS Outcomes System: AB636 Measures, % IMPROVEMENT (+) or (–) indicates direction of desired change Decline in Performance Improvement in Performance

CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley January 2004-July 2009 California CWS Outcomes System: Federal Measures, % IMPROVEMENT (+) or (–) indicates direction of desired change Decline in Performance Improvement in Performance ///110.4%

CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley policy "Our collaboration with the University of California, Berkeley on the Child Welfare Dynamic Report System allows the State of California to make data accessible for analysis by the general public, stakeholders, and policy-makers. The availability of this information permits us to make informed public policy to improve outcomes for children and youth in foster care." John Wagner Director California Department of Social Services

CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley legislation “As a county administrator, I am fully in support of public access to county level child welfare data for many reasons. One in particular that has been most effective for me in my role as a legislative advocate is to be able to cite data on the CSSR when arguing for or against a particular bill impacting child welfare. The fact that we can drill down to a particular population the bill addresses and help inform the debate on both policy and fiscal impacts results in data driven legislation; data that all the stakeholders can view and understand leads to consensus on the facts. This is a major breakthrough in the legislative process that before had to operate on inaccessible data or data embedded in paper reports at the local level that were a challenge to gather and analyze.” Kathy Watkins Legislative Program Manager/Legislation and Research Unit San Bernardino County Human Services System

CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley training “The publicly available data provided by CSSR/CDSS is invaluable for the training system. All of the statewide and regional training systems can integrate actual data into their curricula - this brings training alive for the participants, and reinforces practice that is informed by outcomes.” Barrett Johnson Director Child Welfare In-Service Training Project California Social Work Education Center (CalSWEC) “As a trainer in child welfare, the accessible public data base provides current information to support many of the topics we offer in our Regional Training Academy. We refer to it frequently and are grateful for the gifted folks who created this resource and made it open to those of us who are peripheral but important to supporting good child welfare practice.” Liz Quinnett Program Coordinator Public Child Welfare Training Academy (PCWTA)

CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley program evaluation “I use the data from the UCB/CDSS site almost every week in my job as principal analyst in the Program Evaluation & Research unit. The data is invaluable for analysis of trends over time, answering specific questions posed by Children & Family Services managers, and for routine program monitoring reports. I frequently respond to requests for ad-hoc analyses by working directly with the staff person and showing them how to create reports from the website - following the principle of ‘teach them how to fish’. I also refer staff from community based agencies who ask for data for grant proposals to the website and often show them how to extract data. Tom Clancy Program Evaluation & Research Alameda County Social Services Agency

CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley irresistible information "We commend Dr. Needell and the work of her staff at UCB/CDSS in creating and refining the child welfare services data posted on the public website. From an administrative perspective, the information has been instrumental in identifying trends, program adjustments and training needs. Being available to the public, the data has supported our efforts in program transparency as well as serving to educate the community on client needs and agency services." Ken Jensen Deputy Director Santa Barbara County Dept. of Social Services

CENTER FOR SOCIAL SERVICES RESEARCH School of Social Welfare, UC Berkeley CSSR.BERKELEY.EDU/UCB_CHILDWELFARE Needell, B., Webster, D., Armijo, M., Lee, S., Dawson, W., Magruder, J., Exel, M., Glasser, T., Williams, D., Zimmerman, K., Simon, V., Putnam-Hornstein, E., Frerer, K., Cuccaro-Alamin, S., Winn, A., Lou, C., & Peng, C. (2009). Child Welfare Services Reports for California. Retrieved July 1, 2009, from University of California at Berkeley Center for Social Services Research website. URL: Barbara Needell Presentation Developed by Emily Putnam-Hornstein and Christine Wei-Mien Lou