CHILD WELFARE SYSTEM IMPROVEMENT: PARTNERING FOR CHILD WELFARE SYSTEM IMPROVEMENT: Implementing Federal and State Mandates for Reform XXth ISPCAN International Congress on Child Abuse & Neglect Nagoya, Japan 9/16/2014 Daniel Webster, PhD California Child Welfare Indicators Project University of California, Berkeley Alicia Sandoval Child Welfare Data Analysis Bureau California Department of Social Services Dave McDowell, PhD Child Welfare Outcomes and Accountability Bureau Barbara Needell, PhD CCWIP is supported by the California Department of Social Services and the Stuart Foundation

BACKGROUND The Challenge A Coordinated Reform Effort Addressing Child and Family Service Review (CFSR) improvements Large state with many independent jurisdictions Supporting State DSS with data expertise A Coordinated Reform Effort California Outcomes and Accountability System (Assembly Bill 636) Aligns with federal mandates, with key augmentations Shift from compliance to continuous quality improvement – outcome driven and collaborative California Child Welfare Indicators Project Public reporting of administrative outcome data Developing tools and system outcome measures Technical assistance Public child welfare reform—particularly for a state as large and varied as California is a daunting prospect. This presentation describes ongoing efforts and promising results from a collaboration of staff from CDSS, and UCB as well as the Stuart Foundation [who provide financial support] who have partnered together to approach this important task.  The collaboration has been instrumental in the implementation and continued oversight of the CA outcomes and accountability system which we will briefly describe.  Presentation focus is on the dynamic public website and some of the tools available to help users monitor system improvement. Statewide outcome data show some indication of improvements in almost all systemic outcomes since the advent of these coordinated efforts. This success underscores the benefits of this productive partnership, the value of public access to child welfare data, and can hopefully serve as a model to be adopted by other jurisdictions. An overview of Child Welfare in California… Most populous state California has a population that is roughly the size of Canada (total population is over 36 million, child population is over 9.6 million ) State supervised, county administered child welfare system 58 diverse counties (ranging from the very rural to very urban) AB636 passed in 2001 Individualized county self-improvement plans Quarterly data reports made available to state and county officials Longitudinal data publicly available Public-private collaboration to support key reforms QTRLY DIST OF OUTCOME DTA Includes national standards, but also draws heavily on previous work done by CWDA and UCB using entry cohort measures Retains key process measures (e.g., child visits, time to investigation) STATE CTY PARTNERSHIP Shifts focus from process measured compliance to outcome based review system, but requires linking outcomes to related processes. Requires county collaboration with community partners. Promotes sharing of promising practices among counties

Partnering Roles Collaboration California Department of Social Services California Child Welfare Indicators Project, UC Berkeley Children’s Data Network, USC Children’s Research Center Statewide Data Committee Stuart Foundation CDSS & UC Berkeley share responsibility for child welfare reporting, analysis, and official publications http://cssr.berkeley.edu/ucb_childwelfare/ http://www.dss.cahwnet.gov/research/default.htm

A PORTAL FOR PUBLIC ADMINISTRATIVE DATA http://cssr.berkeley.edu/ucb_childwelfare/

Monitoring system Reform

Making Analysis Transparent Publicly –Available Data Uniform across state, 24/7 availability Aggregate data, with drill down capability Pluses and minuses of public data Continuous Quality Improvement Local staff must be able to question/challenge analysis, and get an explanation A learning not punitive process Continually test data & communicate with users Ongoing feedback loop Pros and Cons of public data Pros Greater performance accountability (Numbers are there for all to see) Community awareness and involvement, encourages public-private partnerships (Funders can be shown “results”) Allows state and counties to track improvements over time and to identify areas where programmatic adjustments are needed (Counties can see the big performance picture) County/County and County/State collaboration (County A is doing well in X by doing Z. Maybe that would work in our county as well…) Cons: Potential for misuse, misinterpretation, and misrepresentation (e.g., inappropriate county comparisons, treating small insignificant differences as meaningful, taking statistics out of context) Available to those with agendas or looking to create a sensational headline (Accurate statistics can be used to bolster inaccurate arguments (or create attention-grabbing headlines)) “Torture numbers, and they’ll confess to anything”

and Meeting New Challenges MOVING FORWARD and Meeting New Challenges An Evolving Landscape with CFSR Round 3 New measures, risk adjustment, improvement standards Rethink and re-tool quarterly report Inform child welfare agency staff Remain responsive & supportive to the field’s needs Nimble adaptability of dynamic website Create new tools Develop new outcome metrics Partner with CDN in new research areas Turning Data into Knowledge Promote familiarity of data throughout agencies Support the link from outcome to practice Educate current and emerging professionals (Graduate courses, Trainings, Data Fellows Program) Creating new tools to respond to user needs --website tiled display Nimble adaptability of site: --if new state programs or foci appear, the ability to quickly change parameters or add measures is a strength of the project Adapting analyses, reporting, and monitoring system reform under an evolving federal review process --new fed measures, children in congregate care 1+ years Educating new and existing professionals in data-driven system improvement --EPH graduate course; in-service trainings on tools; partnering with CH on advanced analytics, --Data Fellows Program with partners at USC, UW (and potentially UMinn.)

Questions? dwebster@berkeley.edu