First 5 Sonoma County Triple P Implementation & Evaluation January 31, 2012 In its 2011-2015 Strategic Plan, First 5 Sonoma County has committed more than $2.8 million to Triple P: Training Materials Clinical consultation Peer support Services In addition, our Commission has extended this plan to 2020 and intends to continue support for countywide implementation of Triple P.

Sonoma County Implementation Goal: Train and sustain 420 providers Comparable annual outcomes Decrease of 165 cases of child maltreatment Decrease of 58 out-of-home placements Decrease of 14 suspicious child injuries We asked California Institute for Mental Health--CIMH--to help us plan implementation of Triple P in Sonoma County. CIMH was formed by California County mental health directors to support the implementation of EBPs in California. Our plan was and is to create a countywide network of Triple P providers who can duplicate the efforts and outcomes achieved in the CDC study in South Carolina.   CIMH projected that, we will need 420 providers, in all levels of Triple P, from Level 2-5 to have the same penetration in the population that was achieved in SC. CIMH also projected that when we have this ongoing number of trained providers, we can expect a comparable reduction in the number of child abuse cases in Sonoma County—approximately 165 fewer per year--and a comparable reduction in out of home placements and suspicious emergency room visits for child injuries. Unfortunately—nothing is that simple—because many factors determine whether a child is removed from the home, for example. However, we expect to have a significant impact on these community indicators in our county.

Triple P Provider Network Status Began trainings in April 2010 Current more than 300 providers trained Levels 2-4 (Level 5 in April) 25% annual attrition rate 24 participating agencies, plus individuals Our initial effort has been to train local providers in Triple P, so they can integrate the practice into the work they already do with families. Participating agencies sign an MOU First 5, agreeing that we will provide the training and support free, and the providers will use Triple P as their parent education tool, and then, twice a year, will report data on the children and families they serve So far we have trained about 300 providers (many of them in 2 or 3 levels of Triple P). Currently, about 24 agencies have had staff trained, some in multiple levels. We are planning for a 25% attrition rate and expect to provide ongoing training and support through 2020 to maintain the needed level of trained providers in the community.

Evidence-Based Program Delivered with fidelity to the program model Evaluation of Triple P Evidence-Based Program Delivered with fidelity to the program model Proven outcomes are achieved Because Triple P is an EBP, evaluation is simplified. We don’t have to replicate expensive studies to evaluate our program. We only have to assure the EBP is delivered with fidelity to the model.   Thus, If Triple P is delivered as it was designed, we can expect to achieve the same level of outcomes shown in the earlier studies.

Program Performance “Dashboard” Reports Demographics Service delivery Measuring Fidelity Program Performance “Dashboard” Reports Demographics Service delivery Client improvement How do we measure fidelity? CIMH develops dashboard reports for many EBPs. These reports work like the dashboard of a car--they tell us how well the program is running. Our trained providers give us demographic information about their clients, as well as service delivery information, such as the number of sessions, the number of dropouts, etc. In addition, in the therapeutic levels--Levels 4 & 5--we ask them to do pre- and post intervention testing, using validated evaluation tools—the Eyberg Child Behavior Inventory and the Parenting Scale—to track improvement in the children and parents.

Initial Evaluation of Triple P Level 4/5 Improvement Measures Eyberg Child Behavior Inventory (ECBI) pre- and post-treatment results: 33.3% improvement in frequency of child behavior problems 65.5% improvement in parent’s perception that the child’s behavior is problematic Averages for both scores dropped from above the clinical cutoff to well below Parenting Scale 23.3% improvement in parenting and disciplinary styles As you can see, both measures show strong results. The ECBI shows 33% improvement in the frequency of child problem behavior and 65.5% improvement in the parent’s perception of the child’s behavior as problematic. We are particularly excited about that 65.5% change in parent perception about the child’s behavior. This improvement is huge. We believe that it indicates an increase in parent understanding of child development and behavior appropriate to the child’s age. When the parent understands child development, his or her response to formerly “problematic” behavior is more appropriate. In addition, both scores move from levels in the clinical range down to the normal range of child behavior and parenting success.   The parenting scale is showing 23.3 % improvement in parenting and disciplinary styles that frequently lead to children’s disruptive behavior problems. All of these scores indicate that our program is succeeding.

Initial Evaluation of Triple P Data show differences in level of change in ECBI Intensity Scale scores (frequency of problem behaviors) between Caucasian and Latino clients. ECBI scores showed a higher rate of improvement for Latino clients than for Caucasian clients—37.4%:22.6%.* *Only 13 Caucasian children compared to 39 Latino children. In addition, we can also observe the data across populations to determine the applicability of the program in diverse cultures. What we see is that Triple P works well with at-risk Latino families. ECBI scores show a higher rate of improvement for Latino clients than for Caucasians 37.4% to 22.6%. However, we believe this is preliminary data because the number of Caucasian clients is much lower. Latino clients are clearly benefiting, we just don’t have enough data on Caucasian clients to say definitively that they aren’t benefiting as much.

Initial Evaluation of Triple P The Check-Engine Light Is Blinking: Entry rate may be low Dropout rate may be high Number of sessions is high in specific agencies Working with agencies to resolve these issues The dashboard does show us areas where improvement could be made: Entry rate may be low Dropout rate may be high Number of sessions is high in specific agencies (Investigation has shown that practitioners are reporting total months of client involvement rather than just Triple P interventions.) But these are resolvable issues that can be improved by working with the agencies involved.

Initial Evaluation of Triple P Conclusions: Outcomes are strong and clinically relevant Triple P is effective with at-risk Latino families Most providers are delivering Triple P with fidelity to the model On track to attain our desired outcomes The results tell us that: Outcomes strong and clinically relevant Effective with Latino families at highest risk—a population targeted in our strategic plan Being delivered appropriately We are on track to attain our desired outcomes once we achieve countywide implementation. We know that Triple P has been proven around the world. These results tell us that Triple P also works effectively with Sonoma County children and families.