CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis Ed Byrnes, Ph.D. Eastern Washington University Michael Lawson, M.S. University of California at Davis

Evaluation Questions 1.What is the CPS recidivism rate for Birth & Beyond families in comparison to similar families who were involved with CPS? 2.How does the time to onset of CPS recidivism differ between Birth & Beyond families and similar families who were involved with CPS?

Evaluation Design Quasi Experimental External Comparison Group Fixed Follow Up Periods Data Sources –B & B Database –Sacramento County CPS Database Between Groups Comparability –Multiple Points of Comparison Outcomes –CPS Recidivism and Onset of Recidivism Client Predictors of Outcomes –Demographic Characteristics –Assessment Results

Birth and Beyond Group 384 B & B Families All Had A Prior CPS Referral All Had At Least One B & B Home Visit All Had A First Home Visit On Or Before July 11, 2006 For One Year Follow Up

CPS Comparison Group 327 Families NOT B & B Clients All Had A First CPS Referral Between January 1, 1999 and July 11, 2006 Sample Trimming –Children’s Ages –Most Severe CPS Allegation –Children’s Language and Race Stratified Random Sampling –Most Severe CPS Allegation –Children’s Gender, Language and Race

Between Groups Comparability No Significant Differences in –Children’s Race –Children’s Primary Language –Children’s Gender Very Unlikely to Influence Results Children’s Ages Differed Significantly –CPS Comparison Group Had More Neonates –CPS Comparison Group Was Then Older –Children’s Age and Group Membership Correlated at.25, a Weak Relationship –Significance An Artifact of Sample Size of 711 Families If Children’s Ages Have Any Influence on Results It Is Quite Small

Between Groups Comparability

CPS Recidivism Data From CPS Database All Referrals With A Definitive Finding –Substantiated, Inconclusive and Unfounded –Reflect Contact With The CPS System One Year Follow Up Period –From First Home Visit for B & B Group –From First CPS Referral for Comparison Group –Contrast Voluntary Additional Services with Usual CPS Processing

CPS Recidivism Of the 384 B & B families, 140 (36%) had a new CPS referral during the one year follow up period Of the 327 CPS Comparison Group families 159 (49%) had a new CPS referral during the one year follow up period This difference was statistically significant B & B participation versus regular CPS services was weakly correlated with CPS Recidivism (Phi =.12) Participating in B & B home visiting influenced CPS recidivism in the desired direction

CPS Recidivism

CPS Recidivism Onset Data From CPS Database Same Criteria As CPS Recidivism One Year Follow Up Period Survival Analysis Time to First CPS Referral –From First Home Visit for B & B Group –From First CPS Referral for Comparison Group

CPS Recidivism Onset Average time elapsed from the beginning of services to their first CPS referral during the one year follow up period –For B & B families was 279 days (sd = 6.5) –For CPS Comparison Group families was 256 days (sd = 7.45) This difference was statistically significant Relationship between group membership and time to the onset of CPS recidivism was moderate-to-strong –Magnitudes of the test statistics Participating in B & B home visiting influenced the time to the onset of CPS recidivism in the desired direction

CPS Recidivism Onset

B & B and CPS Comparison Groups have similar patterns of cumulative CPS recidivism rates through the first 150 days After 150 days the cumulative CPS recidivism curves diverge The relationship between participating in B & B home visiting and a delayed onset of CPS recidivism is more beneficial as time progresses

Predicting CPS Outcomes Additional Evaluation Questions –What demographic, service and assessment characteristics of cases are strongly related to having a new CPS referral within 150 days of the first home visit? –What demographic, service and assessment characteristics of cases are strongly related to having a new CPS referral within one year of the first home visit? Members of the CPS comparison group never engaged in B & B services they could not be included in these analyses

Predicting CPS Outcomes Logistic Regression Primer –Predict the occurrence of an event Risk and Protection –Can be quantified using Logistic Regression Odds Ratios –Value of 1 means no relationship between a factor and an event –Value greater than 1 means a risk factor for an event –Value less than 1 means a protective factor against an event

Predicting CPS Outcomes Outcome Variables –New CPS Referral Within 150 Days –New CPS Referral Within 1 Year –Same Definitions As Recidivism and Onset Predictor Variables –Demographics –Services –Assessment

Predicting CPS Outcomes Model Building Process Two Stages –Model by Variable Category for Each Outcome –Final Model for Each Outcome Criteria For Final Model Inclusion –Statistical Significance –Odds Ratio Greater Than 2 Evaluating Models –Goodness of Fit, Improved Case Classification, Outliers In The Solution v. Over-identification –Sample Size –Explained Variance: How Much Variability In The Outcome Can Be Attributed To Variables In The Model – 20% or higher is acceptable

Predicting CPS Outcomes Predicting 150 Day CPS Recidivism No Service Variables Met Final Model Criteria Predictors At Intake –Being a first time parent –Being pregnant –Being under the age of 25 –Being a member of the Black race –Being a member of the White race –Being a member of the Hispanic race –Meeting the criteria for clinical depression –Having a high total score on the APPI (indicating more skillful parenting) –Having a low score on the MSSI (indicating a low level of social support)

Predicting CPS Outcomes Predicting 150 Day CPS Recidivism 85 Cases With Complete Data 36% Explained Variance Met Other Model Quality Criteria Risk Factors –Pregnant at the Time of Intake –Being Black –Having Low Levels of Social Support Protective Factor –Being Under Age 25

Predicting CPS Outcomes

Predicting 1 Year CPS Recidivism No Service Variables Met Final Model Criteria Predictors At Intake –Being fluent in English –Being pregnant –Having a low total score on the APPI (indicating less skillful parenting) –Experiencing severe domestic violence Predictors From Follow Up Assessment –Having a high score on the CAGE questionnaire (indicating a higher likelihood of alcohol dependence) –Experiencing severe domestic violence

Predicting CPS Outcomes Predicting 1 Year CPS Recidivism 93 Cases With Complete Data 25% Explained Variance Met Other Model Quality Criteria Risk Factors –Pregnant at the Time of Intake –Having a Higher Final CAGE Questionnaire Score –Having Low Parenting Skills at Intake No Protective Factors

Predicting CPS Outcomes

Dynamic Risk Assessment Results Are Not Very Stable Over Time –Pretest – Posttest Correlations Are Weak To Modest Risk and Protective Factors That B & B Clients Experience Are Dynamic Rather Than Static –B & B Home Visitors Must View Assessment As An Ongoing Process

Dynamic Risk Chronbach's Alpha and Test-Retest Reliability Internal ConsistencyTest-Retest Reliability pre post r AAPI MSSI CES-D

CPS Recidivism Results B & B participation was associated with lower aggregate CPS recidivism rates B & B participation was associated with a delayed onset of CPS recidivism Given the broad ecologies of child maltreatment, these results can be viewed as promising

Discussion Questions What challenges have you encountered in conducting quasi experimental evaluation research, and how have you addressed these? Since California has a county administered child welfare system what efforts can we make to encourage uniformity of data for greater generalizability of results?

Discussion Questions What are the trade offs between using matched (case control) designs versus designs that rely on inclusion and exclusion algorithms for comparison groups? Since California and it's counties, like so many other states and locales, are facing deep budget shortfalls, how can we continue to promote evidence based practice as a priority?

Dissemination Plan Incorporate feedback about our study from this symposium into a final manuscript. Submit the manuscript for publication. Peer reviewed journals being considered include: –Children and Youth Services Review; –Research on Social Work Practice.

Follow Up Contact Ed Byrnes, Ph.D. (509) Michael Lawson, M.S.