Moving Forward with Wisconsin’s Community Response Program Prepared for Wisconsin Children’s Trust Fund Erik Bakken, Michele Dickinson, Austin Frerick, Kate Grannemann, Marianne Griffin, and Ye Wang Michele

1: How do current practices of CRP sites influence potential for evaluation? 2: Which states have programs similar to Wisconsin’s CRP? 3: How should CTF evaluate CRP to inform statewide implementation? Research Questions Michele

Risk Factors for Perpetration Community High Poverty Residential Instability High Unemployment Poor Social Connections Family Social Isolation Family Disorganization Violence in the Home Parenting Stress Poor Parent-Child Relationships Negative Interactions Individual Substance Abuse Mental Health Issues Lack of Parenting Skills History of Maltreatment Young Age Low Education Single Parent Low Income Austin

CRP Service Areas Domestic Violence Services Employment/ Job Assistance Family Medical Needs Financial Support Household or Family Needs Housing Substance Abuse Services Parent Education & Child Development Mental Health Services Austin

Referral for Alleged Child Maltreatment Screened Out Referred to CRP No Further Action Screened In Austin

Referral for Alleged Child Maltreatment Screened Out Referred to CRP No Further Action Screened In Traditional CPS Investigation CPS Case Closed Case Opened for Ongoing Services Austin

Austin Referred to CRP Screened Out No Further Action Referral for Alleged Child Maltreatment Screened Out Referred to CRP No Further Action Screened in Traditional CPS Investigation CPS Case Closed Case Opened for Ongoing Services Alternative Response Case Closed Austin

1: How do current practices of CRP sites influence potential for evaluation? 2: Which states have programs similar to Wisconsin’s CRP? 3: How should CTF evaluate CRP to inform statewide implementation? Research Questions Kate

Areas of Variation #1: Point of Referral Kate GROUP 1 Screened Out Referral for Alleged Child Maltreatment Screened Out GROUP 1 No Further Action Screened in Traditional CPS Investigation CPS Case Closed GROUP 2 Case Opened for Ongoing Services Alternative Response Case Closed GROUP 3 Kate

Involvement in Referral Process Areas of Variation #2: Involvement in Referral Process CPS Staff & CRP Staff Community Response Program Only CPS Staff Community Response Program Kate

Service Type & Provision Areas of Variation #3: Service Type & Provision Service Area Site A Site B Site C Site D Domestic violence services R Employment/job assistance DO/R Family medical needs Financial support DO Household or family needs Housing Mental health services Parent education and child development Substance abuse services Kate R: Referral to non-profit organization or other agency DO: Directly offered

Consistency in Defining Success ● Short Term: Completion of family’s service goals ● Long Term: Reduction in Re-Referrals to CPS Kate

1: How do current practices of CRP sites influence potential for evaluation? 2: Which states have programs similar to Wisconsin’s CRP? 3: How should CTF evaluate CRP to inform statewide implementation? Research Questions Michele

All Referrals Accepted Referrals Screened Alabama, Alaska, California, Delaware, District of Columbia, Idaho, Iowa , Kansas, Maryland, New York, North Carolina, North Dakota, Ohio, South Dakota, Wyoming   Total: 15 Arizona, Arkansas , Colorado, Connecticut, Florida, Georgia, Hawaii, Illinois, Indiana*, Kentucky, Louisiana, Maine, Massachusetts, Michigan, Minnesota, Mississippi, Missouri, Montana, Nebraska, Nevada, New Hampshire*, New Jersey, New Mexico, Oklahoma, Oregon, Pennsylvania, Rhode Island, South Carolina, Tennessee*, Texas, Utah, Vermont, Virginia, Washington, West Virginia, Wisconsin    Total: 36 Michele * Denotes insufficient data

Alternative and Traditional Response All Referrals Accepted Referrals Screened  Delaware District of Columbia Maryland New York North Carolina Ohio Wyoming   Total: 7  Arizona Colorado Connecticut Kentucky Louisiana Minnesota Nevada Oklahoma Texas Vermont Virginia Washington Wisconsin Total: 13  Alabama Alaska  California Idaho Iowa Kansas North Dakota South Dakota Total: 8 Arkansas Florida Georgia Hawaii Illinois Indiana* Maine Massachusetts Michigan Mississippi Missouri Montana Nebraska New Hampshire* New Jersey New Mexico Oregon Pennsylvania Rhode Island South Carolina Tennessee* Utah West Virginia Total: 23 Michele * Denotes insufficient data

Total: 6 Total: 15 Total: 1 Total: 7 Total: 2 Total: 8   Alternative and Traditional Response Traditional Response All Referrals Accepted Referrals Screened No Formal Response for Screened-Out Referrals Delaware District of Columbia Maryland New York Ohio Wyoming         Total: 6 Arizona Louisiana Oklahoma Vermont Virginia Washington Alabama Alaska Idaho Kansas North Dakota South Dakota     Arkansas Florida Hawaii Maine Massachusetts Michigan Mississippi Montana Nebraska New Hampshire* New Mexico Oregon Rhode Island Utah West Virginia  Total: 15 Formal Response for Screened-Out Referrals North Carolina Total: 1 Colorado Connecticut Kentucky Minnesota Nevada Texas Wisconsin Total: 7 California Iowa Total: 2 Georgia Illinois Indiana* Missouri New Jersey Pennsylvania South Carolina Tennessee* Total: 8 Michele * Denotes insufficient data

2011 2014 Michele

1: How do current practices of CRP sites influence potential for evaluation? 2: Which states have programs similar to Wisconsin’s CRP? 3: How should CTF evaluate CRP to inform statewide implementation? Research Questions Michele

Recommendations Randomized Controlled Trial Cost-Benefit Analysis Kate Introduces

Observational Quasi-Experimental Randomization   Observational Quasi-Experimental Randomization Dosage County Design Overview Observation of differences in outcomes for CRP participants as compared to non-participants The measurement of treatment received and the level of problems families had prior to service The measurement of differences in outcomes in CRP and non-CRP counties Random assignment before consent that creates two similar groups to measure differences between treatment and non-treatment Austin

Internal Threats to Validity   Observational Quasi-Experimental Randomization Dosage County Design Overview Observation of differences in outcomes for CRP participants as compared to non-participants The measurement of treatment received and the level of problems families had prior to service The measurement of differences in outcomes in CRP and non-CRP counties Random assignment before consent that creates two similar groups to measure differences between treatment and non-treatment Internal Threats to Validity History bias Maturation bias Omitted variable bias Selection bias Hawthorne effects History Attrition Austin

Internal Threats to Validity   Observational Quasi-Experimental Randomization Dosage County Design Overview Observation of differences in outcomes for CRP participants as compared to non-participants The measurement of treatment received and the level of problems families had prior to service The measurement of differences in outcomes in CRP and non-CRP counties Random assignment before consent that creates two similar groups to measure differences between treatment and non-treatment Internal Threats to Validity History bias Maturation bias Omitted variable bias Selection bias Hawthorne effects Attrition Cost $ $$ $$$ Austin

Internal Threats to Validity   Observational Quasi-Experimental Randomization Dosage County Design Overview Observation of differences in outcomes for CRP participants as compared to non-participants The measurement of treatment received and the level of problems families had prior to service The measurement of differences in outcomes in CRP and non-CRP counties Random assignment before consent that creates two similar groups to measure differences between treatment and non-treatment Internal Threats to Validity History bias Maturation bias Omitted variable bias Selection bias Hawthorne effects Attrition Cost $ $$ $$$ Strengths Limited change required at agency level and buy-in is typically easier to achieve Demonstrates whether various levels of program participation result in different outcomes for families Provides a non-participant comparison group and shows differences in effects Allows for causal inferences regarding program effects because it provides a non-participant comparison group Austin

Recommendations Randomized Controlled Trial Cost-Benefit Analysis Michele

Project GAIN Cost-Savings Analysis Societal (non-program) savings Estimated Program Savings Societal (non-program) savings Total Savings Annual Savings $62,000- 108,000 $679,000- 694,000 $741,000- 802,000 Kate

Cost-Benefit Analysis Data Needs CPS administration costs CPS and CRP staff salaries and benefits Re-referral rates among CRP participants and non-participants Substantiated re-referral rates among CRP participants and non-participants Out-of-home placements rates among CRP participants and non-participants Estimate of time spent by CPS workers on re-referred cases, re-referred cases that are substantiated, and re-referred cases that result in out-of-home placement Out-of-home placement costs CRP costs Michele

For further information Recommendations Randomized Controlled Trial Cost-Benefit Analysis For further information Contact Michele Dickinson at (608) 516-6992 or michele.dickinson@wisconsin.gov Visit this website after June 1, 2014 for a PDF copy of our report: www.lafollette.wisc.edu/publications/workshops.html Michele

Extra

Variable Lower Bound Upper Bound Savings from reduced re-referrals (total dollars per case) Low Estimate $1,496 $1,278 High Estimate $1,538 $2,012 Number of screened-out cases (total number) 6,661 CRP take-up rate (percent participating) 39% 54 % Case reduction rate (percent reduction) 5% 15% Program costs (total annual dollars) $380,000 $430,000 Estimated lifetime societal costs (health care, mental, productivity, criminal justice, etc.)- endangerment only (present value of costs at 3% discount rate) $97,952 $210,012 Extra