1. Evaluation of the New York State Collocation Program: Findings from the Implementation Study Eunju Lee, Rose Greene, and Bud LePage Center for Human Services Research School of Social Welfare, University at Albany and New York State Office of Children and Family Services

2. 2 Presentation Overview  Description of NYS Collocation Program  Evaluation Methodology  Findings from the Process Study  Next Step

3. New York State Collocation Program

4. 4 Description: Historical Background  RFP in Fall 2000 using TANF prevention funds  Support of new community-based services for vulnerable children and families  Focus on prevention  Collocation programs began in July 2001

5. 5 Description: Program Design  Place AOD specialist (CASAC) in local child welfare offices  Assist child welfare staff with cases involving parental alcohol and/or substance abuse  Early identification, joint CW/AOD assessments, joint family service planning, timely referrals, and case management services

6. 6 Description: Program Goals  Decreased foster care placements  Reduced repeat indicated child abuse or maltreatment reports of families  Reduced length of stay in out-of-home care  Increased number of family reunifications from foster care placements

7. 7 Description: Lead Agencies  9 program sites run by 5 agencies  Upstate:  Finger Lakes Addictions Counseling & Referral Agency, Inc. (FLACRA)  Otsego Chemical Dependencies Clinic  St Mary's Hospital  NYC:  Women in Need, Inc. (WIN)  Veritas Therapeutic Community, Inc.

8. 8 Description: Map of Program Sites

9. 9

10. 10 Description: Map of Program Sites

11. Other Studies

12. 12 Literature Review Delaware’s Title IV-E Waiver Project  Multidisciplinary Team Treatment Project  Substance abuse counselor located in DFS unit  Program in place 5 years  Reduced foster care days and costs  Source: Dillard, D. (March, 2002) Final Evaluation Report

13. 13 Literature Review Collocation of Mental Health and Health in MI  Randomized controlled trials studying patients with major depression receiving broad-based collaborative treatment by both PCPs and MHPs  Collocation of MHP and PCP practices in same building was strongly associated with increased interaction and collaboration  Source: Valenstein et al.(1999) Journal of Family Practice

14. 14 Literature Review NJ’s School-Based Youth Services Program  Provides range of services to children and adolescents in schools including mental health and substance abuse counseling, health care, employment preparation, crisis intervention, and summer programs  Improved educational aspirations and psycho-social attributes  Source: K. Hooper-Briar & H. Lawson (Eds.) (1996) Expanding Partnerships for Vulnerable Children, Youth and Families

15. Methodology

16. 16 Methodology: Logic Model

17. 17 Methodology: Overview  Funded by Children’s Bureau, DHHS in 2003  Three year evaluation project  Both implementation and outcome studies  Implementation study  7 Program Sites  Focus Group, Interviews and Reports  Outcome Study  1 NYC site 1 upstate site  Case Record review

18. 18 Methodology: Implementation vs. Outcome  Study the implementation at the system level  Identify factors for successful implementation and barriers to success  Provide insights on the design of the outcome study  Examine the effects on parents and children  Assess program effectiveness on substance abuse and child welfare outcomes  Identify factors that facilitate or hinder in achieving outcomes

19. 19 Methodology: Outcome Study  A quasi-experimental design using case record reviews Retrospective longitudinal design Follow the two groups of families in the CPS system over 1 year Treatment group -- families who came in contact with CPS after collocation Control group – families who came in contact with CPS before collocation

20. Findings from Implementation Study

21. 21 Implementation Study: Data Collection  State level interviews  Site visits (7 sites) Focus groups of child welfare workers Focus groups of child welfare supervisors Interviews with Program Coordinator Interviews with administrators of the treatment agency Interviews with child welfare administrators

22. 22 Implementation Study: Analytical Approach  Utilized evaluation framework to describe and assess program processes  Compared program operations to the logic model  Compared practices among the sites

23. 23 Implementation Findings: Design and Service Initiation  The program framework was identified in the RFP but was left up to the localities to design the program mechanics  Involving the right people in the program design has important implications  Funding uncertainties resulted in early start- up difficulties  Hiring the right person to be AOD Specialist was important  The newly hired AOD Specialists experienced a slow start

24. 24 Implementation Findings: Reaching the Target Population  Two Questions:  Are certain targeted groups not receiving services?  Are services delivered to individuals outside of the targeted group?  The crux of the program is the identification of the target population

25. 25 Implementation Findings: Reaching the Target Population  To reach the goal of increasing the identification of substance abusers presumes there are clients with AOD issues that are unknown to child welfare staff  The program is also designed to serve substance abusers known to the child welfare system

26. 26 Implementation Findings: Reaching the Target Population  Client identification occurred in four ways (each with their own limitations) Through the initial hotline call On the initial investigation Following the initial investigation Families in the system

27. 27 Implementation Findings: Reaching the Target Population  Not all targets are being served  There are individuals being served who are not part of the targeted group  This precludes obtaining some program goals (e.g., family reunification)  This also leads to outcomes in other areas

28. 28 Implementation Findings: Other issues in client identification  Must consider motivation of child welfare staff to refer clients.  Caseworkers and their supervisors are gatekeepers  What are the motivations?  Decreased workload  Evidence to indicate case  Moving client through the system  Helping families

29. 29 Implementation Findings: Client Assessment  Child welfare investigation provides “window of opportunity.”  “Good Cop/Bad Cop” approach  Value of home visits in assessment process

30. 30 Implementation Findings: Treatment Referral  Factors in deciding where clients are sent for treatment  Sponsoring treatment agency  Matching client to appropriate program  Purchased service agreement  Getting clients engaged in services  Monitoring compliance

31. 31 Implementation Findings: Follow Up and Other Services  How long AOD Specialist remains with the case differed by location  NYC was exclusively a program to identify clients and refer them to treatment  Upstate programs provided case management

32. 32 Implementation Findings: Relationships Between System  Collocation is a particular form of collaboration  This type of collaborative arrangement is not an equal partnership  Substance abuse is entering another system  Changes have resulted at the systems level  Changes have resulted at the worker level

33. 33 Implementation Findings: Effects of Collocation  Can collocation achieve the intended outcomes?  The potential effects can be determined by the population served and activities provided.  Intermediate (substance abuse) outcomes  Long term (child welfare) outcomes

34. 34 Implementation Findings: Lessons Learned  Involve front line workers in the design  Consider front line motivations  Provide structure for early operations  Collocation matters  Treatment agency involvement makes a difference

35. Next Step

36. 36 Next Step: Challenges  High expectations on positive outcomes from stakeholders  Selecting sites for outcome study: 1 upstate site and 1 NYC site  Drawing the sample for outcome study  Outcome measures  Access to case records  Developing Case Record Extraction Form

37. 37 Next Step: Study Sample Issues  Over-sampling of the cases served by AOD specialists  How to select Comparison Group  Sample Size (tentative N=400)  NYC and Upstate comparison

38. 38 Next Step: Outcome Measures  Rates of SA identification  Rates of treatment engagement  Rates of treatment completion  % of CPS re-reports over one year  % of placement after one year