1. How does Cross-System Collaboration Between the Child Welfare and Mental Health Systems Influence Children’s Mental Health Service Use and Service Effectiveness? Findings From a Two-Phase, Mixed Methods Study The Leadership Symposia on Evidence-Based Practice in Human Services January 30, 2009 San Diego, California Kathy Lemon Osterling Ph.D., MSW Assistant Professor School of Social Work San Jose State University kathy.lemon-osterling@sjsu.edu

2. Study Contributors Research and Curriculum:  Alice M. Hines, Ph.D. Principal Investigator  Peter Allen Lee, Ph.D. Co-Investigator  Kathy Lemon Osterling, Ph.D. Co-Investigator  Marty Tweed, MSW, Curriculum Consultant Santa Clara County Advisory Committee:  Judi Boring, formerly from Santa Clara County DFCS  Judy Bushey, County consultant to CA DSS  Amando Cablas, Ph.D., Santa Clara Valley Health & Hospital System, Community Health Services  Doug Klinkerman, Santa Clara Valley Health & Hospital System, Information Systems

3. Purpose of Workshop Describe factors that help or hinder cross-system collaboration between public child welfare and mental health systems Describe the influence of children’s mental health service use on family reunification and family maintenance Discuss the potential role of cross-system collaboration in improving mental health service use and effectiveness for children involved in both systems Discuss how study findings can be used to promote evidence-based/evidence-informed practice

4. Background: Why is Cross-System Collaboration Important? 1) High need for mental health services among children involved with the child welfare system: Experiences of trauma, maltreatment, being removed from caregivers 1 50%-80% have a mental health diagnosis vs. 25% of the general child population 2

5. Background: Why is Cross-System Collaboration Important? 2) Mental health services for the child may improve family reunification outcomes When children are experiencing mental health problems, reunification may be seen by professionals overseeing the case as having the potential to worsen children’s mental health symptoms 3 Research suggests that children’s emotional problems and externalizing behavioral problems are linked with a reduced likelihood of reunification, and an increased likelihood of reentry into out-of-home placement 3,4

6. Background: Why is Cross-System Collaboration Important? 3) Both systems share the same client (the child) and have the same overall goal of improving child well-being Mental health interventions are among the most commonly used service by children involved in the child welfare system 5

7. Background: Why is Cross-System Collaboration Important? 4) Strong cross-system collaboration may improve mental health service delivery and effectiveness of services Effective communication between systems improves assessment and service planning 6 Coordinated case plans ensure that both systems are working toward the same goals Family reunification timelines (ASFA) create a need for timely access to mental health services for children

8. Purpose of Study 1) Examine the influence of public mental health services on family reunification and family maintenance outcomes among children involved with the child welfare system (Quantitative) 2) Explore factors that impede or enhance cross- system collaboration between the child welfare and mental health systems (Qualitative)

9. Literature Review: Collaboration General definition of collaboration: Partnerships (i.e. relationships) that promote mutually beneficial goals, that often cannot be achieved when individuals act alone.

10. Literature Review: What are Critical Practice Areas in Collaboration? The Ten Bridges 7 Screening and assessment Engagement and retention Services Building community supports Values Budgets and program sustainability Information systems and outcomes Joint accountability and shared outcomes Training and staff development Links to other agencies

11. Literature Review: Theoretical Framework Collaboration occurs at multiple system levels 8 Mezzo Level: Differing professional orientations and overlapping, but distinct system structures and goals Macro Level: Organizational support in the form of resources, funding and planning processes to guide collaborative efforts Micro Level: Characteristics & interpersonal processes that affect establishment of trust between individuals

12. Literature Review: Theoretical Framework Collaboration is a developmental process 9

13. Literature Review: Previous Research One study using data from NSCAW found that increased coordination (defined as “concrete linkages” such as co-location of services, joint training, formal committees, shared office space, & coordinated service provision) between child welfare and mental health services was associated with: 1) Stronger associations between service need and service use and 2) A reduction in racial/ethnic disparities in service use 10

14. Literature Review: Previous Research However, another study using data from 24 counties in Tennessee found that increases in service coordination were actually linked to decreases in service quality (i.e. comprehensiveness, availability, continuity and responsiveness) 11 Possible interpretations: diffusion of responsibility, and a poorly structured collaboration (service coordination teams not directly working with families)

15. Literature Review: Previous Research The quality and structure of the collaboration are important Wraparound and System of Care models of service delivery are linked with positive outcomes (improved functioning of child and family and less restrictive living situations) 12 However, most children involved in child welfare system receive outpatient mental health counseling

16. Qualitative Methods: Research Questions Phase I: What factors impede or enhance collaboration between the child welfare and mental health systems? Phase 2: Prior to and during early implementation of the Mental Health Services Act, what factors impede or facilitate collaborative practice between the child welfare and mental health systems, with a particular focus on transition age youth?

17. Qualitative Methods: Research Design, Sampling Procedures & Sample Research Design: Case study Sampling Procedures: Combination of random sampling and convenience sampling Sample: Over the 2 phases: Total of 15 interview participants and 25 interviews (10 participants were interviewed twice) Interview participants included 6 child welfare, and 6 mental health program managers or supervisors and 3 managers from community-based organizations Total of 6 focus groups, 3 with child welfare line workers, and 3 with mental health line workers

18. Qualitative Methods: Data Collection, Data Analysis, Reliability & Validity Data Collection: Semi-structured interview guide & face-to-face interviews Data Analysis: Analysis involving multiple reviews of transcripts to identify main themes and process of creating coded categories to understand findings Reliability & Validity: Advisory group served as peer reviewers, discussion and review of coding process by research team

19. Quantitative Methods: Research Questions Phases 1 and 2 1) What are the characteristics of children and youth involved in the child welfare system and the public mental health system? 2) Among children and youth involved in the child welfare system and the mental health systems, what is the influence of outpatient mental health services for children on family reunification and family maintenance outcomes?

20. Research Design: Correlational study with secondary data from a merged administrative dataset Sampling Procedures: Phase 1: All children and youth whose child welfare case was closed over a 12-month period (Jan-Dec 2004) Phase 2: All youth between the ages of 15 and 19, whose child welfare cases were closed over an 18-month period between the ages of 15 and 19 (Nov 2005-April 2007) Sample: Phase 1: N = 1,127 (total sample), N = 519 (received public mental health services, 46%) Sample: Phase 2: N = 638 (total sample), N = 147 received public mental health services, 23%) Quantitative Methods: Research Design, Sampling Procedures & Sample

21. Quantitative Study Methods: Measurement of Key Variables Independent Variables: Outpatient mental health services for child or youth (may include individual, group, or family therapy, or psychiatric medications) Mental health treatment completion Dependent Variable: Case closure type of family stabilized: Family reunification and family maintenance Other Variables: Demographic variables  Gender  Race/ethnicity System-Related Factors  Type of maltreatment  Age at entry into the CWS  Number of hours of outpatient mental health services Type of Mental Health Problem:  Adult-type disorder (e.g. mood disorder)  Adjustment disorder  Childhood disorder

22. Quantitative Study Methods: Reliability and Validity, Data Analysis Reliability & Validity: No formal tests of reliability of variables because data came from administrative databases Operational definitions of variables were developed and verified with input from Advisory Group Data Analysis: Binary statistics to compare characteristics of children and youth in both the child welfare and mental health systems compared to those only in child welfare Logistic regression to test influence of outpatient mental health treatment completion on family stabilization, while controlling for the influence of demographic variables, system-related factors and type of mental health problem (analysis using sub- sample of children receiving outpatient services)

23. Qualitative Results Challenges in Collaboration Divisions in System Goals, Mandates & Policies: Both Phases Budget or Resource Problems: Both Phases Communication & Service Coordination Problems: Only Phase 1 Workload: Only Phase 2

24. Qualitative Results: Divisions in System Goals, Mandates & Policies (Both Phases) Child Welfare: “There are inherent tensions. In mental health, they usually rely on the client to identify issues. In child welfare, we have specific issues that we feel need to be addressed because of timelines” (Program Manager, Phase 1). “We have different mandates. We [child welfare system] are enforcers. There are different cultures because of this; they are more the ‘good guys.” (Program Manager, Phase 1). Mental Health: “Child welfare deals with the safety of the child and the desire to have the child be adjusted and returned to a normalized living environment. Sometimes there is an under emphasis on mental health problems and the [child welfare] system can create problems” (Program Manager, Phase 2). “We just have different needs and different perspectives on the system—legal vs. mental health” (Line Worker, Phase 1).

25. Child Welfare: “There are no opportunities for line staff to communicate with one another. There is no annual or bi-annual communication forum to talk about what each system is doing” (Program Manager). “If a therapist is working with a family, they should meet regularly with the social worker and develop treatment goals and discuss how needs are being met. The way it is now is the therapist sends a written report to the court and doesn’t talk to the social worker; there is no mechanism to talk together” (Supervisor). Mental Health: “There’s a lack of structure. I’ve never been in a meeting with social workers” (Line Worker). “Social workers may be under pressure to address certain therapeutic goals that might be set up by attorneys or judges—not by the therapist or the clinicians, and it makes it difficult, because we may not necessarily agree” (Line Worker) Qualitative Results: Communication & Service Coordination Problems (Phase 1)

26. Qualitative Results: Budget or Resource Problems (Both Phases) Child Welfare: “ For families in the voluntary [services], they often only have services for three months, and we sometimes have to spend a lot of time getting them Medi-Cal, and then there is often a waiting list to see the mental health worker, so we can eat up the time that they have just trying to get them services” (Line Worker, Phase 1) “The parents cannot get counseling if they do not have custody of the child [because of funding restrictions], but they cannot get their children back unless they get counseling” (Line Worker, Phase 1) Mental Health: “ The budget cuts are really impeding the collaborations, especially for mental health. [The child welfare agency] didn’t really get a cut because social services took it out of other parts of the organization. It’s really overwhelming. I’m personally spending about 40-50% of my time on dealing with the budget cut” (Program Manager, Phase 2) “The budget reduction is huge…We can’t talk about the future when it’s all about how many people we’re laying off. I see despair at all levels” (Program Manager, Phase 2)

27. Qualitative Results: Workload Stressors (Phase 2) Child Welfare: “Collaboration is difficult because workers are under pressure to provide services. Child welfare workers have to complete court reports and there is very little time to be creative and look for resources. Trying to take time to pursue resources might be a luxury. Workers get in the habit of using the top 3, 4, or 5 resources they use all the time” (Supervisor). “We try to structure collaboration into work time, but there is always a crisis” (Supervisor). Mental Health: “Time is a big impediment to collaboration. Especially with Mental Health, we’re always ready for the unexpected. We have that mindset of always being prepared, which is a distraction to thinking about the future because we’re always putting out fires” (Program Manager)

28. Qualitative Results Infrastructure to Support communication & Service Coordination: Both Phases Service Integration: Both Phases Support & Planning from organizational Leaders: Only Phase 1 A High Value Placed on Collaboration Among Workers & in Organization: Both Phases Factors that Facilitate Collaboration

29. Qualitative Results: Infrastructure to support communication & Service Coordination (Both phases) Child Welfare: “ It would be beneficial if all the professionals involved in the youth’s life could hear all the concerns. This already happens but more is needed. It’s hard to do with two different departments. Everyone is invited but not everyone comes. Maybe making this mandatory would help” (Supervisor, Phase 2). “We have an educational rights specialist to advocate for kids who aren’t getting what they need in the educational system. Having something similar in mental heath would help a lot” (Line Worker, Phase 1). Mental Health: “If the institution had some mechanism for collaboration such as meetings when there is an opportunity for lining up policy, programming and practices in a way that will allow for better collaboration at the line staff level” (Line Worker, Phase 1). “We need consistent meetings from all parties involved, from the very beginning. Problems can be solved if we set up mechanisms to do that” (Line Worker, Phase 1).

30. Qualitative Results: Service Integration (Both Phases) Child Welfare: “The location of the mental health unit at the shelter helps with collaboration. When you have someone on site that is a big factor” (Line Worker, Phase 1). “What we should do is band together to go after funding sources. Instead of wasting our resources going after the same pot, we should get together and discuss who is best fitted to receive a certain funding, and then we should all get behind that one agency” (Program Manager, Phase 1). “The need for cross-training is bad. A lot of mental health workers don’t understand the court process and how difficult that can be” (Supervisor, Phase 1). Mental Health: “I think we need to have more shared staff, more co-located staff, more co- managers, even merging departments” (Program Manager, Phase 2). “Some people who work in mental health have very little idea of how DFCS works. Training from DFCS on what drives their system, their legal mandates, safety issues, and court issues. This would create a better appreciation” (Program Manager, Phase 1)

31. Qualitative Results: Support & Planning From Organizational Leaders (Phase 1) Child Welfare: “It flows from the Board of Supervisors and county executives to make collaboration a value in our county so that it is monitored” (Supervisor, Phase 1) Mental Health: “ We need better coordination, the top levels need to establish policies” (Line Worker, Phase 1). “Having a strong, committed Board of Supervisors helps collaboration— they say they want collaboration and we do it. Having the support of executive directors is very important. They have power and can make things happen” (Program Manager, Phase 1).

32. Qualitative Results: A High Value Placed on Collaboration Among Workers & in the Organization (Both Phases) Child Welfare: “We’re selling ourselves short by not seriously trying to collaborate…We could make a huge dent on our social problems by collaborating” (Program Manager, Phase 1). “We need to see each other as going down the same path” (Program Manager, Phase 1). Mental Health: “We both have the same objective, so how can we make our services blend? How can we form a partnership so that we can bring resources, funding streams, and so forth to the table and then provide the services that we are good at providing. There’s a lot to gain from collaboration” (Program Manager, Phase 2). [Collaboration can be facilitated by] “realizing that the two populations we serve are one” (Program Manager, Phase 2).

33. Quantitative Results: Children and youth referred to mental health system were older at entry into the child welfare system

34. Quantitative Results: Higher rate of physical abuse for children referred to mental health services

35. Quantitative Results: Higher rate of sexual abuse for children referred to mental health services

36. Quantitative Results: Adult-type disorder most common

37. Quantitative Results: Number of hours of outpatient services low for transition age youth

38. Quantitative Results: Influence of outpatient mental health services for children on family reunification and family maintenance outcomes Among children and youth participating in outpatient mental health services, those who completed mental health services were not more likely to have a family stabilization outcome than children and youth who did not complete outpatient mental health services.

39. Summary & Interpretation Study Finding: High need for mental health services, especially for youth entering the CWS at an older age, those having experienced physical or sexual abuse, and those with Adult-type disorders Study Finding: Outpatient mental health services not influencing family stabilization outcomes Study Finding: Number of hours of outpatient services low for transition age youth Study Finding: Challenges in cross-system collaboration between child welfare and mental health systems Interpretation: Reduced access to services, possibly influenced by lack of resources & system infrastructures Interpretation: Reduced effectiveness of services, possibly influenced by difficulties with joint treatment planning

40. Practice & Policy Implications Study Finding: Numerous factors that facilitate collaboration were identified Practice/Policy Implication: Identify joint funding projects Practice/Policy Implication: Cross-training on system goals, policies, & mandates Practice/Policy Implication: Continue to place a high value on the importance of collaboration Practice/Policy Implications: Co-locating mental health and child welfare staff Interpretation: Improved access to services and improved effectiveness of services

41. Limitations Cross-system collaboration is not measured in the quantitative study Small sample size for qualitative study Administrative data does not allow for measurement of other important variables (e.g. type of therapeutic intervention, quality of therapeutic relationship, alleviation of symptoms, etc.) Quantitative methods are correlational--although several variables are statistically controlled for in the multivariate models (demographic, system-related, type of mental health problem), this study did not use experimental methods to test the effectiveness of mental health interventions (e.g. “randomized controlled trial”) Study conducted in one county More research is needed to address these limitations

42. Discussion Questions How do child welfare and mental health systems collaborate in your county? A number of factors that help collaboration or hinder collaboration were discussed in this presentation. Have you observed these factors operating in your own county? What other factors influence collaboration? Are there differences in collaboration challenges depending on what systems are collaborating (e.g. substance abuse, criminal justice, public assistance, etc…)? (see handout on 2-phase study on collaborative practice between child welfare and substance abuse systems, ). How can this research be used to improve practice in your county? How can these findings be used to promote evidence-based or evidence-informed collaborative practice between child welfare and mental health systems?

43. References 1 Halfon, N., Zepeda, A., & Inkelas, M. (2002). Mental health services for children in foster care. UCLA Policy Brief. 2 Halfon, N., Mendonca, A., & Berkowitz, G. (1995). Health status of children in foster care: The experience of the Center for the Vulnerable Child. Archives of Pediatric and Adolescent Medicine,149, 386-392. U.S. Department of Health and Human Services (1999). Mental Health: A Report of the Surgeon General. Rockville, MD: Author. 3 Landsverk, J., Davis, I., Ganger, W., Newton, R., & Johnson, I. (1996). Impact of child psychosocial functioning on reunification from out-of-home placement. Children and Youth Services Review, 18(4/5), 447.462. 4 Jones, L. (1998). The social and family correlates of successful reunification of children in foster care. Children and Youth Services Review, 20(4), 305-323. 5 Hines, A. M., Lee, P. A. Osterling, K. L., & Drabble, L. (2007). Factors predicting family reunification for African American, Latino, Asian, and White families in the child welfare system. Journal of Child and Family Studies, 16(2), 275-289. 6 Prince, J., & Austin, M. J. (2005). Inter-agency collaboration in the child welfare and mental health systems. Social Work in Mental Health, 4, 1-16. 7 Young, N.K. & Gardner, S.L. (2002) TAP 27: Navigating the Pathways: Lessons and Promising Practices in Linking Alcohol and Drug Services and Child Welfare. Washington DC: Substance Abuse and Mental Health Services Administration

44. References 8 Whittington, C. (2003). A model of collaboration. In J. Weinstein & C. Whittington & T. Leiba (Eds.), Collaboration in social work practice (pp. 39-61). London: Jessica Kingsley Publishers. 9 Gardner, S. L. & the National Center on Substance Abuse and Child Welfare (2008). Materials describing developmetal process of collaboration: http://www.cffutures.com/presentations.shtml 10 Hurlburt, M. S., Leslie, L. K., Landsverk, J., Barth, R. P., Burns, B. J., Gibbons, R. D., Slymen, D. J., Zhang, J. (2004). Contextual predictors of mental health service use among children open to child welfare. Archives of General Psychiatry, 61, 1217-1224. 11 Glisson, C. & Hemmelgarn, A. (1998). The effects of organizational climate and interorganizational coordination on the quality and outcomes of children’s service systems. Child Abuse and Neglect, 22(5), 401-421. 12 Bruns, E., Stuer, J., Force, M., & Burchard, J. (2005). Adherence to wraparound principles and association with outcomes. Journal of Child and Family Studies, 14(4), 521-534. Burns, B.J., Schoenwald, S.K., Burchard, J.D., Faw, L, & Santos, A. (2000). Comprehensive community-based interventions for youth with severe emotional disorders: multisystemic therapy and the wraparound process. Journal of Child and Family Studies, 9(3), 283-314.