1. 1 Psychological Symptoms among Young Maltreated Children: Do Services Make a Difference? The research for this presentation was funded by the Administration on Children, Youth, and Families of the U.S. Department of Health and Human Services. Points of view or opinions in this presentation and accompanying documents are those of the presenter and do not necessarily represent the official position or policies of the U.S. DHHS. Results are preliminary, please cite only with permission. Contact Julie McCrae for further information: jsmccrae@email.unc.edu Findings from the National Survey of Child and Adolescent Well-being (NSCAW) Julie S. McCrae, Shenyang Guo, & the NSCAW Research Group Presented at the Ninth Annual International Research Conference on the Victimization of Children & Youth Portsmouth, NH  July 10-13, 2005

2. 2 Background Mental disorders are a public health problem (Murray & Lopez, 1996) Maltreatment may be the single most preventable and intervenable contributor to mental disorders in this country (DeBellis, 2001) Child welfare agencies have been called upon to improve mental health-related efforts Results of federal reviews and from the NSCAW study so far show that –many children are underserved ( Burns et al., 2004) –It is unclear whether children benefit from child welfare and other services in terms of psychosocial outcomes (U.S. DHHS, 2002)

3. 3 Study Purpose To investigate the relationship between mental health service receipt and change in children’s psychopathology symptoms 3 years following a maltreatment investigation Focus on young children

4. 4 Sample and Measures Children ages 3 to 10 at baseline (BL) Time pointn Baseline2,325 12 mos1,902 18 mos1,924 36 mos1,931 Services: Child and Adolescent Services Assessment (CASA), adapted version Outcomes: Child Behavior Checklist (CBCL) change scores T2 (36 mos) – T1 (baseline)

5. 3 Methods to Assess Service Impact –Correlations View change between T1 and T2 in relation to services –Linear Regression (RESULTS NOT SHOWN) View change in relation to services and other factors –Propensity Scores Matching using difference-in-differences approach View change in relation to services with service groups that are more comparable Selection Bias

6. 6 Results: Proportion of Children (1) with Bord/Clinical-level Behavior Problems and (2) Received MHS by 36 Months n = 1989-1948; proportions are weighted

7. 7 How much have children changed? Baseline (T1) to 36 months (T2) Service condition Unadjusted Mean Change: T2 – T1 InternalizingExternalizingCBCL Total MHS (n=959) -.99-.27-.57 No MHS (n=965) -2.57-1.51-2.12 Mean difference 1.58*1.24*1.55* Children who receive MHS improve slightly; unserved children improve significantly more *p<.05 (t-test, unadjusted mean difference, unweighted data)

8. 8 How much have symptomatic (at BL) 3 to 10-year-olds changed? Service condition Unadjusted Mean Change: T2 – T1 Internalizing (n=664) Externalizing (n=790) CBCL Total (n=1155) MHS-6.65-5.20-5.33 No MHS-11.7-7.99-8.20 Mean difference 5.05*2.79*2.87* *p<.05 (t-test, unadjusted mean difference, unweighted data) Symptomatic children improve by 5-12 points; unserved children improve significantly more

9. 9 Analysis #3: Viewing Service Impact by Using a Matching Estimator Purpose: Correct for selection bias Method: Propensity Score Matching, Difference-in- Differences Approach, Local Linear Regression weights 2-step procedure 1.Create propensity score using logistic regression 2.Use matching estimator to weight cases according to their propensity score (propensity to receive services), and then re-view children’s outcomes in relation to services

10. 10 Logistics used for Propensity Scores Independent variables All children (n=1683) Bord/clinical children (n=751) OR (95% CI) Child age (years)1.05 (1.00, 1.11)1.08 (.99, 1.17) Gender: Male1.31 (1.04, 1.65)*1.38 (.96, 1.99) Race: Black.73 (.57,.94)*.74 (.51, 1.09) Treatment hx: Yes4.81 (3.74, 6.20)***4.02 (2.71, 5.95)*** Worker report: emot/behav prob 2.03 (1.53, 2.71)***2.02 (1.31, 3.10)** Caregiver mh prob1.45 (1.14, 1.84)**1.19 (.83, 1.72) Taking psych meds3.15 (1.93, 5.16)***1.79 (.99, 3.21) Cumulative risk1.38 (1.19, 1.60)***1.26 (1.00, 1.59)* Sexual abuse1.40 (1.01, 1.95)*1.41 (.84, 2.37) ^Also included in the models: physical abuse (ns)

11. 11 Accounting for selection bias, children who receive MHS improve significantly more than children who do not receive MHS only with regard to total problem behavior Results of PSM approach All 3 to 10-year-olds Service condition Adjusted Mean Change: T2 – T1 InternalizingExternalizingCBCL total MHS (n=779) -1.15-1.43-.85 No MHS (n=813) -1.44-2.00-.56 Mean difference.29.56-1.41* *p<.05 (t-test, adjusted mean difference, unweighted data)

12. 12 Results of PSM Approach: Symptomatic Children Service Group Adjusted Mean Change: T2 – T1 Internalizing (n=551) Externalizing (n=656) CBCL Total (n=715) MHS-1.52 -1.42 No MHS-.07-.30-.15 Mean difference -1.45*-1.23*-1.28* Accounting for selection bias, symptomatic children who receive MHS improve significantly more than children who do not receive MHS *p<.05 (t-test, adjusted mean difference, unweighted data)

13. 13 Findings 1 1.Behavior change among all 3 to 10-year- olds is very slight over 36 months –Change is attributable to services pre- matching, with served children improving significantly less than unserved children, regardless of symptom type –Post-PSM, served & unserved groups not distinguishable with regard to internalizing & externalizing behaviors, but results flip with regard to total problem behavior - served children experience more positive change change is still slight among all children

14. 14 Findings 2 2. Much more change is observed among symptomatic children –Decreases in problem behavior, on average, in the range of 5 to 12 points –Pre-PSM: Served children improve significantly less than unserved children –Post-PSM: Results flip – unserved children remain roughly the same, while served children improve about 1½ points

15. 15 3. PSM is influential in these analyses Particularly with regard to symptomatic children Selection bias confirmed Findings 3

16. 16 Implications Able to make the case that more young children ought to receive MHS –Do see positive change in behavior problems over 36 months with service Whether the amount of change is clinically meaningful for large numbers of children is not clear Targeted intervention is recommended –There is not evidence that all young, maltreated children ought to receive MHS –Services may have the most impact on internalizing behaviors

17. 17 Limitations Not likely that all selection bias is corrected No account for events in the interim Inability to include service dose Estimates are not weighted to the national child welfare population The research for this presentation was funded by the Administration on Children, Youth, and Families of the U.S. Department of Health and Human Services. Points of view or opinions in this presentation and accompanying documents are those of the presenter and do not necessarily represent the official position or policies of the U.S. DHHS. Results are preliminary, please cite only with permission Sincere thanks to the Children’s Bureau for funding this dissertation research.