1. FFT in California: Evaluation Outcomes Cricket Mitchell, PhD CIMH Consultant April 3, 2008

2. Overview FFT in California Youth Referred for FFT Youth Who Never Began Youth Who Are Currently in FFT Youth Who Dropped Out of FFT Youth Who Completed FFT Comparisons Between Youth Who Dropped Out of and Youth Who Completed FFT Process and Outcome Information for Youth Who Completed FFT

3. FFT in California 2695 youth referred for FFT as of January 2008 28 CiMH Development Team FFT sites established to date –4 cohorts –Sites in varying stages of training and consultation 167 therapists trained to date –Range of 1-87 FFT clients each

4. FFT in California Clinical Services System (CSS) –Currently the only source of all CiMH FFT Development Team data Strengths –Important clinical tool specific to FFT model –Tracks data at multiple levels Descriptives Process Outcomes Limitations –Hundreds of therapists entering data; increases potential for error –Data owned and managed by FFT, Inc.

5. 2695 Youth Referred for FFT 396 (14.7%) Never Began or Have Not Yet Begun 2299 (85.3%) FFT Participants –687 (29.9% of participants) currently in process –548 (23.8% of participants) dropped out of FFT –703 (30.6% of participants) completed all phases of FFT –361 (15.7% of participants) status cannot be determined based on available data

6. Never Began or Have Not Yet Begun 14.7% (n=396) of all youth referred for FFT 9.7% (n=260) of all youth referred for FFT have a therapist-documented “Never Began” reason –52% declined services, never began –32% made appointment, client never attended –16% never contacted

7. FFT Participants 85.3% (n=2229) of all youth referred for FFT Status defined by available CSS data –Dropped Out of FFT A therapist-identified dropout reason OR a therapist- identified never began reason with at least 1 session –Completed FFT No therapist-identified dropout or never began reason, at least 1 session, AND any family member- or therapist- completed outcome indicator –Currently in Process No therapist-identified dropout or never began reason, at least 1 session, AND no family member- or therapist- completed outcome indicator

8. FFT Participants Phase defined by available CSS data –Early One or more sessions in Engagement & Motivation AND no sessions in Behavior Change or Generalization –Mid One or more sessions in Behavior Change AND no sessions in Generalization –Late One or more sessions in Generalization

9. FFT Participants Early Dropouts –23.8% (n=548) Mid Dropouts –9.0% (n=206) Late Dropouts –2.7% (n=52) Early Completers –2.0% (n=45) Mid Completers –2.5% (n=58) Late Completers –30.6% (n=703)

10. FFT Participants: Currently in Process 29.9% (n=687) of participants Average age = 14.9 years 62.4% Male Ethnicity*: –38% Hispanic –33% White –22% Black – 4% Biracial – 2% Asian –<1% American Indian –<1% Other *Percentages may not equal 100 due to rounding.

11. FFT Participants: Currently in Process Current phase of FFT –Early 45.1% (n=310) –Mid 37.7% (n=259) –Late 17.2% (n=118)

12. FFT Participants: Dropouts 23.8% (n= 548) of all FFT participants –Early Dropouts –If include youth with ambiguous status (Mid- and Late-Dropouts; and Early- and Mid-Completers) attrition rate = 39.5% Therapists documented “Dropout” reasons for 97.3% (n=533): –49% quit/drop out even after contact –27% incarcerated –12% moved prior to completing program – 7% runaway – 5% phone not working/can’t contact

13. FFT Participants: Dropouts 23.8% (n= 548) of all FFT participants Average age = 16.0 59.5% Male Ethnicity*: –39% White –35% Hispanic –16% Black – 6% Biracial – 2% Asian – 1% American Indian – 1% Other –<1% Alaskan Native *Percentages may not equal 100 due to rounding.

14. FFT Participants: Completers 30.6% (n= 703) of all FFT participants Average age = 16.3 61.2% Male Ethnicity*: –41% White –38% Hispanic –12% Black – 5% Biracial – 2% Asian – 2% Other – 1% American Indian *Percentages may not equal 100 due to rounding.

15. Any Differences Between Dropouts and Completers? Age –No differences Gender –No differences Ethnicity –No differences

16. Any Differences Between Dropouts and Completers? Baseline Functioning – Youth Outcome Questionnaires (YOQ) –No differences in average scores for Youth Self- Report, Mother Reports, or Father Reports –No differences in the percentage of youth above and below the clinical cutpoint for Mother Reports or Father Reports –Significant differences in the percentage of youth above and below the clinical cutpoint for Youth Self-Report (see graph on following slide)

17. Any Differences Between Dropouts and Completers? Youth who dropped out of FFT self-reported YOQ scores that were above the clinical cutpoint at higher rates than youth who completed FFT. Chi-Square (df=1) = 8.08, p<.01

18. Any Differences Between Dropouts and Completers? Family members’ perceptions of the therapeutic process, therapeutic progress, and the therapeutic alliance – Counseling Process Questionnaires (CPQ) – Engagement & Motivation Phase –No differences in the average scores for the 16 core items for Youth, Mothers, or Fathers –No differences in the average percent of improvement between items 17 and 18 for Youth or Fathers –Significant differences in the average percent of improvement between items 17 and 18 for Mothers (see graph on following slide)

19. Any Differences Between Dropouts and Completers? During the Engagement & Motivation phase, mothers of youth who completed FFT reported a higher rate of improvement from when they first came to FFT until “now” (the time the CPQ was completed) than mothers of youth who dropped out of FFT. t (df=439) = -3.19, p<.01

20. Any Differences Between Dropouts and Completers? Therapist Outcome Measure (TOM) –Significant differences in therapist reports of general family change since the beginning of FFT (see graph on following slide)

21. Any Differences Between Dropouts and Completers? Chi-Square (df=6) = 716.34, p<.001

22. FFT Completers: Process Information and Clinical Outcomes 30.6% (n= 703) completed FFT –Late Completers Baseline Clinical Functioning: –YOQs from Youth, Mothers, and Fathers Process Information: –CPQs for each of the three FFT phases Clinical Outcomes: –YOQs from Youth, Mothers, and Fathers –Clinical Outcome Measures (COM) from Youth, Mothers, and Fathers –Therapist Outcome Measure (TOM) from Therapists Are there any differences in baseline functioning, process information, or clinical outcomes by gender and ethnicity?

23. Baseline Clinical Functioning Average YOQ Scores* for Youth Who Completed FFT (n=703) nRangeAverage (sd) Youth Self-Report584-11 – 17154.50 (+ 31.4) Mother Report542-13 – 19670.71 (+ 36.0) Father Report197-1 – 16169.86 (+ 37.2) *Scores range from -16 – 240; Higher scores indicate higher levels of impairment, or more problematic general functioning.

24. Baseline Clinical Functioning

25. Any Differences in Baseline Clinical Functioning by Gender or Ethnicity? Youth general functioning as measured by the Average Scores on the Youth Outcome Questionnaires (YOQs) –YOQ-Self Report – Average Scores Significant differences by gender (see following slides) No differences by ethnicity –YOQ-Mother Report – Average Scores No differences by gender Significant differences by ethnicity (see following slides) –YOQ-Father Report – Average Scores No differences by gender or ethnicity

26. Any Differences in Baseline Clinical Functioning by Gender or Ethnicity? Female youth who completed FFT had higher self-reported YOQ scores at baseline than Male youth who completed FFT. F (df=2) = 5.27, p<.01

27. Any Differences in Baseline Clinical Functioning by Gender or Ethnicity? Mothers of youth who completed FFT reported higher YOQ scores at baseline for White youth and youth with other ethnicities than for Black and Hispanic youth. F (df=4) = 5.00, p<.01

28. Any Differences in Baseline Clinical Functioning by Gender or Ethnicity? Youth general functioning as measured by the Clinical Cutpoint Information on the Youth Outcome Questionnaires (YOQs) –YOQ-Self Report – Clinical Cutpoint Information Significant differences by gender (see following slides) No differences by ethnicity –YOQ-Mother Report – Clinical Cutpoint Information No differences by gender Significant differences by ethnicity (see following slides) –YOQ-Father Report – Clinical Cutpoint Information No differences by gender Significant differences by ethnicity (see following slides)

29. Any Differences in Baseline Clinical Functioning by Gender or Ethnicity? Significantly more Females than Males had YOQ-Self Report scores at baseline that were above the clinical cutpoint. Chi-Square (df=2) = 10.24, p<.01

30. Any Differences in Baseline Clinical Functioning by Gender or Ethnicity? Significantly more White youth and youth of other ethnicities had YOQ- Mother Report scores at baseline that were above the clinical cutpoint, as compared to Black and Hispanic youth. Chi-Square (df=4) = 16.33, p<.01

31. Any Differences in Baseline Clinical Functioning by Gender or Ethnicity? Significantly more Black youth, White youth, and youth of other ethnicities had YOQ-Father Report scores at baseline that were above the clinical cutpoint, as compared to Hispanic youth. Chi-Square (df=4) = 11.36, p<.05

32. FFT Process Information *Responses range from 1-7; Higher scores indicate higher rates of agreement. Average CPQ Responses* for the Core 16 Items Youth Who Completed FFT (n=703) Respondent FFT Phase Engagement & Motivation Behavior ChangeGeneralization Youth 5.03 (n=435) 5.42 (n=418) 5.60 (n=277) Mother 5.28 (n=383) 5.69 (n=381) 5.90 (n=249) Father 5.17 (n=162) 5.56 (n=147) 5.75 (n=92)

33. FFT Process Information

34. Any Differences in FFT Process Information by Gender or Ethnicity? CPQ Core 16 Items –No differences by gender or ethnicity for Youth or Mother Reports across all three phases of FFT –No differences by gender for Father Reports across all three phases of FFT –No differences by ethnicity for Father Reports in Behavior Change or Generalization Phases –Significant differences by ethnicity for Father Reports in Engagement & Motivation Fathers of Hispanic youth reported significantly higher average CPQ scores on Items 1-16 than fathers of White youth, indicating higher rates of agreement with the core 16 CPQ items. –Hispanic (n=43) = 5.53 (+.97) –White (n=87) = 4.98 (+.77)

35. Any Differences in FFT Process Information by Gender or Ethnicity? Percent Improvement Between CPQ Items 17 and 18 –No differences by gender or ethnicity for Youth, Mother, or Father Reports across all three phases of FFT

36. FFT Clinical Outcomes Youth reported 22% improvement; Mothers reported 26% improvement; and Fathers reported 18% improvement in youth general functioning. *All pre/post comparisons statistically significantly different, p<.001.

37. FFT Clinical Outcomes Thirty-eight of the 174 (22%) youth who were below the clinical cutpoint on the YOQ prior to participation in FFT moved into the clinical range (above the cutpoint) after FFT. Seventy-seven of the 216 (36%) youth who were at or above the clinical cutpoint on the YOQ prior to participation in FFT moved into the non-clinical range (below the cutpoint) after FFT.

38. FFT Clinical Outcomes Seven of the 96 (7%) youth who were below the clinical cutpoint on the YOQ prior to participation in FFT moved into the clinical range (above the cutpoint) after FFT. Ninety-six of the 265 (36%) youth who were at or above the clinical cutpoint on the YOQ prior to participation in FFT moved into the non-clinical range (below the cutpoint) after FFT.

39. FFT Clinical Outcomes Six of the 27 (22%) youth who were below the clinical cutpoint on the YOQ prior to participation in FFT moved into the clinical range (above the cutpoint) after FFT. Twenty of the 87 (23%) youth who were at or above the clinical cutpoint on the YOQ prior to participation in FFT moved into the non-clinical range (below the cutpoint) after FFT.

40. FFT Clinical Outcomes

41. Any Differences in FFT Clinical Outcomes by Gender or Ethnicity? Average YOQ Total Scores –No differences by gender or ethnicity for Youth Self-Report, Mother Reports, or Father Reports YOQ Clinical Cutpoint Status –No differences by gender or ethnicity for Youth Self-Reports or Father Reports –No differences by ethnicity for Mother Reports –Significant differences by gender for Mother Reports (see graph on following slide)

42. Any Differences in FFT Clinical Outcomes by Gender or Ethnicity? According to Mother Reports, the majority of females had YOQ scores above the clinical cutpoint after FFT, while the majority of males had YOQ scores below the clinical cutpoint after FFT. Chi-Square (df=2) = 14.43, p<.01

43. Any Differences in FFT Clinical Outcomes by Gender or Ethnicity? COMs & TOM – General Family Change Since the Beginning of FFT –No differences by gender or ethnicity for Youth, Mother, Father, or Therapist Reports

44. Summary Based on available CSS data… At least 10% of referred youth never began FFT –An additional 5% have either not yet begun, or have never began and a reason has not been documented 30% of referred youth are currently participating in FFT At least 24% of referred youth dropped out of FFT –The attrition rate increases if youth are included whose status cannot be determined based on the available data. At least 31% of referred youth have completed FFT –The completion rate increases if youth are included whose status cannot be determined based on the available data.

45. Summary Youth who never began, are currently in process, dropped out, and completed FFT do not vary by demographic characteristics (age, gender, ethnicity) Two interesting differences were identified among youth who dropped out of and youth who completed FFT: –Youth who dropped out of FFT self-reported YOQ scores that were above the clinical cutpoint at higher rates than youth who completed FFT. –During the Engagement & Motivation phase, mothers of youth who completed FFT reported a higher rate of improvement from when they first came to FFT until “now” (the time the CPQ was completed) than mothers of youth who dropped out of FFT.

46. Summary FFT Baseline Clinical Functioning –The majority of youth who completed FFT had baseline YOQ scores above the clinical cutpoint YOQ-Self Report = 56% YOQ-Mother Report = 75% YOQ-Father Report = 73% –Interesting differences by gender and ethnicity were identified in baseline functioning Females had higher average YOQ-Self Report scores than Males; they also had a greater proportion of Self- Report scores above the clinical cutpoint White youth and youth of Other ethnicities had higher average YOQ-Mother report scores than Black and Hispanic youth; they also had a greater proportion of Mother-Report scores above the clinical cutpoint Black youth, White youth, and youth of Other ethnicities had a greater proportion of YOQ-Father Report scores above the clinical cutpoint than Hispanic youth

47. Summary FFT Process Information –Family member reports on the core 16 CPQ items do not vary across informant or across the three phases of FFT –Family member reports of percent improvement (Items 17 and 18 on the CPQ) show a progressive increase across the three phases of FFT

48. Summary FFT Clinical Outcomes –Youth demonstrated significant improvements in general functioning YOQ-Self Report –22% improvement in average scores –net shift of 10% from clinical to non-clinical range YOQ-Mother Report –26% improvement in average scores –net shift of 25% from clinical to non-clinical range YOQ-Father Report –18% improvement in average scores –net shift of 12% from clinical to non-clinical range –The majority of all family members’ and therapists’ responses on the COMs and TOM report that “General Family Change Since the Beginning of FFT” is “A Lot or Very Much Better”

49. Additional Information Cricket Mitchell, PhD cricketmitchell@hotmail.com