1. The Clinical Services Systems – our tool to evaluate services
California Symposium, April, 2012
Kimberly Mason, MSW, LCSW

2. Clinical Services System (CSS)
Established with the onset of widespread dissemination of the model
Went online 2001
Important way to track what is happening in sessions and outcomes for families receiving services
If it is not documented – it didn’t occur

3. Entering Client Information
We use this to track the gender, ethnicity and types of referral problems that sites are serving – this becomes important as programs look for funding and we look for ways to improve training and services
You should begin entering information when you start working the case to open for services.

4. Pre Assessments
Outcome Questionnaire – Measurement of Individual Functioning. Cut-Off Score is Given to family members 18 or older
Youth Outcome Questionnaire – Parent/Guardian fills in about youth behavior. Cut-Off Score is 46
Youth Outcome Questionnaire – Self Report – Youth fills out about their own behavior. Cut-Off Score is 47

5. Open Date Documentation
You give the OQ, YOQ and YOQ-SR and enter information into the CSS
You enter client information into the CSS.
This begins the process of data collection
Definitions –
Referral Date – enter the date that the referral came to you as a therapist to open the case
Open Date – the date that consent for treatment paperwork is signed

6. Progress Notes
You must Complete a Progress Note on each family Session
Progress Notes are Phase Dependant and we track the number of sessions in each phase
We also track the time between sessions – especially the time between the first 3 sessions

7. Progress Notes
Engagement/Motivation Notes – Supervisor/Consultant looks to these to see what techniques being used, progress towards goals, relational assessments and planning for next session
Behavior Change Note – Look to these for the overall behavior change plan – skills targeted and then the actual teaching, modeling or directing of those skills
Generalization Note – Look for the overall plan, progress towards goals, linkage back to referral sources

8. Family Self-Report
This is the Opportunity for the family to give feedback on their progress in therapy
First 3 questions are related to how they see their family and their hope for the future
The last 4 questions are about your alliance with the family
The Family Self-Report should be given to each family member participating in therapy at the first two sessions of every phase

9. Therapist Self-Report
This is the opportunity for therapist to gauge their progress towards meeting goals.
Therapist Self-Report should be completed whenever the family completes a FSR
We track the completion of these documents as part of Dissemination Adherence

10. Closing a Case
We track length of time in the program, therefore it is very important that a case be closed in CSS when services are completed.
It is also essential that you close the case using definitions that are present in the CSS

11. Step by Step of Closing Case
Make sure that you have the correct client selected and then go to Add New Termination
Enter Termination Date - date of last clinical contact with the family (for families that complete FFT) or date of last ATTEMPT at clinical contact with the family (for families that do not complete FFT).
Termination date is not considered the date you enter the termination data into the CSS.

12. Select Type –
Finished Counseling –A Completed Case is a case where the family completes all three phases of FFT.  Completed Cases are given a Finished Result at Termination. COMs are entered and a TOM is completed by the therapist on the CSS Termination Page.
Dropped-Out of Counseling - A Non-Completed Case is a case where treatment is interrupted and the family does not complete all three phases of FFT. Efforts to reengage the family do not result in a subsequent session.  A Non-Completed Case is considered a drop out.  These cases are given both a Drop Out Reason, a Time of Drop Out description, and a TOM on the CSS Termination page. Non-Completed Cases do not receive a Finished Result but receive a completed TOM.
Never Seen - Cases that were Never Seen are defined as those cases that never had a FFT session.

13. Select from the following:
Youth Remains in the Home/Family – At the time of case closure is youth living at home
Youth is enrolled in an educational/vocational school program or working – At the time of case closure youth has either completed schooling, is working or is enrolled in some type of educational program
Youth is free of New Law Violations since FFT Treatment began? – During the course of FFT did youth receive an new law violations – this would not include violations of probation or contact with law enforcement – only if youth was charged with new offense

14. Before Completing Closure for Successfully Completed Cases you need to have the COM-A, COM-P and TOM Completed as these will be basis for the case closure status
Client Outcome Measure – Adolescent (COM-A) – completed by youth at time of successful case closure and measures youth belief about family change.
Client Outcome Measure – Parent (COM-P) – completed by parent/guardian at time of successful case closure and measures parent belief about family change
Therapist Outcome Measure (TOM) – completed at case closure regardless of closing status and measures therapist belief about family change

15. *In the event.
Positive Outcome-Completed all three phases and received outcome ratings on COMs and TOMs that were consistently 3s, 4s and 5s among all family members and therapists.* Risk factors have been significantly diminished and/or protective factors strengthened as measured by the COMs and TOM.
No or Non-Significant Outcome- Completed all three phases and received outcome ratings on COMs and TOMs that were consistently 1s and 2s among all family members and therapist.* Family is basically functioning the same or nearly the same as when FFT began. No (or minimal) meaningful changes are evident among risk and/or protective factors as measured by the COMs and TOM.
Worse Outcome- Completed all three phases and received outcome ratings on COMs and TOMs that were mostly 0s among all family members and therapist.* Risk factors have increased and/or protective factors have diminished as measured by the COMs and TOM.This field is required! *In the event that COMs are not available, the TOM may be used as a basis for outcomes.

16. If Case Does not close Successfully
Drop – out Time: Enter the time of the unsuccessful case closure. We track this to look for trends of when families leave services
Drop – Out Reason: There are 8 different choices. Select the one that matches closest for the family

17. Drop-Out Definitions
Administrative Discharge: This non-completion category is for families that were discharged by the FFT agency because they (a) did not meet the criteria for FFT; (b) were incarcerated for pre-referral reasons only; or (c) funding for treatment was terminated. This termination is NOT considered a treatment failure AND is NOT included in the formula for calculating the percent of successfully completed cases
Incarcerated: This non-completion category is for families that were terminated because the youth was placed in a justice commitment facility during the course of treatment and/or was scheduled to go to a commitment facility at the end of treatment for activities/violations that occurred once treatment started. If clinical contact occurs during the detention or incarcerated period the case may remain open. This termination is considered a treatment failure AND is included in the formula for calculating percent of successfully completed cases.

18. Moved: This non-completion category is for families that were terminated because the youth was moved outside the service delivery area during the course of FFT treatment. This termination is NOT considered a treatment failure AND is NOT included in the formula for calculating the percent of successfully completed cases
Placed out of Home: This non-completion category is for families that were terminated because the youth was placed in foster care or long-term psychiatric placement (or to a similar treatment setting) before the completion of services. This termination is considered a treatment failure AND is included in the formula for calculating percent of successfully completed cases.

19. Quit after First Session: This non-completion category is for families that were terminated because the family quit after receiving at least one FFT face-to-face session. This category captures all cases that DROPOUT of FFT prior to the planned discharge. This termination is considered a treatment failure AND is included in the formula for calculating the percent of successfully completed cases. Previously labeled “Quit after Contact”.
Runaway: This non-completion category is for families that were terminated because the youth ran away from home for an extended period of time preventing services from being completed. This termination is considered a treatment failure AND is included in the formula for calculating the percent of successfully completed cases.

20. Youth Deceased: This non-completion category is for families that were terminated because the youth died during the course of treatment. This termination is considered a treatment failure AND is included in the formula for calculating the percent of successfully completed cases.
Referred to Other Services: This non-completion category is for families that were terminated because (a) the referral source ended treatment due to overlapping services; or (b) the family was initially referred to FFT and other services and decided to go to the other services. This termination is NOT considered a treatment failure AND is NOT included in the formula for calculating the percent of successfully completed cases.

21. If Selecting Never Seen Case Closure Status
Administrative Discharge: This “never seen” category is for families who were discharged by the FFT agency because they (a) did not meet the criteria for FFT; (b) did not meet referral criteria for the program/agency; (c) were incarcerated before treatment started; or (d) funding for treatment was terminated. These families never had a FFT session.
Declined Services: This “never seen” category is for families who refused to authorize FFT services. These families never had a FFT session.
Never Attended Initial Appointment: This “never seen” category is for families who authorized and agreed to receive FFT services but never attended any scheduled FFT sessions.
Not Able to Contact: This “never seen” category is for families who were not able to be contacted by the therapist. The therapist attempted all possible ways of contacting the family, and no contact was successfully made. These families never had a FFT session.
Other Treatment Conflict: This “never seen” category is for families who did not start FFT treatment due to already being enrolled in different treatment services. These families never had a FFT session.
Youth Whereabouts Unknown: This “never seen” category is for families who authorized/agreed to FFT services but the youth could not be located. These families never had a FFT session.

22. Post Assessments Outcome Questionnaire Youth Outcome Questionnaire
Youth Outcome Questionnaire – Self Report
Client Outcome Measure Parent(COM-P)
Client Outcome Measure Adolescent(COM-A)
Therapist Outcome Measure

23. Data Markers-
Utilization – Track average number of families for each therapist over given time period – Target is families for full-time therapist and 5-7 for part-time
Successful Completions – Take total cases closed and subtract non treatment failures(administrative discharge, move and youth referred to other services). Divide total cases closed successfully by adjusted cases closed number = successful completion percentage.
Target – 70% for Phase 1 team and 80% for Phase 1 and Beyond
Unsuccessful Completions – we review reasons case unsuccessful and at what point in the treatment

24. Data Markers Treatment Pacing – We track
Referral to 1st Session: Date therapist begins working the case to open and time of first session. Target is 7 days or less
1st to 2nd Session: Target is 7 days or less
2nd to 3rd Session: Target is 7 days or less

25. Data Markers
Total Days in Program – length of time between referral and case closure date. Target is an average of days
Average Dissemination Adherence – Average of the weekly ratings given by supervisor when therapist staffs case. This rating is of CSS documentation, treatment pacing, and contacts with referral sources. Target is 4 or above
Fidelity – Average of the weekly ratings give by supervisor when therapist staff case. This is clinical adherence + clinical competence. Target is 3 or above

26. Data Markers We track the completion of Pre and Post Assessments
OQ’s and YOQ’s – Target is 90% or above
COM’s – Target is 90% or above
TOM’s – Target is 100%

27. Data Markers for Supervisors
We track the total number of dissemination adherence and fidelity ratings given over a quarter. The target is to average 11 ratings per therapist each quarter.
We track the total number of Global Therapist Ratings given over the course of a year. The target is 1 Global Therapist Rating per therapist each quarter.

28. Tracking Data Results In:
California FFT 2011 – January 1, 2011-December 31, N=1381 families

29. Completions by Category

30. Average Days In Program for Successfully Completed Cases

31. Average Days between Referral and First Session

32. CSS usage – taken from weekly case report for active cases