The Clinical Services Systems – our tool to evaluate services California Symposium, April, 2012 Kimberly Mason, MSW, LCSW

Clinical Services System (CSS) Established with the onset of widespread dissemination of the model Went online 2001 Important way to track what is happening in sessions and outcomes for families receiving services If it is not documented – it didn’t occur

Entering Client Information We use this to track the gender, ethnicity and types of referral problems that sites are serving – this becomes important as programs look for funding and we look for ways to improve training and services You should begin entering information when you start working the case to open for services.

Pre Assessments Outcome Questionnaire – Measurement of Individual Functioning. Cut-Off Score is 63. Given to family members 18 or older Youth Outcome Questionnaire – Parent/Guardian fills in about youth behavior. Cut-Off Score is 46 Youth Outcome Questionnaire – Self Report – Youth fills out about their own behavior. Cut-Off Score is 47

Open Date Documentation You give the OQ, YOQ and YOQ-SR and enter information into the CSS You enter client information into the CSS. This begins the process of data collection Definitions – Referral Date – enter the date that the referral came to you as a therapist to open the case Open Date – the date that consent for treatment paperwork is signed

Progress Notes You must Complete a Progress Note on each family Session Progress Notes are Phase Dependant and we track the number of sessions in each phase We also track the time between sessions – especially the time between the first 3 sessions

Progress Notes Engagement/Motivation Notes – Supervisor/Consultant looks to these to see what techniques being used, progress towards goals, relational assessments and planning for next session Behavior Change Note – Look to these for the overall behavior change plan – skills targeted and then the actual teaching, modeling or directing of those skills Generalization Note – Look for the overall plan, progress towards goals, linkage back to referral sources

Family Self-Report This is the Opportunity for the family to give feedback on their progress in therapy First 3 questions are related to how they see their family and their hope for the future The last 4 questions are about your alliance with the family The Family Self-Report should be given to each family member participating in therapy at the first two sessions of every phase

Therapist Self-Report This is the opportunity for therapist to gauge their progress towards meeting goals. Therapist Self-Report should be completed whenever the family completes a FSR We track the completion of these documents as part of Dissemination Adherence

Closing a Case We track length of time in the program, therefore it is very important that a case be closed in CSS when services are completed. It is also essential that you close the case using definitions that are present in the CSS

Step by Step of Closing Case Make sure that you have the correct client selected and then go to Add New Termination Enter Termination Date - date of last clinical contact with the family (for families that complete FFT) or date of last ATTEMPT at clinical contact with the family (for families that do not complete FFT). Termination date is not considered the date you enter the termination data into the CSS.

Select Type – Finished Counseling –A Completed Case is a case where the family completes all three phases of FFT.  Completed Cases are given a Finished Result at Termination. COMs are entered and a TOM is completed by the therapist on the CSS Termination Page. Dropped-Out of Counseling - A Non-Completed Case is a case where treatment is interrupted and the family does not complete all three phases of FFT. Efforts to reengage the family do not result in a subsequent session.  A Non-Completed Case is considered a drop out.  These cases are given both a Drop Out Reason, a Time of Drop Out description, and a TOM on the CSS Termination page. Non-Completed Cases do not receive a Finished Result but receive a completed TOM. Never Seen - Cases that were Never Seen are defined as those cases that never had a FFT session.

Select from the following: Youth Remains in the Home/Family – At the time of case closure is youth living at home Youth is enrolled in an educational/vocational school program or working – At the time of case closure youth has either completed schooling, is working or is enrolled in some type of educational program Youth is free of New Law Violations since FFT Treatment began? – During the course of FFT did youth receive an new law violations – this would not include violations of probation or contact with law enforcement – only if youth was charged with new offense

Before Completing Closure for Successfully Completed Cases you need to have the COM-A, COM-P and TOM Completed as these will be basis for the case closure status Client Outcome Measure – Adolescent (COM-A) – completed by youth at time of successful case closure and measures youth belief about family change. Client Outcome Measure – Parent (COM-P) – completed by parent/guardian at time of successful case closure and measures parent belief about family change Therapist Outcome Measure (TOM) – completed at case closure regardless of closing status and measures therapist belief about family change

*In the event. Positive Outcome-Completed all three phases and received outcome ratings on COMs and TOMs that were consistently 3s, 4s and 5s among all family members and therapists.* Risk factors have been significantly diminished and/or protective factors strengthened as measured by the COMs and TOM. No or Non-Significant Outcome- Completed all three phases and received outcome ratings on COMs and TOMs that were consistently 1s and 2s among all family members and therapist.* Family is basically functioning the same or nearly the same as when FFT began. No (or minimal) meaningful changes are evident among risk and/or protective factors as measured by the COMs and TOM. Worse Outcome- Completed all three phases and received outcome ratings on COMs and TOMs that were mostly 0s among all family members and therapist.* Risk factors have increased and/or protective factors have diminished as measured by the COMs and TOM.This field is required! *In the event that COMs are not available, the TOM may be used as a basis for outcomes.

If Case Does not close Successfully Drop – out Time: Enter the time of the unsuccessful case closure. We track this to look for trends of when families leave services Drop – Out Reason: There are 8 different choices. Select the one that matches closest for the family

Drop-Out Definitions Administrative Discharge: This non-completion category is for families that were discharged by the FFT agency because they (a) did not meet the criteria for FFT; (b) were incarcerated for pre-referral reasons only; or (c) funding for treatment was terminated. This termination is NOT considered a treatment failure AND is NOT included in the formula for calculating the percent of successfully completed cases Incarcerated: This non-completion category is for families that were terminated because the youth was placed in a justice commitment facility during the course of treatment and/or was scheduled to go to a commitment facility at the end of treatment for activities/violations that occurred once treatment started. If clinical contact occurs during the detention or incarcerated period the case may remain open. This termination is considered a treatment failure AND is included in the formula for calculating percent of successfully completed cases.

Moved: This non-completion category is for families that were terminated because the youth was moved outside the service delivery area during the course of FFT treatment. This termination is NOT considered a treatment failure AND is NOT included in the formula for calculating the percent of successfully completed cases Placed out of Home: This non-completion category is for families that were terminated because the youth was placed in foster care or long-term psychiatric placement (or to a similar treatment setting) before the completion of services. This termination is considered a treatment failure AND is included in the formula for calculating percent of successfully completed cases.

Quit after First Session: This non-completion category is for families that were terminated because the family quit after receiving at least one FFT face-to-face session. This category captures all cases that DROPOUT of FFT prior to the planned discharge. This termination is considered a treatment failure AND is included in the formula for calculating the percent of successfully completed cases. Previously labeled “Quit after Contact”.   Runaway: This non-completion category is for families that were terminated because the youth ran away from home for an extended period of time preventing services from being completed. This termination is considered a treatment failure AND is included in the formula for calculating the percent of successfully completed cases.

Youth Deceased: This non-completion category is for families that were terminated because the youth died during the course of treatment. This termination is considered a treatment failure AND is included in the formula for calculating the percent of successfully completed cases. Referred to Other Services: This non-completion category is for families that were terminated because (a) the referral source ended treatment due to overlapping services; or (b) the family was initially referred to FFT and other services and decided to go to the other services. This termination is NOT considered a treatment failure AND is NOT included in the formula for calculating the percent of successfully completed cases.

If Selecting Never Seen Case Closure Status Administrative Discharge: This “never seen” category is for families who were discharged by the FFT agency because they (a) did not meet the criteria for FFT; (b) did not meet referral criteria for the program/agency; (c) were incarcerated before treatment started; or (d) funding for treatment was terminated. These families never had a FFT session.   Declined Services: This “never seen” category is for families who refused to authorize FFT services. These families never had a FFT session. Never Attended Initial Appointment: This “never seen” category is for families who authorized and agreed to receive FFT services but never attended any scheduled FFT sessions. Not Able to Contact: This “never seen” category is for families who were not able to be contacted by the therapist. The therapist attempted all possible ways of contacting the family, and no contact was successfully made. These families never had a FFT session. Other Treatment Conflict: This “never seen” category is for families who did not start FFT treatment due to already being enrolled in different treatment services. These families never had a FFT session. Youth Whereabouts Unknown: This “never seen” category is for families who authorized/agreed to FFT services but the youth could not be located. These families never had a FFT session.

Post Assessments Outcome Questionnaire Youth Outcome Questionnaire Youth Outcome Questionnaire – Self Report Client Outcome Measure Parent(COM-P) Client Outcome Measure Adolescent(COM-A) Therapist Outcome Measure

Data Markers- Utilization – Track average number of families for each therapist over given time period – Target is 10-12 families for full-time therapist and 5-7 for part-time Successful Completions – Take total cases closed and subtract non treatment failures(administrative discharge, move and youth referred to other services). Divide total cases closed successfully by adjusted cases closed number = successful completion percentage. Target – 70% for Phase 1 team and 80% for Phase 1 and Beyond Unsuccessful Completions – we review reasons case unsuccessful and at what point in the treatment

Data Markers Treatment Pacing – We track Referral to 1st Session: Date therapist begins working the case to open and time of first session. Target is 7 days or less 1st to 2nd Session: Target is 7 days or less 2nd to 3rd Session: Target is 7 days or less

Data Markers Total Days in Program – length of time between referral and case closure date. Target is an average of 60-180 days Average Dissemination Adherence – Average of the weekly ratings given by supervisor when therapist staffs case. This rating is of CSS documentation, treatment pacing, and contacts with referral sources. Target is 4 or above Fidelity – Average of the weekly ratings give by supervisor when therapist staff case. This is clinical adherence + clinical competence. Target is 3 or above

Data Markers We track the completion of Pre and Post Assessments OQ’s and YOQ’s – Target is 90% or above COM’s – Target is 90% or above TOM’s – Target is 100%

Data Markers for Supervisors We track the total number of dissemination adherence and fidelity ratings given over a quarter. The target is to average 11 ratings per therapist each quarter. We track the total number of Global Therapist Ratings given over the course of a year. The target is 1 Global Therapist Rating per therapist each quarter.

Tracking Data Results In: California FFT 2011 – January 1, 2011-December 31, 2011 N=1381 families

Completions by Category

Average Days In Program for Successfully Completed Cases

Average Days between Referral and First Session

CSS usage – taken from weekly case report for active cases