1. Cricket Mitchell, PhD Senior Associate, CIMH
TF-CBT Program Performance
and Outcome Evaluation
Cricket Mitchell, PhD
Senior Associate, CIMH
TFCBT Evaluation Training

2. TF-CBT Evaluation: What You Should Have
Guide to Evaluation and All Measures (PDF File)
TFCBT Outcome Evaluation Guide and Forms-3
“Quick-Start” instructions for evaluation
Measures of symptom-specific and global youth mental health functioning (PTSD-RI and YOQ)
Database (Excel File)
TFCBT Data Entry Shell2
Holds all data for all children served
Submit to CIMH twice a year; keep entering new data into same database (ongoing record)
TFCBT Evaluation Training

3. Overview of Training
Standardized Measures Used in the TF-CBT Evaluation
UCLA Post Traumatic Stress Reaction Index©
(PTSD-RI)
Description, Administration, Scoring, Clinical Utility
Youth Outcome Questionnaires©
(YOQ & YOQ-SR)
Instructions for TF-CBT Data Entry & Data Submissions
TFCBT Evaluation Training

4. Overview of Training
Standardized Measures Used in the TF-CBT Evaluation
UCLA Post Traumatic Stress Reaction Index©
(PTSD-RI)
Description, Administration, Scoring, Clinical Utility
Youth Outcome Questionnaires©
(YOQ & YOQ-SR)
Instructions for TF-CBT Data Submissions
TFCBT Evaluation Training

5. Importance of Program Performance and Outcome Evaluation
Assessment is the beginning of developing a relationship with the child and parents
Demonstrates a desire to know what the child is experiencing
By incorporating standardized measures of functioning, the efficiency and thoroughness of assessment is enhanced
TFCBT Evaluation Training

6. Importance of Program Performance and Outcome Evaluation
Using standardized measures of functioning…
Assists in initial clinical impressions
Provides valuable information to guide treatment/interventions
Assesses sufficiency of treatment delivered
Demonstrates treatment-related improvements in child functioning
TFCBT Evaluation Training

7. UCLA Post-Traumatic Stress Reaction Index© (PTSD-RI)
TFCBT Evaluation Training

8. CIMH PTSD-RI Training Content for today’s training courtesy of:
National Center for Child Traumatic Stress at UCLA
2004 article in Current Psychiatry Reports by Alan Steinberg, Melissa Brymer, Kelly Decker and Robert Pynoos (included in LA PEI MAP Evaluation Guide and Forms PDF file)
National Child Traumatic Stress Network
Video-taped training on the administration and scoring of the PTSD-RI (Alan Steinberg, William Saltzman and Melissa Brymer)
Personal communications with Laura Murray
LA PEI MAP Evaluation Training: PTSD-RI

9. PTSD-RI Description Symptom-specific measure of functioning
Assesses the frequency of occurrence of children’s post-traumatic stress reactions
Parent/caregiver report for children 3 and older
Self-report for children 7 and older
Readability is age 12
Preferred method of administration is to read aloud to informant
Sensitive to clinical change over time
Valid and reliable
TFCBT Evaluation Training

10. PTSD-RI Description Part I: 14 items Part II: 13 items
Assesses lifetime history of exposure to trauma
Yes or No
If more than one trauma, focus on event most currently bothersome
Part II: 13 items
Assesses objective and subjective features of the trauma exposure
Maps on to DSM-IV Criteria A1 & A2
Part III: 20 items
Assesses the frequency of PTS symptoms during the past month
Maps on to DSM-IV Criteria B, C & D
5-point Likert scale response options
0 = None (of the time)
1 = Little (of the time)
2 = Some (of the time)
3 = Much (of the time)
4 = Most (of the time)
TFCBT Evaluation Training

11. Let’s take a look at the PTSD-RI
Let’s take a look at the PTSD-RI... (pgs 6-10 in the TF-CBT Outcome Evaluation Guide and Forms pdf)
LA PEI MAP Evaluation Training: PTSD-RI

12. PTSD-RI Administration
Readability is age 12
Preferred method of administration is to read measure aloud to informant
Ask parents and youth to respond to the questionnaires as honestly as possible
Informants can easily be influenced by the attitude of the person administering the scale
Let them know that this questionnaire will help you, as a clinician, better understand how the child is doing overall
Ask parents and children to complete all items
“Don’t Know” responses are not scored
LA PEI MAP Evaluation Training: PTSD-RI

13. PTSD-RI Administration
Part I & II: Lifetime history of exposure
Part III: “How much of the time during the past month?”
For Part III, guide the respondent through the Frequency Rating Scale prior to administration
Tear off Page 5 so they can see it while they respond
Explain each response option
Ensure understanding of each response option
“Suppose I ask you how often in the past month you…
… had a headache?”
… did your homework?”
… had green hair?”
LA PEI MAP Evaluation Training: PTSD-RI

14. Let’s take a look at the PTSD-RI Scoring Worksheet
Let’s take a look at the PTSD-RI Scoring Worksheet... (pg 27 in the TF-CBT Outcome Evaluation Guide and Forms pdf)
LA PEI MAP Evaluation Training: PTSD-RI

15. PTSD-RI Scoring Each DSM-IV Criterion is established separately
DSM-IV Criterion A
Exposure to trauma (Part 1)
At least 1 “Yes” on Q#s 1-13
Criterion A1 (Part II)
To be met, >1 “Yes” on Q#s 15-21
Criterion A2 (Part II)
To be met, >1 “Yes” on Q#s 22-26
Criterion A
To be met, exposure to trauma and A1 and A2 met
LA PEI MAP Evaluation Training: PTSD-RI

16. PTSD-RI Scoring DSM-IV Criteria B, C, and D
Based on Part III: Questions 1-20
Transfer each item’s response score onto the scoring sheet next to the appropriate Question #
PTSD-RI items map directly on to DSM-IV criteria
Except Q14 & Q20 which assess associated features
Severity score for each Criterion is the sum of the items that map on to that Criterion
Each Criterion is met (to assist in your diagnostic impressions) if a minimum number of symptoms are “present” (see slides 17-19)
Symptom Cutoff Score >3
A score of 3 or 4 (much or most of the time) indicates that a symptom is “present”
LA PEI MAP Evaluation Training: PTSD-RI

17. PTSD-RI Scoring DSM-IV Criterion B (re-experiencing)
Met if >1 of the 5 symptoms present* (*score 3 or 4)
DSM-IV Criteria
PTSD-RI Items
B1) recurrent and intrusive thoughts
B2) recurrent distressing dreams
B3) acting or feeling as if event recurring
B4) intense psychological distress at cues
B5) physiological reactivity to cues
Q#3*
Q#5
Q#6
Q#2
Q#18
LA PEI MAP Evaluation Training: PTSD-RI

18. PTSD-RI Scoring DSM-IV Criterion C (avoidance)
Met if >3 of the 7 symptoms present* (*score 3 or 4)
DSM-IV Criteria
PTSD-RI Items
C1) avoids thoughts, feelings or talks
C2) avoids activities, places or people
C3) inability to recall important aspect
C4) decreased interest in activities
C5) feelings of detachment
C6) restricted range of affect
C7) sense of foreshortened future
Q#9
Q#17
Q#15
Q#7
Q#8
Q#10 or Q#11
Q#19
LA PEI MAP Evaluation Training: PTSD-RI

19. PTSD-RI Scoring DSM-IV Criterion D (increased arousal)
Met if >2 of the 5 symptoms present* (*score 3 or 4)
DSM-IV Criteria
PTSD-RI Items
D1) difficulty falling or staying asleep
D2) irritability or outbursts of anger
D3) difficulty concentrating
D4) hypervigilance
D5) exaggerated startle response
Q#13
Q#4
Q#16
Q#1
Q#12
LA PEI MAP Evaluation Training: PTSD-RI

20. PTSD-RI Scoring PTSD Severity Overall/Total Score
Based on Part III: Questions 1-20
Transfer each item’s response score onto the scoring sheet next to the appropriate Question #
For one item score, transfer only the higher number of two Question responses
Q#s 10 or 11
Note that the Parent Scoring Worksheet also states to select higher score between Q#s 3 and 21.
Items 14 and 20 are omitted (Associated Features)
A total of 17 items are summed to determine the PTSD Severity Overall/Total Score
Post-TF-CBT administration only includes Part III – the frequency of symptom items
LA PEI MAP Evaluation Training: PTSD-RI

21. Clinical Utility of the PTSD-RI
PTSD Severity Overall/Total Score
Possible scores range from 0 to 68
Clinical cutpoint is 38 or higher
Scores in the high 20’s and 30’s indicate sub-clinical, yet significant levels of PTS reactions that are appropriate for intervention
LA PEI MAP Evaluation Training: PTSD-RI

22. Clinical Utility of the PTSD-RI
Asks about a broad range of traumatic events
Primary reason for referral may not be only history of trauma exposure
Structured questions can help elicit additional information about exposure to traumatic events
Often, children have not been asked directly about traumatic events they’ve experienced
LA PEI MAP Evaluation Training: PTSD-RI

23. Clinical Utility of the PTSD-RI
Although not a diagnostic tool, the PTSD-RI can inform clinical impressions
Items map directly onto DSM-IV Criteria for PTSD,
Each item/question in Part III is labeled with subscript indicating the specific DSM-IV criterion (letter and number) that it assesses
e.g., 1D4 … 3B1 … 7C4
AF = Associated Feature (i.e., guilt, avoidance)
LA PEI MAP Evaluation Training: PTSD-RI

24. Clinical Utility of the PTSD-RI
Informs clinician about specific post-traumatic stress reactions that are most bothersome to this particular child
Helps prioritize symptoms for intervention
Guides specific techniques that will be used
Guides psycho-education
Not all symptoms need to be normalized for each child presenting with PTSD or PTS reactions
LA PEI MAP Evaluation Training: PTSD-RI

25. Clinical Utility of the PTSD-RI
Research has shown that certain types of treatment approaches are better for certain aspects of PTS symptoms
e.g.,
avoidance responds best to in vivo types of exposure
sleep disturbances would suggest the use of behavioral regimens and/or relaxation techniques
significant rumination and self-blame would indicate the need for cognitive interventions
LA PEI MAP Evaluation Training: PTSD-RI

26. Clinical Utility of the PTSD-RI
Comparisons of pre/post scores reveal areas of clinical improvement
e.g.,
Does the child’s Overall/Total PTSD Severity Score decrease substantially?
Does the child’s symptomotology improve in all domains of post-traumatic stress reactions?
LA PEI MAP Evaluation Training: PTSD-RI

27. Youth Outcome Questionnaires© (YOQ & YOQ-SR)
TFCBT Evaluation Training

28. CIMH YOQ & YOQ-SR Training
Information on the administration, scoring, and clinical utility of the YOQ & YOQ-SR was obtained from each measure’s respective Administration and Scoring Manual published by OQ Measures, LLC
LA PEI MAP Evaluation Training: YOQs

29. YOQ & YOQ-SR Description
General measure of functioning
Assesses the global mental health functioning of children
Parent/caregiver report for children ages 4-18
Self-report for adolescents 12-18
Sensitive to clinical change in short periods of time
Valid and reliable
LA PEI MAP Evaluation Training: YOQs

30. YOQ & YOQ-SR Description
Parallel versions of the same measure
64 items
5-point Likert scale response options*
Never or Almost Never
Rarely
Sometimes
Frequently
Almost Always or Always
*response values vary by item
Six Scale Scores
Intrapersonal Distress (ID)
Somatic (S)
Interpersonal Relations (IR)
Social Problems (SP)
Behavioral Dysfunction (BD)
Critical Items (CI)
Total Score
LA PEI MAP Evaluation Training: YOQs

31. Let’s take a look at the YOQ-SR
Let’s take a look at the YOQ-SR... (pgs in the TF-CBT Outcome Evaluation Guide and Forms pdf)
LA PEI MAP Evaluation Training: YOQs

32. YOQ & YOQ-SR Administration
“… during the past 7 days.”
Ask parents and youth to fill out the questionnaires as honestly as possible
Informants can easily be influenced by the attitude of the person administering the scale
Let them know that this questionnaire will help you, as a clinician, better understand how the child is doing overall
Ask parents and youth to complete all items
LA PEI MAP Evaluation Training: YOQs

33. YOQ & YOQ-SR Scoring
For each item, transfer the value corresponding to the selected response into the box at the right-hand side of the page titled, ‘For Office Use Only’
Each item loads onto one of the six scales (e.g., ID, SP, IR)
Note that some items have negative response option values
Sum the items in each scale on Side1
Add the numbers in all boxes under the heading ID and enter that subtotal into the ID box at the bottom of the page
Repeat for each scale
Sum the items in each scale on Side 2
Transfer the subtotals from Side 1 to Side 2
Sum the subtotals to determine Scale Scores
Sum the Scale Scores to determine Total Score
Note that it is possible to have negative values for scores
LA PEI MAP Evaluation Training: YOQs

34. YOQ & YOQ-SR Scoring Missing Data (items that are left blank)
If 5 or more items are missing, consider the questionnaire invalid
Substitute a mean item response for the missing item
Determine in which scale the missing item belongs
Add up the other items in that scale, and determine their average
Substitute the average score for the missing response
LA PEI MAP Evaluation Training: YOQs

35. Clinical Utility of the YOQ & YOQ-SR
Total Score
Possible scores range from -16 to 240
Clinical cutpoints
46 or higher on the YOQ
47 or higher on the YOQ-SR
Lower scores indicate more normative, non-clinical, aspects of general mental health functioning
Elevations on certain scales indicate areas of specific distress for the child
LA PEI MAP Evaluation Training: YOQs

36. Clinical Utility of the YOQ & YOQ-SR
Intrapersonal Distress (ID) Scale
Possible scores range from -4 to 68
Clinical cutpoints
16 or higher on the YOQ
17 or higher on the YOQ-SR
Assesses the amount of emotional distress in the child, including anxiety, depression, fearfulness, hopelessness, and self-harm
High scores indicate a considerable degree of intrapersonal distress in the child
LA PEI MAP Evaluation Training: YOQs

37. Clinical Utility of the YOQ & YOQ-SR
Somatic (S) Scale
Possible scores range from 0 to 32
Clinical cutpoints
5 or higher on the YOQ
6 or higher on the YOQ-SR
Indicates change in somatic distress or physical complaints
High scores indicate the parent/caregiver is aware of, or the youth is experiencing, a high number of somatic symptoms; while low scores indicate either absence or unawareness of them
LA PEI MAP Evaluation Training: YOQs

38. Clinical Utility of the YOQ & YOQ-SR
Interpersonal Relations (IR) Scale
Possible scores range from -6 to 34
Clinical cutpoints
4 or higher on the YOQ
3 or higher on the YOQ-SR
Assesses issues relevant to the child’s relationship with parents, other adults, and peers
High scores indicate significant interpersonal difficulty; while low scores reflect a cooperative, pleasant interpersonal demeanor
LA PEI MAP Evaluation Training: YOQs

39. Clinical Utility of the YOQ & YOQ-SR
Social Problems (SP) Scale
Possible scores range from -2 to 68
Clinical cutpoints
3 or higher on the YOQ & YOQ-SR
Assesses problems that are socially related including aggression and delinquency
A feature of these items is that they are slow to change; whereas, content tapped by many of the other scales often changes over a period of time as a result of treatment intervention
LA PEI MAP Evaluation Training: YOQs

40. Clinical Utility of the YOQ & YOQ-SR
Behavioral Dysfunction (BD) Scale
Possible scores range from -4 to 40
Clinical cutpoints
12 or higher on the YOQ
11 or higher on the YOQ-SR
Assesses inattention, hyperactivity, impulsivity, concentration, ability to organize tasks, and ability to handle frustration
LA PEI MAP Evaluation Training: YOQs

41. Clinical Utility of the YOQ & YOQ-SR
Critical Items (CI) Scale
Possible scores range from 0 to 36
Clinical cutpoints
5 or higher on the YOQ
6 or higher on the YOQ-SR
Assesses areas such as paranoia, obsessive-compulsive behaviors, hallucination, delusions, suicide, mania, and eating disorders
LA PEI MAP Evaluation Training: YOQs

42. Clinical Utility of the YOQ & YOQ-SR
Assesses a variety of specific areas of difficulty in youth mental health functioning
Assists in initial clinical impressions
Provides valuable information to guide treatment/interventions
Comparisons of pre/post scores reveal areas of clinical improvement as well as areas of potential unmet need
LA PEI MAP Evaluation Training: YOQs

43. Summary of YOQ/YOQ-SR Score Ranges and Clinical Cutpoints
YOQ/YOQ-SR Scale
Range of possible scores
Clinical Cutpoint for YOQ
Clinical Cutpoint for YOQ-SR
Intrapersonal Distress (ID)
-4 to 68 16 17
Somatic (S)
0 to 32 5 6
Interpersonal Relations (IR)
-6 to 34 4 3
Social Problems (SP)
-2 to 30
Behavioral Dysfunction (BD)
-4 to 40 12 11
Critical Items (CI)
0 to 36
Total Score
-16 to 240 46 47
TFCBT Evaluation Training

44. TF-CBT Data Entry & Data Submissions
TFCBT Evaluation Training

45. TF-CBT Data Entry Shell2 – ‘Demos, etc’ Spreadsheet
Client Information:
Early Therapy Information:
 End of Therapy Information:
(DSM-IV code)
Date of
Total #
Completed
Client ID
DOB
Gender
Ethnicity
Primary
Axis I
Therapist ID
Referral
First Session
Last Session
of Sessions
TF-CBT?
The last three fields are for end-of-therapy information.
Data should only be entered when a child is no longer in TF-CBT.
TFCBT Evaluation Training

46. TF-CBT Data Entry Tips – ‘Demos, etc’ Spreadsheet (slide 1 of 2)
Client ID automatically populates into the other spreadsheets (pre and post outcome data on the PTSD-RI and YOQ/YOQ-SR measures)
Use a unique identifier for each child (e.g., chart #)
Pull-down menus are provided for all categorical data (gender, ethnicity, and Completed TF-CBT?)
DSM-IV Axis I diagnosis should be entered as the numerical code
Enter only one DSM-IV numerical code
If more than one Axis I diagnosis applies, enter the Primary Axis I
TFCBT Evaluation Training

47. TF-CBT Data Entry Tips – ‘Demos, etc’ Spreadsheet (slide 2 of 2)
The last three fields (Date of Last Session, Total # of Sessions, and Completed TF-CBT?) are end-of-therapy fields only
They should be blank/empty for children currently being seen
They should be filled in for children no longer in TF-CBT
Completed TF-CBT?
Select ‘yes’ if the child completed the intervention
Select ‘no’ if the child dropped out of TF-CBT prior to completing the intervention (e.g., moved away, stopped coming)
TFCBT Evaluation Training

48. TF-CBT Data Entry Excel File – Pre & Post PTSD-RI Spreadsheets
Pre-TF-CBT
PARENT REPORT:
CHILD REPORT:
PTSD-RI
Total
Client ID
Severity Score
Post-TF-CBT
PARENT REPORT:
CHILD REPORT:
PTSD-RI
Total
Client ID
Severity Score
TFCBT Evaluation Training

49. TF-CBT Data Entry Tips – Pre and Post PTSD-RI Spreadsheets
There is a separate spreadsheet for the Pre- PTSD-RI data and the Post- PTSD-RI data
Enter the PTSD Overall/Severity Score for the Parent and Child informants into their respective fields
Do not enter text into these fields
If data are missing, leave the fields blank
TFCBT Evaluation Training

50. TF-CBT Data Entry Excel File – Pre & Post YOQ Spreadsheets
Pre- TF-CBT
PARENT/CAREGIVER REPORT:
YOUTH SELF-REPORT:
YOQ (Scale Scores and Total Score)
YOQ-SR (Scale Scores and Total Score)
Client ID ID S IR SP BD CI
Total
Post- TF-CBT
PARENT/CAREGIVER REPORT:
YOUTH SELF-REPORT:
YOQ (Scale Scores and Total Score)
YOQ-SR (Scale Scores and Total Score)
Client ID ID S IR SP BD CI
Total
TFCBT Evaluation Training

51. TF-CBT Data Entry Tips – Pre and Post YOQ Spreadsheets
There is a separate spreadsheet for the Pre- YOQ & YOQ-SR data and the Post- YOQ & YOQ-SR data
Enter the Scale Scores and the Total Score for the Parent and Adolescent informants into their respective fields
Do not enter text into these fields
If data are missing, leave the fields blank
TFCBT Evaluation Training

52. TF-CBT Data Submissions
Data submissions occur twice a year
The end of February (reflecting children served thru January)
The end of August (reflecting children served through July)
Note that this is the anticipated schedule; actual data submission dates may vary slightly
An notice is sent approximately one month in advance of each data submission deadline
After data are submitted, sites continue to enter new data into the same database (always reflects an ongoing, historical record of children served)
Every effort is made to distribute reports within two months of data submission
Aggregate and site-specific reports
TFCBT Evaluation Training

53. Questions
TFCBT Evaluation Training

54. The End
Contact Information
Cricket Mitchell, PhD
Cell phone:
TFCBT Evaluation Training