Cricket Mitchell, PhD Senior Associate, CIMH TF-CBT Program Performance and Outcome Evaluation Cricket Mitchell, PhD Senior Associate, CIMH TFCBT Evaluation Training

TF-CBT Evaluation: What You Should Have Guide to Evaluation and All Measures (PDF File) TFCBT Outcome Evaluation Guide and Forms-3 “Quick-Start” instructions for evaluation Measures of symptom-specific and global youth mental health functioning (PTSD-RI and YOQ) Database (Excel File) TFCBT Data Entry Shell2 Holds all data for all children served Submit to CIMH twice a year; keep entering new data into same database (ongoing record) TFCBT Evaluation Training

Overview of Training Standardized Measures Used in the TF-CBT Evaluation UCLA Post Traumatic Stress Reaction Index© (PTSD-RI) Description, Administration, Scoring, Clinical Utility Youth Outcome Questionnaires© (YOQ & YOQ-SR) Instructions for TF-CBT Data Entry & Data Submissions TFCBT Evaluation Training

Overview of Training Standardized Measures Used in the TF-CBT Evaluation UCLA Post Traumatic Stress Reaction Index© (PTSD-RI) Description, Administration, Scoring, Clinical Utility Youth Outcome Questionnaires© (YOQ & YOQ-SR) Instructions for TF-CBT Data Submissions TFCBT Evaluation Training

Importance of Program Performance and Outcome Evaluation Assessment is the beginning of developing a relationship with the child and parents Demonstrates a desire to know what the child is experiencing By incorporating standardized measures of functioning, the efficiency and thoroughness of assessment is enhanced TFCBT Evaluation Training

Importance of Program Performance and Outcome Evaluation Using standardized measures of functioning… Assists in initial clinical impressions Provides valuable information to guide treatment/interventions Assesses sufficiency of treatment delivered Demonstrates treatment-related improvements in child functioning TFCBT Evaluation Training

UCLA Post-Traumatic Stress Reaction Index© (PTSD-RI) TFCBT Evaluation Training

CIMH PTSD-RI Training Content for today’s training courtesy of: National Center for Child Traumatic Stress at UCLA 2004 article in Current Psychiatry Reports by Alan Steinberg, Melissa Brymer, Kelly Decker and Robert Pynoos (included in LA PEI MAP Evaluation Guide and Forms PDF file) National Child Traumatic Stress Network Video-taped training on the administration and scoring of the PTSD-RI (Alan Steinberg, William Saltzman and Melissa Brymer) Personal communications with Laura Murray LA PEI MAP Evaluation Training: PTSD-RI

PTSD-RI Description Symptom-specific measure of functioning Assesses the frequency of occurrence of children’s post-traumatic stress reactions Parent/caregiver report for children 3 and older Self-report for children 7 and older Readability is age 12 Preferred method of administration is to read aloud to informant Sensitive to clinical change over time Valid and reliable TFCBT Evaluation Training

PTSD-RI Description Part I: 14 items Part II: 13 items Assesses lifetime history of exposure to trauma Yes or No If more than one trauma, focus on event most currently bothersome Part II: 13 items Assesses objective and subjective features of the trauma exposure Maps on to DSM-IV Criteria A1 & A2 Part III: 20 items Assesses the frequency of PTS symptoms during the past month Maps on to DSM-IV Criteria B, C & D 5-point Likert scale response options 0 = None (of the time) 1 = Little (of the time) 2 = Some (of the time) 3 = Much (of the time) 4 = Most (of the time) TFCBT Evaluation Training

Let’s take a look at the PTSD-RI Let’s take a look at the PTSD-RI... (pgs 6-10 in the TF-CBT Outcome Evaluation Guide and Forms pdf) LA PEI MAP Evaluation Training: PTSD-RI

PTSD-RI Administration Readability is age 12 Preferred method of administration is to read measure aloud to informant Ask parents and youth to respond to the questionnaires as honestly as possible Informants can easily be influenced by the attitude of the person administering the scale Let them know that this questionnaire will help you, as a clinician, better understand how the child is doing overall Ask parents and children to complete all items “Don’t Know” responses are not scored LA PEI MAP Evaluation Training: PTSD-RI

PTSD-RI Administration Part I & II: Lifetime history of exposure Part III: “How much of the time during the past month?” For Part III, guide the respondent through the Frequency Rating Scale prior to administration Tear off Page 5 so they can see it while they respond Explain each response option Ensure understanding of each response option “Suppose I ask you how often in the past month you… … had a headache?” … did your homework?” … had green hair?” LA PEI MAP Evaluation Training: PTSD-RI

Let’s take a look at the PTSD-RI Scoring Worksheet Let’s take a look at the PTSD-RI Scoring Worksheet... (pg 27 in the TF-CBT Outcome Evaluation Guide and Forms pdf) LA PEI MAP Evaluation Training: PTSD-RI

PTSD-RI Scoring Each DSM-IV Criterion is established separately DSM-IV Criterion A Exposure to trauma (Part 1) At least 1 “Yes” on Q#s 1-13 Criterion A1 (Part II) To be met, >1 “Yes” on Q#s 15-21 Criterion A2 (Part II) To be met, >1 “Yes” on Q#s 22-26 Criterion A To be met, exposure to trauma and A1 and A2 met LA PEI MAP Evaluation Training: PTSD-RI

PTSD-RI Scoring DSM-IV Criteria B, C, and D Based on Part III: Questions 1-20 Transfer each item’s response score onto the scoring sheet next to the appropriate Question # PTSD-RI items map directly on to DSM-IV criteria Except Q14 & Q20 which assess associated features Severity score for each Criterion is the sum of the items that map on to that Criterion Each Criterion is met (to assist in your diagnostic impressions) if a minimum number of symptoms are “present” (see slides 17-19) Symptom Cutoff Score >3 A score of 3 or 4 (much or most of the time) indicates that a symptom is “present” LA PEI MAP Evaluation Training: PTSD-RI

PTSD-RI Scoring DSM-IV Criterion B (re-experiencing) Met if >1 of the 5 symptoms present* (*score 3 or 4) DSM-IV Criteria PTSD-RI Items B1) recurrent and intrusive thoughts B2) recurrent distressing dreams B3) acting or feeling as if event recurring B4) intense psychological distress at cues B5) physiological reactivity to cues Q#3* Q#5 Q#6 Q#2 Q#18 LA PEI MAP Evaluation Training: PTSD-RI

PTSD-RI Scoring DSM-IV Criterion C (avoidance) Met if >3 of the 7 symptoms present* (*score 3 or 4) DSM-IV Criteria PTSD-RI Items C1) avoids thoughts, feelings or talks C2) avoids activities, places or people C3) inability to recall important aspect C4) decreased interest in activities C5) feelings of detachment C6) restricted range of affect C7) sense of foreshortened future Q#9 Q#17 Q#15 Q#7 Q#8 Q#10 or Q#11 Q#19 LA PEI MAP Evaluation Training: PTSD-RI

PTSD-RI Scoring DSM-IV Criterion D (increased arousal) Met if >2 of the 5 symptoms present* (*score 3 or 4) DSM-IV Criteria PTSD-RI Items D1) difficulty falling or staying asleep D2) irritability or outbursts of anger D3) difficulty concentrating D4) hypervigilance D5) exaggerated startle response Q#13 Q#4 Q#16 Q#1 Q#12 LA PEI MAP Evaluation Training: PTSD-RI

PTSD-RI Scoring PTSD Severity Overall/Total Score Based on Part III: Questions 1-20 Transfer each item’s response score onto the scoring sheet next to the appropriate Question # For one item score, transfer only the higher number of two Question responses Q#s 10 or 11 Note that the Parent Scoring Worksheet also states to select higher score between Q#s 3 and 21. Items 14 and 20 are omitted (Associated Features) A total of 17 items are summed to determine the PTSD Severity Overall/Total Score Post-TF-CBT administration only includes Part III – the frequency of symptom items LA PEI MAP Evaluation Training: PTSD-RI

Clinical Utility of the PTSD-RI PTSD Severity Overall/Total Score Possible scores range from 0 to 68 Clinical cutpoint is 38 or higher Scores in the high 20’s and 30’s indicate sub-clinical, yet significant levels of PTS reactions that are appropriate for intervention LA PEI MAP Evaluation Training: PTSD-RI

Clinical Utility of the PTSD-RI Asks about a broad range of traumatic events Primary reason for referral may not be only history of trauma exposure Structured questions can help elicit additional information about exposure to traumatic events Often, children have not been asked directly about traumatic events they’ve experienced LA PEI MAP Evaluation Training: PTSD-RI

Clinical Utility of the PTSD-RI Although not a diagnostic tool, the PTSD-RI can inform clinical impressions Items map directly onto DSM-IV Criteria for PTSD, 309.81 Each item/question in Part III is labeled with subscript indicating the specific DSM-IV 309.81 criterion (letter and number) that it assesses e.g., 1D4 … 3B1 … 7C4 AF = Associated Feature (i.e., guilt, avoidance) LA PEI MAP Evaluation Training: PTSD-RI

Clinical Utility of the PTSD-RI Informs clinician about specific post-traumatic stress reactions that are most bothersome to this particular child Helps prioritize symptoms for intervention Guides specific techniques that will be used Guides psycho-education Not all symptoms need to be normalized for each child presenting with PTSD or PTS reactions LA PEI MAP Evaluation Training: PTSD-RI

Clinical Utility of the PTSD-RI Research has shown that certain types of treatment approaches are better for certain aspects of PTS symptoms e.g., avoidance responds best to in vivo types of exposure sleep disturbances would suggest the use of behavioral regimens and/or relaxation techniques significant rumination and self-blame would indicate the need for cognitive interventions LA PEI MAP Evaluation Training: PTSD-RI

Clinical Utility of the PTSD-RI Comparisons of pre/post scores reveal areas of clinical improvement e.g., Does the child’s Overall/Total PTSD Severity Score decrease substantially? Does the child’s symptomotology improve in all domains of post-traumatic stress reactions? LA PEI MAP Evaluation Training: PTSD-RI

Youth Outcome Questionnaires© (YOQ & YOQ-SR) TFCBT Evaluation Training

CIMH YOQ & YOQ-SR Training Information on the administration, scoring, and clinical utility of the YOQ & YOQ-SR was obtained from each measure’s respective Administration and Scoring Manual published by OQ Measures, LLC LA PEI MAP Evaluation Training: YOQs

YOQ & YOQ-SR Description General measure of functioning Assesses the global mental health functioning of children Parent/caregiver report for children ages 4-18 Self-report for adolescents 12-18 Sensitive to clinical change in short periods of time Valid and reliable LA PEI MAP Evaluation Training: YOQs

YOQ & YOQ-SR Description Parallel versions of the same measure 64 items 5-point Likert scale response options* Never or Almost Never Rarely Sometimes Frequently Almost Always or Always *response values vary by item Six Scale Scores Intrapersonal Distress (ID) Somatic (S) Interpersonal Relations (IR) Social Problems (SP) Behavioral Dysfunction (BD) Critical Items (CI) Total Score LA PEI MAP Evaluation Training: YOQs

Let’s take a look at the YOQ-SR Let’s take a look at the YOQ-SR... (pgs 37-38 in the TF-CBT Outcome Evaluation Guide and Forms pdf) LA PEI MAP Evaluation Training: YOQs

YOQ & YOQ-SR Administration “… during the past 7 days.” Ask parents and youth to fill out the questionnaires as honestly as possible Informants can easily be influenced by the attitude of the person administering the scale Let them know that this questionnaire will help you, as a clinician, better understand how the child is doing overall Ask parents and youth to complete all items LA PEI MAP Evaluation Training: YOQs

YOQ & YOQ-SR Scoring For each item, transfer the value corresponding to the selected response into the box at the right-hand side of the page titled, ‘For Office Use Only’ Each item loads onto one of the six scales (e.g., ID, SP, IR) Note that some items have negative response option values Sum the items in each scale on Side1 Add the numbers in all boxes under the heading ID and enter that subtotal into the ID box at the bottom of the page Repeat for each scale Sum the items in each scale on Side 2 Transfer the subtotals from Side 1 to Side 2 Sum the subtotals to determine Scale Scores Sum the Scale Scores to determine Total Score Note that it is possible to have negative values for scores LA PEI MAP Evaluation Training: YOQs

YOQ & YOQ-SR Scoring Missing Data (items that are left blank) If 5 or more items are missing, consider the questionnaire invalid Substitute a mean item response for the missing item Determine in which scale the missing item belongs Add up the other items in that scale, and determine their average Substitute the average score for the missing response LA PEI MAP Evaluation Training: YOQs

Clinical Utility of the YOQ & YOQ-SR Total Score Possible scores range from -16 to 240 Clinical cutpoints 46 or higher on the YOQ 47 or higher on the YOQ-SR Lower scores indicate more normative, non-clinical, aspects of general mental health functioning Elevations on certain scales indicate areas of specific distress for the child LA PEI MAP Evaluation Training: YOQs

Clinical Utility of the YOQ & YOQ-SR Intrapersonal Distress (ID) Scale Possible scores range from -4 to 68 Clinical cutpoints 16 or higher on the YOQ 17 or higher on the YOQ-SR Assesses the amount of emotional distress in the child, including anxiety, depression, fearfulness, hopelessness, and self-harm High scores indicate a considerable degree of intrapersonal distress in the child LA PEI MAP Evaluation Training: YOQs

Clinical Utility of the YOQ & YOQ-SR Somatic (S) Scale Possible scores range from 0 to 32 Clinical cutpoints 5 or higher on the YOQ 6 or higher on the YOQ-SR Indicates change in somatic distress or physical complaints High scores indicate the parent/caregiver is aware of, or the youth is experiencing, a high number of somatic symptoms; while low scores indicate either absence or unawareness of them LA PEI MAP Evaluation Training: YOQs

Clinical Utility of the YOQ & YOQ-SR Interpersonal Relations (IR) Scale Possible scores range from -6 to 34 Clinical cutpoints 4 or higher on the YOQ 3 or higher on the YOQ-SR Assesses issues relevant to the child’s relationship with parents, other adults, and peers High scores indicate significant interpersonal difficulty; while low scores reflect a cooperative, pleasant interpersonal demeanor LA PEI MAP Evaluation Training: YOQs

Clinical Utility of the YOQ & YOQ-SR Social Problems (SP) Scale Possible scores range from -2 to 68 Clinical cutpoints 3 or higher on the YOQ & YOQ-SR Assesses problems that are socially related including aggression and delinquency A feature of these items is that they are slow to change; whereas, content tapped by many of the other scales often changes over a period of time as a result of treatment intervention LA PEI MAP Evaluation Training: YOQs

Clinical Utility of the YOQ & YOQ-SR Behavioral Dysfunction (BD) Scale Possible scores range from -4 to 40 Clinical cutpoints 12 or higher on the YOQ 11 or higher on the YOQ-SR Assesses inattention, hyperactivity, impulsivity, concentration, ability to organize tasks, and ability to handle frustration LA PEI MAP Evaluation Training: YOQs

Clinical Utility of the YOQ & YOQ-SR Critical Items (CI) Scale Possible scores range from 0 to 36 Clinical cutpoints 5 or higher on the YOQ 6 or higher on the YOQ-SR Assesses areas such as paranoia, obsessive-compulsive behaviors, hallucination, delusions, suicide, mania, and eating disorders LA PEI MAP Evaluation Training: YOQs

Clinical Utility of the YOQ & YOQ-SR Assesses a variety of specific areas of difficulty in youth mental health functioning Assists in initial clinical impressions Provides valuable information to guide treatment/interventions Comparisons of pre/post scores reveal areas of clinical improvement as well as areas of potential unmet need LA PEI MAP Evaluation Training: YOQs

Summary of YOQ/YOQ-SR Score Ranges and Clinical Cutpoints YOQ/YOQ-SR Scale Range of possible scores Clinical Cutpoint for YOQ Clinical Cutpoint for YOQ-SR Intrapersonal Distress (ID) -4 to 68 16 17 Somatic (S) 0 to 32 5 6 Interpersonal Relations (IR) -6 to 34 4 3 Social Problems (SP) -2 to 30 Behavioral Dysfunction (BD) -4 to 40 12 11 Critical Items (CI) 0 to 36 Total Score -16 to 240 46 47 TFCBT Evaluation Training

TF-CBT Data Entry & Data Submissions TFCBT Evaluation Training

TF-CBT Data Entry Shell2 – ‘Demos, etc’ Spreadsheet Client Information: Early Therapy Information:  End of Therapy Information:   (DSM-IV code) Date of Total # Completed Client ID DOB Gender Ethnicity Primary Axis I Therapist ID Referral First Session Last Session of Sessions TF-CBT? The last three fields are for end-of-therapy information. Data should only be entered when a child is no longer in TF-CBT. TFCBT Evaluation Training

TF-CBT Data Entry Tips – ‘Demos, etc’ Spreadsheet (slide 1 of 2) Client ID automatically populates into the other spreadsheets (pre and post outcome data on the PTSD-RI and YOQ/YOQ-SR measures) Use a unique identifier for each child (e.g., chart #) Pull-down menus are provided for all categorical data (gender, ethnicity, and Completed TF-CBT?) DSM-IV Axis I diagnosis should be entered as the numerical code Enter only one DSM-IV numerical code If more than one Axis I diagnosis applies, enter the Primary Axis I TFCBT Evaluation Training

TF-CBT Data Entry Tips – ‘Demos, etc’ Spreadsheet (slide 2 of 2) The last three fields (Date of Last Session, Total # of Sessions, and Completed TF-CBT?) are end-of-therapy fields only They should be blank/empty for children currently being seen They should be filled in for children no longer in TF-CBT Completed TF-CBT? Select ‘yes’ if the child completed the intervention Select ‘no’ if the child dropped out of TF-CBT prior to completing the intervention (e.g., moved away, stopped coming) TFCBT Evaluation Training

TF-CBT Data Entry Excel File – Pre & Post PTSD-RI Spreadsheets Pre-TF-CBT PARENT REPORT: CHILD REPORT: PTSD-RI Total Client ID Severity Score   Post-TF-CBT PARENT REPORT: CHILD REPORT: PTSD-RI Total Client ID Severity Score   TFCBT Evaluation Training

TF-CBT Data Entry Tips – Pre and Post PTSD-RI Spreadsheets There is a separate spreadsheet for the Pre- PTSD-RI data and the Post- PTSD-RI data Enter the PTSD Overall/Severity Score for the Parent and Child informants into their respective fields Do not enter text into these fields If data are missing, leave the fields blank TFCBT Evaluation Training

TF-CBT Data Entry Excel File – Pre & Post YOQ Spreadsheets Pre- TF-CBT PARENT/CAREGIVER REPORT:   YOUTH SELF-REPORT: YOQ (Scale Scores and Total Score) YOQ-SR (Scale Scores and Total Score) Client ID ID S IR SP BD CI Total Post- TF-CBT PARENT/CAREGIVER REPORT:   YOUTH SELF-REPORT: YOQ (Scale Scores and Total Score) YOQ-SR (Scale Scores and Total Score) Client ID ID S IR SP BD CI Total TFCBT Evaluation Training

TF-CBT Data Entry Tips – Pre and Post YOQ Spreadsheets There is a separate spreadsheet for the Pre- YOQ & YOQ-SR data and the Post- YOQ & YOQ-SR data Enter the Scale Scores and the Total Score for the Parent and Adolescent informants into their respective fields Do not enter text into these fields If data are missing, leave the fields blank TFCBT Evaluation Training

TF-CBT Data Submissions Data submissions occur twice a year The end of February (reflecting children served thru January) The end of August (reflecting children served through July) Note that this is the anticipated schedule; actual data submission dates may vary slightly An email notice is sent approximately one month in advance of each data submission deadline After data are submitted, sites continue to enter new data into the same database (always reflects an ongoing, historical record of children served) Every effort is made to distribute reports within two months of data submission Aggregate and site-specific reports TFCBT Evaluation Training

Questions TFCBT Evaluation Training

The End Contact Information Cricket Mitchell, PhD Email: cmitchell@cimh.org Cell phone: 858-220-6355 TFCBT Evaluation Training