Using the GAIN Recommendation and Referral Summary (GRRS) to Support Clinical Decision-making Michael L. Dennis, Ph.D. Dionna Christian GAIN Coordinating Center (GCC) Chestnut Health Systems, Bloomington, IL Presentation for SAMHSA’s Center for Substance Abuse Treatment (CSAT) Adolescent Treatment Grantee meeting, Baltimore, MD, February 23-25, 2004.

ACKNOWLEDGEMENT The development of the GAIN Recommendation and Referral Summary was supported with funds from the Center for Substance Abuse Treatment (Contract no. 270-2003-00006), Robert Wood Johnson Foundation (Grant no. 47266), and the National Institute on Alcohol Abuse and Alcoholism (Grant no. R01 AA10368). It would not have happened without the input from dozens of grantees (particularly Susan Godley and the staff of Chestnut Health System’s Bloomington, IL adolescent treatment unit) and additional hard work of several individual beta testers (who proofread over two dozen G-RRS against original interviews and made many invaluable suggestions for improving it) and the grants that sponsored their work, this includes: Lora Passetti, Matt Orndorff, Jenny Hammond and Bobbie Jo Allen from Chestnut Health Systems (CSAT grant no TI14456), Doug Smith from University of Iowa (CSAT grant no TI13354), Alex Kuprion from Seven Counties (CSAT grant no TI13345), Benjamin Chambers and Wayne Scott from Multnomah County (RWJF grant no 45054), and Robert Ketterlinus from Philadelphia Health Management Corporation (CSAT grant no TI14376). The information and the opinions expressed herein are solely those of the authors and do not represent official positions of the government, RWJF, or any other organization. The opinions are those of the author do not reflect official positions of the government . Available on-line at www.chestnut.org/li/apss

The Progression of Substance Use Problems Multiple Problem Clients Clinical Disorder Problem Use Frequent Use Bingeing Opportunistic Use Experimentation No Use Severity

Multiple Co-occurring Problems Are the Norm and Increase with Level of Care Source: CSAT’s Cannabis Youth Treatment (CYT), Adolescent Treatment Model (ATM), and Persistent Effects of Treatment Study of Adolescents (PETS-A) studies

Severity is Related to Other Problems 100% 80% 71% 57% 25% 42% 30% 60% 37% 40% 22% 13% 22% 20% 5% 0% Health Problem Acute Mental Acute Attention Conduct Distress* Distress* Traumatic Deficit Disorder* Distress* Hyperactivity Disorder* Abuse/Partial Remission (n=322) Past Year Dependence (n=278) * p<.05 Source: Tims et al 2002

Objectives Provide an overview of how to use the GAIN Recommendation and Referral Summary (G-RRS) to support clinical decision-making. Review procedures of downloading, installing and customizing the G-RRS Discuss issues in implementing the G-RRS

Part I Overview of how to use the GAIN Recommendation and Referral Summary (G-RRS) to support clinical decision-making First we examined the reliablility of the scales and subscales. We first looked at these at the age, level of care and study level (10 groups from earlier slides). What we will show here are the weighted mean alphas by age and level of care group. 2nd, for each of the six study datasets, we conducted an exploratory factor analysis to see if the factor structure we are proposing would replicate within each study dataset. Although we are not reporting these results here, the basic structure seemed to replicate for all but 1 study where the level of crimes reported was so low that the factor analysis did not work. 3rd, we combined the 6 datasets, identified the 4 age/level of care groups, and re-ran the factor analysis on each of the 4 groups. That is what I will be presenting here. Finally, using the combined dataset, we conducted a series of Confirmatory Factor Analyses using AMOS to replicate earlier work with this much larger and more heterogeneous dataset. The hypothesized structure. . . . (GO TO NEXT SLIDE)

The Global Appraisal of Individual Needs or “GAIN” is actually a series of standardized instruments designed to integrate the assessment for both clinical (e.g., diagnosis, bio-psycho-social assessment, placement, and treatment planning) and program evaluation (needs assessment, clustering, fidelity, outcomes, and benefit cost) purposes.

CSAT’s Adolescent Treatment Program Grantees and Collaborators Other Collaborators Cannabis Youth Treatment (CYT) Adolescent Treatment Model (ATM) RWJF Reclaiming Futures Program Strengthening Communities for Youth (SCY) RWJF Other RWJF Grantees Adolescent Residential Treatment (ART) NIAAA/NIDA Other Grantees Effective Adolescent Treatment (EAT) Other CSAT Grantees

Main Interpretative Reports to Support Diagnosis, Placement, and Treatment Planning GAIN Referral and Recommendation Summary (G-RRS) - Text based narrative in MS Word designed to be edited and shared with specialist, clinical staff from other agencies, insurers and lay people. Individual Clinical Profile (ICP) – more detailed report in MS Access designed to help triage problems and help the clinician go back to the GAIN for more details if necessary (generally not edited or shared)

G-RRS Organization Presenting Concerns and Identifying Information DSM-IV/ICD-9 Diagnoses  Evaluation Procedure Substance Use Diagnoses and Treatment History (ASAM criteria A) Level of Care and Service Needs (ASAM Six Dimensional Criteria B) Summary Recommendation

General Can use the client name, initials or another term supplied by the person running the report Can use the site’s organizational name or another term supplied by the person running the report The G-RRS comes out in a MS Word Document file (*.doc) that can be read, edited and saved by most word processing programs. The report include three types of prompts identifying areas where counselors: Often add additional information or comments from other sources of information Have to reconcile and finalize potentially conflicting diagnoses Have to make preliminary treatment planning recommendations The ICP report parallels the G-RRS and provides more detailed information to supplement it and/or to cross reference back to the GAIN for more information.

General - Continued The G-RRS summarizes data collected and follows existing rules; it is a tool to feed into and support clinical judgment – not to replace it. The G-RRS can only generate reports using the data collected. A G-RRS based on the full (90-120 minute) version of the GAIN contains more details (e.g., name of school, employer, probation officer) than a G-RRS based on the core (60-90 minute) version of the GAIN. Sites can add in questions that are not in their core but that they want to have for the G-RRS. Sites can also remove sections of the report that they do not want and/or modify some of the labels (e.g., signature lines)

1. Presenting Concerns and Identifying Information Basic demographics (age, race, gender, marital status, children), appearance/disabilities, source and reason for referral, current living and vocational status Provides fixed coded responses plus the clients verbatim words (IN CAPS). Prompts to add any additional information related to reason for referral, custody arrangements, living situation, current address, parents' marital status, addresses of relevant parents/guardians Full version includes

2. DSM-IV/ICD-9 Diagnoses Self Reports Based Measures and Codes for Summary of current treatment, medication and allergies to aid interpretation Axis 1. Substance use disorders, major depression, generalized anxiety, ADHD, CD, and pathological gambling to criteria, screening for mood/anxiety disorders, suicide risk, traumatic distress Axis 2. Screening for personality disorders by cluster Axis 3. Lifetime history by ICD-9 area and check for common drug-health interactions Axis 4. Traumatic victimization, check for major axis IV bio-psycho-social stressors, and checks for other high-stress events

Diagnosis – Continued Other Also reports the additional staff diagnoses reported on GAIN Diagnosis page Ability to document Axis 5 Past year and Past 90 day staff ratings for GAF, SOFAS, GARF Ability to acknowledge other sources of information Can collapse, modify or delete diagnoses Prompt to reconcile and confirm diagnoses ICP the rules why each diagnosis, specifier and rule out was printed The manual lists all diagnoses, specifiers and rule outs that were checked, including the rules for when they are to be printed

3. Evaluation Procedure Reviews type of administration, environmental context, ratings of the client’s behaviors during the meeting, validity concerns and any addition source of information reported on the GAIN’s diagnosis page. Prompt to enter any other sources of information consulted as part of evaluation (e.g. urine test results, records, referral letters, family assessments, probation reports, etc)

4. Substance Use and Treatment History (ASAM PPC-2R Criteria A) Detailed text narrative age of first use, preferred substance, substances for which the client perceives a need for treatment For each DSM-IV substance use disorder diagnosis (in order of clinical severity from the S9 grid) Diagnosis and specific symptoms reported in the past month, year and lifetime Recency, frequency and peak amount of use (if collected) the date and amount of last use (required for some insurance) Where a class of drugs (e.g., amphetamines), the specific drugs reported A list of other substance used (but for which diagnosistic criteria are not met) and prompt to add more identified through biometric (e.g., urine, saliva, hair) testing or collateral reports. History of substance abuse treatment, including (if collected) a detailed treatment history (program, level of care, intake and discharge date)

5. Placement (ASAM PPC-2R Criteria B) Arranged by six dimensions of ASAM Criteria B: Acute Alcohol/Drug Intoxication and Withdrawal Potential Biomedical Conditions and Complications Emotional, Behavioral, or Cognitive Conditions and Complications Readiness to Change Relapse, Continued Use, or Continued Problem Potential Recovery Environment General ICP gives code for why each text statements suggesting the need for a higher level of care was printed ICP gives scale scores with triaged ranges (low, moderate, high score) by areas of ASAM ICP gives days of behavior and service utilization reported in each section of the GAIN GAIN manual lists all statements evaluated Treatment planning embedded by section

Conceptualization of Treatment Planning Need for Each ASAM area Problem Severity None Past Current Treatment History No Problem Monitor (relapse prevention) Need for initial or low invasive treatment Monitor (check understanding of problem) Need for re-intervention and possibly more intensive treatment Readiness to Step down or Discharge Responding to treatment, Monitor for discharge Not responding, needs more intensive/ different treatment

B1. Acute Alcohol/Drug Intoxication and Withdrawal Potential Lifetime history of withdrawal and seizures Current (past week) withdrawal scale score and symptoms endorsed Recency of use, with flags on use in the past 48 hours Frequency of use, with flags on any daily use (45+/90 days) and weekly use of opioids (13+/90 days) Lifetime history of detoxification and days of detoxification in the past 90 days. Prompt to recommend one of the following: Monitor for change in intoxication or withdrawal symptoms, Ambulatory detoxification services related to withdrawal, Inpatient detoxification services related to current intoxication and withdrawal.

B2. Biomedical Conditions and Complications Overall Health and Pain Assessment (includes allergies) Nutrition and Exercise (includes body mass index and eating disorder behaviors) Sexual Activity and Orientation (includes preferences, activity, and contraceptive use) Treatment History for Health Problems (lifetime and past 90 day use of hospitals, emergency rooms, outpatient surgery, outpatient treatment, and medication; in full, current treatment duration and provider) Prompt to recommend one of the following: Monitoring for change physical health (and medication compliance) the following specific accommodations for medical conditions required to participate in treatment: List out a more detailed medical assessment (including nutritional guidance), referral for the following specific medical services: List out.

B3. Emotional, Behavioral, or Cognitive Conditions and Complications Emotional Conditions (internal mental distress scale score, past 12 month symptoms related to somatic, depression, suicide, anxiety, or trauma disorders; recency and prevalence of problems, suicide risk). Behavioral Conditions (behavior complexity scale score, past 12 month symptoms related to inattention, hyperactivity, and conduct disorders; recency and prevalence of problems). Arguing and Aggression (past 12 month symptoms of oral and physical violence; recency and prevalence of problems) Illegal Activity and Juvenile Justice Systems Involvement (lifetime and past 90 day number and type of arrests; recency and prevalence of being on/in probation, parole, detention, jail, house arrest, electronic monitoring; current status; prompt to enter next court date or other important legal system date; receny and type of illegal activity engaged in during the past year, prevalence of illegal activity and relationship to substance use).

B3. Continued Cognitive Conditions (including Cognitive Impairment Score at the time of the evaluation, involvement in special education, and any other observed indications of cognitive impairment or developmental disabilities) Treatment History for Emotional, Behavioral, or Cognitive Problems (prior diagnoses; lifetime and past 90 day use of mental hospitals, emergency rooms, outpatient treatment, and medication; in full, current treatment duration and provider) Prompt to recommend one of the following: monitoring for change in mental health (and medication compliance) the following specific accommodations for psychological conditions required to participate in treatment: List out a more detailed psychological assessment referral for the following specific psychological services: List out

B4. Readiness to Change Perceived pressure to be in treatment and source of pressure Treatment Motivation scale score Treatment Resistance scale score (if collected) Reasons for Quitting (RFQ) given, perceived ability to stop (or stay abstinent). Note: RFQ typically only used the Personal Feedback Report is also being used in MET/CBT or other MI Prompt to recommend one of the following: monitoring for change in readiness for change the following assistance to help address treatment resistance: list out individual motivational enhancement sessions the following specific services to help maintain motivation to stay in recovery: list out

B5. Relapse, Continued Use, or Continued Problem Potential List of individual risk factors that predict continued use/problems or relapse, including Low self-efficacy to resist Low problem orientation (i.e., helplessness) Daily use of anything or weekly use of opioids Using substances to forget about traumatic memories First used substances or got drunk under the age of 15 Reporting 3 or more symptoms of dependence/abuse in the past month Continued substance use despite prior treatment Prompt to recommend one of the following: monitoring for change in relapse potential relapse prevention skills groups increased structure to reduce environmental risks of relapse the following specific steps to reduce continued use/relapse potential: list out

B6. Recovery Environment Family/Home Environment (including who they live with, level of clients involvement with parents and own children, use in the home and time in a controlled environment) School and Work Environment (including recency and prevalence of school and work, problems there in the past 12 months, pattern of grades, income spend on alcohol/drugs and if collected, the name of school/employer and type of job Social Network Environment (For each of above and peers that they spend most of their social time with, the extent to which people in living, vocational and social circles were getting drunk, using drugs, committing illegal activity, fighting, vocationally engaged, had a treatment history, and considered themselves in recovery) Sources of Social Support (if collected, open-end and closed list) Personal Strengths (if collected, open-end and closed list).

B6. Continued Spirituality (including religious affliation, strength and centrality of spiritual believes) Satisfaction with Environment (extent satisfied with living situation, family, sexual partners, work/school, free time activities, coping/support) Victimization (Lifetime history, severity, recency and current fears about being attached with a weapon, beaten, sexually abuse, or emotionally abused; Prompt to comment on any reports/follow-up done) Prompt to recommend one of the following: monitoring for change in recovery environment a residential or more structured treatment setting to temporarily control environmental risks the following specific steps to reduce recovery environment risks: list out the following specific steps to take further advantages of sources of support/personal strengths: list out

6. Summary Recommendation Summary of current systems clients is involved in and that treatment needs to be be coordinated with Any level of care recommendation from GAIN placement worksheet Prompt to : enter level of care recommendation comment on any special barriers to placement and what might be done about them. comment on need to coordinate care with other treatment or agencies. Signatures Staff notes from assessment

Using the ICP to help with the G-RRS Identify the criteria on which the diagnosis or statement is made Examining scale scores in a given area to better understand the severity or what is going on Complete breakout of demographics, behaviors, service utilization More detailed information for treatment planning

Individual Clinical Profile (ICP) Organization Identifiers DSM-IV/ICD-9 Diagnoses  Demographics (including appearance, housing situation, prior treatment, involvement in other systems, potential validity concerns, staff notes) ASAM placement flags ASAM placement profile worksheet Behaviors and Service Utilization Treatment Planning Worksheet (including client and staff rating or urgency, what the client has asked for help with, and things that most agencies/accrediting agencies would expect to be in the treatment plans) Note – this is an access report, not intended for general distribution and only reports on data that was collected

[Notes] on why the statements were printed Notice the addition of the conditions why statement was printed. Key: Tx-treatment Sx-Symptom 3+ 3 or more > - greater than < - less than CAPS – quote from staff or client From Phillip ICP page 1

ICP Demographics section lists out code and all values Example of Code-Response label Gives status even if none or negative Cannot give page numbers as it varies by version – but can jump directly there in ABS with variable name From Phillip ICP page 3

ICP ASAM Flags bulleted out Minimal Criteria for level of care and basis for printing the statement Red flags indicating the need for more services in the area or a higher level of care and the basis for printing the statement Manual has a list of all statements evaluated From Phillip ICP page 5

ICP ASAM Profile ASAM Criteria Scale Name [basis] Score or Skipped Circle Score and Connect Dots Score or Skipped * Bad Data Scale triaged into Low, Medium, or High Severity Scales file as more on purpose, interpretation, source, and psychometrics From Phillip ICP page 6

Simple Behavior/Utilization Measures Left side gives behaviors in the past 90 days Right side gives utilization in the past 90 days Organized by Section of the GAIN Gives page number, item number -- skipped, RF refused DK don’t know From Phillip ICP page 9

Help with Treatment Planning Compares Client and Staff Urgency Ratings Circle Score and Connect Dots X Specific things the client has asked for Other Actions or Things Typically Expected by Agencies or Accrediting Agencies From Phillip ICP page 10

GI Scales and Variable File 1000+ page electronic encyclopedia in MS Excel with documentation for each GAIN scale, subscale, index, created variable/text statements used in the G-RRS, ICP and our research to date For each variable, documentation includes: Scale/variable name (and any related/earlier versions) Time Period(s) covered Section of the GAIN Question (items, page in full version) Scale measurement type (Cut-points for triage) Purpose (s) Short Description Interpretation Supplemental References on source, norms, psychometrics Comments GAIN V5 SPSS Syntax: Prior SPSS Syntax: (if different) Actual questions (from version 5)

Key Methodological Work Underway ASAM placement recommendations based on expert and statistical models Identification of multi-problem clusters or “Code types” Modeling Change over time in relations to the treatment hinge and the cycle of relapse, treatment re-entry and recovery Propensity score models to predict outcomes and serve as a synthetic “average treatment” comparison group First we examined the reliablility of the scales and subscales. We first looked at these at the age, level of care and study level (10 groups from earlier slides). What we will show here are the weighted mean alphas by age and level of care group. 2nd, for each of the six study datasets, we conducted an exploratory factor analysis to see if the factor structure we are proposing would replicate within each study dataset. Although we are not reporting these results here, the basic structure seemed to replicate for all but 1 study where the level of crimes reported was so low that the factor analysis did not work. 3rd, we combined the 6 datasets, identified the 4 age/level of care groups, and re-ran the factor analysis on each of the 4 groups. That is what I will be presenting here. Finally, using the combined dataset, we conducted a series of Confirmatory Factor Analyses using AMOS to replicate earlier work with this much larger and more heterogeneous dataset. The hypothesized structure. . . . (GO TO NEXT SLIDE)

Validity Checks Currently Available Staff ratings of understanding, misrepresentation, appearance/behaviors during assessment, and context Consistency Reports Counts of missing/refused items Out of normative responses on time, key items Additional Scales in the Works Inconsistency scale Endorsing rare items (faking bad/general severity) Not endorsing common items (faking good/a typical profile) Predicting false negative relative to urine tests

Other Computer Generated Clinical Reports GAIN-Q Referral and Recommendation Summary (GRRS) – text based summary to support preliminary diagnosis and placement based on the GAIN-Quick Personal Feedback Reports (PFR) – text based summary to support the motivational interviewing component of MET/CBT based on the GAIN-I or GAIN-Q Validity reports to identify areas for clarification and potential problems Other site specific clinical reports (e.g., pre-filling existing paperwork like a health assessment, TEDS report etc) Data elements can be transferred into existing MIS and used in other reports/systems as well.

GAIN/ABS just part of a Trans-Enterprise MIS Service Logs Appt Tracking School MIS Mgmt Reports Host MIS Welfare MIS Host Acct Sys JJS MIS Assessment Building System: GAIN, Screener And Other Measures Host Lab Some of you may have seen this slide as it is one that David Hodgkins uses. It is a busy slide, but basically all means is that I will be talking about getting the data to the evaluators at NGIT and/or CHS. Evaluator or Data Manager GRL, Other Data Cross Site Evaluation

Part II Downloading and Installing the G-RRS application (see Dionna’s presentation) First we examined the reliability of the scales and subscales. We first looked at these at the age, level of care and study level (10 groups from earlier slides). What we will show here are the weighted mean alphas by age and level of care group. 2nd, for each of the six study datasets, we conducted an exploratory factor analysis to see if the factor structure we are proposing would replicate within each study dataset. Although we are not reporting these results here, the basic structure seemed to replicate for all but 1 study where the level of crimes reported was so low that the factor analysis did not work. 3rd, we combined the 6 datasets, identified the 4 age/level of care groups, and re-ran the factor analysis on each of the 4 groups. That is what I will be presenting here. Finally, using the combined dataset, we conducted a series of Confirmatory Factor Analyses using AMOS to replicate earlier work with this much larger and more heterogeneous dataset. The hypothesized structure. . . . (GO TO NEXT SLIDE)

Part III Implementing the G-RRS First we examined the reliablility of the scales and subscales. We first looked at these at the age, level of care and study level (10 groups from earlier slides). What we will show here are the weighted mean alphas by age and level of care group. 2nd, for each of the six study datasets, we conducted an exploratory factor analysis to see if the factor structure we are proposing would replicate within each study dataset. Although we are not reporting these results here, the basic structure seemed to replicate for all but 1 study where the level of crimes reported was so low that the factor analysis did not work. 3rd, we combined the 6 datasets, identified the 4 age/level of care groups, and re-ran the factor analysis on each of the 4 groups. That is what I will be presenting here. Finally, using the combined dataset, we conducted a series of Confirmatory Factor Analyses using AMOS to replicate earlier work with this much larger and more heterogeneous dataset. The hypothesized structure. . . . (GO TO NEXT SLIDE)

Implementation Issues While many staff will be very excited about having the G-RRS help them do paper work, there are several issues that will need to be resolved on a site-by-site basis. We recommend starting with a small team of people (from multiple agencies if a complex project) that can try using the G-RRS and think through how it interacts with existing systems, requirements, and policies during a 4 to 8 week start up phase.

Some Likely Questions Who will run the report? (Do they have the necessary hardware and software?) How will the report get to the clinician that needs to use it? How does this clinician get access to the ICP and full GAIN when they need it? Where will the reports be saved (and what security safeguards are in place to protect documents with private health information)? Does the report need a signature (e.g, from the person or persons responsible for finalizing the diagnosis, placement and treatment plans)? If so, who needs to sign it and by when? When does the report need to be in the clinical file? (How does the report get into the clinical file?)

Some Likely Questions Who edits the report/needs training on how to edit the report? Where can the information needed to fill in the prompts be found? Does the site want to drop or change the wording of some prompts? What parts of the report (if any) should be expanded or deleted routinely? (If deleted routinely, should they just be deleted from the template?) Who needs to approve implementation of the report? Who needs to be notified that the report will be implemented (e.g., records staff, counselors)? Who will be the point person within the site to ask questions about the G-RRS? (typically a GAIN trainer, ABS administrator or clinical coordinator). See also FAQs in hand out on install G-RRS applications

Getting Help Manuals and forms on line at the Adolescent Program Support Site (www.chestnut.org/li/apss ) or the generic GAIN (www.chestnut.org/li/gain ) or ABS (www.chestnut.org/li/abs ) websites. For questions related to installing or modifying the G-RRS or ABS in general, contact our ABS SUPPORT team at abssupport@chestnut.org or call Dionna Christian at 309-820-3543 ext. 83400. For questions on administering the GAIN, specific GAIN items, interpreting the GAIN, or questions on QA, contact our GAIN SUPPORT team at gainsupport@chestnut.org or call Michelle White at 309-820-3543 ext. 83439. For GAIN or ABS license questions, contact Joan Unsicker at junsicker@chestnut.org or 309-820-3543 ext. 83413. For GAIN, GRL, WAI or TxSI data submission questions, contact Melissa Ives at mives@chestnut.org or 309-820-3543 ext. 83408. For other information or information on holding a future training in your area, to answer any questions not covered above, or if you have problems using any of the above contact information, contact Michelle White at mwhite@chestnut.org or 309-820-3543 ext 83439.

Contact Information Michael L. Dennis, Ph.D. Lighthouse Institute, Chestnut Health Systems 720 West Chestnut, Bloomington, IL 61701 Phone: (309) 827-6026, Fax: (309) 829-4661 E-Mail: mdennis@chestnut.org Website: www.chestnut.org/li/GAIN GAIN Training Coordinator: Michelle White at 309-827-6026 or mwhite@chestnut These slides are available from the Adolescent Program Support Site (www.chestnut.org/li/APSS ) for CSAT/RWJF grantees and GAIN site (www.chestnut.org/li/gain) for others