Using Research/Evaluation Questions to Define Data Collection and Findings: Findings from the FY 2004 KTOS Follow-up Study Robert Walker, Allison Mateyoke-Scrivener, & Jennifer Cole University of Kentucky Center on Drug and Alcohol Research

Question 1: What are the demographic characteristics of KTOS sample? Almost any study asks for these data Almost any study asks for these data As with all research questions and decisions about measures, you must know how fine a detail is needed – e.g., on ethnicity. As with all research questions and decisions about measures, you must know how fine a detail is needed – e.g., on ethnicity. Does your survey discriminate between race and ethnicity? Does your survey discriminate between race and ethnicity? Can subjects have more than one race? Can subjects have more than one race? More than one ethnicity? More than one ethnicity?

Demographics Intake questionBreakdownN=888 GenderMale62.4% Mean age33.8 RaceWhite Black Other 86.7% 11.5% 1.8% Ethnicity: Hispanic /Latino 1 0.9% Marital statusNever married Married Divorced Separated/Widowed 38.6% 21.0% 27.8% 12.6% Current living arrangementHoused Institutionalized Other 87.9% 8.7% 3.4% Percent of sample who live with: Dependent child(ren) Anyone with an alcohol problem Anyone with a drug problem 31.4% 8.8% 8.3%

Question 2: Policy Q #1 is about abstinence The study must address the sophistication level of the users of the data. The study must address the sophistication level of the users of the data. Current political climates have little tolerance for harm reduction ways of understanding treatment outcomes. Current political climates have little tolerance for harm reduction ways of understanding treatment outcomes. Moralistic views of substance abuse call for abstinence, not reduced use. Moralistic views of substance abuse call for abstinence, not reduced use.

Increase in Percent of Clients with Alcohol Abstinence During Preceding 30 Days (n=888) Remained abstinent Newly abstinent Total % alcohol abstinent at follow-up NValid %N AlcoholMale (n = 549) Female (n = 333) Total (N = 882) % 56.2% 47.6% % 24.6% 25.1% 67.7% 80.8% 72.7% Alcohol to Intoxication Male (n = 551) Female (n = 332) Total (N = 883) % 73.8% 66.7% % 18.1% 21.5% 86.0% 91.9% 88.2%

How to present findings that highlight change Raw numbers help, but often do not help the reader understand the effect of seemingly modest differences in numbers. Raw numbers help, but often do not help the reader understand the effect of seemingly modest differences in numbers. Findings should include ways of showing the significance of change both statistically and through the use of different ways to think about change values. Findings should include ways of showing the significance of change both statistically and through the use of different ways to think about change values.

Rate of Percent Increase in Alcohol Abstinent Clients (n=888)

Increase in Percent of Clients who Reported Abstinence from Illegal Drugs in the Past 30 Days (Excluding Alcohol) Remained abstinentNewly abstinentTotal % abstinent at follow-up NValid % of the total sample N Male (n = 553) % %87.5% Female (n = 332) % %90.7% Total (N = 885) % %88.7% a Significance established using z test for proportions. *p <.01. **p <.001

Question 3: Are there differences by drug type? Measures in KTOS instrument continue to drill down to greater detail on specific drugs. Measures in KTOS instrument continue to drill down to greater detail on specific drugs. Differences in drug use patterns may suggest differences in the rates of becoming abstinent as well. Differences in drug use patterns may suggest differences in the rates of becoming abstinent as well. Taking each drug separately may help discriminate which drugs appear more difficult for recovery. Taking each drug separately may help discriminate which drugs appear more difficult for recovery.

Remained abstinentNewly abstinentTotal % abstinent at follow-up NValid % of the total sample N Male (n = 552) %6612.0%97.0% Female (n = 330) %7021.2%97.3% Total (N = 882) % %97.0% a Significance established using z test for proportions. *p <.01. **p <.001 Increase in Percent of Clients who Report Past 30 Day Tranquilizer Abstinence and Rate of Change

Increase in Percent of Clients who Report Past 30 Day Opiate Abstinence and Rate of Change Remained abstinentNewly abstinentTotal % abstinent at follow-up NValid % of the total sample N Male (n = 551) %539.6%96.4% Female (n = 331) %6820.5%96.6% Total (N = 882) % %96.5% a Significance established using z test for proportions. *p <.01. **p <.001

Increase in Percent of Clients who Report Stimulant Abstinence from Intake to Follow-up and Rate of Change Remained abstinentNewly abstinent NValid % of the total sample N Male (n = 553) %193.4% Female (n = 332) %236.9% Total (N = 885) %424.7% a Significance established using z test for proportions. *p <.01. **p <.001

Sub-question: For those who continue to use, are there any reductions in frequencies? Validated instruments such as the ASI use a composite scores of past 30 days use to examine level of drug use. Validated instruments such as the ASI use a composite scores of past 30 days use to examine level of drug use. However, none of these instruments control for time in jail or hospital. However, none of these instruments control for time in jail or hospital. For example, consider two clients, both of whom report using marijuana for 15 of the past 30 days. For example, consider two clients, both of whom report using marijuana for 15 of the past 30 days. 1 client was in jail for 15 days in the past 30 days 1 client was in jail for 15 days in the past 30 days The other client had been on the street for 30 of the past 30 days The other client had been on the street for 30 of the past 30 days Using the ASI, both clients would be evaluated the same Using the ASI, both clients would be evaluated the same In KTOS, we use a proportion of days to estimate use levels In KTOS, we use a proportion of days to estimate use levels Client 1 would use 100% Client 1 would use 100% Client 2 would use 50% Client 2 would use 50%

Percent Reduction in Proportion of Past 30 Days that Substances were Used (n=888)

Question 4: Does Treatment Result in Increased Employment? The Role of Functionality Variables Apart from drug use changes, functionality measures can be used to partially confirm clinical results. Apart from drug use changes, functionality measures can be used to partially confirm clinical results. Given extensive research on substance use and employment, the equations should be: Given extensive research on substance use and employment, the equations should be: Decrease in drug use = Increased employment Decrease in drug use = Increased employment Increased employment consistent with decreased drug use Increased employment consistent with decreased drug use

Percent of Clients in Each Employment Category

Rate of Change in Employment (n=888) aSignificance established using z test for proportions. *p <.01. **p <.001

Question 5: Are there any changes in arrests after treatment? Another Functionality Variable As with employment, the equation typically is: As with employment, the equation typically is: Decreases in drug use = Decreases in criminal activity and therefore, decreases in arrests. Decreases in drug use = Decreases in criminal activity and therefore, decreases in arrests. Likewise, decreases in arrests tend to confirm that decreases in substance use have occurred. Likewise, decreases in arrests tend to confirm that decreases in substance use have occurred.

Percent of Clients Arrested in the Past 12 Months

Rate of change in arrests Significance established using z test for proportions. *p <.01. **p <.001

Question 6: Are there changes in mental health symptoms after treatment? Given robust associations of mental health problems associated with substance use, the equation should be: Given robust associations of mental health problems associated with substance use, the equation should be: Decreases in substance use should = Decreases in mental health problems. Decreases in substance use should = Decreases in mental health problems. Likewise, decreases in mental health problems should be consistent with decreased mental health problems. Likewise, decreases in mental health problems should be consistent with decreased mental health problems.

Changes in Mental Health Ratings (n=888) Past 12 monthsIntakeFollow-up Serious depression42.1%35.4% Serious anxiety48.7%41.1% Trouble understanding/concentrating41.3%36.4% Trouble controlling violent behavior12.2%9.9% Suicidal thoughts12.1%7.4% Attempted suicide4.8%2.0% Prescribed psychiatric medications31.7%26.0%

Rate of change in MH symptoms Significance established using z test for proportions. *p <.01. **p <.001

Question 7: Are there any cost offsets to society from treatment? Substance abuse treatment, more than any other behavioral treatment, has had to show that it saves taxpayers money. Substance abuse treatment, more than any other behavioral treatment, has had to show that it saves taxpayers money. The changes in employment and arrests offer the clearest behaviors to measure in terms of cost changes from intake to follow-up. The changes in employment and arrests offer the clearest behaviors to measure in terms of cost changes from intake to follow-up.

Total Cost of Publicly-funded Treatment Services

Past 12-month Arrest at Baseline and Follow- up and Estimated Victim Costs of Crime (n = 888) Arrests by type of crime Estimated cost per arrest Past 12 month arrests at intake Cost of crimes at intake Past 12 month arrests at follow-up Cost of crimes at follow-up Reduction in cost Drug$3,580291$1,041,78098$350,840 $690,940 Property$15,418377$5,812,586127$1,958,086 $3,854,500 Violence$35,745166$5,933,67056$2,001,720 $3,931,950 DUI$25,225167$4,212,57556$1,412,600 $2,799,975 Total1001$17,000,611337$5,723,246 $11,277,365 Reduction in Nights in Jail and Related Costs Jail time and costsAt intakeAt follow-upReduction Overall number of nights spend in jail in the past 12 months 32,7239,41023,313 (71.2%) Annualized total estimated cost of jail stays at $38.03 per night $891,047$256,234$634,813 (28.8%)

Increased Employment and Earnings (n=888) Employment variableBaselineFollow-upIncrease Number of clients working full or part- time Annualized – days worked in the past 30 days for follow-up sample 83,508125,49641,988 Total annualized hours of paid work668,0641,003,968335,904 Annualized total estimated labor value at $7.00 per hour times total hours $4,676,448$7,027,776$2,351,328 Estimated tax value 6%) of labor income $280,586$421,667$141,080

Conclusion The reductions in self-reported arrests for Kentucky clients Combined with estimates for reduced victim costs of their crimes Reduced costs of jail time and - Increased earnings and tax revenues Suggest a cost offset benefit for Kentucky taxpayers estimated at a ratio of 5.35 to 1. In other words, Kentucky saved $5.35 for every dollar spent on treatment during 2004.