1. Management Practices in Addiction Treatment Programs K. John McConnell, PhD Oregon Health & Science University Supported by a grant from NIDA 1R01DA020832

2. Two research threads Substance Abuse: – A call for research on the “business” of addiction treatment (Kimberly & McLellan, JSAT 2006) Economics: – Typically treat “management quality” as a fixed effect – New research to measure management practice (Bloom & Van Reenen, QJE 2007) Innovative telephone survey to “score” management practice Strongly correlated with firm performance (>700 firms in 4 countries)

3. Study Objective Adapt management practice survey tool to addiction treatment programs Are better management practices associated with better client treatment?

4. Study Sample Programs from the NIATx 200 project – NIDA 1R01DA020832 – Approximately 200 agencies in 5 States (MA, MI, NY, OR, WA) randomized to different levels of support (coaching, interest circles, learning sessions, and combinations of all 3) We use data on 147 surveys by 3 interviewers – ~20 programs not interviewed because they were part of a multi-site agency – Excluded 5 programs with different interviewers – Excluded 5 with interview scored as “not reliable” – Excluded 20 with missing data

5. Steps to measure management practices 1. Developing management practice scoring – Adapted scorecard for 14 questions on 4 areas: Client intake and retention Quality improvement & data monitoring Program targets Incentives & employee management – 60 minute phone interview of program executive sponsors 2. Obtaining unbiased responses – Interviews run by 3 MPH level research assistants – “Double-blind”: Interviewers do not know program performance Interviewees are not informed (in advance) they are scored – “Open-ended” question format

6. (13) Promoting high performers a.Tell me about your promotion system. b.What about poor performers – do they get promoted more slowly? Are there any examples you can think of? c.How would you identify and develop (i.e. train) your star performers? d.If two people both joined the agency 5 years ago and one was much better than the other would he/she be promoted faster? Scoring grid: Score 1 People are promoted primarily upon the basis of tenure Score 3 People are promoted upon the basis of performance Score 5 We actively identify, develop and promote our top performers Examples: An agency based on an individual’s commitment to the company measured by experience. An agency has no formal training program. People learn on the job and are promoted based on their performance on the job. At one agency each employee is given a red light (not performing), amber light (doing well and meeting targets) a green light (consistently meeting targets; very high performer)

7. Analysis Negative binomial, clustered std. err. Outcome: Time to treatment – Days from first phone call to first appointment Pseudo-patient phone calls by NIATx researcher Caller identified herself as NIATx researcher Average of 7 calls made to each program Independent variables – Average z-score of management practices across 14 questions z-score removes scaling issues between questions – Employee FTE – State & interviewer fixed effects – Day of week for pseudo-patient phone call – Month of survey

8. Results

9. Distribution of management practice scores for 147 addiction treatment programs

10. Double-scored interviews (N = 14)

11. Model results Main analysis: – Time-to-treatment 1081 observations on 147 programs Management score significantly associated with time-to-treatment (P <0.01) Sub-analyses – 7 of 14 management practices were correlated with better time to treatment – Of the 4 areas, 3 were associated with better time-to-treatment Client intake and retention Quality improvement & data monitoring Program targets – One was not Incentives & employee management ?

12. Limitations Endogeneity Double scoring vs. test-retest Programs selected from NIATx 200 project

13. Concluding thoughts Management practices can be measured and are associated with client treatment Are they associated with financial outcomes? Do employee incentives matter or are substance abuse treatment programs “different”? NIATx aims for process improvement; can we design interventions to improve management practices?

14. Thank you University of Wisconsin – David Gustafson, PI – Andy Quanbeck – Alice Pulvermacher – Jay Ford – Anna Wheelock – Renee Hill Oregon Health & Science University – Dennis McCarty – Kim Hoffman – Marie Shea – Gretchen Luhr – Susan Rosenkranz – Traci Rieckmann Stanford University – Nick Bloom Research supported by NIDA R01DA020832