1. Estimating Mental Illness in an Ongoing National Survey Joe Gfroerer, Sarra Hedden, Peggy Barker, Jonaki Bose Center for Behavioral Health Statistics and Quality, SAMHSA Jeremy Aldworth RTI International COPAFS Meeting March 16, 2012

2. Outline of Presentation Summary of National Survey on Drug Use and Health (NSDUH) Design of Mental Health Surveillance Study (MHSS) Results Methodological issues

3. National Survey on Drug Use and Health (NSDUH) Sponsor: Substance Abuse and Mental Health Services Administration (SAMHSA), Center for Behavioral Health Statistics and Quality (CBHSQ) Purpose: Estimate prevalence, correlates and trends of substance use in U.S. History: Conducted since 1971, annually since 1990 3

4. NSDUH Design Representative nationally and in each state Civilian, noninstitutional population, age 12+ Face-to-face interview using ACASI 68,000 respondents each year; oversample age 12-25 $30 incentive Response rates (weighted, 2010): 88% of selected households completed screener 74% of selected persons completed interview 4

5. NSDUH Sample Design: Target Sample Sizes by State and Age Group Completed Interviews per State Large states (8): 3,600 per year Small states (43): 900 per year Completed Interviews by Age Group 1/3 of sample in each age group (12-17, 18-25, 26+) 5

6. NSDUH Questionnaire Use of alcohol, tobacco, and illicit drugs Substance use disorders (DSM-IV) Substance use and mental health treatment Health conditions, service utilization Demographics Mental health (MDE, suicide) 6

7. NSDUH Mental Health Surveillance Study (MHSS) SAMHSA legislation requires the agency to produce methods to estimate serious mental illness (SMI) (and serious emotional disturbance (SED) in children) TAG (2006) recommended NSDUH for SMI (and NHIS for SED) MHSS implemented in 2008 NSDUH 7

8. SAMHSA Definition of Serious Mental Illness (SMI) among Adults Any DSM-IV mental disorder (other than developmental and substance use disorders) WITH serious functional impairment (both in past year)

9. Estimating SMI in NSDUH A complete diagnostic assessment to determine SMI is not feasible in NSDUH interview Would require too many questions Interviewers are not clinicians Alternative approach used by SAMHSA: Administer clinical interviews on a subsample of NSDUH respondents, to diagnose SMI Include short scales in main NSDUH interview, to be used as predictors of SMI in a model: K6, WHODAS Develop a regression model, based on subsample data, and apply to main sample data to predict SMI for each respondent

10. Kessler 6-item Nonspecific Psychological Distress Scale (K-6) Included in NSDUH and several other large national surveys Developed specifically for use in large surveys Discriminates between cases and non-cases in community samples Demonstrates consistency across population groups Responses 0-4 for each item; combined score 0-24

11. Percentage Distribution of K6 Scores among Persons Aged 18 or Older: 2008 Percentage K6 Score

12. Measuring Impairment in NSDUH Main Sample: WHODAS WHO Disability Assessment Schedule (WHODAS) 16 items assessing functional impairments in various domains Reduced to 8 items for NSDUH based on IRT analysis Responses: 0 to 3 for each item

13. Clinical Interview Subsample At end of NSDUH interview, a request for 2nd interview on mental health is made to respondents selected for the clinical followup interview $30 incentive N=500 to 1500 per year Nationally representative, stratified sample Interview conducted by a trained clinical interviewer, by telephone, 2-4 weeks after main interview 13

14. Clinical Interview Content Structured Clinical Interview for DSM-IV (SCID): 15 specific mental disorders are covered Global Assessment of Functioning scale (GAF) 14

15. Estimation Step 1: Determine Best Weighted Logistic Regression Model Using Clinical Interview Subsample Let π = Pr(“true” SMI│X 1, X 2 ) logit(π) =    +   X 1 +   X 2 X 1 = recoded K6 score (0-17) X 2 = recoded WHODAS score (0-8)

16. Estimation Step 2: Determine Minimum- Bias Cutpoint from Clinical Interview Data 1. Based on model, each CI respondent has predicted Pr(SMI+) = 2. Based on clinical interview, each CI respondent has a “true” SMI diagnosis 3. Select cutpoint,, for which false positives equal false negatives in the CI subsample - If then predicted SMI status = positive - If then predicted SMI status = negative

17. Final Model Based on 2008 Clinical Interview Data logit( ) = -4.7500  + 0.2098X 1 + 0.3839X 2 Where X 1 = recoded K6 score (0-17) X 2 = recoded WHODAS score (0-8) Cutpoint: = 0.26972 17

18. Estimation Step 3: Apply Model to Main Sample 1. Based on model, and reported K6 and WHODAS scores, each NSDUH respondent has predicted Pr(SMI+) = 2. If then SMI status = yes If then SMI status = no

19. ROC Statistics: Final SMI Model with K6 and WHODAS vs. Alternative Model with K6 Only 19 Model Parameters Predicted Rate False Pos. Rate False Neg. Rate Sensi- tivity Speci- ficity Area Under ROC Curve K6.046.029.028.387.971.679 K6 and WHODAS.047.024.023.506.976.741

20. Levels of Mental Illness Level of MI in Past YearDefinition Low/Mild Mental Illness (LMI) Any disorder, and GAF>59 Moderate Mental Illness (MMI) Any disorder, and GAF 51-59 Serious Mental Illness (SMI) Any disorder, and GAF<51 TOTAL/Any Mental Illness (AMI) Any disorder Secondary purpose of the MHSS was to generate estimates of “any mental illness” and to designate levels of severity:

21. Estimating Other Levels of Mental Illness Various models were compared Result: The SMI model, with different cutpoints, was found to predict as well as any other model

22. AMI/ SMI Prediction Based on Recoded K6 and WHODAS Scores 8 7 6 5 4 3 2 1 0 01234567891011121314151617 Recoded K6 Score Recoded WHODAS Score SMI LMI or MMI No MI

23. Prevalence of Mental Health Problems among Adults (18+): 2010 Percent with disorder/problem in past year 23 46 mil 11 mil 15 mil 9 mil

24. Any Mental Illness in the Past Year among Adults Aged 18 or Older, by Age and Gender: 2010 Percent with Any Mental Illness (AMI) in the Past Year 24 Fig MH 2.1 Age Group Gender

25. Serious Mental Illness in the Past Year among Adults Aged 18 or Older, by Age and Gender: 2010 Percent with Serious Mental Illness (SMI) in the Past Year 25 Fig MH 2.2 Age Group Gender

26. Receipt of Mental Health Services among Adults Aged 18 or Older, by Level of Mental Illness: 2010 26 Fig MH 2.9 Percent Receiving Mental Health Services in the Past Year

27. Past Year Substance Use among Adults Aged 18 or Older, by Any Mental Illness: 2010 27 Percent Using Substance Fig MH 4.1 Marijuana Illicit Drugs 1 Psychotherapeutics InhalantsCocaine HeroinHallucinogens 1 Illicit Drugs include marijuana/hashish, cocaine (including crack), heroin, hallucinogens, inhalants, or prescription-type psychotherapeutics used nonmedically.

28. Past Year Substance Dependence or Abuse and Mental Illness among Adults Aged 18 or Older: 2010 28 Fig MH 4.2 SUD = substance use disorder. SUD, No Mental Illness 11.2 Million SUD and Mental Illness 9.2 Million 20.3 Million Adults Had SUD 45.9 Million Adults Had Mental Illness 36.7 Million Mental Illness, No SUD

29. Past Year Substance Dependence or Abuse among Adults Aged 18 or Older, by Level of Mental Illness: 2010 Percent Dependent or Abusing Substance 29 Fig MH 4.4

30. Issue: Trend Measurement Options: Update models, parameters, and/or cutpoints each year Small annual sample high variance Continue to accumulate clinical interview data and evaluate models; update model when there is evidence that estimates can be substantially improved Will need to update all prior estimates

31. Prevalence of Mental Illness among Adults (18+): 2008 to 2010 Percent in past year 31

32. Issue: Nonresponse Bias and Weighting CI Sample Disposition, 2008-2009: Unwtd. N Unwtd. Pct. Wtd. Pct. TOTAL 3,062 100.0 Respondents 2,027 66.2 59.5 Immediate refusal 420 13.7 24.3 Agreed, but noncontact 477 15.6 12.5 Other nonresponse 138 4.5 3.7

33. Nonresponse Bias Assessment: Rates of Key Measures among Respondents, Refusals, and Noncontacts: Clinical Interview Sample, 2008-9 33 Percent in Past Year

34. Nonresponse Bias Assessment: Age and Family Income among Respondents, Refusals, and Noncontacts: Clinical Interview Sample, 2008-9 34 Percent

35. Other Issues What is the best sample design? Optimize for modeling? Prevent extreme weights What is best estimation method? Variance estimation not straightforward Estimate prevalences of specific disorders from the clinical interview sample?

36. Conclusions MHSS provides the only current data on trends in mental illness and its co-occurrence with substance use Estimates have been widely cited and used in analyses of the impact of health care reform Methods can be replicated in other surveys But more work needed to refine the models and estimation methods