Estimating Mental Illness in an Ongoing National Survey

Joe Gfroerer, Sarra Hedden, Peggy Barker, Jonaki Bose Center for Behavioral Health Statistics and Quality, SAMHSA

Jeremy AldworthRTI International

COPAFS MeetingMarch 16, 2012

Outline of Presentation

• Summary of National Survey on Drug Use and Health (NSDUH)

• Design of Mental Health Surveillance Study (MHSS)

• Results

• Methodological issues

National Survey on Drug Use and Health (NSDUH)

• Sponsor: Substance Abuse and Mental Health Services Administration (SAMHSA), Center for Behavioral Health Statistics and Quality (CBHSQ)

• Purpose: Estimate prevalence, correlates and trends of substance use in U.S.

• History: Conducted since 1971, annually since 1990

3

NSDUH Design• Representative nationally and in each state

• Civilian, noninstitutional population, age 12+

• Face-to-face interview using ACASI

• 68,000 respondents each year; oversample age 12-25

• $30 incentive

• Response rates (weighted, 2010):

• 88% of selected households completed screener

• 74% of selected persons completed interview

4

NSDUH Sample Design: Target Sample Sizes by State and Age Group

• Completed Interviews per State

• Large states (8): 3,600 per year

• Small states (43): 900 per year

• Completed Interviews by Age Group

• 1/3 of sample in each age group (12-17, 18-25, 26+)

5

NSDUH Questionnaire

• Use of alcohol, tobacco, and illicit drugs

• Substance use disorders (DSM-IV)

• Substance use and mental health treatment

• Health conditions, service utilization

• Demographics

• Mental health (MDE, suicide)

6

NSDUH Mental Health Surveillance Study (MHSS)

• SAMHSA legislation requires the agency to produce methods to estimate serious mental illness (SMI) (and serious emotional disturbance (SED) in children)

• TAG (2006) recommended NSDUH for SMI (and NHIS for SED)

• MHSS implemented in 2008 NSDUH

7

SAMHSA Definition of Serious Mental Illness (SMI) among Adults

Any DSM-IV mental disorder (other than developmental and substance use disorders)

WITH

serious functional impairment

(both in past year)

Estimating SMI in NSDUH• A complete diagnostic assessment to determine

SMI is not feasible in NSDUH interview • Would require too many questions

• Interviewers are not clinicians

• Alternative approach used by SAMHSA:• Administer clinical interviews on a subsample of NSDUH

respondents, to diagnose SMI

• Include short scales in main NSDUH interview, to be used as predictors of SMI in a model: K6, WHODAS

• Develop a regression model, based on subsample data, and apply to main sample data to predict SMI for each respondent

Kessler 6-item Nonspecific

Psychological Distress Scale (K-6)

• Included in NSDUH and several other large national surveys

• Developed specifically for use in large surveys

• Discriminates between cases and non-cases in community samples

• Demonstrates consistency across population groups

• Responses 0-4 for each item; combined score 0-24

Percentage Distribution of K6 Scores among Persons Aged 18 or Older: 2008

10.710.69.4

7.96.1

5.34.0 3.3 2.7 2.5 2.0 2.6

1.6 1.4 1.1 1.1 1.0 1.40.5 0.5 0.4 0.3 0.2 0.8

22.4

0

5

10

15

20

25

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

Percentage

K6 Score

Measuring Impairment in NSDUH Main Sample: WHODAS

• WHO Disability Assessment Schedule (WHODAS)

• 16 items assessing functional impairments in various domains

• Reduced to 8 items for NSDUH based on IRT analysis

• Responses: 0 to 3 for each item

Clinical Interview Subsample

• At end of NSDUH interview, a request for 2nd interview on mental health is made to respondents selected for the clinical followup interview

• $30 incentive

• N=500 to 1500 per year

• Nationally representative, stratified sample

• Interview conducted by a trained clinical interviewer, by telephone, 2-4 weeks after main interview

13

Clinical Interview Content

• Structured Clinical Interview for DSM-IV (SCID): 15 specific mental disorders are covered

• Global Assessment of Functioning scale (GAF)

14

Estimation Step 1: Determine Best Weighted Logistic Regression Model Using Clinical Interview Subsample

Let π = Pr(“true” SMI│X1, X2)

logit(π) = + X1 + X2

• X1 = recoded K6 score (0-17)

• X2 = recoded WHODAS score (0-8)

Estimation Step 2: Determine Minimum-Bias Cutpoint from Clinical Interview

Data1. Based on model, each CI respondent has predicted

Pr(SMI+) =

2. Based on clinical interview, each CI respondent has a “true” SMI diagnosis

3. Select cutpoint, , for which false positives equal false negatives in the CI subsample

- If then predicted SMI status = positive

- If then predicted SMI status = negative

0

0

0

Final Model Based on 2008 Clinical Interview Data

logit( ) = -4.7500+ 0.2098X1 + 0.3839X2

Where X1 = recoded K6 score (0-17)

X2 = recoded WHODAS score (0-8)

Cutpoint: = 0.26972

17

0

Estimation Step 3: Apply Model to Main Sample

1. Based on model, and reported K6 and WHODAS scores, each NSDUH respondent has predicted Pr(SMI+) =

2. If then SMI status = yes

If then SMI status = no

0

0

ROC Statistics: Final SMI Model with K6 and WHODAS

vs. Alternative Model with K6 Only

19

Model Parameters

Predicted Rate

False Pos. Rate

False Neg. Rate

Sensi-tivity

Speci-ficity

Area Under ROC Curve

K6 .046 .029 .028 .387 .971 .679

K6 and WHODAS

.047 .024 .023 .506 .976 .741

Levels of Mental Illness

Level of MI in Past Year Definition

Low/Mild Mental Illness (LMI)

Any disorder, and GAF>59

Moderate Mental Illness (MMI)

Any disorder, and GAF 51-59

Serious Mental Illness (SMI)

Any disorder, and GAF<51

TOTAL/Any Mental Illness (AMI)

Any disorder

Secondary purpose of the MHSS was to generate estimates of “any mental illness” and to designate levels of severity:

Estimating Other Levels of Mental Illness

• Various models were compared

• Result: The SMI model, with different cutpoints, was found to predict as well as any other model

AMI/ SMI Prediction Based on Recoded K6 and WHODAS Scores

8

7

6

5

4

3

2

1

0

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Recoded K6 Score

Rec

oded

WH

OD

AS

Sco

re

SMI

LMI or MMI

No MI

Prevalence of Mental Health Problems among Adults (18+): 2010

Percent with disorder/problem in past year

23

20

5

6.8

3.8

0

5

10

15

20

25

Any Mental Illness Serious MentalIllness

Major DepressiveEpisode

Serious Thoughts ofSuicide

46 mil 11 mil 15 mil 9 mil

Any Mental Illness in the Past Year among Adults Aged 18 or Older, by Age and Gender: 2010

Percent with Any Mental Illness (AMI) in the Past Year

24

FigMH2.1

20.0

29.9

22.1

14.3

16.8

23.0

0

5

10

15

20

25

30

35

18 or Older 18 to 25 26 to 49 50 or Older Male Female

Age Group Gender

Serious Mental Illness in the Past Year among Adults Aged 18 or Older, by Age and Gender: 2010

Percent with Serious Mental Illness (SMI) in the Past Year

25

FigMH2.2

5.0

7.7

5.8

3.2 3.4

6.5

0

1

2

3

4

5

6

7

8

9

18 or Older 18 to 25 26 to 49 50 or Older Male Female

Age Group Gender

Receipt of Mental Health Services among Adults Aged 18 or Older, by Level of Mental Illness: 2010

26

FigMH2.9

Percent Receiving Mental Health Services in the Past Year

Past Year Substance Use among Adults Aged 18 or Older, by Any Mental Illness: 2010

27

Percent Using Substance

FigMH4.1

Marijuana

Illicit Drugs1 Psychotherapeutics

InhalantsCocaine

HeroinHallucinogens

1 Illicit Drugs include marijuana/hashish, cocaine (including crack), heroin, hallucinogens, inhalants, or prescription-type psychotherapeutics used nonmedically.

Past Year Substance Dependence or Abuse and Mental Illness among Adults Aged 18 or Older: 2010

28

FigMH4.2

SUD = substance use disorder.

SUD,No Mental

Illness

11.2 Million

SUD and Mental Illness

9.2 Million

20.3 Million Adults Had SUD

45.9 Million Adults Had Mental Illness

36.7 Million

Mental Illness, No SUD

Past Year Substance Dependence or Abuse among Adults Aged 18 or Older, by Level of Mental Illness: 2010

Percent Dependent or Abusing Substance

29

FigMH4.4

Issue: Trend Measurement

Options:

• Update models, parameters, and/or cutpoints each year• Small annual sample high variance

• Continue to accumulate clinical interview data and evaluate models; update model when there is evidence that estimates can be substantially improved• Will need to update all prior estimates

Prevalence of Mental Illness among Adults (18+): 2008 to 2010

Percent in past year

31

19.5

4.4

19.9

4.8

20.0

5.0

0

5

10

15

20

25

Any Mental Illness Serious Mental Illness

2008 2009 2010

Issue: Nonresponse Bias and Weighting

CI Sample Disposition, 2008-2009:

Unwtd. N

Unwtd. Pct.

Wtd. Pct.

TOTAL 3,062 100.0 100.0

Respondents 2,027 66.2 59.5

Immediate refusal 420 13.7 24.3

Agreed, but noncontact 477 15.6 12.5

Other nonresponse 138 4.5 3.7

Nonresponse Bias Assessment:Rates of Key Measures among Respondents, Refusals, and

Noncontacts: Clinical Interview Sample, 2008-9

33

Percent in Past Year

3.7

5.6

15.0

17.1

1.3 1.9

9.9

5.54.9 5.3

15.5

18.2

02468

101214161820

Suicide Thoughts Perceive MH TxNeed

Rec'd MHTreatment

Marijuana Use

Respondent (60%) Refusal (24%) Noncontact (13%)

Nonresponse Bias Assessment:Age and Family Income among Respondents, Refusals,

and Noncontacts: Clinical Interview Sample, 2008-9

34

Percent

16.8

12.09.2 9.7

19.8

29.7

0

5

10

15

20

25

30

35

Age 18-25 <$20K Income

Respondent (60%) Refusal (24%) Noncontact (13%)

Other Issues

• What is the best sample design?• Optimize for modeling?

• Prevent extreme weights

• What is best estimation method?• Variance estimation not straightforward

• Estimate prevalences of specific disorders from the clinical interview sample?

Conclusions

• MHSS provides the only current data on trends in mental illness and its co-occurrence with substance use

• Estimates have been widely cited and used in analyses of the impact of health care reform

• Methods can be replicated in other surveys

• But more work needed to refine the models and estimation methods