A Brief Self-administered A Brief Self-administered Substance Use Screening Tool for Substance Use Screening Tool for Primary Care: Two-site validation Primary Care: Two-site validation study of the Substance Use Brief study of the Substance Use Brief

Screen (SUBS) Screen (SUBS)

Jennifer McNeely, Shiela M. Strauss, Charles M. Cleland, Richard Saitz, Joseph J. Palamar, John Rotrosen, Marc N. Gourevitch

No relevant financial relationships to disclose

ObjectivesObjectives1. Discuss potential advantages of a self-

administered screening approach2. Describe development of the Substance Use

Brief Screen (SUBS)3. Present results of a validation study in primary

care 3. Discuss feasibility and application

Self-administered Self-administered screening approachscreening approach

1. Efficiency

2.Fidelity

3. Patient comfort

Sterling, Addict Med Clin Pract 2012Friedmann, J Gen Int Med 2000Friedmann, Arch Int Med 2001Anderson, Alcohol Alcoholism 2004McCormick, J Gen Int Med 2006

Bradley, JGIM, 2011Williams, ASCP, 2013

Tourangeau, Pub Op Quart, 1996Wight, AIDS Educ Prev, 2000

Substance Use Brief Screen (SUBS)Substance Use Brief Screen (SUBS)

Instrument development Instrument development processprocess

Pilot Study Modify computer interface

Inform feasibility, acceptability

Focus Groups

Test-retest Reliability Excellent agreement

Validation: comparison to reference standard

measures

McNeely, et al., INEBRIA 2011; AMERSA 2011McNeely et al., SGIM 2012McNeely et al., Substance Abuse, in pressSpear, Shedlin, McNeely, AHSR 2013McNeely et al., J Sub Abuse Treat, under review

Cognitive Interviews Select language and format

Overview of Study Overview of Study ProceduresProcedures

Screening (computer) Validation Measures (interviewer)

ReferralsIncentive

• ASSIST• Timeline Follow

Back• MINI-Plus• Fagerstrom• REALM• Demographics

Oral fluid tests

Second Consent

• SUBS• SISQ-alcohol, SISQ-drug

Reference standard Reference standard measuresmeasures

  ASSIST TLFB (30 days)

MINI-Plus Fagerstr

om Test*

Nic-Alert oral fluid *

Intercept oral fluid*

Tobacco – unhealthy use +     + +  

Tobacco - disorder       +    

Alcohol – unhealthy use + + +      

Alcohol - disorder     +      Illicit drug – unhealthy use + + +     +

Illicit drug - disorder     +      

Rx drug – unhealthy use + + +     +

Rx drug – disorder     +      

* Collected at Site A only

Statistical AnalysisStatistical Analysis1. Comparison of SUBS item to

composite reference standards 2. Examine site differences3. Calculate sensitivity, specificity, AUC:oTobacco oAlcoholo Illicit drugsoPrescription drugs

4. Subgroup analysesAny drugs

Study Sites and Study Sites and RecruitmentRecruitment

• Adult primary care clinics• 2 urban safety net hospitals • Patients presenting for medical visits• Consecutive recruitment

Eligibility Criteria: • Age 21-65• Current clinic patient• Fluent in English• No disability preventing computer use

Participant RecruitmentScreened: N = 3063

Eligible: N = 1198

1865 were excludedLanguage: 972Age: 439Not a patient: 229Other: 225

Completed interview: N = 586Site A: 390*Site B: 196

610 declinedNo time: 499Other: 111

2 had lost data

*348 (89%) Site A participants agreed to saliva test

Characteristics of the 586 Characteristics of the 586 participantsparticipants

Age (years) Mean = 46, SD = 12Range = 21-65

Sex (%) MaleFemaleTransgender

49.849.80.4

Race/Ethnicity (%) Black/African American HispanicWhite/Caucasian Other

50.221.718.79.4

Country of Birth (%)

United StatesOutside of United States

66.433.6

Education and Health LiteracyEducation and Health Literacy Highest Level of Education Health Literacy Level

Prevalence of substance usePrevalence of substance use

Substance Lifetime use %

Current use%

Tobacco 64 37Alcohol 85 56Any Drug 62 26

Illicit 59 23Prescription 24 9

Unhealthy useUnhealthy useSubstance + on SUBS

N (%)

+ on Reference

N (%)

Sensitivity

%(95% CI)

Specificity

%(95% CI)

AUC

(95% CI)

Tobacco 144 (37) 136 (35) 98 (94-100) 96 (92-98) .97(.95-100)

Alcohol 252 (43) 189 (32) 85 (79-90) 77 (73-81) .81 (.78-.84)

Illicit Drugs 133 (23) 148 (26) 81 (74-87) 97 (95-98) .89 (.86-.92)

Rx Drugs 74 (13) 54 (9) 56 (41-69) 92 (89-94) .74 (.67-.80)

Any Drugs 169 (29) 160 (28) 83 (76-88) 91 (88-94) .87 (.84-.90)

Unhealthy use: saliva test Unhealthy use: saliva test resultsresults

(n=390)(n=390)Substance + on SUBS

N (%)

+ on Reference

N (%)

Sensitivity

% (95% CI)

Specificity

%(95% CI)

AUC

Tobacco 144 (37) 121 (31) 99(96-100)

91(87-94)

.951

Alcohol 167 (43) 67 (17) 94(85-98)

68(62-73)

.809

Illicit Drugs 77 (20) 83 (21) 80(69-88)

96(94-98)

.879

Rx Drugs 45 (12) 25 (6) 56(35-76)

91(88-94)

.737

Any Drugs 108 (28) 85 (22) 86(76-92)

89(85-92)

.874

NC NC

77 92

Substance Use Disorder Substance Use Disorder

Substance + on SUBS

N (%)

+ on Reference

N (%)

Sensitivity

%(95% CI)

Specificity

%(95% CI)

AUC

(95% CI)

Tobacco 144 (37) 49 (13) 100 (93-100) 72 (67-77) .86 (.84-.89)

Alcohol 252 (43) 77 (13) 94 (86-98) 65 (60-69) .79 (.76-.83)

Illicit Drugs 133 (23) 95 (16) 82 (73-89) 89 (86-91) .85 (.81-.90)

Rx Drugs 74 (13) 27 (5) 59 (39-78) 89 (86-92) .74 (.65-.84)

Any Drugs 171 (30) 98 (17) 86 (77-92) 82 (78-85) .84 (.80-.88)

Subgroup AnalysisSubgroup AnalysisSubgroups anticipated to have greater difficulty with self-administered screening:•Male•Age greater than 50•Hispanic/Latino•Primary language other than English•Born outside US•Education or health literacy lower than high school level

Subgroup AnalysisSubgroup Analysis• Lower sensitivity for unhealthy drug use in

females (p<0.01)

• Higher sensitivity for tobacco use in Hispanics (p<0.01)

Sensitivity SpecificityFemale 70.9 (51.7, 82.4) 94.3 (90.5, 97.0)

Male 88.6 (80.9, 94.0) 87.0 (81.2, 91.5)

Sensitivity SpecificityHispanic 100.0 (96.2,

100.0)94.6 (90.3, 97.4)

Non-Hispanic 92.7 (80.1, 98.5) 98.5 (91.8, 100.0)

LimitationsLimitations• Safety net primary care populations• English speaking only• Tested in research context, with

assurance of confidentiality• Low prevalence of prescription drug

misuse

ConclusionsConclusions• Substance Use Brief Screen (SUBS)

accurately identified unhealthy substance us in primary care patients

• Additional assessments may be needed for prescription drug misuse

• Computerized self-administered approach has potential to ease implementation barriers to substance use screening in primary care

AcknowledgementsAcknowledgementsFunding:• K23 Career Development

Award NIDA K23 DA030395• NYU-HHC CTSI Translational

Pilot GrantNIH/NCATS UL1 TR000038 • The MITRE Corporation

(contract from ONC and SAMHSA)

Staff and others:•Julianne Cameron•Arianne Ramautar•Derek Nelsen•Linnea Russell•Seville Meli•Jacqueline German•Ritika Batajoo•Catherine Federowicz•Marshall Gillette•Charlie Jose•Emily Maple•Keshia Toussaint•Study participants