Upload
mauricio-tardiff
View
227
Download
10
Embed Size (px)
Citation preview
Global Appraisal of Individual Needs (GAIN): How it acts as a key piece of infrastructure
for supporting the move towards evidenced based practice
Michael Dennis, Ph.D.Chestnut Health Systems,
Normal, IL
Presentation at the “Adolescent Treatment Summit”, Saratoga Springs, New York October 19-20, 2009. The meeting and presentation are sponsored by St. Peter’s
Addiction Recovery Center (SPARC) and New York State Office of Alcoholism and Substance Abuse Services (OASAS). This presentation reports on treatment & research
funded by the Center for Substance Abuse Treatment (CSAT), Substance Abuse and Mental Health Services Administration (SAMHSA) under contracts 270-2003-00006
and 270-07-0191, as well as several individual CSAT, NIAAA, NIDA and private foundation grants. The opinions are those of the author and do not reflect official
positions of the consortium or government. Available on line at www.chestnut.org/LI/Posters or by contacting Joan Unsicker at 448 Wylie Drive,
Normal, IL 61761, phone: (309) 451-7801, Fax: (309) 451-7763, e-mail: [email protected]
1. Provide an overview of the role of the GAIN as a piece of infrastructure in support the move toward both evidence based practice and practice based evidence
2. Describe each of the measures, the reports that they use to help the assessment guide clinical decision making and illustrate how they provide a successively more detailed picture of client needs
3. Highlight our current work to using actuarial estimates of outcomes to improve placement decisions
4. Summarize the status of efforts to make the data available for secondary analysis and translate the software, measures and reports from English into Spanish, French, Portuguese and other languages
Goals of this Presentation are to
Part 1. Provide an overview of the role of the GAIN as a piece of infrastructure in support the move toward both evidence based practice and practice based evidence
The Global Appraisal of Individual Needs (GAIN) is ..
A family of instruments ranging from screening, to quick assessment to a full Biopsychosocial and monitoring tools
Designed to integrate clinical and research assessment
Designed to support clinical decision making at the individual client level
Designed to support evaluation and planning at program level
Designed to support secondary analyses and comparisons across individuals and programs
As of June 30, 2009, there were 1127 administrative units (agencies, grantees, counties, states) collaborating to use the GAIN in the U.S.,
State or County System GAIN-Short Screener GAIN-Quick GAIN-Full
Canada and other countries*
* 1-10 Sites Other Countries: Brazil, China, Mexico, Japan
State or County System GAIN-Short Screener GAIN-Quick GAIN-Full
So what does it mean to move the field towards Evidence Based Practice (EBP)?
Introducing explicit intervention protocols that are– Targeted at specific problems/subgroups and outcomes– Having explicit quality assurance procedures to cause adherence
at the individual level and implementation at the program level
Introducing reliable and valid assessment that can be used – At the individual level to immediately guide clinical judgments
about diagnosis/severity, placement, treatment planning, and the response to treatment
– At the program level to drive program evaluation, needs assessment, performance monitoring and long term program planning
Having the ability to evaluate client and program outcomes – For the same person or program over time, – Relative to other people or interventions
Key Issues that we try to address with the GAIN Instruments and Coordinating Center
1. High turnover workforce with variable education background related to diagnosis, placement, treatment planning and referral to other services
2. Heterogeneous needs and severity characterized by multiple problems, chronic relapse, and multiple episodes of care over several years
3. Lack of access to or use of data at the program level to guide immediate clinical decisions, billing and program planning
4. Missing, bad or misrepresented data that needs to be minimized and incorporated into interpretations
5. Lack of Infrastructure that is needed to support implementation and fidelity
1. High Turnover Workforce with Variable Education
Questions spelled out and simple question format
Lay wording mapped onto expert standards for given area
Built in definitions, transition statements, prompts, and checks for inconsistent and missing information.
Standardized approach to asking questions across domains
Range checks and skip logic built into electronic applications
Formal training and certification protocols on administration, clinical interpretation, data management, coordination, local, regional, and national “trainers”
Above focuses on consistency across populations, level of care, staff and time
On-going quality assurance and data monitoring for the reoccurrence or problems at the staff (site or item) level
Availability of training resources, responses to frequently asked questions, and technical assistance
Outcome: Improved Reliability and Efficiency
2. Heterogeneous Needs and Severity
Multiple domains Focus on most common
problems Participant self description of
characteristics, problems, needs, personal strengths and resources
Behavior problem recency, breadth , and frequency
Utilization lifetime, recency and frequency
Dimensional measures to measure change with interpretative cut points to facilitate decisions
Items and cut points mapped onto DSM for diagnosis, ASAM for placement, and to multiple standards and evidence- based practices for treatment planning
Computer generated scoring and reports to guide decisions
Treatment planning recommendations and links to evidence-based practice
Basic and advanced clinical interpretation training and certification
Outcome: Comprehensive Assessment
3. Lack of Access to or use of Data at the Program Level
Data immediately available to support clinical decision making for a case
Data can be transferred to other clinical information system to support billing, progress reports, treatment planning and on-going monitoring
Data can be exported and cleaned to support further analyses
Data can be pooled with other sites to facilitate comparison and evaluation
PC and web based software applications and support
Formal training and certification on using data at the individual level and data management at the program level
Data routinely pooled to support comparisons across programs and secondary analysis
Over three dozen scientists already working with data to link to evidence-based practice
Outcome: Improved Program Planning and Outcomes
4. Missing, Bad or Misrepresented Data
Assurances, time anchoring, definitions, transition, and question order to reduce confusion and increase valid responses
Cognitive impairment check Validity checks on missing,
bad, inconsistency and unlikely responses
Validity checks for atypical and overly random symptom presentations
Validity ratings by staff
Training on optimizing clinical rapport
Training on time anchoring Training answering questions,
resolving vague or inconsistent responses, following assessment protocol and accurate documentation.
Utilization and documentation of other sources of information
Post hoc checks for on-going site, staff or item problems
Outcome: Improved Validity
5. Lack of Infrastructure
Direct Services
Training and quality assurance on administration, clinical interpretation, data management, follow-up and project coordination
Data management
Evaluation and data available for secondary analysis
Software support
Technical assistance and back up to local trainer/expert
Development
Clinical Product Development
Software Development
Collaboration with IT vendors (e.g., WITS)
Over 36 internal & external scientists and students
Workgroups focused on specific subgroup, problem, or treatment approach
Labor supply (e.g., consultant pool, college courses)
Outcome: Implementation with Fidelity
Across measures, the GAIN has a Common Factor Structure of Psychopathology
Source: Dennis, Chan, and Funk (2006) CFI=.92, RMSEA=.06 allowing for age
Alcohol and Other Drug Abuse, Dependence and Problem Use are Age Related
Source: 2002 NSDUH and Dennis & Scott, 2007
0
10
20
30
40
50
60
70
80
90
100
12-13
14-15
16-17
18-20
21-29
30-34
35-49
50-64
65+Other drug or heavy alcohol use in the past year
Alcohol or Drug Use (AOD) Abuse or Dependence in the past year
Age
Severity Category
Over 90% of use and
problems start between the ages of
12-20
It takes decades before most recover or die
Per
cent
age
Co-occurring Mental Health Problems are Common, but the Type of Problems also Changes with Age
Source: Chan, YF; Dennis, M L.; Funk, RR. (2008). Prevalence and comorbidity of major internalizing and externalizing problems among adolescents and adults presenting to substance abuse treatment. Journal of Substance Abuse Treatment, 34(1) 14-24 .
Internalizing Disorders go up
with age
Externalizing Disorders go down
with age (but do NOT go away)
Progressive Continuum of Measurement (Common Measures)
Screening to Identify Who Needs to be “Assessed” (5-10 min)– Focus on brevity, simplicity for administration & scoring– Needs to be adequate for triage and referral– GAIN Short Screener for SUD, MH & Crime– ASSIST, AUDIT, CAGE, CRAFT, DAST, MAST for SUD– SCL, HSCL, BSI, CANS for Mental Health– LSI, MAYSI, YLS for Crime
Quick Assessment for Targeted Referral (20-30 min)– Assessment of who needs a feedback, brief intervention or referral for
more specialized assessment or treatment– Needs to be adequate for brief intervention– GAIN Quick – ADI, ASI, SASSI, T-ASI, MINI
Comprehensive Biopsychosocial (1-2 hours) – Used to identify common problems and how they are interrelated– Needs to be adequate for diagnosis, treatment planning and placement
of common problems– GAIN Initial (Clinical Core and Full)– CASI, A-CASI, MATE
Specialized Assessment (additional time per area)– Additional assessment by a specialist (e.g., psychiatrist, MD, nurse,
spec ed) may be needed to rule out a diagnosis or develop a treatment plan or individual education plan
– CIDI, DISC, KSADS, PDI, SCAN
Screener Quick C
omprehensive S
pecial
More E
xtensive / Longer/ E
xpensive
18
Part 2. Describe each of the measures, the reports that they use to help the assessment guide clinical decision making and illustrate how they provide a successively more detailed picture of client needs
Next slides will
Describe the difference in the breadth of information you get with different levels of assessment
Summarize validation studies to date
Illustrate the difference using data from a single sample (Reclaiming futures project)
Demonstrate that multi-morbidity is the norm and varies by type of client and program
GAIN-Short Screener (GSS)
Administration Time: A 3- to 5-minute screener Purpose: Used in general populations to
– identify or rule-out clients who will be identified as having any behavioral health disorders on the 60-120 min versions of the GAIN
– triage area of problem– serve as a simple measure of change– Easy for administration and interpretation by staff with minimal
training or direct supervision Mode: Designed for self- or staff-administration, with paper and pen,
computer, or on the web Scales: Four screeners for Internalizing Disorders, Externalizing
Disorders, Substance Disorders, Crime/Violence, and a Total
Response Set: Recency of 20 problems rated past month (3), 2-12 months ago (2), more than a year ago (1), never (0)
Interpretation: Combined by cumulative time period as: – Past month count (3s) to measure of change– Past year count (2s or 3s) to predict diagnosis– Lifetime count (1s, 2s or 3s) as a measure of peak severity– Can be classified within time period low (0), moderate (1-2)
or high (3)– Can also be used to classify remission as – Early (lifetime but not past month)– Sustained (lifetime but not past year)
Reports: Narrative, tabular, and graphical reports built into web based GAIN ABS and/or ASP application for local hosting
GAIN-Short Screener (GSS) (continued)
GAIN-Short Screener (GSS)
GAIN SS Psychometric Properties
Total Disorder Screener (TDScr)
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Prevalence (% 1+ disorder)
Sensitivity (% w disorder above)
Specificity (% w/o disorder below)
(n=6194 adolescents)
Low Mod. High
At 3 or more symptoms we get 99% prevalence, 91% sensitivity, & 89% specificity
Using a higher cut point increases prevalence and specificity, but
decreases sensitivity
Total score has alpha of .85 and is
correlated .94 with full GAIN version
Source: Dennis et al 2006
GSS Performance by Subscale and Disorders
Prevalence Sensitivity Specificity Screener/Disorder 1+ 3+ 1+ 3+ 1+ 3+ Internal Disorder Screener (0-5) Any Internal Disorder 81% 99% 94% 55% 71% 99% Major Depression 56% 87% 98% 72% 54% 94% Generalized Anxiety 32% 56% 100% 83% 44% 83% Suicide Ideation 24% 43% 100% 84% 41% 79% Mod/High Traumatic Stress 60% 82% 94% 60% 55% 90%
External Disorder Screener (0-5) Any External Disorder 88% 97% 98% 67% 75% 96% AD, HD or Both 65% 82% 99% 78% 51% 85% Conduct Disorder 78% 91% 98% 70% 62% 90%
Substance Use Disorder Screener (0-5) Any Substance Disorder 96% 100% 96% 68% 73% 100% Dependence 65% 87% 100% 91% 30% 82% Abuse 30% 13% 89% 25% 14% 28%
Crime Violence Screener (0-5) Any Crime/Violence 88% 99% 94% 49% 76% 99% High Physical Conflict 31% 46% 100% 70% 38% 77% Mod/High General Crime 85% 100% 94% 51% 71% 100%
Total Disorder Screener (0-5)Any Disorder 97% 99% 99% 91% 47% 89% Any Internal Disorder 58% 63% 100% 98% 8% 28% Any External Disorder 68% 75% 100% 99% 10% 37% Any Substance Disorder 89% 92% 99% 92% 20% 51% Any Crime/Violence 68% 73% 100% 96% 10% 32%
Recommend Triage as
0=Not likely1-2 Possible3+=Likely
Moderate (1+) gives best result
for sensitivity
High (3+) gives best result for
specificity
GAIN SS Total Score is Correlated With Level Of Care Placement: Adolescents
0%
1%
2%
3%
4%
5%
6%
7%
8%
9%
10%
11%
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
TDScr Score
% w
ithi
n L
evel
of
Car
e an
d A
ge G
roup OP/IOP (n=2499)
Residential (n=1965)
Low
Mod High ->OP/IOP
Median=6.0Residential
Median=10.5
GAIN SS Total Score is Correlated With Level Of Care Placement: Adults
GAIN SS Can Also be Used for Monitoring
109
11
910
8
32 2
0
4
8
12
16
20
Intake 3Mon
6Mon
9Mon
12Mon
15Mon
18Mon
21Mon
24Mon
Total Disorder Screener (TDScr)
12+ Mon.s ago (#1s)
2-12 Mon.s ago (#2s)
Past Month (#3s)
Lifetime (#1,2,or 3)
Track Gap Between Prior and current
Lifetime Problems to identify “under
reporting”
Track progress in reducing current
(past month) symptoms)
Monitor for Relapse
GAIN Short Screener Profile: Reclaiming Futures (Range based on 0/1-2/3+ Symptoms)
33% 37%48%
38%
81%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
InternalizingDisorderScreener
ExternalizingDisorderScreener
SubstanceDisorderScreener
Crime/ViolenceScreener
TotalDisorderScreener
Low
Mod.
High
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
GAIN Short Screener Number of Problems Mod/Hi in Reclaiming Futures
40%
22%
22%
9%7%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
No. ofProblems
No SR prob
1 Prob.
2 Probs.
3 Probs.
4 Probs.
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
93% endorsed one or more problems
(40% 4 or more)
Construct Validity of GSS Internalizing Disorder Screener
0%10%20%30%40%50%60%70%80%90%
100%
% Days with MHproblem
Mod/High onEmotional Problem
Scale (EPS)
Mod/High onInternal MentalDistress Scale
(IMDS)
Internalizing Disorder Screener (IDScr)
Fu
ll G
AIN
mea
sure
0 1 2 3 4 5
Source: Education Service District 113 (n=979) and King County (n=1002)
Construct Validity of GSS Externalizing Disorder Screener
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
% Days withbehavioralproblems
Mod/High onEmotional Problem
Scale (EPS)
High on BehaviorComplexity Scale
(BCS)
Externalizing Disorder Screener (EDScr)
Fu
ll G
AIN
mea
sure
0 1 2 3 4 5
Source: Education Service District 113 (n=979) and King County (n=1002)
Construct Validity of GSS Substance Disorder Screener
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
% Days of AOD use
Past Year Abuse orDependence
Past YearDependence
Substance Disorder Screener (SDScr)
Fu
ll G
AIN
mea
sure
0 1 2 3 4 5
Source: Education Service District 113 (n=979) and King County (n=1002)
Construct Validity of GSS Crime/Violence Screener
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
% Days of illegalactivities
Mod/High onIllegal Activity
Scale (IAS)
High onCrime/Violence
Scale (CVS)
Crime and Violence Screener (CVScr)
Fu
ll G
AIN
mea
sure
0 1 2 3 4 5
Source: Education Service District 113 (n=979) and King County (n=1002)
Adolescent Rates of High (2+) Scores on Mental Health (MH) or Substance Abuse (SA) Screener by Setting
in Washington State
77% 86
%
73%
75%
61%67
%
83%
62%
75%
60%
57%
40% 46
%
12%
12%
47%
37%
35%
12%
11%
0%10%20%30%40%50%60%70%80%90%
100%
Substance AbuseTreatment(n=8,213)
Student AssistancePrograms(n=8,777)
Juvenile Justice(n=2,024)
Mental HealthTreatment (10,937)
Children'sAdministration
(n=239)
Either High on Mental Health High on Substance High on Both
Source: Lucenko et al (2009). Report to the Legislature: Co-Occurring Disorders Among DSHS Clients. Olympia, WA: Department of Social and Health Services. Retrieved from http://publications.rda.dshs.wa.gov/1392/
Problems could be easily identified Comorbidity is common
Adult Rates of High (2+) Scores on Mental Health (MH) or Substance Abuse (SA) Screener
by Setting in Washington State
81%
78%
65%
64% 69
%
18%
68% 73
%
43%
44%
69%
17%
69%
51%
53%
51%
17%
4%
56%
46%
31%
31%
17%
3%
0%10%20%30%40%50%60%70%80%90%
100%
SubstanceAbuse
Treatment(n=75,208)
Eastern StateHospital(n=422)
Corrections:Community(n=2,723)
Corrections:Prison
(n=7,881)
Mental HealthTreatment(55,847)
ChildrensAdministration
(n=1,238)
Either High on Mental Health High on Substance High on Both
Source: Lucenko et al (2009). Report to the Legislature: Co-Occurring Disorders Among DSHS Clients. Olympia, WA: Department of Social and Health Services. Retrieved from http://publications.rda.dshs.wa.gov/1392/
Lower than expected rates of SA in Mental Health & Children’s
Admin
35%
12%
11%
56%
34%
15%
9%
47%
0%10%20%30%40%50%60%70%80%90%
100%
Substance AbuseTreatment (n=8,213)
Juvenile Justice(n=2,024)
Mental HealthTreatment (10,937)
Children'sAdministration
(n=239)
GAIN Short Screener Clinical Indicators
Source: Lucenko et al (2009). Report to the Legislature: Co-Occurring Disorders Among DSHS Clients. Olympia, WA: Department of Social and Health Services. Retrieved from http://publications.rda.dshs.wa.gov/1392/
Adolescent Client Validation of Hi Co-occurring from GAIN Short Screener vs Clinical Records
by Setting in Washington State
Two page measure closely approximated all found in the clinical record after the next two years
Adult Client Validation of Hi Co-occurring from GAIN Short Screener vs Clinical Records
by Setting in Washington State
17%
3%
59%
39%
22%
56%
0%
10%20%
30%40%
50%
60%70%
80%90%
100%
Substance Abuse Treatment(n=75,208)
Mental Health Treatment(55,847)
Childrens Administration(n=1,238)
GAIN Short Screener Clinical Indicators
Source: Lucenko et al (2009). Report to the Legislature: Co-Occurring Disorders Among DSHS Clients. Olympia, WA: Department of Social and Health Services. Retrieved from http://publications.rda.dshs.wa.gov/1392/
Higher rate in clinical record in Mental Health and Children’s Administration
(But that was past on “any use” vs. “abuse/dependence” and 2 years vs. past year
Other ValidationsConfirmatory Factor Analysis Dennis, Chan & Funk (2006) found that the 20 item GSS and its four subscales
were highly correlated (.84 to .94) with the full scale, had 90% sensitivity and over 90% area under the curve relative to the full GAIN; Confirmatory factors analysis also found it to be consistent with the overall model of psychopathology after allowing for age (CFI=.92; RMSEA=.06).
Substance Disorders: McDonnell and colleagues (2009) found that the 5-item GAIN SS Substance
Disorder Screener had 92% sensitivity and 85% correct classification relative to the Diagnostic Inventory Scale for Children (DISC) Predictive Scales (DPS; Lucas et al 2001) and 88% sensitivity and 88% correct classification relative to the CRAFFT (Knight et al 2001)
Internalizing Disorders: McDonnell and colleagues (2009) found that the 5-item GAIN SS Internalizing
Disorder Screener had 100% sensitivity and 75% correct classification relative to the Youth Self Report (YSR; Achenbach et al, 2001) and that the 5-item GAIN SS Externalizing Disorder Screener had 89% sensitivity and 65% correct classification to the YSR.
Riley and colleagues (2009) found that the 5-item GAIN SS’s Internalizing Disorder Screener had 92% sensitivity and 80% area under the curve relative to the Structured Clinical Interview for DSM (SCID) and was more efficient relative to 11 item Addiction Severity Index (ASI) psychiatric composite score (McLellan et al., 1992), 10 item K10 (Kessler et al., 2002) and the 87 item Psychiatric Diagnostic Screening Questionnaire (PDSQ; Zimmerman and Mattia, 2001)
GAIN Quick (GQ)
Administration Time: 20-30 minutes (depending on severity and whether reasons for quitting module used)
Training Requirements: 1 day (train the trainer) plus certification within 1-2 months
Mode: Generally Staff Administered on Computer (can be done on paper or self administered with proctor)
Purpose: Designed for use in targeted populations to support brief intervention or referral for further assessment or behavioral intervention. Not originally designed for follow-up.
Scales: The GQ has total scale (99-symptoms) and 15 subscales (including more detailed versions of the GSS scales and subscales plus scales for service utilization, sources of psychosocial stress, and health problems).
GAIN Quick (GQ) (Continued)
Response Set: Breadth (past year symptom counts for behavior and lifetime for utilization) and Prevalence (past 90 days)
Interpretation: – Items can be used individually or to create specific
diagnostic or treatment planning statements– Items can be summed into scales or indices for each behavior
problem or and for recent service utilization overall– All scales, indices and selected individual items have
interpretative cut-points to facilitate clinical interpretation and decision making
– Some people repeat just days items for follow-up.
Reports: Narrative, tabular, graphical, validity and “motivational interviewing” reports built into web based GAIN ABS; Program level reports available in SPSS/Excel
GAIN Quick Profile of Reclaiming Futures Sites (Range based on 0-24% / 25-74% / 75-100% of Symptoms)
18% 24%
26%
29%
22% 29
%
25%
28%
5%24%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%G
ener
al F
acto
rs
Sour
ces
of S
tres
s
Hea
lth
Dis
tres
s
*Gen
eral
Lif
e P
rob
Dep
ress
ion
Suic
ide
Ris
k
Anx
iety
-Tra
uma
Sx
*Int
erna
lizi
ng
Hyp
er-I
natt
enti
on
Con
duct
Dis
orde
r
Gen
eral
Cri
me
*Ext
erna
lizi
ng
AO
D U
se &
Abu
se
AO
D D
epen
denc
e
Subs
tanc
e P
robl
em
*Tot
al S
core
Low (0-24%)
Mod (25-75%)
High (76-100%)
Source: Reclaiming Futures Chicago, IL, Dayton, OH, Portland, OR and Santa Cruz, CA sites (n=475). * Summary Measure
RiskStressHealth
More detail within
each area
GAIN Quick Number of Problems Mod/Hi
69%
13%
8%8%3%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
No. ofProblems
No SR prob
1 Prob.
2 Probs.
3 Probs.
4 Probs.
97% endorsed one or more problems(69% 4 or more
problems)
Source: Reclaiming Futures Chicago, IL, Dayton, OH, Portland, OR and Santa Cruz, CA sites (n=475).
Validations
Titus and colleagues (2008) found that the internal consistency of the full scales ranged from .82 to .90 among adults and adolescents with alpha above .7 for 7 of 8 subscale scores for adults and 7 of 8 subscale scores for the adolescents.
Titus and colleagues (2008) found that the mental health scales from the GAIN quick have good internal consistency (.86 to .90), are correlated with the full GAIN dimensional measures (.92 to .97) and .99 to 100% sensitivity relative to the full GAIN.
GAIN Quick (GQ): In Transition
Strengths: Length (20-30min) in desired range, range of topics, efficiently categorizes, narrative reports to support screening, brief intervention, and referral to treatment
Problems: – Lacks scales or recency to support analyses or outcomes
related to “change over time”– Item response choices do not provide information about
lifetime problems– Current Personal Feedback Report focuses only on substance
use and does not address the other content areas of the GAIN-Q
– Only about 60% of the items can be directly imported into the GAIN-I
GAIN Quick (GQ): In Transition
Plans for Version 3:– Keep focus on screening, brief intervention and referral to
treatment– Break out sections for Crime/Violence, HIV risk, Work and
School problems– Subsume GSS and add similar screeners in other GAIN Q
areas with recency response to address change and lifetime issues
– Change measures for each symptom count and days items– Create reasons for change items in each area to support brief
intervention, reducing number of items in substance use– Make all questions importable into full GAIN– Expand narrative report to have more treatment planning
statements and to allow motivational interviewing within each area
Plans for Version 4: Add computer adaptive testing (CAT) component to get at more detailed diagnosis where needed
GAIN Initial (GI) Administration Time: Core version 60-90 minutes; Full version 110-140
minutes (depending on severity) Training Requirements: 3.5 days (train the trainer) plus recommend formal
certification program (administration certification within 3 months of training; local trainer certification within 6 months of training); Advanced clinical interpretation recommended for clinical supervisors and lead clinicians
Mode: Generally Staff Administered on Computer (can be done on paper or self administered with proctor)
Purpose: Designed to provide a standardized biopsychosocial for people presenting to a substance abuse treatment using DSM-IV for diagnosis, ASAM for placement, and needing to meet common (CARF, COA, JCAHO, insurance, CDS/TEDS, Medicaid, CSAT, NIDA) requirements for assessment, diagnosis, placement, treatment planning, accreditation, performance/outcome monitoring, economic analysis, program planning and to support referral/communications with other systems
Scales: The GI has 9 sections (access to care, substance use, physical health, risk and protective behaviors, mental health, recovery environment, legal, vocational, and staff ratings) that include 103 long (alpha over .9) and short (alpha over .7) scales, summative indices, and over 3000 created variables to support clinical decision making and evaluation. It is also modularized to support customization
Response Set: Breadth (past year symptom counts for behavior and lifetime for utilization), Recency (48 hours, 3-7 days, 1-4 weeks, 2-3 months, 4-12 months, 1+ years, never) and Prevalence (past 90 days), patient and staff ratings
Interpretation: – Items can be used individually or to create specific diagnostic or
treatment planning statements– Items can be summed into scales or indices for each behavior
problem or type of service utilization– All scales, indices and selected individual items have interpretative
cut-points to facilitate clinical interpretation and decision making
GAIN Initial (GI) (continued)
Reports: – Validity Report (VR): identifying missing/bad data and
potentially problematic areas of assessment– Individual Clinical Profile (ICP): lab report with graphical
and tabular summary with links back to the items– GAIN Recommendation and Referral Summary (GRRS):
Draft of biopsychosocial narrative for clinician to use for initial assessment summary, diagnosis, placement and treatment planning
– Personal Feedback Report (PFR): used to support Motivational Interviewing (MI) / Motivational Enhancement Therapy (MET)
– Program Profile: program level report that allows comparison of client characteristics, services received and outcomes between programs, cohorts or types of clients.
GAIN Initial (GI) (continued)
GAIN Initial Profile: Substance Problems Past Year(Range based range of clinical/logical/statistical rules)
31%19%15%
7%0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Sub.
Use
/In
duce
dP
rob.
Abu
se
Dep
ende
nce
Sub.
Pro
b.P
ast
Yea
r
Low
Mod.
High
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
GAIN Initial Profile: Substance Problems by Time(Range based range of clinical/logical/statistical rules)
39%31%
13%2%0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Sub.
Pro
b.L
ifet
ime
Sub.
Pro
b.P
ast
Yea
r
Sub.
Pro
b.P
ast
Mon
th
Wit
hdra
wal
Sx P
ast
Wee
k
Low
Mod.
High
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
GAIN Initial Profile: Motivation and Readiness(Range based range of clinical/logical/statistical rules)
0% 7%19%
32%
76%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Self
-E
ffic
acy
Tre
atm
ent
Res
ista
nce
Tre
atm
ent
Pre
ssur
e
Tre
atm
ent
Mot
ivat
ion
Pro
blem
Ori
enta
tion
Low
Mod.
High
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
GAIN Initial Profile: Crime/Violence(Range based range of clinical/logical/statistical rules)
25%33%
5% 8%
51%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Con
flic
t-T
acti
c
Pro
pert
yC
rim
e
Inte
rper
sona
lC
rim
e
Dru
g C
rim
e
Cri
me
Vio
lenc
e
Low
Mod.
High
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
GAIN Initial Profile: Environmental Risk(Range based range of clinical/logical/statistical rules)
39%
64%54%
28%
0%
10%
20%30%
40%
50%
60%
70%80%
90%
100%
Liv
ing
Env
iron
men
tR
isk
Voc
atio
nal
Env
iron
men
tR
isk
Soci
alE
nvir
onm
ent
Ris
k
Env
iron
men
tal
Ris
k
Low
Mod.
High
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
GAIN Initial Profile: Internalizing Disorders(Range based range of clinical/logical/statistical rules)
3%15%
1% 9% 9%24%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Som
atic
Dep
ress
ion
Suci
de R
isk
Anx
iety
-F
ear
Tru
ama
Inte
rnal
izin
g
Low
Mod.
High
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
GAIN Initial Profile: Externalizing Disorders(Range based range of clinical/logical/statistical rules)
20%12% 14% 20%
0%
10%20%
30%
40%50%
60%
70%
80%90%
100%
Inat
tent
iven
ess
Hyp
erac
tivi
ty-
impl
usiv
e
Con
duct
Dis
orde
r
Ext
erna
lizi
ng
Low
Mod.
High
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
GAIN Initial Profile: Personality Disorders(Range based range of clinical/logical/statistical rules)
53%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Cau
tiou
s(C
lust
er A
)
Impl
usiv
e(C
lust
er B
)
Wor
ryin
g(C
lust
er C
)
Tot
alP
erso
nali
ty
Low
Mod.
High
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
GAIN Initial Profile: General Factors / Stress(Range based range of clinical/logical/statistical rules)
26%44%
10% 12%0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Em
ploy
men
tP
rob.
S
choo
lP
rob.
Vic
tim
izat
ion
P
erso
nA
xis
IV
O
ther
Axi
sIV
Low
Mod.
High
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
GAIN Initial Profile: Other Problem Scales(Range based range of clinical/logical/statistical rules)
12%2% 4%
17%0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Hea
lth
Gam
blin
g
Soci
alSu
ppor
t
Lif
eSa
tisf
acti
on
Low
Mod.
High
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
GAIN Initial Profile: Measures of Behavior Change (Range based range of clinical/logical/statistical rules)
41%
3% 10% 14%
41%23%
3%4%0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%Su
bsta
nce
Use
Hea
lth
Em
otio
ns
Rec
over
yE
nvir
onm
ent
Ille
gal
Act
ivit
y
Scho
ol
Wor
k
Fin
anci
al
Low
Mod.
High
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
GAIN Initial Number of Problems Mod/Hi
98%0%0% 1%1%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
No. ofProblems
No problems
1 Prob.
2 Probs.
3 Probs.
4 Probs.
99% endorsed one or more problems
(98% 4 or more)
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
GAIN Treatment Planning/Placement Grid
Problem Recency/Severity
None Past Current (past 90 days)*
Low-Mod | High Severity Treatm
ent H
istory
Non
e Past C
urren
t .
1. No Problem2. Past problem Consider monitoring and relapse prevention.
3. Low/Moderate problems; Not in treatmentConsider initial or low invasive treatment.
4. Severe problems;Not in treatment Consider a more intensive treatment or intervention strategies.
0. Not LogicalCheck under- standing of problem or lying and recode.
5. No current problems; Currently in treatmentReview for step down or discharge.
6. Low/Moderate problems; Currently in treatment Review need to continue or step up.
7. Severe problems; Currently in treatmentReview need for more intensive or assertive levels.
* Current for Intoxication & Withdrawal = Past 7 days
Reclaiming Futures ASAM Placement Cells
0% 20% 40% 60% 80% 100%
B1.Intox/Withd.
B2 Biomedical
B3.Psych/Beh
B4.Readiness
B5.Rel. Pot.
B6.Environ.
Inconsistent No problem Past Prob Low/Mod ProbHigh Prob No Prob in Tx L/M Prob in Tx H Prob in Tx
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
Other Common Treatment Planning Needs: Reclaiming Futures
90%
89%
87%
80%
76%
74%
73%
70%
66%
62%
61%
59%
58%
57%
56%
0%
10
%
20
%
30
%
40
%
50
%
60
%
70
%
80
%
90
%
10
0%
Continuing care
Case management
Recovery Environment Risk
Coping w/ Psychosocial Stressors
Accessing Treatment
Child Maltreatment
Disatisfaction with Environment
Behavior Control
School Problems
Anger Management
Vocational Assistance
Detox or Withdrawal
Recovery Coach
HIV risk reduction (sex)
Tobacco CessationSource: Reclaiming Futures (n=192)
64
Part 3. Highlight our current work to using actuarial estimates of outcomes to improve placement decisions
CSAT Adolescent Treatment GAIN Data from 203 level of care x site combinations
Outpatient
General Group Home
Short-Term Residential
Outpatient Continuing CareIntensive Outpatient
Long-term ResidentialModerate-Term Residential
Early InterventionOtherCorrections
Levels of Care
Source: Dennis, Funk & Hanes-Stevens, 2008
Ratings of Problem Severity (x-axis) by Treatment Utilization (y-axis) by Population Size (circle size)
12%
20%
14%
8%
14%
12%
-0.20
0.00
0.20
0.40
0.60
0.80
1.00
-0.20 0.00 0.20 0.40 0.60 0.80 1.00
Average Current Problem Severity
Ave
rage
Cur
rent
Tre
atm
ent U
tili
zati
on
.
A Low-Low
B Low- Mod
C Mod-Mod
DHi-Low
EHi-Mod
F. Hi-Hi (CC)
G. Hi-Mod(Env Sx/ PH Tx)
9%
H. Hi-Hi(Intx Sx; PH/MH Tx) 12%
Variance Explained in 10 NOMS Outcomes
\1 Past month \2 Past 90 days *All statistically Significant
26%
24%
11%
25%
15%
33%
26%
18%
14%
8%
24%
0% 5% 10% 15% 20% 25% 30% 35%
No AOD Use
No AOD related Prob.
No Health Problems
No Mental Health Prob.
No Illegal Activity
No JJ System Involve.
Living in Community
No Family Prob.
Vocationally Engaged
Social Support
Count of above
Percent of Variance Explained
Source: CSAT 2007 AT Outcome Data Set (n=11,013)
Predicted Count of Positive Outcomes by Level of Care: Cluster A Low - Low (n=1,025)
2
3
4
5
6
7
8
9
10
Outpatient Higher LOC
2
3
4
5
6
7
8
9
10
Predicted Count of Positive Outcomes by Level of Care: Cluster A Low - Low (n=1,025)
Person “A” does better in Outpatient
Person “B” does better in Higher Levels of Care
Best Level of Care*: Cluster A Low - Low (n=1,025)
Source: CSAT 2007 AT Outcome Data Set (n=11,013)
Best Level of Care*: Cluster A Low - Low (n=1,025)
99.6%
0.4%0%
20%
40%
60%
80%
100%
120%
Outpatient Higher LOC
% B
est
Pre
dic
ted O
utc
om
es
* Based on Maximum Predicted Count of Positive Outcomes
Source: CSAT 2007 AT Outcome Data Set (n=11,013)
Best Level of Care*: Cluster B Low - Mod (n=1,654)
0.4%
14.1%
75.1%
10.5%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Outpatient IOP OPCC Residential
% B
est P
redi
cted
Out
com
es
* Based on Maximum Predicted Count of Positive Outcomes
Best Level of Care*: Cluster C Mod-Mod (n=1209)
Source: CSAT 2007 AT Outcome Data Set (n=11,013)
Best Level of Care*: Cluster C Mod-Mod (n=1209)
30.2%
7.6%
23.6%
38.6%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Outpatient IOP OPCC Residential
% B
est
Pre
dic
ted O
utc
om
es
* Based on Maximum Predicted Count of Positive Outcomes
Source: CSAT 2007 AT Outcome Data Set (n=11,013)
Best Level of Care*: Cluster D Hi-Low (n=687)
38.3%33.8%
27.9%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Outpatient IOP/OPCC Residential * Based on Maximum Predicted Count of Positive Outcomes
Best Level of Care*: Cluster F Hi-Hi (CC) (n=968)
Source: CSAT 2007 AT Outcome Data Set (n=11,013)
Best Level of Care*: Cluster F Hi-Hi (CC) (n=968)
81.5%
8.6%
0.0%
9.9%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Outpatient IOP OPCC Residential
% B
est
Pre
dic
ted O
utc
om
es
* Based on Maximum Predicted Count of Positive Outcomes
Source: CSAT 2007 AT Outcome Data Set (n=11,013)
Best Level of Care*: Cluster Cluster H Hi-Hi (Intx/PH/MH) (n=1,017)
78.2%
17.2%
0.0%4.6%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Outpatient IOP OPCC Residential
% B
est P
redi
cted
Out
com
es
* Based on Maximum Predicted Count of Positive Outcomes
Source: CSAT 2007 AT Outcome Data Set (n=11,013)
Best Level of Care*: Cluster E Hi-Mod (n=1,190)
1.1%
88.3%
10.6%
0.0%0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Outpatient IOP OPCC Residential
% B
est P
redi
cted
Out
com
es
* Based on Maximum Predicted Count of Positive Outcomes
Best Level of Care*: Cluster G Hi-Mod (Env/PH) (n=749)
Source: CSAT 2007 AT Outcome Data Set (n=11,013)
Best Level of Care*: Cluster G Hi-Mod (Env/PH) (n=749)
94.1%
5.9%0.0%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Outpatient IOP/OPCC Residential
* Based on Maximum Predicted Count of Positive Outcomes
Best (x) by Actual (y) Level of Care Placement
3132
797
1968
2339
0
500
1000
1500
2000
2500
3000
3500
Outpatient (n=3132)
Intensive Outpatient(n=797)
OP - ContinuingCare (n=1968)
Residential(n=2339)
Higher
Best
Lower
553/771=72%unmet need
218/224=97% to targeted
771/982=79% in need
Exploring Need, Unmet Need, & Targeting of Mental Health Services in AAFT
Size of the Problem
Extent to which services are currently being targeted
Extent to which services are not reaching those in most need
At Intake .
After 3 mon
No/Low
Need
Mod/High
Need
Total
Any Treatment 6 218 224
No Treatment 205 553 758
Total 211 771 982
Mental Health Problem (at intake) vs. Any MH Treatment by 3 months
79%
97%
72%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
% of Clients WithMod/High Need
(n=771/982)*
% w Need but No ServiceAfter 3 months
(n=553/771)
% of Services Going toThose in Need
(n=218/224)
*3+ on ASAM dimension B3 criteriaSource: 2008 CSAT AAFT Summary Analytic Dataset
Why Do We Care About Unmet Need?
If we subset to those in need, getting mental health services predicts reduced mental health problems
Both psychosocial and medication interventions are associated with reduced problems
If we subset to those NOT in need, getting mental health services does NOT predict change in mental health problems
Conversely, we also care about services being poorly targeted to those in need.
Residential Treatment need (at intake) vs. 7+ Residential days at 3 months
36%
52%
90%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
% of Clients WithMod/High Need
(n=349/980)*
% w Need but NoService After 3 months
(n=315/349)
% of Services Going toThose in Need (n=34/66)
Opportunity to redirect
existing funds through better
targeting
Source: 2008 CSAT AAFT Summary Analytic Dataset
82
Part 4. Summarize the status of efforts to make the data available for secondary analysis and translate the software, measures and reports from English into Spanish, French, Portuguese and other languages
83
We currently pool data from Center for Substance Abuse Treatment (CSAT) grantees annual and make it available for secondary analysis:
– Requires abstract length proposal/ feasibility– Requires agreement to respect privacy and not attempt to re-identify– We will get permission from any active grantees– No cost to the end user
Over 36 scientist and evaluators have already accessed the data and about 1-2 more come get approval each month
We can also negotiate access to additional data from individual grantees and/or regional projects
Secondary Analysis
84
Status of Translations
LanguageShortScreener
Other Instruments Software Reports
English Done Done Done Done
Spanish Done Done In progress In progress
French In progress In progress Planned Planned
Portuguese Done In progress Not yet Not Yet
Hmong, Japanese, Russian, Pilipino, Punjabi, Vietnamese
Done Not yet Not yet Not Yet
85
Acknowledgments and Contact Information This presentation was supported by analytic runs provided by Chestnut Health Systems for the
Substance Abuse and Mental Health Services Administration's (SAMHSA's) Center for Substance Abuse Treatment (CSAT) under Contracts 207-98-7047, 277-00-6500, 270-2003-00006 and 270-2007-00004C using data provided by the following 152 grantees: TI11317 TI11321 TI11323 TI11324 TI11422 TI11423 TI11424 TI11432 TI11433 TI11871 TI11874 TI11888 TI11892 TI11894 TI13190TI13305 TI13308 TI13313 TI13322 TI13323 TI13344 TI13345 TI13354 TI13356 TI13601 TI14090 TI14188 TI14189 TI14196 TI14252 TI14261 TI14267 TI14271 TI14272 TI14283 TI14311 TI14315 TI14376 TI15413 TI15415 TI15421 TI15433 TI15438 TI15446 TI15447 TI15458 TI15461 TI15466 TI15467 TI15469 TI15475 TI15478 TI15479 TI15481 TI15483 TI15485 TI15486 TI15489 TI15511 TI15514 TI15524 TI15524 TI15527 TI15545 TI15562 TI15577 TI15584 TI15586 TI15670 TI15671 TI15672 TI15674 TI15677 TI15678 TI15682 TI15686 TI16386 TI16400 TI16414 TI16904 TI16928 TI16939 TI16961 TI16984 TI16992 TI17046 TI17070 TI17071 TI17334 TI17433 TI17434 TI17446 TI17475 TI17476 TI17484 TI17486 TI17490 TI17517 TI17523 TI17535 TI17547 TI17589 TI17604 TI17605 TI17638 TI17646 TI17648 TI17673 TI17702 TI17719 TI17724 TI17728 TI17742 TI17744 TI17751 TI17755 TI17761 TI17763 TI17765 TI17769 TI17775 TI17779 TI17786 TI17788 TI17812 TI17817 TI17825 TI17830 TI17831 TI17864 TI18406 TI18587 TI18671 TI18723 TI19313 TI19323 TI655374. Any opinions about this data are those of the authors and do not reflect official positions of the government or individual grantees. It is available at www.chestnut.org/li/posters. Comments or questions can be addressed to Michael Dennis, Chestnut Health Systems, 448 Wylie Drive, Normal, IL 61761. Phone 1-309-451-7801; E-mail: [email protected].
More information on the GAIN is available at www.chestnut.org/li/gain or by e-mailing [email protected] .