Upload
alamea
View
17
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Using Research/Evaluation Questions to Define Data Collection and Findings: Findings from the FY 2004 KTOS Follow-up Study. Robert Walker, Allison Mateyoke-Scrivener, & Jennifer Cole University of Kentucky Center on Drug and Alcohol Research. - PowerPoint PPT Presentation
Citation preview
Using Using Research/Evaluation Research/Evaluation
Questions to Define Data Questions to Define Data Collection and Findings:Collection and Findings:
Findings from the FY Findings from the FY 2004 KTOS Follow-up 2004 KTOS Follow-up
StudyStudyRobert Walker, Allison Mateyoke-Robert Walker, Allison Mateyoke-
Scrivener, & Jennifer ColeScrivener, & Jennifer Cole
University of Kentucky University of Kentucky
Center on Drug and Alcohol Center on Drug and Alcohol ResearchResearch
Question 1: What are the Question 1: What are the demographic demographic
characteristics of KTOS characteristics of KTOS sample?sample?
Almost any study asks for these dataAlmost any study asks for these data As with all research questions and As with all research questions and
decisions about measures, you must decisions about measures, you must know how fine a detail is needed – e.g., know how fine a detail is needed – e.g., on ethnicity.on ethnicity. Does your survey discriminate between Does your survey discriminate between
race and ethnicity?race and ethnicity? Can subjects have more than one race?Can subjects have more than one race? More than one ethnicity?More than one ethnicity?
DemographicsDemographics
Intake question Breakdown N=888
Gender Male 62.4%
Mean age 33.8
Race WhiteBlackOther
86.7%11.5%1.8%
Ethnicity: Hispanic /Latino1
0.9%
Marital status Never marriedMarriedDivorcedSeparated/Widowed
38.6%21.0%27.8%12.6%
Current living arrangement
HousedInstitutionalizedOther
87.9%8.7%3.4%
Percent of sample who live with:
Dependent child(ren) Anyone with an alcohol
problemAnyone with a drug
problem
31.4%8.8%8.3%
Question 2: Policy Q #1 is Question 2: Policy Q #1 is about abstinenceabout abstinence
The study must address the The study must address the sophistication level of the users of sophistication level of the users of the data.the data.
Current political climates have little Current political climates have little tolerance for harm reduction ways of tolerance for harm reduction ways of understanding treatment outcomes.understanding treatment outcomes.
Moralistic views of substance abuse Moralistic views of substance abuse call for abstinence, not reduced use.call for abstinence, not reduced use.
Increase in Percent of Clients with Alcohol Abstinence
During Preceding 30 Days(n=888)
Remained abstinent
Newly abstinent
Total % alcoholabstinent at
follow-upN Valid % N Valid %
Alcohol Male (n = 549)
Female (n = 333)
Total (N = 882)
233187420
42.4%56.2%47.6%
13982
221
25.3%24.6%25.1%
67.7%80.8%72.7%
Alcohol to
Intoxication
Male (n = 551)
Female (n = 332)
Total (N = 883)
344245589
62.4%73.8%66.7%
13060
190
23.6%18.1%21.5%
86.0%91.9%88.2%
How to present findings How to present findings that highlight changethat highlight change
Raw numbers help, but often do not help Raw numbers help, but often do not help the reader understand the effect of the reader understand the effect of seemingly modest differences in seemingly modest differences in numbers.numbers.
Findings should include ways of showing Findings should include ways of showing the significance of change both the significance of change both statistically and through the use of statistically and through the use of different ways to think about change different ways to think about change values.values.
Rate of Percent Increase in Rate of Percent Increase in Alcohol Abstinent ClientsAlcohol Abstinent Clients
(n=888)(n=888)
59.7%**
37.8%**43.8%**
24.5%**
52.7%**
32.2%**
010203040506070
Alcohol Alcohol to intoxication
Males Females Total
Increase in Percent of Clients who Reported Abstinence from Illegal Drugs in the Past 30 Days
(Excluding Alcohol)Remained abstinent
Newly abstinent Total % abstinent at follow-
upN Valid %
of the total
sample
N Valid % of the total sample
Male (n = 553) 330 59.7% 154 27.8% 87.5%
Female (n = 332)
158 47.6% 143 43.1% 90.7%
Total (N = 885) 488 55.1% 297 33.6% 88.7%61.0%**
90.6%**
46.6%**
0 20 40 60 80 100
Males
Female
Total
Males Female Total
aSignificance established using z test for proportions.*p < .01. **p < .001
Question 3: Are there Question 3: Are there differences by drug type?differences by drug type?
Measures in KTOS instrument continue to Measures in KTOS instrument continue to drill down to greater detail on specific drill down to greater detail on specific drugs.drugs.
Differences in drug use patterns may Differences in drug use patterns may suggest differences in the rates of suggest differences in the rates of becoming abstinent as well.becoming abstinent as well.
Taking each drug separately may help Taking each drug separately may help discriminate which drugs appear more discriminate which drugs appear more difficult for recovery.difficult for recovery.
Remained abstinent
Newly abstinent Total % abstinen
t at follow-
up
N Valid % of the total
sample
N Valid % of the total
sample
Male (n = 552) 469 85.0% 66 12.0% 97.0%
Female (n = 330) 251 76.1% 70 21.2% 97.3%
Total (N = 882) 720 81.6% 136 15.4% 97.0%18.9%**
27.9%**
14.1%**
0 5 10 15 20 25 30 35
Males
Females
Total
aSignificance established using z test for proportions.*p < .01. **p < .001
Increase in Percent of Clients who Report Past 30 Day Tranquilizer Abstinence and Rate of Change
Increase in Percent of Clients who Report Past 30 Day
Opiate Abstinence and Rate of ChangeRemained abstinent
Newly abstinent Total % abstinent at follow-
upN Valid %
of the total
sample
N Valid % of the total
sample
Male (n = 551) 478 86.8% 53 9.6% 96.4%
Female (n = 331) 252 76.1% 68 20.5% 96.6%
Total (N = 882) 730 82.8% 121 13.7% 96.5%16.6%**
26.9%**
11.1%**
0 5 10 15 20 25 30 35
Males
Females
Total
aSignificance established using z test for proportions.*p < .01. **p < .001
Increase in Percent of Clients who Report Stimulant Abstinence from Intake to
Follow-up and Rate of Change
Remained abstinent
Newly abstinent
N Valid % of the total
sample
N Valid % of the total sample
Male (n = 553) 527 95.3% 19 3.4%
Female (n = 332)
305 91.9% 23 6.9%
Total (N = 885) 832 94.0% 42 4.7%98.7%**
98.8%**
98.7%**
80 85 90 95 100
Males
Females
Total
aSignificance established using z test for proportions.*p < .01. **p < .001
Sub-question: For those who Sub-question: For those who continue to use, are there any continue to use, are there any
reductions in frequencies?reductions in frequencies? Validated instruments such as the ASI use a composite Validated instruments such as the ASI use a composite
scores of past 30 days use to examine level of drug use.scores of past 30 days use to examine level of drug use. However, none of these instruments control for time in However, none of these instruments control for time in
jail or hospital.jail or hospital. For example, consider two clients, both of whom report For example, consider two clients, both of whom report
using marijuana for 15 of the past 30 days.using marijuana for 15 of the past 30 days. 1 client was in jail for 15 days in the past 30 days1 client was in jail for 15 days in the past 30 days The other client had been on the street for 30 of the past 30 The other client had been on the street for 30 of the past 30
daysdays Using the ASI, both clients would be evaluated the Using the ASI, both clients would be evaluated the
samesame In KTOS, we use a proportion of days to estimate use In KTOS, we use a proportion of days to estimate use
levelslevels Client 1 would use 100%Client 1 would use 100% Client 2 would use 50%Client 2 would use 50%
Percent Reduction in Proportion of Percent Reduction in Proportion of Past 30 Days that Substances were Past 30 Days that Substances were
Used (n=888)Used (n=888)
85.7%
52.3%
72.4%
92.0%
81.6%85.5%
0
20
40
60
80
100
Alcohol** Marijuana** Tranquilizers** Opiates** Cocaine** Stimulants*
Perc
ent
of C
lien
ts
Question 4: Does Question 4: Does Treatment Result in Treatment Result in
Increased Employment? Increased Employment? The Role of Functionality The Role of Functionality
VariablesVariables Apart from drug use changes, Apart from drug use changes,
functionality measures can be used to functionality measures can be used to partially confirm clinical results.partially confirm clinical results.
Given extensive research on substance Given extensive research on substance use and employment, the equations use and employment, the equations should be:should be: Decrease in drug use = Increased Decrease in drug use = Increased
employmentemployment Increased employment consistent with Increased employment consistent with
decreased drug usedecreased drug use
30.5%29.4%
10.9%
40.3% 40.8%
16.1%
38.7%44.6%
10.0%
54.6%
29.2%
15.2%
5
15
2535
45
55
65
Full-time Part-time Anyemployment
Unemployed Disabled Proportion ofdays worked
in last 30
Intake Follow-up
Percent of Clients in Each Employment Category
Rate of Change in Rate of Change in EmploymentEmployment
(n=888)(n=888)
↑51.7%**
↓8.3%**
↑35.5%**
↓28.4%**
↓5.6%
0 10 20 30 40 50 60
Full time
Part time
Any employment
Unemployed
Disabled
aSignificance established using z test for proportions.*p < .01. **p < .001
Question 5: Are there any Question 5: Are there any changes in arrests after changes in arrests after
treatment? Another treatment? Another Functionality VariableFunctionality Variable
As with employment, the equation As with employment, the equation typically is:typically is: Decreases in drug use = Decreases in Decreases in drug use = Decreases in
criminal activity and therefore, criminal activity and therefore, decreases in arrests.decreases in arrests.
Likewise, decreases in arrests tend to Likewise, decreases in arrests tend to confirm that decreases in substance use confirm that decreases in substance use have occurred.have occurred.
Percent of Clients Arrested in the Past 12 Months
61.0%59.5%
27.4%23.2% 27.6%
5.8%8.5%25.3%
0
20
40
60
80
100
Arrested on anycharge
Arrestedspecifically ondrug charge
Arrest on DUIcharge
Spent at least 1night in jail
Intake Follow-up
Rate of change in arrestsRate of change in arrests
↓57.5%**
↓54.8%**
↓78.8%**
↓63.4%**
40 60 80 100
Spent at least 1 night in jail
Arrest on DUI charge
Arrested specifically on drug charge
Arrested on any charge
Percent of Clients
Significance established using z test for proportions.*p < .01. **p < .001
Question 6: Are there changes Question 6: Are there changes in mental health symptoms in mental health symptoms
after treatment?after treatment? Given robust associations of mental Given robust associations of mental
health problems associated with health problems associated with substance use, the equation should substance use, the equation should be:be: Decreases in substance use should = Decreases in substance use should =
Decreases in mental health problems.Decreases in mental health problems. Likewise, decreases in mental health Likewise, decreases in mental health
problems should be consistent with problems should be consistent with decreased mental health problems.decreased mental health problems.
Changes in Mental Health Ratings(n=888)
Past 12 months Intake Follow-up
Serious depression 42.1% 35.4%
Serious anxiety 48.7% 41.1%
Trouble understanding/concentrating
41.3% 36.4%
Trouble controlling violent behavior
12.2% 9.9%
Suicidal thoughts 12.1% 7.4%
Attempted suicide 4.8% 2.0%
Prescribed psychiatric medications
31.7% 26.0%
Rate of change in MH Rate of change in MH symptomssymptoms
Significance established using z test for proportions.*p < .01. **p < .001
↓15.9%*
↓15.6%*
↓32.8%
↓11.9%
↓18.9%
↓38.8%*
↓58.3%*
↓18.0%*
0 10 20 30 40 50 60 70
Depression
Anxiety
Hallucinations
Trouble concentrating
Trouble controlling violent behavior
Suicidal thoughts
Attempted suicide
Prescribed psychiatric medication
Question 7: Are there any Question 7: Are there any cost offsets to society from cost offsets to society from
treatment?treatment? Substance abuse treatment, more Substance abuse treatment, more
than any other behavioral treatment, than any other behavioral treatment, has had to show that it saves has had to show that it saves taxpayers money.taxpayers money.
The changes in employment and The changes in employment and arrests offer the clearest behaviors arrests offer the clearest behaviors to measure in terms of cost changes to measure in terms of cost changes from intake to follow-up.from intake to follow-up.
Total Cost of Publicly-funded Treatment Services
$1,884,689
$2,252,359
$367,670
Substance abuse treatment Other MH Treatment Total Service Cost (n=888)
Past 12-month Arrest at Baseline and Follow-up and Estimated Victim
Costs of Crime (n = 888)Arrests
by type of crime
Estimated cost
per arrest
Past 12
month arrests
at intake
Cost of crimes at
intake
Past 12 month arrests
at follow-up
Cost of crimes
at follow-up
Reduction in cost
Drug $3,580 291 $1,041,780
98 $350,840 $690,940
Property
$15,418 377 $5,812,586
127 $1,958,086
$3,854,500
Violence
$35,745 166 $5,933,670
56 $2,001,720
$3,931,950
DUI $25,225 167 $4,212,575
56 $1,412,600
$2,799,975
Total 1001 $17,000,611
337 $5,723,246
$11,277,365
Reduction in Nights in Jail and Related Costs
Jail time and costs At intake
At follow-up
Reduction
Overall number of nights spend in jail in the past 12 months
32,723 9,410 23,313 (71.2%)
Annualized total estimated cost of jail stays at $38.03 per night
$891,047
$256,234 $634,813
(28.8%)
Increased Employment and Earnings (n=888)
Employment variable Baseline Follow-up
Increase
Number of clients working full or part-time
356 485 129
Annualized – days worked in the past 30 days for follow-up sample
83,508 125,496 41,988
Total annualized hours of paid work
668,064 1,003,968
335,904
Annualized total estimated labor value at $7.00 per hour times total hours
$4,676,448
$7,027,776
$2,351,328
Estimated tax value (@6%) of labor income
$280,586 $421,667
$141,080
ConclusionConclusion• The reductions in self-reported arrests for
Kentucky clients• Combined with estimates for reduced victim
costs of their crimes• Reduced costs of jail time and -• Increased earnings and tax revenues• Suggest a cost offset benefit for Kentucky
taxpayers estimated at a ratio of 5.35 to 1.
• In other words, Kentucky saved $5.35 for every dollar spent on treatment during 2004.