Upload
buinga
View
218
Download
2
Embed Size (px)
Citation preview
Do Stimulant Medications Improve
Educational and Behavioral Outcomes
for Children with ADHD?
Janet Currie, Mark Stabile, Lauren Jones
Motivation
• Over past 20 years, mental health disabilities have overtaken physical disabilities as the leading cause of activity limitations in children.
• U.S. Surgeon General’s report on Mental health:
– 20% of children in the U.S. suffer some impairment from a mental health disorder
– 11% have significant functional impairment
– 5% extreme functional impairments
• ADHD is 3 times more likely than asthma to be contributing to childhood disability in the US.
Kids with ADHD: outcomes and treatment
• Recent literature indicates that children with ADHD have:
– Lower standardized test scores
– More likely to be placed in special education
– More likely to be delinquent (Currie and Stabile, 2006, 2007)
• Untreated children disrupt learning for their classmates (Aizer,
2009)
• In the US about 8% of children diagnosed with ADHD. More than
half on stimulant medication such as Ritalin (CDCP, 2005)
• Millions of children worldwide taking stimulants such as Ritalin
for ADHD
• (Ritalin is the brand name for Methylphenidate, a central nervous
system stimulant that helps with hyperactivity and impulse
control)
Controversy over treatment.
• Despite or perhaps because of the number of children taking
meds, treatment for ADHD is somewhat controversial.
• National Institute of Mental Health (US) recommends
treatment with stimulants and says it is safe under medical
supervision (NIMH 2012)
• However, there remain concerns about safety and side effects
including cardiovascular problems, decreased appetite,
insomnia, headache, dizziness, and mood changes (Nissen,
2006, Schachter, 2001, NIMH, 2012).
• Significant coverage in the NY Times over the past year on
treatment and side effects of treatment for ADHD.
• E.g. In 2006 US Food and Drug Administration voted to
recommend a warning label for cardiovascular risk
Controversy cont.
• Part of the reason for the controversy around treatment is that
is quite difficult to get evidence on the longer term benefits.
• Drugs often prescribed with the goal of helping children do
better in school. But we don’t know much about whether they
do.
• Most drug trials only follow children for a short period of time
• Families choose whether to seek treatment and how long to
stay on – not random.
• If drugs do not actually lead to scholastic benefits in the long
run, then the case for subjecting kids to even a small risk of
side effects is weakened.
Our contributions:
• Assess the short and long term treatment for ADHD with stimulant medication using longitudinal data and a policy experiment which expanded access to coverage for prescription drugs.
• Large sample of children followed from 1994 to 2008
• All children assessed for ADHD so that we do not have to deal with selection into diagnosis
• Exploit exogenous variation in the availability of drugs due to the policy experiment
• Able to use child fixed effects to control for unobservable characteristics of the child.
Findings in Brief
• We find strong evidence that insurance expansion in Quebec
resulted in an increase in the use of Ritalin relative to the rest
of Canada.
• We find little evidence of reported improvements in behavior.
• We find negative effects on some schooling and behavior
measures in both the medium and long run.
What do we know about Stimulants and Outcomes?
• Controlled studies suggest medication improves short
term memory and performance over a few months.
Assumed these will translate into academic gains.
• US National Institute of Mental Health Multimodal
Treatment Study (MTA): random selection of ADHD
kids into 1) behavioral therapy, stimulants plus
behavioral therapy, stimulants alone, usual
community care. No significant difference in
behavior between the four groups.
• Note: all children were treated, is not comparing
“community care” vs. “no care.”
• Studies that track kids who select into treatment
(Zoega, 2009, Barberesi et al 2007) suggest treated
children do better when treatment starts earlier (but
potential endogeneity of treatment start dates).
• Dalsgaard, Nielsen, Simonsen (2012) show positive
effects on ER contact, poisoning, injuries.
• ADHD with treatment on health (Barberesi et al
2013) – higher rate of suicide.
• We know much less about the effects on education.
Data
• National Longitudinal Survey of Children and Youth
• Surveyed children ages 0-11 and their families
beginning in 1994.
• For short term analysis: use full cross sectional data
for all children ages 2-15 where possible.
• Includes ADHD score for all children as defined
above
• Includes indicator variable for taking stimulants
(Ritalin or other similar medication).
• For longer term analysis: use longitudinal sample of
children ages 2-11 in 1994. Track children through to
2008
The Measurement of Mental Health
• Measures taken from Qs asked to parents about symptoms.
• Qs similar to those asked as a first step in a diagnosis of a mental health condition.
• Asked of all children, regardless of whether they have a diagnosis or not.
• Provides a linear measure for children ranging from no symptoms to mild to severe, instead of a 0/1 measure.
Mental Health Var E.g.
• Hyperactivity score:
– HOW OFTEN WOULD YOU SAY THAT “name”:
• Can't sit still, is restless or hyperactive?
• Is distractible, has trouble sticking to any activity?
• Can't concentrate, can't pay attention for long?
• Cannot settle to anything for more than a few moments?
• Is inattentive?
Distribution of Hyperactivity Scores:
0
5
10
15
20
25
Score 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
Outcomes of Interest – Short Term
• Behavior, medium term:
– Behavioral scores
– Relationship with parents
Based on questions to parents.
Behavior Scores
• Anxiety-Emotional Disorder Score. Score ranges from 0-
12. Higher is worse.
• Subset of anxiety related to happiness: How often does
seem:
1. unhappy, sad or depressed
2. not as happy as other kids
3. unable to enjoy him/herself
• How well does child get along with parents over the past
six months? Child gets along “quite well” or “very well”
gets a 1; 0 otherwise
Outcomes of Interest – Short Term
• Education, medium term:
– Math test score (based on 15 questions from
Canadian Achievement Test. Administered in
schools.)
– Parents asked whether child repeated a grade in
past 2 years.
Outcomes – Long Term
• Behavior Longer term:
– Ever diagnosed with a mental/emotional disorder
– Depression score
• Education, Longer term:
– Graduates from high school
– Ever attends post-secondary schooling
Quebec’s Insurance Expansion
• January 1, 1997 Quebec implemented its prescription drug
insurance program. This is on top of regular public health
insurance.
• All residents of Quebec, by law, required to have drug
insurance
• Can have either private coverage through employer or group or
public (new)
• Public plan available for seniors, welfare recipients, and
people without access to a private employer plan.
• Public plan premiums and deductibles are scaled according to
income. Maximum premium in 1997 was $750 with 25% co-
insurance.
Quebec
Rest of Canada
Wave 1 1994
Wave 2 1996
Wave 3 1998
Wave 4 2000
Wave 5 2002
Wave 6 2004
Wave 7 2006
Wave 8 2008
Medium term
outcomes sample
Age 2-11
Age 2-13
Age 4-15
Age 6-16
Age 8-16
Age 10-16
Age 12-16
Age 14-16
Policy Change
Date
Changes in Insurance Coverage in Quebec versus the Rest
of Canada:
Reported Drug Insurance Coverage
Pre-Reform Post -Reform
Year 1996 1998 2002 2003
Quebec 55% 84% 86% 89%
Rest of Canada 65% 72% 74% 76%
Source: Authors’ calculations from NPHS and CCHS
Empirical Strategy
We first estimate a difference-in-difference for the effects of the policy on the
use of stimulants:
Where X includes family income, pmk immigrant, pmk sex, child sex, birth
order, family size, two parents, mother’s age at the birth, whether mom had a
teen birth, and child-age dummies. Includes Year dummies and Province
dummies.
Policy variable is lagged by one period to allow time for take up.
Some versions of the model include a child fixed effect, so that we are
comparing the same children over time.
Empirical Strategy
Next estimate a DDD model, focusing on those children most likely to be
affected by an increase in stimulant use, those with higher pre-policy ADHD
scores:
Prefer DDD as DD may reflect province specific time trends.
Empirical Strategy Cont.
• For longer term outcomes follow child from 1994 to 2008.
• Observe exposure to stimulants between ages 4 and 15.
• Measure whether child is ever on stimulants between 4 and 15.
• DDD term is the number of years in Quebec between ages 4
and 15 post 1996 * 1996ADHD score.
Sample 1 - Outcomes observed before age 16
Variable Quebec Rest of Canada
ADHD score in 1996 2.83 (2.42) 2.65 (2.31)
Stimulants, waves 1 and 2 0.016 0.014
Stimulants, waves 3 and up 0.049 0.023
Medium-term Outcomes
Unhappiness Score (6 pt) 0.47 (0.80) 0.53 (0.89)
Parent relationship 0.93 (0.26) 0.85 (0.36)
Standardized Math Score 10.02 (4.49) 8.27 (4.37)
Repeat a grade since last interview 0.07 0.02
Child and Family Characteristics
Child is male 0.51 0.51
Child is first born 0.55 0.50
Permanent Household income $58,958(33087) $64,518(36938)
Two-parent household 0.82 0.84
Family size 3.99 (0.93) 4.12 (0.93)
Mother age at birth 27.98 (4.78) 28.14 (5.13)
Mother high school grad 0.85 0.90
Mother is working 0.71 0.75
PMK is an immigrant 0.04 0.09
Number of children in sample 1 2,478 10,471
Number of obs. In sample 1 10,622 44,617
Sample 2 - Outcomes observed after age 16
Variable
Lives in Quebec
(cycle 1)
Rest of Canada
(cycle 1)
ADHD score in 1996 2.80(2.44) 2.54(2.32)
Ever Stimulants 0.09 0.05
# Surveys used Stimulants, given ever used 2.21(1.26) 2.06(1.26)
Post-1997 Years used Stim., given ever used 1.91(1.30) 1.56(1.22)
Long-term outcomes
Self-assessed depression score (36 pt) 5.84(4.73) 6.53(4.99)
High school grad 0.85 0.90
Some post-secondary 0.77 0.65
Child and Family Characteristics
Child is male 0.50 0.50
Child is first born 0.53 0.46
Permanent household income $58,711 (34,333) $64,669 (37,075)
Two-parent household in 1994 0.89 0.88
Family size in 1994 3.93 (0.89) 4.06 (0.90)
Mother age at birth 27.77 (4.59) 27.96 (5.09)
Mother high school grad in 1994 0.82 0.87
PMK is an immigrant 0.07 0.05
Number of children in sample 2 1,654 6,989
Standard errors of continuous variables in parentheses.
Table 3: Effect of Law Change on Stimulant Use
Outcome: Uses Stimulants
(1) (2) (3) (4)
DD - FE DD - No FE DDD - FE DDD - No FE
After 1997 -0.0072** -0.0092** -0.0172** -0.0133**
(0.0027) (0.0036) (0.0031) (0.0040)
Quebec -0.0118 0.0154** -0.0256 0.0101**
(0.0209) (0.0027) (0.0243) (0.0039)
After 1997 * Quebec 0.0247** 0.0287** 0.0123** 0.0159**
(0.0046) (0.0062) (0.0030) (0.0052)
1996 ADHD Score - - - 0.0105**
(0.0006)
After 1997*1996 ADHD Score - - 0.0039** 0.0014
(0.0008) (0.0010)
Quebec*1996 ADHD Score - - 0.0051 0.0004
(0.0083) (0.0009)
Aft. 1997*Que.*ADHD Sc. - - 0.0043** 0.0048**
(0.0021) (0.0018)
N 55,239 55,239 55,239 55,239
Age Range 2-15 2-15 2-15 2-15
Table 3: Effect of Law Changes on Stimulant Use, Con’t
Outcome: Ever Used Stimulants
(5) (6)
DD DDD
U16 Survey years after 1997 0.0003 0.0078**
(Elig Yrs) (0.0036) (0.0033)
Quebec in 1994 -0.0120 0.0181
(0.0121) (0.0253)
Elig Yrs * Quebec 0.0196** 0.0073
(0.0034) (0.0085)
1996 ADHD Score - 0.0403**
(0.0047)
Elig Yrs*1996 ADHD Score - -0.0038**
(0.0012)
Quebec*1996 ADHD Score - -0.0201**
(0.0098)
EligYrs*Que.94*96 ADHD Sc. - 0.0056*
(0.0032)
N 8,643 8,643
Table 4: Child Fixed Effects Estimates of Exposure to the Policy on
Contemporaneous Outcomes
Dependant Variable: Did Not Repeat Grade Math score Unhappiness Relationship w
Parents
(1) (2) (3) (4) (5) (6) (7) (8)
After 1997 0.0247** 0.0207** -0.3179** -0.2836** -0.2846** -0.1977** 0.0227** -0.0015
(0.0063) (0.0061) (0.1072) (0.1129) (0.0176) (0.0285) (0.0094) (0.0109)
Quebec 0.0578** 0.0225 0.2730 -0.1511 -0.0023 0.2054 0.0480 -0.0665
(0.0282) (0.0388) (0.2410) (0.2828) (0.1133) (0.1280) (0.1157) (0.1330)
After 1997 * Quebec -0.0581** -0.0228** -0.1883 -0.0694 0.1232** 0.0769 -0.0353** -0.0014
(0.0068) (0.0076) (0.1927) (0.1929) (0.0152) (0.0475) (0.0073) (0.0099)
After 1997*1996 ADHD
Score - 0.0016 - -0.0141** - -0.0326** - 0.0088**
(0.0009) (0.0056) (0.0076) (0.0021)
Quebec*1996 ADHD Score - 0.0117 - 0.1369** - -0.0894** - 0.0789**
(0.0141) (0.0557) (0.0366) (0.0348)
Aft. 1997*Que.*ADHD Sc. - -0.0128** - -0.0403** - 0.0172 - -0.0124**
(0.0016) (0.0062) (0.0145) (0.0033)
N 44,968 44,968 32,515 32,515 36,458 36,458 22,554 22,554
Age Range 4-15 4-15 5-15 5-15 2-11 2-11 4-9 4-9
Magnitudes
• Coefficient on DDD term can be compared to
baseline magnitudes:
• Change in not repeating grades = (-.0128/.93)
= 1.4% decline
• Change in math score = (-.0403/10) = -.004 decline
• Change in Unhappiness = (.0172/.47) = 4% increase
• Change in Relationship w Parents = (-.0124/.93)
= 1.3% decline
Table 5: Effects of Exposure to the Policy on Long-Term
Outcomes
Dependant Variable: Depression Score High School grad Some Post-sec
(1) (2) (3) (4) (5) (6)
U17 Survey years after 1996 0.3696** 0.4226** -0.0028 -0.0066 -0.0005 -0.0096
(Elig Yrs) (0.0793) (0.1149) (0.0047) (0.0061) (0.0096) (0.0122)
Quebec in 1994 0.6756 1.6972** -0.2191** -0.2430** -0.1265** -0.2068**
(0.3817) (0.4795) (0.0307) (0.0448) (0.0377) (0.0662)
Elig Yrs * Quebec -0.0876 -0.3073** 0.0498** 0.0675** 0.0666** 0.0868**
(0.0883) (0.1413) (0.0097) (0.0164) (0.0142) (0.0219)
1996 ADHD Score - 0.3414** - -0.0206** - -0.0447**
(0.0923) (0.0062) (0.0089)
Elig Yrs*1996 ADHD Score - -0.0288 - 0.0019 - 0.0046
(0.0274) (0.0018) (0.0026)
Quebec*1996 ADHD Score - -0.4340** - 0.0125 - 0.0382**
(0.1242) (0.0133) (0.0163)
EligYrs*Que.94*96 ADHD Sc. - 0.0867** - -0.0067 - -0.0084
(0.0402) (0.0043) (0.0048)
N 6,493 6,493 4,676 4,676 4,676 4,676
Magnitudes
Comparing DDD coefficient to baseline yields:
- Depression: (.0867/5.84) = 1.5% increase
- High School Graduation (-.0067/.85) = .8% decrease
- Some Post-Secondary (-.0084/.77) = 1.1% decrease
Table 6: Effects on Stimulant Use by Gender
Outcome: Uses Stimulants
(1) (2)
Boys Girls
After 1997 -0.0255** -0.0077**
(0.0052) (0.0022)
Quebec 0.0070 -0.0651
(0.0325) (0.0391)
After 1997 * Quebec 0.0299** -0.0031
(0.0058) (0.0042)
After 1997*1996 ADHD Sc. 0.0049** 0.0022**
(0.0012) (0.0009)
Quebec*1996 ADHD Score 0.0009 0.0154
(0.0117) (0.0105)
Aft. 1997*Que.*ADHD Sc. 0.0004 0.0078**
(0.0025) (0.0021)
N 27,971 27,268
Age Range 2-15 2-15
Table 6: Effects on Stimulant Use by Gender
Outcome: Ever Used Stimulants
(3) (4)
Boys Girls
U16 Survey years after 1997 0.0107** 0.0032
(Elig Yrs) (0.0052) (0.0038)
Quebec in 1994 -0.0499 0.0580**
(0.0513) (0.0182)
Elig Yrs * Quebec 0.0254 -0.0043
(0.0160) (0.0040)
1996 ADHD Score 0.0525** 0.0228**
(0.0061) (0.0071)
Elig Yrs*1996 ADHD Score -0.0050** -0.0018
(0.0015) (0.0016)
Quebec*1996 ADHD Score -0.0057 -0.0293**
(0.0178) (0.0092)
EligYrs*Que.94*96 ADHD Sc. 0.0021 0.0076**
(0.0056) (0.0021)
Sample Size 4,333 4,310
Age Range 0-9 in 1994 0-9 in 1994
Table 7: Child Fixed Effects Estimate of Exposure to Policy
on Contemporaneous Outcomes, by Gender
Dependant Variable: Unhappiness
Rel. w.
Parents
Did Not Rep.
Gr. Math Sc.
Boys (1) (2) (3) (4)
After 1997 * Quebec 0.1300* -0.0106 -0.0439** -0.1299
(0.0666) (0.0136) (0.0091) (0.1712)
Aft. 1997*Que.*ADHD Sc. -0.0047 -0.0120** -0.0142** -0.0257*
(0.0203) (0.0057) (0.0018) (0.0134)
N 18,484 11,457 22,719 16,191
Girls
After 1997 * Quebec 0.0258 0.0090 -0.0124 -0.0063
(0.0344) (0.0152) (0.0128) (0.2027)
Aft. 1997*Que.*ADHD Sc. 0.0430** -0.0127** -0.0058* -0.0588**
(0.0120) (0.0060) (0.0031) (0.0210)
N 19,974 11,097 22,249 16,324
Age Range 2-11 4-9 4-15 5-15
Table 8: Effects of Exposure to the Policy on Long-Term
Outcomes by Gender
Dependant Variable: Depression Score High School grad Some Post-sec
Boys (1) (2) (3) (4) (5) (6)
Elig Yrs * Quebec -0.0223 0.1049 0.0591** 0.0836** 0.0735** 0.0744**
(0.0841) (0.1932) (0.0171) (0.0242) (0.0187) (0.0271)
EligYrs*Que.94*96 ADHD Sc. - -0.0317 - -0.0091* - -0.0017
(0.0630) (0.0047) (0.0058)
N 3,213 3,280 2,259 2,259 2,259 2,259
Girls
Elig Yrs * Quebec -0.1428 -0.6068** 0.0439** 0.0549** 0.0619** 0.0942**
(0.1450) (0.2221) (0.0137) (0.0210) (0.0195) (0.0237)
EligYrs*Que.94*96 ADHD Sc. - 0.2025** - -0.0041 - -0.0150**
(0.0541) (0.0049) (0.0050)
N 3,280 3,280 2,417 2,417 2,417 2,417
Robustness checks:
• Look at ventilator use – maybe effects are due to some other
drug use?
• Exclude kids with chronic conditions.
• Exclude kids who may have been affected by the roll out of
universal preschool.
• Use a broader measure of psychological problems.
Placebo test
• Exclude Quebec.
• Imagine policy experiments in each province in turn, and in
each year from 1995 to 2005.
• Estimate the model for each of these “experiments.”
• Look at the distribution of estimates and where our estimate
lies in this distribution.
Why might stimulant use have negative effects?
• This natural experiment reflects “community care” which may
not be optimal care. E.g. among those who report using
stimulants, children use them for about 30% of the survey
years we observe them.
• Stigma or increased probability of placement in special
education?
• Medication may substitute for other types of therapy or
educational responses that would be more effective. E.g. if
children are less disruptive, them may be less likely to receive
needed attention. Ritalin improves focus, but does not help
children to decide what to focus on.
Summary
• Policy change in one Canadian province that
improved access to prescription drugs resulted in an
increase in the use of Ritalin.
• We examine the effects of this increase on behavioral
outcomes, child relationships, short-term educational
outcomes and some longer term educational
outcomes.
• We find little evidence of improvements in child
outcomes, and fairly consistent evidence of small
negative effects. Effects on emotional outcomes are
worse for girls.