Upload
emory-ball
View
233
Download
3
Tags:
Embed Size (px)
Citation preview
Statistical Models for the Analysis of Single-Case Intervention Data
Introduction to:
Regression Models
Multilevel Models
Why consider statistical models?
Can provide effect size estimates and confidence intervals for those estimates.
e.g., We are 95% confident the immediate shift in level for Jenny was an increase of between 11 and 14 minutes of time spent reading, or between 1.5 and 2.0 standard deviations
Regression
Imagine a scatter plot showing the relationship between motivation and achievement.
Motivation
Achievement
Regression allows us to summarize the relationship between the variables.
0
20
40
60
80
0 1 2 3 4 5 6
Motivation
Achievement
eXY 10
Often when we think of regression we think of each data point coming from a different individual, but all the observations could come from the same individual.
010203040506070
0 1 2 3 4 5
Time
Behavior
010203040506070
0 1 2 3 4 5
Time
Behavior
eTimeY 10
Rise
Run
0
1Run
Rise
What is the rate of change for Jody?
For single-case studies we expect a discontinuity
0
10
20
30
40
50
60
70
0 1 2 3 4 5 6
Baseline
Intervention
0
10
20
30
40
50
60
70
0 1 2 3 4 5 6
What is the shift in level for Jody?
Effect =
0Phase 1Phase
ePhaseY 10
10
What is the immediate shift in level and the shift in slope
for Jody?
0Phase 1Phase
eTimePhaseTimePhaseY *3210
1
0
0
10
20
30
40
50
60
70
80
90
100
Aslope2
-5 -4 -3 -2 -1 0 1 2 3 4 5
slope3
0
10
20
30
40
50
60
70
80
90
100
Issues to Keep in Mind
You may have to choose at what point in time you should calculate the effect size
0
10
20
30
40
50
60
70
80
90
100
You may want to standardize the effect size
If so, what SD should be use?
1
SDES 1
There needs to be a match between the trajectory specified in the model and what is seen in the data
Effect = b1?
ijijjjij ePhaseY 10
This seems incorrect
What is seen may require specification of a complex growth trajectory
Do you think specification tends to be easier when there are more or less observations in a phase?
Correct model specification requires more than just correctly specifying the growth trajectory
iii ePhaseY 10
Should you assume the errors (ei):are independent?have common variance? are normally distributed?
Cindy
Lucy
George
John
Imagine we have multiple cases
ijijjjij ePhaseY 10
0
10
20
30
40
50
60
70
0 1 2 3 4 5 6 0 1 2 3 4 5 60
102030405060708090
0 1 2 3 4 5 60
102030405060708090
0 1 2 3 4 5 60
102030405060708090
A separate regression could be obtained for each case
ePhaseYGeorge 10
ePhaseYLucy 10
ePhaseYCindy 10
ePhaseYJohn 10
Or a multilevel analysis could be run
Multilevel Model
Multilevel models allow us to answer additional questions:
• What is the average treatment effect?• Does the size of the effect vary across
participants?• What factors relate to effect size?
ijijjjij ePhaseY 10
jj r0000
jj r1101
What is the average effect for the participants?
Average Effect = γ10
ijijjjij ePhaseY 10
jj r0000
jj r1101
Average Baseline Level = γ00
Does the size of the effect vary across participants?
?)( 1 jrVAR
ijijjjij ePhaseY 10
jj r0000
jj r1101
What factors relate to effect size?
ADD
Non-ADD
ijijjjij ePhaseY 10
jjj rADD 001000
jjj rADD 111101
Issues to Keep in Mind
There still needs to be a match between the trajectory specified in the model and what is seen in the data
Effect = b1?
ijijjjij ePhaseY 10
This seems incorrect
Correct model specification requires assumptions about multiple error terms
ijijjjij ePhaseY 10
jj r0000
jj r1101
Should you assume the errors (eij, r0j, r1j) are independent? Normally distributed?
If one standardizes the effect size, what SD should be used for standardization
ijijjjij ePhaseY 10
jj r0000
jj r1101
Within case variance? Between case variance?
Imagine we have multiple studies
24
ijkijkjkjkijk eDY 10 ),0(~ 2eijk Ne
),0(~1
0
ujk
jkN
u
u
jkkjk u0000 jkkjk u1101
kk v0000000
kk v1010010 ),0(~10
00v
k
k Nv
v
Multilevel models were developed for large sample size conditions, but single-case applications tend to have a very small number of cases.
Given small sample sizes the variances (e.g. variance in the treatment effect across participants) will generally be more poorly estimated than the averages (e.g. the average treatment effect).
Example Analysis
Summarize results from 5 studies that examined the effect of intervention on autistic children’s speech
DV: Percent intervals with child speech
IV: Intervention based on increased parent verbalizations
Design: Multiple baseline across participants
Laski, K. E., Charlop, M. H., & Schreibman, L. (1988). Training parents to use the natural language paradigm to increase their autistic children’s speech. Journal of Applied Behavior Analysis, 21, 391-400.
One child from Laski et al.
28
iii eDY 10
21.58ˆ0
95.241̂
2-Level Model for Laski et al.
29
ijijjjij eDY 10
jj u1101 jj u0000
Parameter Estimated Estimate
SE p
Fixed Effects
Average Baseline Level (θ00)
34.17 8.16 .0030
Average Treatment Effect (θ10)
33.57 5.00 .0001
Variance Components
Variance in Baseline Level ( )
577.8 299.1 .0267
Variance in Treatment Effects ( )
185.4 110.5 .0468
Covariance u0 & u1 ( ) -294.1 110.5 .0838
Variance Within Person ( )
147.3 18.7 <.0001
2
0u
2
1u
01u2e
Software Code: SAS, R
30
SAS:proc mixed covtest;class Case;model Y= D / solution ddfm=sat;random intercept D / sub=Case type=un;
R:twolevel <- lmer(Y ~ D + (1 + D | Case), data2)summary(twolevel)
3-Level Model for All Five Studies
31
ijkijkjkjkijk eDY 10 ),0(~ 2eijk Ne
),0(~1
0
ujk
jkN
u
u
jkkjk u0000 jkkjk u1101
kk v0000000
kk v1010010 ),0(~10
00v
k
k Nv
v
3-Level Model Results
32
Parameter Estimated Estimate
SE p
Fixed Effects
Average Baseline Level (γ000) 18.71 6.31 .0345
Average Treatment Effect (γ100) 31.72 9.37 .0309
Variance Components
Between Study Variance in Baseline Level ( )
128.2 128.3 .1588
Between Study Variance in Treatment Effects ( )
377.2 320.7 .1198
Covariance v0 & v1 ( ) 165.5 169.8 .3296
Within Study Variance in Baseline Level ( )
316.9 103.6 .0011
Within Study Variance in Treatment Effects ( )
222.2 81.8 .0033
Covariance u0 & u1 ( ) -47.8 70.1 .4956
Variance Within Person ( ) 328.7 15.7 <.0001
2
0u
2
1u
01u2e
2
0v
2
1v
01v
Software Code: SAS, R
33
SAS:proc mixed covtest;class Study Case;model Y= D / solution ddfm=sat;random intercept D / sub=Study type=un;random intercept D / sub=Case(Study) type=un;
R:threelevel <- lmer(Y ~ D + (1 + D | Study:Case) + (1 + D | Study), data3)summary(threelevel)
Statistical models (regression and multilevel) provide a flexible approach for estimating treatment effects from single-case data, but care must be taken to ensure the model being used is consistent with the data being analyzed.
Conclusion
Applications and IllustrationsBaek, E., & Ferron, J. M. (2013). Multilevel models for multiple-baseline data: Modeling across
participant variation in autocorrelation and residual variance. Behavior Research Methods, 45, 65-74.
Baek, E. K., Moeyaert, M., Petit-Bois, M., Beretvas, S. N., Van den Noortgate, W., & Ferron, J. M. (2014). The use of multilevel analysis for integrating single-case experimental design results within a study and across studies. Neuropsychological Rehabilitation, 24, 590-606.
Ferron, J. M., Moeyaert, M., Van den Noortgate, W., & Beretvas, S. N. (in press). Estimating casual effects from multiple-baseline studies: Implications for design and analysis. Psychological Methods.
Moeyaert, M., Ferron, J., Beretvas, S. N., & Van den Noortgate, W. (2014). From a single-level analysis to a multilevel analysis of single-case experimental designs. Journal of School Psychology, 52, 191-211.
Moeyaert, M., Ugille, M., Ferron, J., Onghena, P., Heyvaert, M., Beretvas, S. N., & Van den Noortgate, W. (in press). Estimating intervention effects across different types of single-subject experimental designs: Empirical illustration. School Psychology Quarterly.
Rindskopf, D., & Ferron, J. (2014). Using multilevel models to analyze single-case design data. In T. R. Kratochwill & J. R. Levin (Eds.), Single-Case Intervention Research: Statistical and Methodological Advances (pp. 221-246). American Psychological Association.
Shadish, W.R., Kyse, E.N., & Rindskopf, D.M. (2013). Analyzing data from single-case designs using multilevel models: new applications and some agenda items for future research. Psychological Methods, 18, 385-405.
Van den Noortgate, W., & Onghena, P. (2003). Combining single-case experimental data using hierarchical linear models. School Psychology Quarterly, 18, 325-346.
Van den Noortgate, W., Onghena, P. (2007). The aggregation of single-case results using hierarchical linear models. The Behavior Analyst Today, 8(2), 196-209.
Van den Noortgate, W., & Onghena, P. (2008). A multilevel meta-analysis of single-subject experimental designs. Evidence-Based Communication Assessment and Intervention, 2, 142-151.