Kristen L. McMaster University of Minnesota Douglas Fuchs, Lynn S. Fuchs, and Donald L. Compton

Responding to Nonresponders: An Responding to Nonresponders: An Experimental Field Trial of Experimental Field Trial of

Identification and Intervention Identification and Intervention MethodsMethods

Responding to Nonresponders: An Responding to Nonresponders: An Experimental Field Trial of Experimental Field Trial of

Identification and Intervention Identification and Intervention MethodsMethods

Kristen L. McMasterUniversity of Minnesota

Douglas Fuchs, Lynn S. Fuchs, and Donald L. ComptonPeabody College of Vanderbilt University

Responsiveness-to-Intervention Symposium

December 4-5, 2003 • Kansas City, Missouri

The National Research Center on Learning Disabilities, a collaborative project of staff at Vanderbilt University and the University of Kansas, sponsored this two-day symposium focusing on responsiveness-to-intervention (RTI) issues.

The symposium was made possible by the support of the U.S. Department of Education Office of Special Education Programs. Renee Bradley, Project Officer. Opinions expressed herein are those of the authors and do not necessarily represent the position of the U.S. Department of Education.

When citing materials presented during the symposium, please use the following: “McMaster, K. L., Fuchs, D., Fuchs, L. S., & Compton, D. L. (2003, December). Responding to nonresponders: An experimental field trial of identification and intervention methods . Paper presented at the National Research Center on Learning Disabilities Responsiveness-to-Intervention Symposium, Kansas City, MO.”

McMaster, K. L., Fuchs, D., Fuchs, L. S., & Compton, D. L. (2003, December). Responding to nonresponders: An experimental field trial of iMcMaster, K. L., Fuchs, D., Fuchs, L. S., & Compton, D. L. (2003, December). Responding to nonresponders: An experimental field trial of identification and intervention methods. Paper presented at the National Research Center on Learning Disabilities Responsiveness-to-Intervdentification and intervention methods. Paper presented at the National Research Center on Learning Disabilities Responsiveness-to-Intervention Symposium, Kansas City, MO.ention Symposium, Kansas City, MO.

22

Recognized FactsRecognized Facts• Over the past 20 years, we’ve learned that most Over the past 20 years, we’ve learned that most

poor readers have phonological processing poor readers have phonological processing problems and poor word recognition skills.problems and poor word recognition skills.

• Programs emphasizing phonological awareness Programs emphasizing phonological awareness and decoding can greatly improve many young and decoding can greatly improve many young children’s reading performance.children’s reading performance.


33

Yet, A Persistent Problem Yet, A Persistent Problem

• 20 – 30% of children at risk20 – 30% of children at risk

• 50% or more among children with 50% or more among children with high-incidence disabilitieshigh-incidence disabilities

A sizable minority of students remain unresponsive to generally effective reading programs


44

Two General Approaches to Two General Approaches to Unresponsive StudentsUnresponsive Students

• Special-education-like approach: Students temporarily Special-education-like approach: Students temporarily removed from the classroom to receive focused removed from the classroom to receive focused supplemental reading instruction from well-trained supplemental reading instruction from well-trained teachers.teachers.

• General-education approach: Begins with classroom General-education approach: Begins with classroom teacher adapting instruction, curricula, and materials. teacher adapting instruction, curricula, and materials. Important because many students’ reading problems are Important because many students’ reading problems are due to poor classroom instruction and because current due to poor classroom instruction and because current reforms emphasize the classroom as a first step in reforms emphasize the classroom as a first step in addressing students’ academic problems.addressing students’ academic problems.


55

Purpose Purpose

• Individualized adaptations of Individualized adaptations of classroom instruction.classroom instruction.

• More intensive one-to-one tutoring.More intensive one-to-one tutoring.

Experimental study of two alternative interventions for students unresponsive to generally effective

classroom instruction:


66

ParticipantsParticipants

8 Metro-Nashville Public Schools: 8 Metro-Nashville Public Schools:

4 high poverty Title I; 4 middle class4 high poverty Title I; 4 middle class 33 first-grade teacher volunteers, stratified by Title I 33 first-grade teacher volunteers, stratified by Title I

vs. middle class schools, randomly assigned within vs. middle class schools, randomly assigned within school to one of 3 conditions: Standard PALS (school to one of 3 conditions: Standard PALS (nn = 11), = 11), PALS + Fluency (PALS + Fluency (n n = 11), and no-treatment controls.= 11), and no-treatment controls.

No-treatment controls did not participate in present No-treatment controls did not participate in present study and Standard PALS and PALS + Fluency classes study and Standard PALS and PALS + Fluency classes combined to create 22 “PALS” study classes.combined to create 22 “PALS” study classes.


77

Identifying Nonresponders Identifying Nonresponders

• 8 students in each of 22 PALS classes chosen as 8 students in each of 22 PALS classes chosen as “lowest-achieving” based on RLN scores and teacher “lowest-achieving” based on RLN scores and teacher judgment (judgment (NN = 176; 22 classes x 8 students). = 176; 22 classes x 8 students).

Risk Pool

Nonresponders• Monitoring of risk pool for first 7 weeks of PALS on:Monitoring of risk pool for first 7 weeks of PALS on:

─ PALS unit tests (percentage of items correct)PALS unit tests (percentage of items correct)

─ Dolch Word List (number of words read correctly in 1 min.)Dolch Word List (number of words read correctly in 1 min.)

─ Nonword Fluency Test (number of phonemes read correctly in Nonword Fluency Test (number of phonemes read correctly in 1 min.)1 min.)


88

Identifying NonrespondersIdentifying Nonresponders(Cont’d) (Cont’d)

• ZZ-scores computed for risk pool’s -scores computed for risk pool’s level of performancelevel of performance and and rate of growthrate of growth on both Dolch on both Dolch and NWF using 88 average achievers’ performance (4 average achievers x 22 classrooms).and NWF using 88 average achievers’ performance (4 average achievers x 22 classrooms).

• Students identified as nonresponders who scored (a) < 90% on the last PALS unit test, and Students identified as nonresponders who scored (a) < 90% on the last PALS unit test, and (b) scored at least .5 (b) scored at least .5 SDsSDs below average achievers’ on Dolch Word List and NWF test. below average achievers’ on Dolch Word List and NWF test.

• 66 nonresponders identified; or 38% of risk pool and 14% of general population (66 nonresponders identified; or 38% of risk pool and 14% of general population (NN = 484; = 484; 22 classes x 22 students per class)22 classes x 22 students per class)


99

Treatment Levels Treatment Levels • Students continue with PALS program in their classrooms, Students continue with PALS program in their classrooms,

35 min. per day, 3 x per wk.35 min. per day, 3 x per wk.

Control PALS

Adapted PALS• Students continue working with a partner in their Students continue working with a partner in their

classrooms during PALSclassrooms during PALS• ““Coach” is trained to provide more modeling and feedbackCoach” is trained to provide more modeling and feedback• Lessons introduce sounds and words at a slower paceLessons introduce sounds and words at a slower pace• 35 min. per day, 3 x per wk.35 min. per day, 3 x per wk.

• Students work with a trained adult tutor outside the Students work with a trained adult tutor outside the classroom during PALSclassroom during PALS

• Students are trained to mastery on sounds and wordsStudents are trained to mastery on sounds and words• 35 min. per day, 3 x per week35 min. per day, 3 x per week

Treatment Duration = 12 weeksTreatment Duration = 12 weeks

Tutoring


1010

Fidelity of TreatmentFidelity of Treatment

• Checklist of teacher and student behaviorChecklist of teacher and student behavior• Fidelity checked in December and March in each of 22 Fidelity checked in December and March in each of 22

classesclasses• Mean fidelity across teachers and students = 92%Mean fidelity across teachers and students = 92%

Control PALS

Adapted PALS• All students and their partners observed once in AprilAll students and their partners observed once in April• Mean fidelity = 86%; range = 49% – 100%Mean fidelity = 86%; range = 49% – 100%

• Each of 8 tutors participated in simulated tutoring sessionsEach of 8 tutors participated in simulated tutoring sessions• Mean fidelity = 97%Mean fidelity = 97%• One tutoring session per student audiotapedOne tutoring session per student audiotaped• Mean fidelity = 96%Mean fidelity = 96%

Tutoring


1111

Findings Findings

• Were the interventions insufficiently long or intense?Were the interventions insufficiently long or intense?

• Was the nature of the interventions too similar?Was the nature of the interventions too similar?

• Was our statistical power too low?Was our statistical power too low?

No statistically significant differences in reading among students in PALS, Adapted

PALS, and Tutoring


1212

Effect Sizes (ESs) Effect Sizes (ESs) • ESs indicated greatest growth among Tutored students on:ESs indicated greatest growth among Tutored students on:

─ Word attackWord attack─ ComprehensionComprehension

• Greatest responsiveness among Tutored students on growth criteria.Greatest responsiveness among Tutored students on growth criteria.• When considering prior evidence of tutoring effectiveness, our results can be When considering prior evidence of tutoring effectiveness, our results can be

interpreted as indicating the superiority of tutoring over classroom adaptations. (The interpreted as indicating the superiority of tutoring over classroom adaptations. (The joint joint pp-value of our findings + others’ findings is low.)-value of our findings + others’ findings is low.)


1313

Effect Sizes by Secondary Effect Sizes by Secondary Intervention Option Intervention Option

MeasureMeasure

Tutoring Tutoring

vs.vs.

Control PALSControl PALS

Adapted PALSAdapted PALS

vs. vs.

Control PALSControl PALS

Adapted PALSAdapted PALS

vs.vs.

TutoringTutoring

Word IDWord ID .43.43 .44.44 -.01-.01

Word AttackWord Attack .38.38 .05.05 .33.33

FluencyFluency .22.22 .18.18 .03.03

ComprehensionComprehension .32.32 -.02-.02 .34.34


1414

Proportion Nonresponders by Secondary Proportion Nonresponders by Secondary Intervention Option and Criterion Intervention Option and Criterion

Intervention Intervention OptionOption

Dual Dual DiscrepancyDiscrepancy

PercentilePercentileRankRank BenchmarkBenchmark GrowthGrowth

(< .5 SD on Dolch + (< .5 SD on Dolch + NWF slope and NWF slope and

level)level)

(< 30(< 30thth on on WID and/or WID and/or

WAT)WAT)

(< 40 on (< 40 on Passage Passage Fluency)Fluency)

(< 10 gain on (< 10 gain on WID and/or on WID and/or on

WAT) WAT)

Control Control PALSPALS ( (nn = 21) = 21) 8181 3838 100100 4848

Adapted Adapted PALSPALS ( (nn = 15) = 15) 8080 5353 100100 4747

Tutoring Tutoring

((nn = 20) = 20) 5050 4545 100100 2525

Prevalence Prevalence RateRate 1212 88 1717 77

Final Status


1515

Adapted PALSAdapted PALS• Wide range of fidelity (given teachers’ inadequate monitoring).Wide range of fidelity (given teachers’ inadequate monitoring).

• For students demonstrating poor responsiveness to generally For students demonstrating poor responsiveness to generally effective classroom instruction, adaptations of that instruction effective classroom instruction, adaptations of that instruction may be inadequate.may be inadequate.

• However, don’t forget the ES for Word ID (.44). Classroom However, don’t forget the ES for Word ID (.44). Classroom adaptations may adequately address some dimensions of reading adaptations may adequately address some dimensions of reading development more than others. Repeated exposure to words, development more than others. Repeated exposure to words, supervised by capable peers, may benefit Word ID development. supervised by capable peers, may benefit Word ID development. By contrast, Word Attack may require trained adult tutors.By contrast, Word Attack may require trained adult tutors.


1616

Criteria for Identifying Criteria for Identifying NonrespondersNonresponders

• Final status percentile rank and growth Final status percentile rank and growth produced similar rates of LD as traditional produced similar rates of LD as traditional methods (7 – 8%).methods (7 – 8%).

• Final status benchmark appears too stringent. Final status benchmark appears too stringent. Students grow well without achieving Students grow well without achieving benchmark; prevalence rate = 17%.benchmark; prevalence rate = 17%.

Documents

Kristen L. McMaster University of Minnesota Douglas Fuchs, Lynn S. Fuchs, and Donald L. Compton