Upload
dale-cummings
View
214
Download
0
Tags:
Embed Size (px)
Citation preview
Larry CondelliStephanie Cronen
American Institutes for Research, USA
LESLLA Sixth Annual Symposium
Cologne, GermanyAugust 26, 2010
1
Overview of PresentationSummary of Study Purpose and
Design Impact of the InterventionContent of InstructionStudent AttendanceDiscussion and Implications of
Findings
2
Evaluation of an ESL Literacy Intervention: Sam and PatStructured language approach Adapted from Wilson Reading
SystemNever before evaluated for ESL Literacy activities organized
around basal reader/workbookHeavily phonics-based
3
Sam and Pat: Instructional Approach
Direct instruction, transparentRules explained, modeled, practiced
Controlled text, vocabulary and grammarWords match phonics already learned
Sequential Easy to hard in defined steps
ESL instruction to support literacy
4
Sam and Pat: Literacy and Language Skills CoveredPhonics for reading and writingSight wordsOral reading for accuracy and fluencyReading comprehensionVocabularySpeaking and listeningGrammar
5
Research QuestionsHow effective is instruction using the
intervention in improving the English reading, speaking and listening skills of low-literate adult ESL learners?
Is the intervention more effective for certain groups of students (e.g., language, literacy level)?
Do differences in level of implementation of Sam and Pat and other instruction relate to variation in impacts?
6
Study Design10 adult ESOL centers across USAPaired intervention and “normal” ESL
literacy classes (34 total)Random assignment of students and
teachers1,344 students participated for one term
Minimum 5 hours/week 10-12 weeks instruction with approach
Other instruction also providedEach class conducted twice over a year
7
Student Flow in the Study
1. Student applies to center2. Assessed for NLL3. Recruited into study4. Gives informed consent 5. Random assignment to class 6. Pretests administered7. Instruction8. Posttests administered
8
IntakeNL Literacy
Post-test
Random Assignment
Recruited into StudyInformed consent
Pre-test
Instruction
Students AssessmentsPhonics and decoding
Word attackLetter/Word ID
Reading comprehensionVocabulary (ROWPVT)Listening, oral expression
9
Sam And Pat Teachers in the StudyAll teachers randomly assigned3-day teacher training on curriculumFollow-up visits by trainersClassroom observations
to monitor fidelityRefresher webinar at
start of second term
10
Students in the Study:“True” Literacy (LESLLA) Little or no literacy in native
languageLimited oral EnglishEducation: 0-3 yearsLanguages:
Haitian-CreoleSpanishBurmese Others
11
Students in the Study: Non-Roman Alphabet LiterateSome Literacy in native language with
non-Roman scriptMean education: 6 and more yearsLimited oral English Languages:
ArmenianArabicFarsiChinese Others
12
Student Test Results
13
Main Impact Analyses
Outcome Treatment Control Difference Effect Size P-value
WJID 440.81 442.09 -1.28 -0.025 0.573
WJWA 466.70 465.76 0.94 0.024 0.595
SARA Dec 13.25 13.37 -0.12 -0.011 0.809
WJPC 432.83 433.52 -0.70 -0.038 0.295
OWLS 17.80 17.79 0.02 0.002 0.974
ROWPVT 28.53 29.55 -1.02 -0.061 0.111
WJPVT 431.65 431.24 0.41 0.020 0.663
Sample size: 1137 students14
No significant impacts on literacy or language outcomes for full sample (below) and subsamples
But all students gained on pre-post tests
Pre-post Differences, All StudentsOutcome Mean
PretestMean Posttest
Difference P-value
WJID 428.315 442.122 13.808 0.000*WJWA 457.400 466.503 9.103 0.000*WJPC 427.061 433.780 6.719 0.000*OWLS 14.239 18.075 3.836 0.000*ROWPVT 22.898 29.285 6.387 0.000*N=1,113Sam & Pat=567Control=546
15
Lack of Impact of Sam and Pat
The study took place in a challenging environment that makes having/finding an impact difficult, but that represents realityLow exposure and instructional time in adult
ESL class Short class duration Limited instructional time and irregular attendance
Training teachers Lack of specialized training in literacy Short training time available
16
Instruction: What is taught?
17
Classroom ObservationsLiteracy
development instruction:Pre-literacyPhonicsFluencyReading
strategies & comprehension
Writing
ESL Instruction:Oral language
developmentGrammar, etc.VocabularySocio-cultural
knowledgeFunctional
literacy18
Instructional Content by Interval (%)Content Sam and Pat
MeanControl Mean Difference P-value
Reading Development
65.5 19.3 46.3 0.00
ESL Acquisition
27.3 67.6 -40.2 0.00
Other Content 5.5 28.8 -23.3 0.00
Link to Outside
1.3 7.4 -6.1 0.071
Use of NL 20.7 43.9 -23.2 0.00
Sample SizeIntervalsObservations
98031
103433
19
Instruction in Literacy ClassesIn control (“normal”) classes, little
literacy taught- Why?Importance of oral language for daily lifeStudent expressed needs Lack of teaching on how to teach LESLLA students
More diversity of instruction in control classesMore NL use and bringing in outsideLess constrained by curriculum and study
needs
20
Attendance
21
Attendance and OutcomesOutcome Regression
CoefficientP=value
WJID 0.104 0.00WJWA 0.071 0.00WJPC 0.043 0.00OWLS 0.027 0.00ROWPVT 0.028 0.00WJPVT 0.056 0.00Mean attendance hours (p= n.s.)
Sam & Pat: 79.4 Control: 71.9
N=1,137 ,Sam & Pat=587, Control=557
22
Effect of Attendance on Test ScoresInstruction and outcomes correlatedInstruction appears to have an effect on
learningRelationship is weakLarger effect on reading outcomes
May mean literacy gains more sensitive to gain, may be testing artifact
23
Implications of Possible Findings for Practice
and Research for LESLLA Students
24
Implications for LESLLA Practice Sam & Pat is no more effective than other
types of ESL literacy instructionUnder what conditions, then, would Sam & Pat be a
good choice for instruction? Attendance relates to instruction, weakly
Is this what you expect? How could it be increased?Teachers of literacy classes teach little literacy
How can we improve this --through curriculum and teacher professional development?
What are some potential explanations for these results from your perspective?
25