View
224
Download
1
Category
Tags:
Preview:
Citation preview
THE PERCENTAGE OF WORDS KNOWN IN A TEXT AND
READING COMPREHENSION
Norbert SchmittUniversity of Nottingham
Xiangying Jiang
West Virginia University
William GrabeNorthern Arizona University
Reading Performance and Vocabulary Knowledge are Strongly Related
.50 - .75 (Laufer, 1992) .78 - .82 (Qian, 1999) .73 - .77 (Qian, 2002)
The Ability to Read Requires A Large Vocabulary
How Much?
Early Research
3,000 word families (Laufer, 1992) 5,000 individual words (Hirsh and Nation, 1992)
5,000 words Laufer (1989)
The Ability to Read Requires A Large Vocabulary
How Much?
Recent Research
8,000-9,000 word families (Hu and Nation 2000; Nation, 2006)
1st 1,000 word families average about 6 members (types per family)
9th 1,000 frequency level average 3 members
SO 8,000 word families = 34,660 individual word forms
Why Different Estimates?3,000 vs 9,000 Word Families
Different criteria of ‘adequate’ comprehension (Laufer – 55%)
Short textsSmall participant numbers (66)Old frequency counts (Dutch)Determination of unknown words
Vocabulary Coverage
Laufer (1989) found 95% coverage was point which best distinguished ‘comprehenders’ vs. ‘noncomprehenders’
95% 3,000 word families
Hu and Nation (2000) tested comprehension at various coverages 80% = No learners had adequate comprehension 90% = Only a few 95% = 35-41%
At 95% coverage, less than half of the students were successful, so required coverage is higher: 98-99%
98-99% 8,000-9,000 word families
Vocabulary Coverage
So the vocabulary coverage requirement is critical:
3,000 vs 9,000 word families
This study will directly explore the relationship between vocabulary coverage and reading comprehension
Vocabulary Coverage / Reading Comprehension
Figure 1 Linear Relationship
0102030405060708090
100
More W ords Known ---->
Vocabulary Coverage
Rea
din
g C
om
pre
hen
sio
n
Vocabulary Coverage / Reading Comprehension
Figure 2 Vocabulary Threshold
0102030405060708090
More W ords Known ---->
Vocabulary Coverage
Rea
din
g C
om
pre
hen
sio
n
Vocabulary Coverage / Reading Comprehension
Figure 3 S-curve
0
10
20
30
40
50
60
70
80
90
More W ords Known---->
Vocabulary Coverage
Rea
din
g C
om
pre
hen
sio
n
Features of Our Study
Longer texts (582 and 757 words) Extensive vocabulary test Extensive reading comprehension tests Controlled for background knowledge of texts 664 participants from different L1s
Selection of Reading Passages
CLIMATE MICE
Length 757 words 582 words
Content Climate change and global warming
A study on the relationship between exercise and mental acuity
Prior knowledge With much prior knowledge
With little prior knowledge
Difficulty Flesch-Kincaid Grade Level: 9.8
Flesch-Kincaid Grade Level: 9.7
Development of the Vocabulary Test
a checklist format (check the words they know)
120 target words sampled from the texts and 30 nonwords
deleted anybody with over 3 nonwords checked (≥2 nonwords same result)
high sampling rate for a good estimate of how much vocabulary each learner knew in each text
Development of the Reading Comprehension Tests
A two-part reading test for each passage 14 multiple-choice items 16 graphic organizer completion items
– graphic organizers were created to reflect the major discourse structures of the text
– fill in partially-completed graphic organizers
Participants
L1 # Levels #
Turkish 292 IEP 135
Chinese 180 Freshman 270
Arabic 101 Sophomore 143
Spanish 33 Junior 43
Hebrew 26 Senior 50
Other lgs 32 Graduate 23
Total 664 664
Procedure
100 minutes for the entire test battery– Biodata survey (5 mins.)– Vocabulary checklist (15 mins.)– Reading the Climate passage and answer
comprehension items (40 mins.)– Reading the Mice passage and answer
comprehension items (40 mins.)
Initial Screening
Eliminated participants who checked more than 3 nonwords
Eliminated participants who attempted less than 5 items in the graphic organizer task for either passage
Scoring Vocabulary percentage
– automatically calculated by entering checklist selections into an EXCEL spreadsheet
Multiple-choice reading comprehension test– 1 point for each correct answer, 0 for incorrect
ones Graphic organizer reading comprehension
test– 1 point for each acceptable answer and 0 for
unacceptable ones– Interrater reliability .99
Reliability Estimates of the Reading Test
.82 for the entire reading test .79 for the Climate reading test .65 for the Mice reading test .59 for the multiple-choice items .81 for the graphic organizer items
Note: based on KR-21, possible underestimation
Vocabulary Coverage vs Reading Comprehension
0
10
20
30
40
50
60
70
80
90
100
90%
91%
92%
93%
94%
95%
96%
97%
98%
99%
100%
Vocabulary Coverage
Co
mp
reh
ensi
on
Per
cen
tag
e
Mean
+1 SD
-1 SD
The Influence of Background Knowledge
0
10
20
30
40
50
60
70
80
90
90%
91%
92%
93%
94%
95%
96%
97%
98%
99%
100%
Vocabulary Coverage
Co
mp
reh
ensi
on
Per
cen
tag
e
CLIMATE
MICE
Graphic Organizer vs Multiple-Choice Tests
0
10
20
30
40
50
60
70
80
90
90% 91% 92% 93% 94% 95% 96% 97% 98% 99% 100%
Vocabulary Coverage
Co
mp
reh
ensi
on
Per
cen
tag
e
GO
MC
Conclusions The vocabulary coverage / comprehension
relationship is essentially linear between 90% - 100% coverage
So coverage requirements depends on comprehension goals
98% coverage is probably necessary, as 70% comprehension is desirable
But even 90% coverage leads to 50% comprehension
100% coverage only lead to 75% comprehension, so successful reading requires more than vocabulary, but high vocabulary levels are clearly a key requirement
Higher background knowledge lead to about 10 percentage-points better comprehension
There is a large amount of variation among learners
Recommended