THE PERCENTAGE OF WORDS KNOWN IN A TEXT AND

READING COMPREHENSION 

Norbert SchmittUniversity of Nottingham

 Xiangying Jiang

West Virginia University 

William GrabeNorthern Arizona University

Reading Performance and Vocabulary Knowledge are Strongly Related

.50 - .75 (Laufer, 1992) .78 - .82 (Qian, 1999) .73 - .77 (Qian, 2002)

The Ability to Read Requires A Large Vocabulary

How Much?

Early Research

3,000 word families (Laufer, 1992) 5,000 individual words (Hirsh and Nation, 1992)

5,000 words Laufer (1989)

The Ability to Read Requires A Large Vocabulary

How Much?

Recent Research

8,000-9,000 word families (Hu and Nation 2000; Nation, 2006)

1st 1,000 word families average about 6 members (types per family)

9th 1,000 frequency level average 3 members

SO 8,000 word families = 34,660 individual word forms

Why Different Estimates?3,000 vs 9,000 Word Families

Different criteria of ‘adequate’ comprehension (Laufer – 55%)

Short textsSmall participant numbers (66)Old frequency counts (Dutch)Determination of unknown words

Vocabulary Coverage

Laufer (1989) found 95% coverage was point which best distinguished ‘comprehenders’ vs. ‘noncomprehenders’

95% 3,000 word families

Hu and Nation (2000) tested comprehension at various coverages 80% = No learners had adequate comprehension 90% = Only a few 95% = 35-41%

At 95% coverage, less than half of the students were successful, so required coverage is higher: 98-99%

98-99% 8,000-9,000 word families

Vocabulary Coverage

So the vocabulary coverage requirement is critical:

3,000 vs 9,000 word families

This study will directly explore the relationship between vocabulary coverage and reading comprehension

Vocabulary Coverage / Reading Comprehension

Figure 1 Linear Relationship

0102030405060708090

100

More W ords Known ---->

Vocabulary Coverage

Rea

din

g C

om

pre

hen

sio

n

Vocabulary Coverage / Reading Comprehension

Figure 2 Vocabulary Threshold

0102030405060708090

More W ords Known ---->

Vocabulary Coverage

Rea

din

g C

om

pre

hen

sio

n

Vocabulary Coverage / Reading Comprehension

Figure 3 S-curve

0

10

20

30

40

50

60

70

80

90

More W ords Known---->

Vocabulary Coverage

Rea

din

g C

om

pre

hen

sio

n

Features of Our Study

Longer texts (582 and 757 words) Extensive vocabulary test Extensive reading comprehension tests Controlled for background knowledge of texts 664 participants from different L1s

Selection of Reading Passages

CLIMATE MICE

Length 757 words 582 words

Content Climate change and global warming

A study on the relationship between exercise and mental acuity

Prior knowledge With much prior knowledge

With little prior knowledge

Difficulty Flesch-Kincaid Grade Level: 9.8

Flesch-Kincaid Grade Level: 9.7

Development of the Vocabulary Test

a checklist format (check the words they know)

120 target words sampled from the texts and 30 nonwords

deleted anybody with over 3 nonwords checked (≥2 nonwords same result)

high sampling rate for a good estimate of how much vocabulary each learner knew in each text

Development of the Reading Comprehension Tests

A two-part reading test for each passage 14 multiple-choice items 16 graphic organizer completion items

– graphic organizers were created to reflect the major discourse structures of the text

– fill in partially-completed graphic organizers

Participants

L1 # Levels #

Turkish 292 IEP 135

Chinese 180 Freshman 270

Arabic 101 Sophomore 143

Spanish 33 Junior 43

Hebrew 26 Senior 50

Other lgs 32 Graduate 23

Total 664 664

Procedure

100 minutes for the entire test battery– Biodata survey (5 mins.)– Vocabulary checklist (15 mins.)– Reading the Climate passage and answer

comprehension items (40 mins.)– Reading the Mice passage and answer

comprehension items (40 mins.)

Initial Screening

Eliminated participants who checked more than 3 nonwords

Eliminated participants who attempted less than 5 items in the graphic organizer task for either passage

Scoring Vocabulary percentage

– automatically calculated by entering checklist selections into an EXCEL spreadsheet

Multiple-choice reading comprehension test– 1 point for each correct answer, 0 for incorrect

ones Graphic organizer reading comprehension

test– 1 point for each acceptable answer and 0 for

unacceptable ones– Interrater reliability .99

Reliability Estimates of the Reading Test

.82 for the entire reading test .79 for the Climate reading test .65 for the Mice reading test .59 for the multiple-choice items .81 for the graphic organizer items

Note: based on KR-21, possible underestimation

Vocabulary Coverage vs Reading Comprehension

0

10

20

30

40

50

60

70

80

90

100

90%

91%

92%

93%

94%

95%

96%

97%

98%

99%

100%

Vocabulary Coverage

Co

mp

reh

ensi

on

Per

cen

tag

e

Mean

+1 SD

-1 SD

The Influence of Background Knowledge

0

10

20

30

40

50

60

70

80

90

90%

91%

92%

93%

94%

95%

96%

97%

98%

99%

100%

Vocabulary Coverage

Co

mp

reh

ensi

on

Per

cen

tag

e

CLIMATE

MICE

Graphic Organizer vs Multiple-Choice Tests

0

10

20

30

40

50

60

70

80

90

90% 91% 92% 93% 94% 95% 96% 97% 98% 99% 100%

Vocabulary Coverage

Co

mp

reh

ensi

on

Per

cen

tag

e

GO

MC

Conclusions The vocabulary coverage / comprehension

relationship is essentially linear between 90% - 100% coverage

So coverage requirements depends on comprehension goals

98% coverage is probably necessary, as 70% comprehension is desirable

But even 90% coverage leads to 50% comprehension

100% coverage only lead to 75% comprehension, so successful reading requires more than vocabulary, but high vocabulary levels are clearly a key requirement

Higher background knowledge lead to about 10 percentage-points better comprehension

There is a large amount of variation among learners