Test design of reading comprehension in college English

A Probe into Test Design of Reading Comprehension in College English

Yingfei Qu

Foreign language School, Huaqiao University, Xiamen Campus, No. 668, Jimei Avenue, Xiamen, Fujian Province, 361021, China

[email protected], [email protected]

Keywords: Test design; Reading comprehension; College English

Abstract. Evaluation is one of the indispensable steps in foreign language teaching. Traditionally, people in China focused too much on the scores and ignored to a large extent the gradual

development. Reading is an important part in college English in China and should be evaluated

scientifically. This paper, based on the theories of language testing, discussed the testing range of

reading in college English.

Introduction

In most countries around the world, educational testing is increasingly administrated on a large

scale. In heavily populated countries a single test may be administrated to millions of students in a single year, this is especially true for China. Generally, a large-scale testing serves three major

functions: 1) selecting and placing students for further educational and career opportunities; 2)

certifying that students posses requisite knowledge and skills, and 3) monitoring the educations of

educational system to insure that it is functioning effectively. In order to construct tests that carry

out system-oriented functions with large numbers of students, test makers have traditionally been

forced to rely on multiple-choice questions. Defenders of multiple-choice tests often point to a

further advantage such as money saving, objective and easy manipulation. Within such a framework,

the users of the test only focus on the final stage of scoring and thus ignore the fact that language

learning achievement is built in a step by step manner. In some cases, the competence even can not

be reflected in the so called objective testing. Reading, as one of the most important part in college

English in China, has always been a hot topic in testing. This paper probes into the merits and demerits of traditional English testing in China, and provides some suggestions in the range and

scope of reading in college English testing.

Relative Theories about Language Assessment

Basic Testing Theories. In resent years, researchers on educational testing and assessment have

become increasingly aware that multiple-choice questions can set up a misleading opposition

between different ways of reading a text. The test is given at two levels: first level, a basic

qualification, and executive level, and advanced qualification.[1] when test maker design questions, they should pay attention on it. Although not much information is available in the specimen material

on the scope of the investigation into the target language use situation, it would appear that some

consultations with experienced business professional were carried out. In fact, the lack of

information about precisely what the criteria for correctness are is a rather serious deficiency in the

specimen material.

In specific purpose language testing, we must be careful to ensure that the intended discourse

domain, that of the target language use situation, is well signaled in the test[1]. Researchers use the

term discourse domain to refer to the internal interpretation of context. They define discourse

domain as a cognitive construct created by a language learner as a context for interlanguage

development and use. There is considerable research in language testing that demonstrates the effects of test method on test performance. The characteristics of the tasks used are always likely to

affect test scores to some degree, so that there is virtually no test that yields only information about

the ability we want to measure.

6th International Conference on Management, Education, Information and Control (MEICI 2016)

© 2016. The authors – Published by Atlantis Press


© 2016. The authors – Published by Atlantis Press 1240

Pragmatic Knowledge. Pragmatic knowledge enables us to create or interpret discourse by

relating utterances or sentences and texts to their meanings, to the intentions or language users, and

to relevant characteristics of language use setting. According to Dai and He [2], there are two areas

of pragmatic knowledge: Pragmalinguistics and Sociopragmatics. Pragamalinguistics is applied to

the more linguistic end of pragmatics, i.e. how the linguistic forms are used to serve specific pragmatic purpose. Sociopragmatics is the sociological interface of pragmatics; it is concerned with

the customary ways in which people of a particular culture behave to achieve a particular purpose.

Functional Knowledge. Functional knowledge is what Bachman calls “illocutionary

competence” [3]. It enables us to interpret relationship between utterances of or sentences and texts

and the intentions of language users. Quite frequently the appropriate interpretation of a given

utterance also involves the language users‟ prior knowledge of the language setting, including the

characteristics of the participants, their background knowledge of the world, etc. Meanwhile, to

understand the so called „illocutionary” act, one must be able to analyze meaning in the language, or

in other words, becomes more analytical in dealing with the indirect meaning. This, of course is the

abilities test designer are interested in during their preparation of the test.

The Nature of Reading Comprehension in College English

Researchers in native language and foreign language teaching have contributed much to the

understanding of reading comprehension. Reading is to get information, but it is not a unilateral

process. As many researchers argued, reading is in reality a bilateral or interactive process in which

there could be certain “talking” between the readers and the author. In other words, while reading,

the reader is trying to find what the author wanted to express. As it is, reading comprehension must

care if the material is authentic or not, that is, if there is real interaction between the reader and the author [4]. On the other hand, considering the process of language acquisition, reading is in fact a

channel to provide students input which aims to promote students‟ learning. One important of the interactive feature of reading is its “schemata”. That is, in reading, readers

tend to use the existing schemata existing in their mind, which is useful in reading. Meanwhile,

reading ability lies in the understanding of the contents. This ability cannot be observed directly as

it is per se the mental manipulation. However, it can be observed indirectly through the examination

of some items by testing. For college English, students are supposed to master more than 5000

thousand words and be able to read passages of about 500 or more words. All this must be taken

consideration in the testing system.

The Issues of Testing Reading Comprehension in College English

Relative Matters about the Design. In spite of the wide range of reading material specially written

or adapted for English language learning purposes, there are few comprehensive systematic

programmers which have been constructed from a detailed analysis of the skills required for

efficient reading. Much test material is still limited to short reading extracts on which general

„comprehension‟ questions are based. Characteristics of the test rubric include the objective,

procedures for responding, structure of the test/ task, time allotment [1].

The term rubric is a somewhat obscure one, originally used in Christian prayer books to refer to

procedural information about the form of worship, separate from the religious content. In testing, the term has been defined by Bachman as characteristics „that specify how test takers are expected

to proceed in taking the test, and include the instructions, time allocation, and test organization. [1]

The characteristic of the rubric relate to the nature of the communicative event, whether in the target

language situation or the language test, including its objective, procedures for responding, structure,

format, the time available for completing it, and evaluation criteria and procedures.

According to what have been discussed, it is clear that multiple-choice testing of reading

comprehension cannot satisfy the requirement of measuring the abilities of reading comprehension

of Chinese college students, the author present some test design to improve this problem on the

basis of some relative assessment theories. The following four items about testing reading





comprehension are proposed.

Completion Items. Completion items measure recall rather than recognition. Although such

items are similar in many ways to open-ended questions in tests of reading comprehension, they

often regarded as belonging more to the objective category of test items. Usually completion items

require the testees to supply a word or a short phrase. Unless great care is taken to ensure that there is only one correct answer, the making will prove very difficult when the tester is confronted with a

variety of answers ranging from acceptable. All valid interpretations, whether or not these were in

the test writer‟s mind at time of the construction of the test, must be regarded as correct. In

designing reading comprehension, the item is like “The author was surprised to meet Dr Steven

because ____” rather than “Why was the author surprised to meet Dr Steven?”This kind of testing

is often used to relatively detailed illustration which call forth students recall rather than recognition

[5]. This is especially useful in classroom evaluation as well as in formal testing.

Rearrangement Items. These two item types are particularly useful for testing the ability to

understand a sequence of steps of steps in a process or events in a narrative. While in an exercise for

classroom practice the students will often be required to rewrite the jumbled sentences in their correct sequence, it is obviously preferable for testing purposes to instruct them to write simply the

numbers or letters of the jumbled sentences.

Type 1: the first of these items takes the form of jumbled sentences which the students have

to “unscramble” and arrange in the correct order.

Type 2: the jumbled sentences in the second item are based on a reading comprehension text

and should be unscrambled in the light of the information contained in the text. The students

must write the letters of the sentences again in their correct order in the boxes at the end of

the item.

Open-ended and Miscellaneous Items. The term “open-ended” is used to refer to those

questions which elicit a completely subjective response on the part of the testees. The response

required may range from a one-word answer to one or two sentences. When making open-ended items which require answers in sentences, it is frequently advisable to award at least two or three

marks for each correct answer. When constructing a reading comprehension test, the test writer

should remember to let the text itself determine the types of items set. Indeed, if this principle is

observed, it follows that different parts of a particular text will frequently require different item

types .It is now becoming common practice for more than one type of item to be used to test

comprehension of the same text.

Short-answer Questions. First of all, think-aloud protocols will be used to elicit the response of

college English students to perspective test material. Second, this reader response will be used to: a)

construct different ways of reading the passage and b) develop SAQs that are appropriate for

Chinese university student.

Third, the SAQs will be pilot tested to determine not only psychometric properties of reliability

and validity but also the degree to which they elicit: a) correct responses that are stable in form and

b) in correct response that can be lexically differentiated from the correct ones (these two conditions

must be met if artificial intelligence techniques are to be used in scoring response to SAQs).

Fourth, on the basis of the pilot testing, the SAQs will be revised and administrated on a large

scale. Finally, a computer-based program will be developed to score selected samples of the revised

SAQs (the goal here is simply to demonstrate that such programme can be developed for scoring

responses to SAQs).

Conclusion

Reading comprehension is an indispensable part in college English testing system. For years in

China, the tests of various types adopted merely multiple choices due to the influence of

psychometric theories of the western world. Although it is admitted that this type of items are

feasible in manipulation and used for years, its disadvantages are found by many researchers. For

instance, it cannot reflect the real abilities of testees. Therefore, in college English teaching

framework, a useful and multi-dimensional testing system must be adopted so that students‟ abilities





like skimming, scanning, understanding of the details, memory, recognition, etc can be measured

objectively. This paper is theoretically based, and it is hoped that in the future, more and more

teachers, researchers will be involved for the reform and practice of testing students‟ reading

comprehension in college English.

References

[1] D., Douglas, Assessing language for specific purposes. Cambridge: Cambridge University

Press, 2000.

[2] W. D., Dai, & Z. X., He. A New Concise Course in Linguistics for Students in English.

Shanghai: Shanghai Foreign Language Education Press, 2010.

[3] L., Bachman. Fundamental Consideration in Language Testing. Oxford: Oxford University Press, 1998.

[4] F. F., Zhao. An Analysis of the Authenticity of Reading Comprehension in CET-4. Journal of

Hubei Correspondence University, Vol.28, 2015(23), Pp168-169.

[5] J. B., Heaton. Writing English Language Tests. Beijing: foreign Language Teaching and

Research Press, 2000.





Documents

Test design of reading comprehension in college English