Language Testing

http://ltj.sagepub.com

Language Testing

DOI: 10.1177/026553229401100303 1994; 11; 267 Language Testing

Suyi Xiao and John W. Oller, JR achieved?

Can relatively perfect translation between English and Chinese be

http://ltj.sagepub.com/cgi/content/abstract/11/3/267 The online version of this article can be found at:

Published by:

http://www.sagepublications.com

can be found at:Language Testing Additional services and information for

http://ltj.sagepub.com/cgi/alerts Email Alerts:

http://ltj.sagepub.com/subscriptions Subscriptions:

http://www.sagepub.com/journalsReprints.navReprints:

http://www.sagepub.co.uk/journalsPermissions.navPermissions:

http://ltj.sagepub.com/cgi/content/refs/11/3/267 Citations

by Bida Cahyono on August 23, 2009 http://ltj.sagepub.comDownloaded from

http://ltj.sagepub.com/cgi/alerts

http://ltj.sagepub.com/subscriptions

http://www.sagepub.com/journalsReprints.nav

http://www.sagepub.co.uk/journalsPermissions.nav

http://ltj.sagepub.com/cgi/content/refs/11/3/267


Can relatively perfect translationbetween English and Chinese beachieved?Suyi Xiao and John W. Oller jr University of New Mexico

It may be hypothesized that translation is always possible among diverselanguages (Lakoff, 1987) or that it is never possible (Givón, 1989). In order toprovide empirical evidence, 250 speakers of English and Chinese were testedin randomly selected blocks of 25 subjects each on one version of an originaland another of a translated text in their respective native languages. A fixed-ratio cloze procedure was used deleting every fifth word from two distinctversions over each text so that every word of both texts was clozed exactlyonce in one of the five versions. English speaking subjects (n = 125)completed two cloze tests in English: one over the text originally written inEnglish and another over the text translated from Chinese. Chinese-speakingsubjects (n = 125) also completed two tests: one over the text originallywritten in Chinese, and the other over the translation from English.MANOVA comparisons between originals and translations irrespective ofgroup showed no significant contrast. There was, however, a small butsignificant (p < .05) contrast between groups, but none between the passagesaveraging over the treatment and group. Results support the universalitytheory (along Peircean lines) and the conclusion that a high degree oftranslation equivalence is achievable across radically distinct languages andcultural settings.

I Introduction

C.S. Peirce proved that symbolic representations must be translat-able. More recently, the translatability of diverse languages has beenargued in the affirmative by Lakoff (1987) but Giv6n (1989) hasreached a negative conclusion. Giv6n contends that translation

requires an inevitable intrusion into the culture into which thetranslation is made and cannot be accomplished very perfectlybecause of cultural differences. However, Chomsky’s theory ofUniversal Grammar (1982; 1988) and the more thorough-goingtheory of C.S. Peirce, especially in his ’logic of relatives’ (cf. Peirce,1897, in Hartshorne and Weiss, 1933; Oller, 1989; Oller, Scott andChesarek, 1991; Oller and Kennedy, in press), enables a purelytheoretical proof that nearly perfect translation is always possible inprinciple. However, it is bound to be more difficult in some



268

particular cases than others, as is demonstrated by Giv6n (1989; alsosee Xiao, 1992).

Underlying the controversy over whether the universality hypoth-esis is correct or not (and to what degree it may be false, if at all) isthe fact that languages differ radically in their surface forms. In fact,they are so different that some theoreticians insist it may even beimpossible to say in one language what has been said originally inanother. On the other side of the controversy is the fact that humanexperience depends on three common elements that cannot easilybe refuted or denied. First, we have a common genetic endowment -a special bioprogram for language that is shared by all normalhuman beings but not by other species (Chomsky, 1982; 1988;Bickerton, 1981; 1990). Secondly, this genetic endowment also givesrise to a unique physiological expression, common to human beings,that partly determines the character of our peculiarly humanexperience (cf. von Uexkfll, 1934 [1992]). Thirdly, our humanexperience relates to (and is partly determined by) an undeniableexternal reality in the apparent space-time continuum which we, as aconsequence, are equipped to know and understand far more

perfectly than other species. This last element cannot be lightlydismissed (Einstein, 1936; 1941; 1944; Peirce, 1878; 1903; Oller, Salesand Harrington, 1969). All of the foregoing, then, gives a wellgrounded logical basis for communication and translation betweendisparate languages and cultures (Oller and Kennedy, in press).

Also, it is important to realize that the theory of universaltranslatability across languages by no means minimizes (in fact, itrefutes any denial of) the striking and fundamental differences thatexist across languages, cultures and societies. Indeed, the universal-ity theory (as construed in Peircean terms) not only takes account ofthe profound differences that exist across languages and cultures butalso, at the same time, demonstrably provides the only logical basis(as argued by Oller and Kennedy, in press) for the recognition ofany factual differences whatsoever, whether across languages andcultures or within the same language and culture. This last fact isproved rigorously by Peirce in his ’logic of relatives’ and has beenextended to linguistic theory by Oller and Kennedy (in press). Theupshot of the latter argument for the theory of translation is that, ifwe deny the common reality shared by distinct cultures, languagesand so forth (however imperfectly that common reality may beunderstood), there can be no logical basis of any kind for definingany cultural differences at all (cf. also Peirce, 1878; Dewey, 1916;Einstein, 1944). Although the differences across languages andcultures may, at least at the surface, seem to be more salient thanany underlying similarities, the universality theory (again, moving



269

along Peircean lines) suggests that human beings really do possess a’common conceptualizing capacity’ (as argued more recently byLakoff, 1987) whereby the differences across languages and culturescan be relatively perfectly expressed in fully translatable symbols.As a result, the ’gaps’ between distinct cultures can be bridged.Communication is more than just barely possible (Peirce, in Hart-shorne and Weiss, 1931-35; Burks, 1957-58; Nida, 1964; Katz, 1978;Chomsky, 1976; 1982; Fisch et al.,1982; Moore et al., 1984; Kloesel etal., 1986; Lakoff, 1987; Given,1989) but, as Peircean semiotic theoryshows, it can be indefinitely improved towards the limit of theperfection of our understanding relative to any particular commonground of experience (cf. Peirce, 1897; Oller and Kennedy, in

press).The empirical literature on translation is sparse, but studies by

Brislin (1970), Oller et al. (1972) and Malakoff and Hakuta (1991),though conducted in various paradigms with different purposes, allprovide empirical evidence supporting the universality theory.These studies have involved languages as different as Chamorro,Palauan, English, Spanish, Thai and Vietnamese. The results indi-cate that reasonably equivalent translations can be achieved acrossextraordinarily different languages and cultures.

This study intends to examine the universality hypothesis empiri-cally by testing groups of native speakers of English and Chinese ontwo distinct narrative passages. One of the narratives was written

originally in English and the other in Chinese. Both were cross-translated to produce four narrative texts (two in English and two inChinese). Under the universality theory, it is predicted that thesetranslations (to the extent that they are carefully done) should benegligibly distinct in their measured impact on interpreters. In

theory, the translation itself assures some equivalence of the under-lying content of the crosstranslated narratives, but we added anempirical measure to enable a more delicate comparison of diverseproducts of translation across languages. Our procedure is to clozeevery word in the English and Chinese versions of two texts. Basedon our theory (and the design to be elaborated below), ourexperimental hypotheses are the following:

HI: Mean scores on the translated texts and the originals (averag-ing across languages and passages) will not differ

significantly.H2: Mean scores on the two passages (averaging across languages

and the translation treatment) will not differ significantly.H3: Mean scores of the Chinese-speaking and English-speaking

subjects (averaging across texts) will not differ significantly. --



270

H4 : The mean scores on the five versions of the translations andthe five versions of the originals (averaging over the two texts)will not differ significantly from each other.

Clearly, H, is critical and, by nature of the design (see Figure 2, isindependent of either H2 or H3. However, one of our reviewers, towhom we are indebted (mainly for pointing out some difficulties inthe translations done here of which Oller was previously unaware),saw an affirmative outcome relative to our second hypothesis as anecessary condition for the experiment. That is, if the texts were notreasonably equivalent to start with, the reviewer thought the designwould fail to enable any test of the quality of translations. There-fore, the reviewer wondered why we had not conducted a pilot studyto be sure of the equivalence of the difficulties of the originalpassages. (Though the reviewer did not say so, the same sort ofargument would apply, if he or she were correct, to the Chinese-speaking and English-speaking groups selected for study. That is,they too, by his or her reasoning, ought to have been quiteequivalent.) The reviewer, at any rate, saw an affirmation of H2 as acondition for H3. But, in fact, none of the hypotheses are codepen-dent in that way. The only logical dependency among the hypoth-eses is that, if Ht proved false, then H4 would also have to be false.Otherwise, any single hypothesis could be affirmed or denied by theempirical outcomes independently of the rest, as is shown in thediscussion of Figures 1 and 2 where we will return briefly to thereviewer’s concern. In the meantime, it is sufficient to note that ourinterest in H2 was actually motivated independently of our interestin H,. H2 has no special bearing on the universality hypothesis, but itdoes have some status in our design, which is discussed below.The last two hypotheses, H3 and H4 are of interest on their own

accounts. We aim (as shown in our design below) to select groups ofsubjects which are equivalent, but nothing in H, or H2 hangs onwhether we succeed or fail in this objective. So, H3 is of tangentialinterest and bears only on questions of crosscultural differences,e.g., do Chinese and Americans at similar stages of their educationreally have similar (or different) discourse processing skills in theirrespective languages? And, to the extent that such similarities (ordifferences) may exist, can they be revealed in cloze tests over

crosstranslated passages? This kind of question is only indirectlyrelated to the universality theory pertaining to translation, but it isrelevant just to that degree. As for H4, it too is of tangential valuethough it is much more directly related to the theory of universality.It is also easy with our design to test this hypothesis (H4) which turnsout to be so stringent that it acquires a special interest which is



271

worth a little advance discussion.No plausible theory of translation (certainly not any of the

Peircean variety to which we subscribe) predicts that all of theindividual parts of every translation (compared at random as in ourcloze approach) will come out to be perfectly equivalent. That is,there is absolutely no plausible basis for supposing that everyrandomly selected sample of 54 items (as in our design) drawn froma given text by a given every nth deletion procedure (every fifthword in our tests; see below) will produce an outcome that will beequivalent to every other sample drawn from that same text (orworse, a translation of it) even if the ’same’ deletion ratio (as muchas this is possible) is employed. In fact, it is an open question(addressed here for the first time, so far as we know) whether acertain number of ’words’ across languages as different as Englishand Chinese really can be deleted across texts according to the sameevery nth word-based procedure. But there is no sufficient basis tosuppose that any given 54-word sample must be the same as anyother even if the procedure were to be applied in just one language,much less in two. No theory of paraphrase (for the case of just onelanguage) or of translation (for cases of two or more languages)predicts the stringent outcome suggested by H4; at least not for testsas short as 54 items. While it is reasonable to expect the tests to besubstantially correlated if they are based on the same text in a givenlanguage (per Oller, 1979; Carroll, 1993), the prediction in H4 is thatthe means of all the distinct samples will nearly perfectly converge.

Therefore, the question implied by H4 is worth examining becauseour theory does predict that if the cloze tests could be extended wellbeyond the 54-item limit, up to, say, 270 items or so (the lengthachieved by all five tests added together for any single passage in ourstudy; see below), then approximate equivalence in mean scoresought to be achieved in many cases. If the length of the texts in anypair of languages (or in many languages) could be extended

indefinitely, and if the translation process were carefully done bycompetent speakers of the languages involved (e.g., as has beendone for many translations, say, of the Bible - one of the longertexts that has been widely translated), our theory predicts a closerand closer approximation to perfect equivalence in mean clozescores with each increase in length up to some indefinite limit. Intheory, perfect equivalence would have to be achieved with aninfinitely extended text provided only that it was grounded in truenarrative representations, as described by Oller (1993b). However,our results show (see below) that a high degree of equivalence canapparently be achieved even with texts that are, by comparison,relatively brief. ~. ~.



272

According to Peircean theory, as elaborated by Oller and Jonz(1994) with respect to cloze tests, three main sources of variance hadto be taken into consideration and controlled in this study. Thesesources pertain to the three main variables in discourse processinggenerally: contributions of 1) the producer, 2) the consumer and 3)the form of the discourse itself. First, we controlled and alsomeasured the quality of the original texts as produced by theirwriters (first position or producer variance); secondly, we controlledand also measured the proficiency of the subjects who workedthrough our tests; and, thirdly, translation quality was measured andassessed.

II Method

Following C.S. Peirce’s theory of logical relatives, Oller and Jonz(1994) have shown that every communicative exchange consists of atriad of pragmatically inter-related but logically distinct and inde-pendent sources of variance. In cloze tests these same sourcesconstitute the three main sources to be observed, to a greater orlesser degree, in all possible experimental applications. The triad ofsources are as shown in Figure 1. Because Peirce was able to provemathematically, through his logic of relatives, that all monadic,dyadic and polyadic relationships found in any context can bereduced to a complex of triads, and since the one diagrammed hereis nothing other than the most common manifestation of his mostbasic triad, we know that this triad underlies all discourse, and thusall applications of cloze procedure.The design features by which we can alternately achieve relatively

perfect control of paired elements of the triad of elements in Figure1 (producer, consumer and discourse), while at the same time

enabling examination of the third source of variance (independentlyof the others), have been commonly misunderstood (as shown byOller and Jonz, 1994, especially in their last chapter where theyanswer the critics of cloze). However, those design features are soimportant to this study, and to others like it (cf. Oller et al., 1972,and sequels) that it may be worth the effort to diagram them. Figure2 does this.The important thing in this study (and in many others like it) is to

orthogonalize (i.e., render logically independent with respect tocombinations of the measures taken) the three main componentsources of variance: 1) the first person or producer variance (in thiscase the passage or story variance); 2) the second-person or

consumer variance (in the case the language or group variance); and



273

Figure 1 .

3) the third-person or discourse variance (in this case the varianceattributable to the translation treatment which is at issue). As shownin Figure 2, the three desired orthogonalizations are achieved by ourdesign by using the means on distinct pairs of cloze passages fortesting each of the main hypotheses to be examined. In each case,because the paired means are based on distinct cloze passagesperfectly distributed on both of the sides and on the top and bottomof the quasi-Latin square, each of the desired contrasts enables thecomparison to be made independently on each one of the threeelements of the communication triad shown in Figure 1. For

instance, the contrast on the translated versus original texts (H,) isindependent of the two groups because the means for the contrast inquestion are based on tests which receive exactly counterbalancedcontributions from both groups of subjects (English-speaking andChinese) and from both stories (robbery and drowning). And soforth, for each of the other hypotheses, reading right around thediagram in Figure 2.With respect to the two hypotheses that one of our reviewers (see

above) thought were codependent, namely, H2 and H3, the diagramin Figure 2 displays graphically the desired logical independence.The hoped-for equivalence of the overall means on the robbery



274

Figure 2

passage (in its Chinese original and English translated variant) andon the drowning passage (in its English original and Chinesetranslated variant) as predicted by H2 has no bearing on thepossibility of discovering a difference between the two groups asaddressed in H3. The contrast (if there is to be any at all) betweenthe two groups, by virtue of the orthogonalization of the variables inthe design, must be strictly a function of the different abilities of thetwo groups and not of the relative difficulties of the two texts. Theoverall mean of each language group (to be compared per H3) is theaverage of the scores of that group on both texts (robbery anddrowning). This average is indifferent to the difficulty of the originaltexts. Since each of the 125 English-speaking subjects performed onjust one cloze test over each of the passages and each of the 125Chinese subjects also performed on a similar pair of tests over bothpassages, the difficulty of the original texts as a factor is, relative tothe performance of the two groups, so perfectly counterbalanced asto be virtually eliminated. The beauty and simplicity of such an



275

orthogonal (within-subjects, quasi-Latin square) design (see Figure2) is that it logically generates the independence desired in order forthe main hypotheses (H, and H2) to be tested quite independently ofeach other.

z

1 Materials

Two narrative texts were selected (the reasons for using narrativesare accounted for by Oller, 1993b). The original texts consisted of astory about a robbery told originally in Chinese and a story about anear drowning incident told in English. Each story was also trans-lated into the opposite language. The English text (referred to asdrowning) was from Ola and D’Aulaire (1987:143-45). The Chinesetext (robbery), based on a true story, was reconstructed frommemory by Xiao. Both passages were edited so as to contain exactly270 words (see the Appendix for both the English and Chinesetexts). The two original passages were each then translated to theopposite language, resulting in four texts, i.e., an English original, anEnglish translation into Chinese, a Chinese original and a Chinesetranslation into English. The translations were initially checked bynative speakers and were judged to be about equal in level ofdifficulty. Subsequently, the English texts were edited (by Xiao) toreduce the overall number of words to 270 in each passage.

In fact, one reviewer (the same one mentioned above) supposedthat we ought to have made certain that the two passages were quiteequivalent before launching our study. But our goal was merely tobe as certain as possible that the translations were equivalent to theoriginal passages. Actually, this is theoretically easier than whatXiao (1992) really attempted (at the initial insistence of Oller, whodid not fully realize the difficulty of getting the number of words tocome out exactly the same). As a result, Xiao aimed to make thepassages of exactly equal length. This is nearly impossible inChinese and English, as the reviewer in question correctly noted.Indeed, there is a controversy still ongoing among Chinese linguistsconcerning what should count as a word. There are two competingapproaches. One of them treats each character, more or less, as asingle lexical item. This method counts some morphemes as wordsthat perhaps should be treated as parts of words. The other

approach comes closer to marking boundaries according to phrasalunits and, consequently, would (in our application to cloze testing)require more than one character to fill some blanks. After consult-ing a friend who is a professor at Nanjing Teachers University - andwho applied both division approaches to the two Chinese texts (boththe original robbery text in Chinese and the drowning text as -



276

translated into Chinese) - Xiao opted for the more lenient approachto word boundaries. Still, he found that the Chinese passage, by thatmethod, had a few too many ’words’ to match the length of theEnglish passage. At this point, an arbitrary decision was made toeliminate from the English versions one or two sentences whichgave tangential details of the narrative (cf. Xiao, 1992). For instance,in the text about the robbery, ’... up to 30 other bystanders werearound, but none came to help him’ any ‘... dragged for 50 metersbefore he rolled clear’ (see the Appendix) were cut from theChinese version. In retrospect, it would have been preferable tohave allowed a few more cloze items in the English versions.However, the arbitrary procedure that Xiao actually used could onlyhave biased the outcome against the predicted results in eachhypothesis to be tested. The same also holds for a couple ofinfelicities that came into the English versions of the texts at thetime of this cutting (see the footnotes in the Appendix). Since nosimilar infelicity existed in the Chinese texts, again these minortranslation flaws could only have biased things against the predic-tions to be tested.At any rate, all four texts were clozed by using a fixed-ratio

method (in defence of which, see Brown, 1983; Bachman, 1982;1985; Jonz, 1987; 1990; Oller and Jonz, 1994). Every fifth word wasdeleted with different starting positions to create five distinct testsover each of the four texts. In fact, one-fifth of the words in each onewas clozed in each of five cloze tests so that the five tests over anysingle passage taken together accounted for every ’word’ in thatpassage (but the definition of ’word’ is problematic - as noted in theprevious paragraph, especially so in Chinese).Each text, in its original version and in the opposite language

translation, yielded five cloze tests. There were 20 tests in all, five foreach of four passages. That is, the English original of the drowningappeared in five cloze versions, and the English translation of thedrowning into Chinese appeared in five versions; the Chinese

original of the robbery appeared in five versions, and the Chinesetranslation of the robbery into English appeared in five versions.Every version had precisely 270 words, and every cloze test hadexactly 54 blanks. There were exactly five cloze tests with 54 itemseach over each original text and over each translation. The clozeitems were arranged so that every word of each of the four passages(the drowning original in English, the drowning translation intoChinese, the robbery original in Chinese and the robbery translationinto English) was clozed. The first cloze test over each passage wascreated by counting up to the fifth word and deleting it and everyfifth word thereafter. The second was created by starting the first



277

deletion on the fourth word of the text and continuing with everyfifth word thereafter. The third was created by deleting the thirdword and every fifth thereafter, and so on for the fourth and fifth tests.The cloze tests were then filled in by two groups of subjects. The

English original and the Chinese translation into English (in fivepairs of distinct cloze tests, one over the drowning story and anotherover the robbery passage) were given to 125 English speakers, whilethe Chinese original (the robbery story in Chinese) and the Englishtranslation into Chinese were similarly distributed to 125 Chinesespeakers. Each pair of tests in each language was completed by 25subjects (25 x 5 pairs = 125 subjects in each of the two groups).

2 SubjectsThere were 250 subjects in all: 125 English speakers from twosections of Linguistics 101 in the spring semester of 1992 at theUniversity of New Mexico. The Chinese speakers, 125 native

speakers of Chinese, were tested in four departments at NanjingTeachers University in the People’s Republic of China. The Chinesegroup consisted of subjects from 18 to 24 years of age with 75females and 50 males. There were 52 freshmen, 31 sophomores and42 juniors. Of these, 52 were science majors, 21 were languagemajors and 52 were art majors. In the English-speaking group, the .

age-range was from 18 to 40. There were 89 females, 36 males, 21freshmen, 39 sophomores, 50 juniors and 15 seniors. The majorsincluded 64 in education, 13 in business, 14 in science, 27 in

languages, communication and philosophy, and two in art.

3 Procedure

The procedure was similar for both groups of subjects, English andChinese. The package of test materials for the English speakersconsisted of an information sheet plus directions for the tests andone of the five cloze versions over the English original (drowning)text, and one of the five versions over the translation from Chinese

(the robbery text). The five cloze-test versions over the originalpassage in English were randomly paired with one of the five clozetests over the translated text from Chinese. The pairs thus createdwere also randomly assigned to subjects in the same classes untileach block of subjects for each pair of tests reached n = 25. In eachcase, the English original text was stapled to the text from Chinesetranslated into English. Subjects could complete the two tests in anyorder they chose. Order of presentation, as the results of Oller andJonz (1994) amply show, is not a significant factor in any case unless



278

the tests are too long and fatigue begins to be a factor. Here, eachsubject completed 108 cloze items in his or her native language(English or Chinese) and required a maximum of about 40 minutesfor the task. The tests were apparently easy enough to fill in thatfatigue was not likely to be a factor. After the materials werehanded out, the subjects were instructed to complete the informa-tion sheet first before going on to the cloze passages. This sheetasked for demographics to enable a subsequent check on the degreeof equivalence between the English speakers (n = 125) and theChinese speakers (n = 125). Subjects were not required to writetheir names. The testing required about 20-40 minutes.The Chinese part of the experiment was conducted at Nanjing

Teachers University in the People’s Republic of China. A professorat that university administered the tests. Materials were mailed tohim and he administered the test according to the same procedureused for the English-speaking subjects at the University of NewMexico. Scoring and data analysis were done in the USA, based onthe exact-word method (for a defence of this choice, see Oller andJonz, 1994). The tests were relatively easy (as could be seen by howquickly subjects completed the 108 items in each pair of tests) andan exact-word criterion offered two advantages: a lower test-ceilingand ease of scoring for the 250 pairs of tests (500 cloze tests times 54items each giving 27000 items to be scored).

IH Results and conclusions

The statistical analyses were based on the MANOVA subroutines inSPSSx version 4.1,1992. The analyses were done on the IBM 9121 atthe University of New Mexico. Table 1 gives means and standarddeviations of the two originals and the two translations averaging

Table 1 Means and standard deviations for original and translated texts



279

across language groups. Table 2 indicates that the overall means ofthe originals did not contrast significantly from the overall means ofthe translations.

Difference across the originals and translations was negligible(Fl,249 = .03, p = .862) as predicted (HI). We also compared thetwo texts, drowning and robbery, for equivalence by averaging overgroups and over translations and originals. Table 3 gives therelevant means and standard deviations and Table 4 shows the

appropriate MANOVA analysis.The robbery and the drowning texts did not contrast significantly

(FI,249 = .57, p = .453). This result confirms H2 the predictedoutcome. It was our intention in designing the experiment to equatethe passages as closely as possible, and evidently we succeeded.But what if translation really introduces ’intrusions’ into the

target language and culture, as Giv6n (1989) has argued that itmust? Then we should find lower scores on the translations than onthe originals. Yet this was not the case. Giv6n’s prediction, there-fore, can be rejected. Furthermore, this rejection is not subject tothe logical limitation imposed on the acceptance of H1, which is,logically speaking and as we noted above, a negative particular. Toprove such a statement true for all possible cases would require theimpossible task of actually measuring all the possibilities. This cannever be done. But with Giv6n’s prediction, which takes the logicalform of an affirmative universal, we only require one particularnegative to reject it entirely.Table 5 gives grand means and standard deviations of the

originals and the translations across the two groups (i.e., the twolanguages, English and Chinese). Table 6 gives the appropriateMANOVA comparisons.The Bartlett-Box F showed the variances across the groups to be

Table 3 Means and standard deviations for the robbery anddrowning stories



280

Table 5 Grand means and standard deviations for originalsand translations across the two groups (i.e., languages)

well within required limits of homogeneity to enable univariateanalyses of variance. The results in Table 6 reject H3. The reason forthis might be the editing of the two English texts (see the Appendixand notes there), or perhaps actual differences across the subjectpopulations (Chinese and Americans), or both. It is impossible tosay for sure. However, because the Chinese texts did not include anyinfelicities that we know of, the infelicities in the English texts (seethe Appendix and the footnotes) could only have biased thingsagainst the English speakers (and against H, and H2 as notedabove).Table 7 shows the means and standard deviations of the five

versions averaged over the originals and five versions averaged overthe translated texts. Table 8 is the MANOVA source table. Whilethe five versions over the original and the five over the translatedtexts did contrast significantly (F4,245 = 2.42, p = .049), thiscontrast was probably the result of the confounding difference(slight though it was) between the Chinese and English texts.

However, when the robbery and drowning passages were con-trasted, where any infelicities owed to editing were distributedabout equally over the two passages (each in their five versions ineach of the two languages), there was no contrast (F1,245 = .03, p =.858). Again owing probably to the confounding factor of differ-ences across groups, there was a significant interaction between thefive cloze versions and the two texts in their original and translatedforms (F4,245 = 5.26, p = .0001). The fact that this interaction, andthe contrast across versions, is confounded by group differences canbe seen by looking back to Table 6.Moreover, Table 9 helps to show the sources of the significant

contrasts in Table 8. Table 9 compares the difference between the

treatment, originals versus translations (averaging over other fac-tors), with that between the two stories, robbery versus drowning(again, averaging over other factors), and with that between the two

Table 6 Analysis of variance for the within-subject effect across the two groups- .- -- - -- -



281

Table 7 Means and standard deviations for the five versionsover the original and translated texts

language groups, Chinese-speaking versus English-speaking (aver-aging over other factors). It can be seen that the relative contrastbetween the two groups is considerably larger than the differenceson the other factors. It is more than four times larger than thecontrast between the two texts and more than 18 times larger thanthe contrast resulting from our treatment (originals versus transla-tions). Therefore, we are confident that the interaction noted in theprevious paragraph is mainly due to the group contrast. Further, asnoted, this latter contrast might result from the infelicities in theEnglish texts.On the whole, the results support the thesis that nearly perfect

translation is possible. Of course, as noted above, no empiricalresults can ever prove a null hypothesis for all possible cases (whichwould make it out to be a universal negative) any more than anynumber of positive cases can ever suffice to prove a universalaffirmative. Our results do not show that there will always be onlynegligible contrasts between translated and original texts as judgedby cloze scores. But they do show that, for the two texts and the twolanguages studied here, translation is about as successful as it could

Table 8 Analysis of variance for the five versions over the originals and translationsand broken down by texts (i.e., robbery and drowning)



282

Table 9 Mean differences between the treatment (originals vs. translations, H,), thetexts (robbery vs. drowning, HZ) and the two groups (Chinese-speaking vs. English-Speaking, H3 )

be expected to be except for the editorial difficulties noted above(and in the Appendix).Our results sustain Taylor’s (1953) original claim that readers are

able to guess missing elements because of a kind of textualcoherence that is somewhat independent of the surface form of thetext, including the particular language in which it may be written. Inaddition, and in keeping with the hypothesized universality of theabstract basis of narrative schemata (Mandler et al., 1980; Oller,1993b), the experiences and events involved in the two narratives(the content schemata) are also fairly common to both cultures.Consequently, both groups were able to figure out the missingelements and fill them in with nearly identical accuracy across thetwo languages. Apparently the content, formal and abstract sche-mata that are available to language users generally (Carrell, 1984;Oller et al., 1993; Oller and Jonz, 1994; Oller, in press) are sufficientto sustain whatever inferences are necessary to guess missingelements in the various surface forms encountered across the two

languages. All of this holds, it would seem, in spite of the obviousdifferences across the language groups, the cultural backgroundsand even the specific stories told. Again, as expected, the universal-ity theory receives the predicted empirical support.

IV References

Bachman, L.F. 1982: The trait structure of cloze test scores. TESOL

Quarterly 16, 61-70.— 1985: Performance on cloze tests with fixed-ratio and rational

deletions. TESOL Quarterly 19, 535-56.Bickerton, D. 1981: Roots of language. Ann Arbor, MI: Karoma.— 1990: Language and species. Chicago, IL: University of Chicago

Press.

Brislin, R.W. 1970: Back-translation for cross-cultural research. Journal ofCrosscultural Psychology 1, 185-216.

Brown, J.D. 1983: A close look at cloze: validity and reliability. In Oller,J.W. jr, editor, Issues in language testing research. Rowley, MA:



283

Newbury House, 237-50. Burks, A.W., editor 1957-58: Collected writings of C.S. Peirce (Vols 7 and

8). Cambridge, MA: Harvard University Press.Carrell, P. 1984: Evidence of a formal schema in second language

comprehension. Language Learning 34, 87-112 (also in Oller, J. W.,editor, Methods that work: ideas for literacy and language teachers.Boston, MA: Heinle & Heinle, 191-20.

Carroll, J.B. 1993: Human cognitive abilities: a survey of factor-analyticstudies. New York: Cambridge University Press.

Chomsky, N. 1976: Reflections on language. London: Temple Smith.— 1982: Some concepts and consequences of the theory of government

and binding. Cambridge, MA: MIT Press.— 1988: Language and problems of knowledge: the Menagua lectures.

Cambridge, MA: MIT Press.Dewey, J. 1916: Essays in experimental logic. Chicago, IL: University of

Chicago Press.Einstein, A. 1936: Physics and reality. In Einstein, A., Out of my later years.

Secaucus, NJ: Citadel, 59-9.— 1941: The common language of science. In Einstein, A., Out of my

later years. Secaucus NJ: Citadel, 111-13.— 1944: Remarks on Bertrand Russell’s theory of knowledge. In P.A.

Schilpp, editor, The philosophy of Bertrand Russell. New York:Tudor Library of Living Philosophers, 277-91.

— 1956: Out of my later years. Secaucus, NJ: Citadel.Fisch, M., Kloesel, C.J.W., Moore, E.C., Roberts, D.D., Ziegler, L.A. and

Atkinson, N.P., editors 1982: Writings of C.S. Peirce: a chronologicaledition. (Vol. 1). Bloomington, IN: Indiana University Press.

Givón, T. 1989: Mind, code, and context: essays in pragmatics. Hillsdale, NJ:Lawrence Erlbaum Associates

Hartshorne, C. and Weiss, P., editors 1931-35: Collected papers of CharlesSanders Peirce (Vols 1-6). Cambridge, MA: Belknapp Press ofHarvard University.

Jonz, J. 1987: Textual cohesion and second language comprehension.Language Learning 37, 409-38.

— 1990: Another turn in the conversation: what does cloze measure?TESOL Quarterly 24, 61-83.

Katz, J.J. 1978: Effability and translation. In Guenthner, F., et al., editors,Meaning and translation. London: Duckworth.

Kloesel, C.J.W., Fisch, M.H., Houser, N., Niklas, U., Simon, M., Roberts,D.D. and Houser, A., editors 1986: Writings of Charles S. Peirce: achronological edition (Vol. 4). Bloomington, IN: Indiana UniversityPress.

Lakoff, G. 1987: Women, fire, and dangerous things: what categories revealabout the mind. Chicago, IL: University of Chicago Press.

Malakoff, M. and Hakuta, K. 1991: Translation skill and meta-linguisticawareness in bilinguals. In Bialystok, E., editor, Language processingin bilingual children. Cambridge: Cambridge University Press.

Mandler, J.M., Scribner, S., Cole, M. and Deforest, M. 1980: Cross-cultural



284

invariance in story recall. Child Development 51, 19-26.Moore, E.C., Fisch, M., Kloesel, C.J.W., Roberts, D.D., Ziegler, L.A. and

Atkinson, N.P., editors 1984: Writings of C.S. Peirce: a chronologicaledition (Vol. 2). Bloomington, IN: Indiana University Press.

Nida, E.A. 1964: Toward a science of translating. Leiden: Brill.Ola, P. and D’Aulaire, E. 1987: Chain of circumstance. Reader’s Digest 131,

143-45.

Oller, J.W. jr 1979: Language tests at school. London: Longman.— editor 1989: Language and experience: classic pragmatism. Lanham,

MD: University Press of America.— editor 1933a: Methods that work: ideas for literacy and language

teachers. Boston, MA: Heinle & Heinle.— 1993b: Reasons why some methods work. In Oller, J.W. jr, editor,

Methods that work: ideas for literacy and language teachers. Boston,MA: Heinle & Heinle, 374-85.

— in press: Connecting formal and content schemata: some results ofrecent research in semiotics. Paper presented at the 1994 AnnualConference of the American Association of Applied Linguistics,Baltimore, Md, March (manuscript, Department of Linguistics, Uni-versity of New Mexico, Albuquerque, NM.

Oller, J.W. jr, Bowen, J.D., Dien, T.T. and Mason, V. 1972: Cloze tests inEnglish, Thai, and Vietnamese: native and non-native performance.Language Learning 22, 1-15.

Oller, J.W. jr, Chihara, T., Chávez-Oller, M.A., Yii, G.K.S., Greenberg, L.and Hurtado de Vivas, R. 1993: The impact of discourse constraintson processing and learning. In Oller, J.W. jr, editor, Methods thatwork: ideas for literacy and language teachers. Boston, MA: Heinle &Heinle, 206-29.

Oller, J.W. jr and Jonz, J., editors 1994: Cloze and coherence. Cranbury,NJ: Bucknell University Press.

Oller, J.W. jr and Kennedy, C. in press: The determinacy of linguisticevidence. Manuscript, Department of Linguistics, University of NewMexico, Albuquerque, NM.

Oller, J.W. jr, Sales, B.D. and Harrington, R.V. 1969: A basic circularity intraditional and current linguistic theory. Lingua 22, 317-28.

Oller, J.W. jr, Scott, R. and Chesarek, S. 1991: Language and bilingualism:more tests of tests. Cranbury, NJ: Bucknell University Press.

Peirce, C.S. 1878: How to make our ideas clear. Popular Science Monthly12, 286-392 (also in Hartshorne, C. and Weiss, P., editors, Collectedpapers of Charles Sanders Peirce (Vols 1-6). Cambridge, MA:Belknapp Press of Harvard University, 248-71).

— 1897: The logic of relatives. The Monist 7, 19-40 (also in HartshorneC. and Weiss, P., editors, Collected papers of Charles Sanders Peirce(Vols 1-6). Cambridge, MA: Belknapp Press of Harvard University,266-87).

— 1903: Lecture IV. Scholastic realism. In Hartshorne C. and Weiss, P.,editors, Collected papers of Charles Sanders Peirce (Vols 1-6).Cambridge, MA: Belknapp Press of Harvard University, 64-67.



285

Taylor, W. 1953: Cloze procedure: a new tool for meaning readability.Journalism Quarterly 30, 415-33.

von, Uexküll, J. 1934, reissued in 1992: A stroll through the worlds ofanimals and men: a picture book of invisible worlds (illustrated byG. Kriszat). Semiotica 89, 319-91.

Xiao, S. 1992: A universality approach to translation studies. Unpublisheddoctoral dissertation, University of New Mexico.



286

Appendix

English original (’drowning’ text)Nancy and Elizabeth had just bought candy bars and were walkinghome. As they passed the pond, the girls spotted a white ball lyingon what seemed to be a solid covering of ice. They ran to get it.While Nancy was still near the edge, Elizabeth slid across the

frozen pond toward the ball. Suddenly Nancy heard Elizabethscream, ’Help me!’Without a moment’s hesitation, Nancy ran toward her friend, who

had fallen through the ice. But as Nancy grabbed at Elizabeth’sjacket, the ice cracked beneath her, and she too plunged into thenumbing water.

In a house near the pond, a woman was watching TV. She thoughtshe heard what sounded like a cry for help. She went into thekitchen and looked out the window. She saw movement on thesurface of the pond.The two children yelled for help, but soon Elizabeth fell silent and

slipped beneath the water. Nancy lost her grip on Elizabeth andbegan struggling to keep her own head above water. She sank, andpushed herself to the surface. Finally the icy water filled her lungs.Both girls quit breathing, and their heartbeats stopped.The woman ran to the phone and dialed the emergency number.

Minutes later, the police and firemen arrived. Nancy who wasfloating just beneath the surface, was pulled out first. One of thefiremen plunged into the icy water. He found Elizabeth’s limp bodyon the bottom. He pulled her to the surface. On the way to thehospital, Nancy was revived with artificial respiration. Elizabethrevived later at the hospital. After eight weeks, returned home.’

English translation of the Chinese original (‘robbery’ text)One day Wang was walking home. He passed a parked car with twomen and a woman inside. The men got out and demanded moneyfrom him. Wang said he had none and kept walking, but the menpursued him.

Nearby, a young man named Zhang was sitting in his pick-up. Hewas parked there, waiting for loading. He watched as Wang and hispursuers began fighting on the sidewalk. Being a big man, Wangmanaged to get free of the men. But when he moved into the street,the woman in the car started to drive toward him to hit him.

Zhang had never seen Wang before. Up to 30 other by-standerswere around, but none came out to help him. Zhang got out of his

’ The subject can be omitted in Chinese but is omitted here in the English version as well,creating a grammatical error.



287

pick-up. He punched one of the robbers, then the other. Next he ranto the other side of the street and asked a bus driver to call the

police. As he turned around, Zhang saw the car headed straight forhim. He tried to jump onto the hood, but was thrown beneath thevehicle and dragged 50 meters before he rolled clear. At this time,the police arrived and arrested the three robbers. Zhang washospitalized in intensive care for a week.Zhang spent three months gradually recovering from his injuries.

Because of damage to his feet, he still finds it painful to stand. Buthe has no regrets. When asked why he was doing so,2 he said what hewas thinking at the moment was ’I have got to help the guy, eventhough it is at the risk of my life.’

2 Here the phrase ’why he was doing so’ seems to suggest his having difficulty standing, butwhat is intended (and indicated in the Chinese version) is ’why he did what he did’ relative tothe incident with Wang.



288

Chinese original



289

Chinese original



Documents

Language Testing