Comprehension of Arithmetic Word Problems: A Comparison of ... · The specific approach used in this study is to observe the comprehension processes of students whom we assume have

Journal of Educational Psychology1995, Vol.87, No. 1, 18-32

Copyright 1995 by the American Psychological Association, Inc0022-0663/95/S3.00

Comprehension of Arithmetic Word Problems: A Comparison ofSuccessful and Unsuccessful Problem Solvers

Mary Hegarty, Richard E. Mayer, and Christopher A. MonkUniversity of California, Santa Barbara

It is proposed that when solving an arithmetic word problem, unsuccessful problem solversbase their solution plan on numbers and keywords that they select from the problem (thedirect translation strategy), whereas successful problem solvers construct a model of thesituation described in the problem and base their solution plan on this model (the problem-model strategy). Evidence for this hypothesis was obtained in 2 experiments. In Experiment1, the eye fixations of successful and unsuccessful problem solvers on words and numbers inthe problem statement were compared. In Experiment 2, the degree to which successful andunsuccessful problem solvers remember the meaning and exact wording of word problemswas examined.

Why are some students successful in solving word prob-lems whereas others are unsuccessful? To help answer thisquestion, we begin with the well-established observationthat many students from kindergarten through adulthoodhave difficulty in solving arithmetic word problems thatcontain relational statements, that is, sentences that expressa numerical relation between two variables (Hegarty,Mayer, & Green, 1992; Lewis & Mayer, 1987; Riley,Greeno, & Heller, 1983; Verschaffel, De Corte, & Pauwels,1992). For example, Appendix A shows a successful and anunsuccessful solution to a two-step word problem contain-ing a relational statement about the price of butter at twostores. We refer to this as an inconsistent version ofthe problem because the relational keyword (e.g., "less")primes an inappropriate arithmetic operation (subtractionrather than addition), whereas in a consistent problem, therelational term in the second problem statement primes therequired arithmetic operation (e.g., "more" when the re-quired operation is addition). A substantial proportion ofcollege students, who could be called unsuccessful problemsolvers, use the wrong arithmetic operation on inconsistentproblems but perform correctly on consistent problems(Hegarty et al., 1992; Lewis, 1989; Lewis & Mayer, 1987;Verschaffel et al., 1992). We interpret this finding as evi-dence that problem comprehension processes play an im-portant role in the solution of arithmetic word problems.

In this article, we compare the reading comprehensionprocesses used by problem solvers who make errors oninconsistent problems with those of problem solvers who do

Mary Hegarty, Richard E. Mayer, and Christopher A. Monk,Department of Psychology, University of California, SantaBarbara.

We thank Stephanie Thompson for her assistance with protocoltranscription.

Correspondence concerning this article should be addressedto Mary Hegarty or Richard E. Mayer, Department of Psychol-ogy, University of California, Santa Barbara, California93106. Electronic mail may be sent via Internet to [email protected].

not make errors on inconsistent problems, and we refer tothese two groups as successful and unsuccessful problemsolvers, respectively.1 We hypothesize that when con-fronted with an arithmetic story problem, unsuccessfulproblem solvers begin by selecting numbers and keywordsfrom the problem and base their solution plan on these—aprocedure we call the direct-translation strategy. In con-trast, we hypothesize that successful problem solvers beginby trying to construct a mental model of the situation beingdescribed in the problem and plan their solution on the basisof this model—a procedure we call the problem modelstrategy. Our goal is to examine the hypothesis that unsuc-cessful problem solvers are more likely to use a directtranslation strategy whereas successful problem solvers aremore likely to use a problem-model strategy. We acknowl-edge that the direct-translation strategy might be just onesource of unsuccessful problem solving and that the prob-lem-model strategy might be just one source of successfulproblem solving.

Rationale

The domain of mathematical problem solving is becom-ing an exciting domain for researchers conducting cognitivestudies of problem solving (Campbell, 1992; Mayer, 1989,1992; Schoenfeld, 1985, 1987). Although the creation of ageneral theory of problem solving—that was based on gen-eral problem-solving heuristics—was a major goal in the1970s (Newell & Simon, 1972), more recent research in thestudy of expertise points to the crucial role of domain-specific knowledge and processes in a complete account ofproblem solving (Chi, Glaser, & Farr, 1988; Ericsson &Smith, 1991; Smith, 1991; Sternberg & Frensch, 1991). Ourgoal is to provide an account of the domain-specific strat-egies that successful and unsuccessful problem solvers de-velop with practice on solving arithmetic problems and of

1 It would also be appropriate to label successful and unsuccess-ful problem solvers respectively as successful and less successfulproblem solvers.

18

COMPREHENSION OF WORD PROBLEMS 19

how these strategies account for individual differences inperformance.

In addition to its theoretical significance, the topic ofmathematical problem solving has important practical im-plications for the status of science and mathematics educa-tion in the United States. Although improving mathematicalproblem-solving skills of students is one of the six nationaleducational goals, there is disturbing evidence that Ameri-can students are currently not keeping pace with their co-horts in other industrialized nations (Lapointe, Mead, &Phillips, 1989; McKnight et al., 1987; Robitaille & Garden,1989). For example, in one comprehensive study, studentsin the highest performing classroom of those sampled in theUnited States scored lower than students in the lowestscoring classroom of those sampled in Japan, and follow-upstudies revealed that American students are particularlyweak in mathematical problem-solving performance(Stevenson & Stigler, 1992; Stigler, Lee, & Stevenson,1990). Basic research on how successful problem solverscomprehend word problems could contribute to the solutionof this national problem.

In building cognitive theories of problem solving, it isuseful to distinguish between processes involved in con-structing a problem representation and processes involvedin solving a problem (Mayer, 1992). Cognitive research inmathematics learning sometimes emphasizes solution pro-cesses such as computational procedures and problem-solving strategies (Anderson, 1983; Siegler & Jenkins,1989). An equally important goal, that we adopted in thepresent study, is to develop an account of the ways in whichproblem solvers understand problems, that is, constructproblem representations (Hinsley, Hayes, & Simon, 1977;Kintsch & Greeno, 1985; Mayer, 1982; Nathan, Kintsch, &Young, 1992; Reed, 1987). By focusing on comprehensionprocesses, we in no way wish to diminish the crucial role ofother cognitive skills involved in mathematical problemsolving (such as computational procedures). Our motivationfor studying problem comprehension processes derivesfrom growing evidence that most problem solvers havemore difficulty constructing a useful problem representationthan in performing the computations necessary to solvethe problem (Cardelle-Elawar, 1992; Cummins, Kintsch,Reusser, & Weimer, 1988; Dossey, Mullis, Lindquist, &Chambers, 1988; Robitaille & Garden, 1989; Stern, 1993).

The specific approach used in this study is to observe thecomprehension processes of students whom we assumehave acquired the basic skills involved in arithmetic prob-lem solving and for whom these skills are highly practiced,that is, college students. This approach allows us to focus onfundamental difficulties in problem representation that can-not be attributed to poor computational skills, lack of gen-eral reading comprehension skill, lack of general knowl-edge, or unfamiliarity with word problems. By studyingdifficulties in problem comprehension that persist even inan adult population, we can identify cognitive processes thatmay need more attention when children are first acquiringproblem-solving skills. Therefore, although our participantsare adults, our theory may be relevant to younger learners aswell. For example, Riley et al. (1983) and Verschaffel et al.,

(1992) have found that elementary school children havedifficulty in solving one-step word problems containingrelational statements.

Another unique characteristic of our approach is that wemonitored students' eye fixations as they planned solutionsto mathematics problems, which allowed us to gain insightsinto the nature of the comprehension processes (De Corte,Verschaffel, & Pauwels, 1990; Hegarty et al., 1992; Little-field & Rieser, 1993; Verschaffel et al., 1992).

Two Comprehension Strategies for Word Problems

We contrast two general approaches to understandingmathematical word problems that have been suggested byprevious researchers (Hegarty et al., 1992): a short-cutapproach and a meaningful approach that is based on anelaborated problem model. In the short-cut approach, whichwe refer to as direct translation, the problem solver attemptsto select the numbers in the problem and key relationalterms (such as "more" and "less") and develops a solutionplan that involves combining the numbers in the problemusing the arithmetic operations that are primed by the key-words (e.g., addition if the keyword is "more" and subtrac-tion if it is "less"). Thus, the problem solver attempts todirectly translate the key propositions in the problem state-ment to a set of computations that will produce the answerand does not construct a qualitative representation of thesituation described in the problem.

In the meaningful approach, which we refer to as theproblem model approach, the problem solver translates theproblem statement into a mental model of the situationdescribed in the problem. The problem model differs froma text base in that it is an object-based representation, ratherthan a proposition-based representation. This mental modelthen becomes the basis for the construction of a solutionplan.

The direct-translation approach is familiar in several re-search literatures as the method of choice for less successfulproblem solvers. Direct translation has also been referred toas "compute first and think later" (Stigler et al., 1990, p. 15),the keyword method (Briars & Larkin, 1984), and numbergrabbing (Littlefield & Rieser, 1993). Research on expert-novice differences reveals that novices are more likely tofocus on computing a quantitative answer to a story problem(such as in physics) whereas experts are more likely initiallyto rely on a qualitative understanding of the problem beforeseeking a solution in quantitative terms (Chi et al., 1988;Smith, 1991; Steinberg & Frensch, 1991). Similarly, cross-national research on mathematical problem solving suggeststhat use of the direct translation approach might be respon-sible for the poorer performance of American children rel-ative to that of Japanese children. American children aremore likely than Japanese children to engage in short-cutapproaches to word problems, and instruction in U.S.schools is more likely than instruction in Japanese schoolsto emphasize computing correct numerical answers ratherthan understanding the problem (Stevenson & Stigler, 1992-Stigler et al., 1990).

20 M. HEGARTY, R. MAYER, AND C. MONK

The direct-translation approach makes minimal demandson working memory, and it does not depend on extensiveknowledge of problem types. However, direct translationleads to incorrect answers when information implicit in thesituation described by the problem is relevant to the solu-tion, because students who use the direct-translation ap-proach fail to represent this situation. In this article, wepropose that the direct-translation approach accounts forerrors on two-step compare problems such as that shown inAppendix A.

Closer Examination of the Comprehension Process

In this section, we consider the stages of comprehensionfor the butter problem in Appendix A by contrasting howthis problem would be comprehended by a student using thedirect-translation approach versus how it would be compre-hended by a student using the problem-model approach.Each of these approaches involve several stages, which arediagrammed in Figure 1. According to the model in Figure1, to prepare to solve this problem, a problem solver con-structs a text base, extracts a mathematics-specific repre-sentation, and develops a solution plan.

Stage 1: Construction of the Text Base

As in most theories of comprehension (Johnson-Laird,1983; Just & Carpenter, 1987; van Dijk & Kintsch, 1983)we assume that the text in a mathematics problem is pro-cessed in increments. At each increment, we assume that the

Direct TranslationApproach

Problem ModelApproach

Read sentence

Construct / updatesemantic network

Select numbersand keywords

Construct/ updateproblem model

Develop solutionplan

Execute solutionplan

Figure 1. Model of comprehension processes for arithmeticword problems.

problem solver reads a statement, that is, a clause or sen-tence expressing a piece of information about one of thevariables or values in the problem. In constructing a textbase, the problem solver must represent the propositionalcontent of this statement and integrate it with the otherinformation in his or her current representation of theproblem.

First, the problem solver represents the individual state-ments. In this process, the solver may use knowledge of thetypes of statements that occur in mathematics problems,which have been formally analyzed by Mayer (1981). Theseinclude assignments, which express a value for a certainvariable; relations, which express the quantitative relationbetween two variables; and questions, which express thatthe value of a certain variable is unknown. For example, theabove problem can be analyzed into two assignments, arelation and a question:

Assignment 1: ((equals) COST OF BUTTER AT LUCKY, 65 CENTS)

Relation: ((equals) COST OF BUTTER AT LUCKY,

(minus) 2 CENTS, (COST OF BUTTER AT VONS))

Assignment 2: ((equals) BUTTER YOU NEED, 4 STICKS)

Question: ((unknown) COST OF BUTTER AT VONS)

Units of measure and scale conversion must also be encodedas part of each statement.

As problem solvers read later statements in the problem,they must also integrate the new information in this state-ment with their current text base. This integration involvesmaking referential connections between the different state-ments in the problem. For example, in the above problem,the solver must understand that this in the second sentencerefers to the price of butter at Lucky, which was given in thefirst sentence. This process probably depends on generalcomprehension processes for computing coreference (Clark,1969; Ehrlich & Rayner, 1983). Thus, a primary task of theproblem solver is to translate each statement into an internalpropositional representation and to integrate this internalrepresentation with the representation of other statements inthe problem to construct a semantic network representation.

Stage 2: Construction of a Mathematics-SpecificRepresentation

At the second stage of the comprehension of each state-ment, the problem solver is guided by the goal of solvinga mathematics problem and constructs a representation,which we refer to as the mathematics-specific representa-tion. It is at this stage that solvers using the direct-transla-tion approach differ from solvers using the problem-modelapproach.

In the direct-translation approach, the second stage con-sists of the problem solver's decision as to whether thestatement currently being processed contains a key fact, forexample, either a number such as 65 or a keyword such asmore in the relational statement in the problem about sticks


of butter given in Appendix A. We propose that at thisstage, problem solvers using direct translation delete allinformation from the text base except the numbers and thekey words. Thus, the outcome of this stage over severalcycles is a representation that contains less information thanthe text base, that is, only the keywords and numbers.

In the problem-model approach, problem solvers attemptto construct or update their problem model at this stage ofcomprehension. Because a problem model is an object-centered representation, the solver must determine whetherthe statement currently being processed refers to a newobject or an object that is already represented in his or herproblem model. For example, consider how a solver mightconstruct a model of the first two statements of the butterproblem:

At Lucky, butter costs 650 per stick.

This is 20 less per stick than butter costs at Vons.

The problem model has previously been characterized as anarray of objects (Riley & Greeno, 1988; Riley et al., 1983)or as an array of symbols representing variables on a num-ber line where the position of the variable represents itsvalue (Lewis, 1989). Although the data presented in thisarticle do not differentiate between different versions of theproblem-model approach, we use the number-line formathere for illustration purposes. The number-line format isalso suitable for the larger and continuous quantities re-ferred to in these problems (i.e., it is difficult to think of howa price of 65(2 might be represented as an array of objects).

The first statement mentions one variable, the price ofbutter at Lucky. We assume that when this is read, thesolver constructs a representation of a number line with asymbol for "Lucky" (representing the price of butter atLucky) at the number 65 on the number line. The secondstatement refers to two quantities, but at Stage 1 of com-prehension, the solver has determined that "this" refers to"the price of butter at Lucky", so the solver must add onlyone symbol to the model, a symbol to represent the price ofbutter at Vons. Because the position of "Vons" must repre-sent the fact that the price of butter at Lucky is 20 more thanthe price of butter at Vons, the symbol for "Vons" is placedtwo units above the symbol for "Lucky" on the number line.Thus, the problem model contains two symbols representingthe prices of butter at Lucky and Vons, and the relationshipbetween these prices is represented by their relative posi-tions on the number line. Note that in this representation,each number is tied to the variable that it represents as theposition of the variable on the number line represents itsvalue. In contrast, in the mathematics-specific representa-tion constructed with direct translation, the numbers are nottied to variables.

In summary, participants who construct a problem modelchange the format of their representation from a proposi-tion-based to an object-based representation at this stage ofproblem comprehension. In contrast, participants who usethe direct-translation approach construct a more impover-ished propositional representation at this stage, that is, a

representation that contains less information than the textbase and that can be based on erroneous relationshipinformation.

Stage 3: Construction of Solution Plan

Once a problem solver has represented the informationthat he or she believes to be relevant to solving a problem,the solver is ready to plan the arithmetic computationsnecessary to solve the problem. In the case of the problemabout the price of butter, the correct solution plan is to firstadd 2c to the price of butter at Lucky (650 per stick) andthen multiply the result of this computation by 4. A problemsolver using the direct-translation approach must base his orher solution on keywords, such as less in the relationalstatement and the numbers in the problem. Because less isprobably associated with subtraction the solver will proba-bly come up with the wrong solution plan, that is, to subtract20 from the price of butter at Lucky, instead of adding 20.If the problem solver makes such an error, he or she willhave no way of detecting this.

In contrast, a problem solver using the problem-modelapproach has a richer representation on which to base his orher solution plan, including an object-based representationof the cost of a stick of butter at Lucky in relation to the costof butter at Vons. Not only will this guarantee an accuratesolution plan, but a successful problem solver might keepthe problem model in working memory to monitor thesolution process. For example, a solver can infer from thequalitative problem model that the cost of butter at Vons ismore than the cost at Lucky. Therefore, if the computationsyield a per-unit cost that is less than 650, the solver canimmediately detect an error in the computation of the an-swer and repair his or her computations. Thus, anotherimportant function of the qualitative problem model is thatit serves as an aid to problem solvers for monitoring thesolution process.

Experiment 1

Although we propose three stages in the comprehensionof a problem, a previous study (Hegarty et al., 1992) re-vealed that it sometimes takes problem solvers several iter-ations of reading the problem statement to accomplish thesesolution steps. This pattern suggests that problem solversmight have to switch back and forth between differentproblem-solving stages in the process of comprehending aproblem. In that study we monitored participants' eye fix-ations as they solved two-step compare problems, such asthe butter problem shown in Appendix A. The data fromHegarty et al. (1992) revealed that after reading through theproblem statement, the participants made many regressionsto lines of the problems—sometimes reading a line four ormore times—before announcing a solution plan for theproblem. Furthermore, we observed a selection effect instudents' regressions, such that they were more likely toreinspect numbers than words. This effect is explored more


fully in the present study because it can be used to evaluatethe comprehension strategy being used by problem solvers.

The selection effect observed in the eye fixations ofstudents solving mathematics problems (Hegarty et al.,1992) is also symptomatic of the direct-translation strategy.A student using direct translation would focus most heavilyon the numbers and relational terms in a problem rather thanon the other words, because the student bases his or hersolution plan entirely on this information. In contrast, astudent using the problem-model strategy would pay moreattention to the other words in the problem, which are usedin constructing a problem model. In particular, participantsusing the problem-model strategy should pay particularattention to the variable names in the problem, becauseconstructing a qualitative problem model involves repre-senting the variables according to their relative magnitude,and in this representation, the numbers are tied to theirappropriate variable names (in contrast to the direct-trans-lation representation, in which the numbers are not tied tovariables). A variable name is defined as a proper namereferring to one of the quantities in the problem (e.g., Luckyand Vons in the sample problem presented earlier). Notethat these predictions hold, regardless of whether the prob-lem model consists of an array of objects (Riley & Greeno,1988; Riley et al., 1983) or symbols representing the posi-tion of variables on a number line (Lewis, 1989; Okamoto,1992). Consequently, although the numbers and keywordsare also important for participants using a problem modelapproach, participants using this approach should inspectthe numbers and keywords in relation to the other words inthe problem less often than participants using the direct-translation approach. Therefore, the absence of a selectioneffect is more reflective of a problem-model strategy.

We asked participants to solve a large set of problems,which allowed us to observe the effects of practice onstudents' comprehension strategies. On the one hand, wemight expect participants to be more likely to use a problemmodel after practice on solving a set of similar problems.With practice, participants might form problem schemas,which would relieve the working memory demands of stor-ing all the relevant information in the problem statementand therefore enable the participants to use the morememory-demanding problem-model strategy. On the otherhand, without feedback on their errors, unsuccessful prob-lem solvers might continue to use the erroneous directtranslation strategy with practice, which would suggest thatthey need to be explicitly taught a more meaning-basedapproach to comprehending mathematics problems.Therefore, although we could make no a priori predic-tions about the effects of practice on comprehension strat-egies, the study of such practice effects has important ed-ucational implications.

The present study provides new evidence about individualdifferences in the comprehension strategies of successfuland unsuccessful problem solvers. In particular, we proposethat successful problem solving is more likely to result froma problem-model strategy for comprehending word prob-lems, whereas unsuccessful problem solving is more likelyto result from a direct-translation strategy. On the basis of

this hypothesis, we predict different patterns of eye fixa-tions: Unsuccessful problem solvers will be more likelythan successful problem solvers to look at numbers andrelational terms when they reread part of the problem,whereas successful problem solvers will be more likely thanunsuccessful problem solvers to fixate variable names (e.g.,Lucky and Vons) when they reread a part of the problem.

Method

Participants and design. The participants were 38 undergrad-uates recruited from the psychology subject pool at the Universityof California, Santa Barbara. The design was a mixed factorialdesign consisting of two between- and three within-subjects fac-tors. The between-subjects factors were presentation order andproblem-solving success. Participants were randomly assigned toone of four counterbalanced presentation orders, such that thenumbers of participants in each presentation-order condition wereapproximately equal. Participants were classified as successfulproblem solvers if they made 0 or 1 conversion errors on 16compare problems and as unsuccessful problem solvers if theymade 4 or more conversion errors on these problems. All of theerrors were conversion errors, that is, the participant stated anincorrect operation in his or her solution plan, (e.g., addition whenthe relational term in the problem was more and the correct answerinvolved subtraction). The data from two successful and twounsuccessful problem solvers who received each presentation or-der were selected for analyses, so that the analyses are based on theperformance of 16 participants. The data selected were those of thetwo most successful problem solvers (those who made the fewestconversion errors) and the two least successful problem solvers(those who made the most conversion errors) in each condition.

All participants solved a series of problems that included 16target problems that were generated with a 2 X 2 X 4 within-subjects design, so all comparisons involving problem types arewithin-subjects comparisons. The first within-subjects factor waslanguage consistency: In half of the problems the relational term(e.g., less than) was consistent with the operation to be performed(e.g., subtraction) and in the other half it was inconsistent (e.g., theproblem contained less than and addition was required). Thesecond within-subjects factor was whether or not the relationalterm was lexically marked: Half the problems contained an un-marked relational term (e.g., more than) and half contained amarked term (e.g., less than). The third factor was order ofproblems. Problems were presented in four blocks of 12 problems,each of which included 4 target problems, one from each of thefour markedness- consistency combinations and one from each ofthe four cover stories in Table 1.

Materials and apparatus. The materials consisted of four setsof 48 arithmetic word problems, containing 32 filler and 16 targetproblems. The four problem sets contained the same four blocks of12 problems and, we constructed them by varying, within a Latinsquare, the order in which the four blocks were presented. Eachproblem set consisted of four blocks of 12 problems, withfiller problems in positions 1, 2, 4, 5, 7, 8, 10, and 12 and targetproblems in positions 3, 6, 9, and 11 for each block. Each blockcontained the same 12 cover stories, but the numerical values ofthe filler problems and the wording of the target problems differedfrom block to block.

The filler problems included various one-, two-, and three-stepword problems, as shown in Appendix B. Each target problemconsisted of three sentences, presented in four lines, as shown inTable 1. For each target problem, the first sentence was an assign-


Table 1Consistent and Inconsistent Versions of Target Problems

Consistent (Less) Inconsistent (Less)

At Lucky, butter costs 65 cents per stick.Butter at Vons costs 2 cents more per stick than butter at Lucky.If you need to buy 4 sticks of butter,how much will you pay at Vons?

At ARCO, gas costs $1.13 per gallon.Gas at Chevron costs 5 cents less per gallon than gas at ARCO.If you want to buy 5 gallons of gas,how much will you pay at Chevron?

Federal Express charges $1.75 per pound for packet delivery.United Parcel charges 20 cents less per pound than Federal Express.If you want to send a 12-pound package,how much will you pay at United Parcel?

At McDonalds, workers earn $6.00 per hour.Workers at Wendy's earn 50 cents less per hour than workers at

McDonald's.If you work for 8 hours,how much will you earn at Wendy's?

At Lucky, butter costs 65 cents per stick.This is 2 cents less per stick than butter at Vons.If you need to buy 4 sticks of butter,how much will you pay at Vons?

At ARCO, gas costs $1.13 per gallon.This is 5 cents less per gallon than gas at Chevron.If you want to buy 5 gallons of gas,how much will you pay at Chevron?

Federal Express charges $1.75 for packet delivery.This is 20 cents less per pound than United Parcel.If you want to send a 12-pound package,how much will you pay at United Parcel?

At McDonalds, workers earn $6.00 per hour.

This is 50 cents less per hour than workers at Wendy's.If you work for 8 hours,how much will you earn at Wendy's?

Note. The remaining problems were identical to these except that more was substituted for less in the second line of each problem.

ment statement expressing the value of some variable, the secondsentence was a relational statement expressing the value of asecond variable in relation to the first variable, and the thirdsentence (presented in 2 lines) asked a question about the value ofthe some quantity in terms of the second variable. The answer tothe question always involved multiplication or division of thevalue of the second variable by a quantity given in the thirdsentence. The second sentence varied across problems in its con-sistency and lexical marking, thus yielding the four problem typesas given in the rows of Table 1. There were four cover stories fortarget problems as given in the columns of Table 1.

The stimuli were presented on a DEC VR 260 monochromevideo monitor that was situated approximately three feet (0.91 m)from the participant. The participant's eye fixations were moni-tored with an Iscan corneal-reflectance and pupil-center eyetracker (Model RK-426) that sampled the position of the partici-pant's gaze every 16 ms and that output the x and y coordinates ofthis position to a DEC Vaxstation 3200. The Vaxstation alsocontrolled the presentation of problems. The position of the par-ticipant's gaze was instantaneously displayed on a second videomonitor (out of sight of the participant) by a pair of crosshairs(indicating the x and y coordinates) superimposed on the stimulusdisplay that the participant was viewing. The display on thissecond video monitor was recorded on videotape with a VHSvideo camera and recorder. The video equipment was also used torecord the participant's verbal statement of the solution plan bymeans of a microphone, situated approximately one foot (.30 m)from the participant.

Procedure. Each participant was randomly assigned to a testversion and was tested individually. First, the experimenter pre-sented written and verbal instructions. The participant was told thata word problem would appear on the computer screen and that hisor her task was to tell how he or she would solve the problem butnot to carry out any actual arithmetic operations. To illustrate theseinstructions, the participant was given the following sample prob-lem: "Joe has 3 marbles. Tom has 5 more marbles than Joe. Howmany marbles does Tom have?" They were told that an acceptableresponse was "I would add 5 and 3 to get the answer." Participantsthen practiced announcing a solution plan to another problem

before the experiment commenced. The task of giving a solutionplan, rather than a final answer was used because some of thenumbers in the problems were too large for participants to men-tally compute a final solution, and the collection of eye-fixationsnecessitated that participants keep their heads still during theexperiment, so that they could not make written calculations. Aprevious study showed that participants easily followed theseinstructions and that they made errors on this task that were similarto errors that they made on the task of completely solving aproblem (Hegarty et ah, 1992).

Following these instructions, the participant was seated in adentist's chair facing the display screen and microphone. A head-rest was fitted comfortably to the participant's head. The partici-pant was asked to move as little as possible during the experiment.The participant was asked to fixate an asterisk that appeared in thetop left corner of the screen and to push a button in order to beginand end each trial. As soon as the button was pressed, a wordproblem appeared on the display screen. The participant silentlyread the problem, stated how he or she would solve it, and thenpushed a button. No time limitations were imposed. This proce-dure was the same for each of the 48 problems. When the partic-ipant completed the test, he or she was debriefed and dismissed. Atypical session lasted from 30 to 40 min.

Results

Errors. We recorded the time to specify a solution planand whether or not each participant made an error in spec-ifying a solution plan on each of the 16 target problems.Unsuccessful problem solvers committed four or more er-rors on the 16 target problems, and successful problemsolvers committed 0 or 1 errors. As in previous studies,unsuccessful problem solvers generated a higher proportionof errors on inconsistent than consistent problems, (Ms =.62 and .24, respectively), f(7) = 5.69, p < .01, whereassuccessful problem solvers by definition produced almostno errors.


Response times. An analysis of response times revealedthat unsuccessful problem solvers tended to spend moretime solving the problems than did successful problemsolvers, although the difference was significant only by adirectional test (Ms = 21.7 s and 14.6 s, respectively),t (14) = 1.78, p = .05, one-tailed. These data indicate thatthere was no speed-accuracy tradeoff. Because of the dif-ferences in accuracy between the two groups of participantsfor different problem types, the response-time data are dif-ficult to interpret and were not analyzed further.

Scoring of eye-fixation data. The sequence of eye fixa-tions was transcribed from the video recording by a researchassistant who was unaware of the ability classification of theparticipants and the hypotheses of the study. This transcrip-tion provided an eye-fixation protocol for each participantsolving each of the 16 target problems. The protocols listed,in sequence, each line of the problem at which the partici-pant gazed and indicated which words were fixated on thatline at that point in the participant's reading of the problem.Sample protocols are presented in Appendixes C and D. Foreach student we scored three objective measures from theprotocol for each target problem: the number of times thestudent looked back at a number (such as "4"), at a rela-tional term (such as less than), and at a variable name (suchas Lucky) after the initial reading of the problem.

Do successful and unsuccessful problem solvers usedifferent strategies? An examination of students' eye-fixation protocols provides a test of the hypothesis thatunsuccessful problem solvers are more likely to use a direct-translation strategy and successful problem solvers are morelikely to use a problem-model strategy. In particular, weexamine the prediction that the selection effect will bestronger for unsuccessful than for successful problem solv-ers. Figure 2 summarizes the mean number of times perproblem that unsuccessful and successful problem solverslooked back at a number or a relational term and the mean

20-1

15-

10-

B Unsuccessful

• Successful

Variable Names Numbers &Relational Tenns

Figure 2. Mean number of times per problem that successful andunsuccessful problem solvers looked back at numbers or relationalterms and at variable names. There were three numbers, onerelational term, and two or three variable names in eachproblem.

number of times that they looked back at a variable name inthe problem. According to our hypothesis, unsuccessfulproblem solvers should focus more on numbers and rela-tional terms in the problem compared to other material—because the direct-translation strategy is based on abstract-ing numbers and relational terms from the problemstatement—whereas the successful problem solvers shouldshow a more balanced focus that includes the variablenames needed to construct a coherent representation of thesituation being described in the problem, as well as thenumbers needed to solve the problem.

Consistent with our predictions, unsuccessful problemsolvers reexamined numbers and relational terms signifi-cantly more often than did successful problem solvers (seeFigure 2), r(14) = 2.37, p < .05. More specifically, unsuc-cessful problem solvers reexamined numbers an average of16.3 times per problem as compared with 11.2 times forsuccessful problem solvers, f(14) = 2.06, p = .059, and theyreexamined relational terms an average of 2.3 times perproblem as compared with 1.3 times for the successfulproblem solvers, f(14) = 2.07, p = .058. In contrast, un-successful and successful problem solvers did not differsignificantly in how often they reexamined the names of thevariables in the problem (see Figure 2), \t\ < 1. In all ofthese analyses, the data are collapsed over presentationorder, because order did not have any significant effects onreexamination or any significant interactions with problem-solving success (p > .10 in all cases).

Because unsuccessful problem solvers made more regres-sions in general, it was important for us to show that theyinspected numbers and relational terms on a greater propor-tion of their regressions. This was the case, that is, unsuc-cessful problem solvers looked at a number or relationalterm in more of their regressions (66.3%) than did success-ful problem solvers, who looked at a number or relationalterm in 59.4% of their regressions, f(14) = 4.59, p < .001.These percentages were computed for each participant bydividing the mean number of times per problem that anumber or relational term was reexamined by the meannumber of times a number, relational term or variable namewas reexamined.

The picture that emerged from this analysis was thatunsuccessful problem solvers struggle more than do suc-cessful problem solvers to construct a representation of theproblem but spend their additional effort mainly in reexam-ining numbers and relational terms rather than in reex-amining other informative words.2 This relatively higher

2 We expected situational terms such as "gas," "butter," "pack-age," and "workers" to produce the same pattern of results aswords describing the variable names: Unsuccessful problem solv-ers using direct translation will not need to focus especially onthese words, but successful problem solvers will find these wordshelpful in their attempts to build a problem model. As withvariable names, the groups did not differ significantly in thenumber of times they looked back to the situational terms "gas,""butter," "package," or "workers," f(14) = 1.02, p > .10; the samepattern was present for each block as is indicated by the lack ofGroup X Block interaction, F(3, 24) = 1.59, p > .20, MSE = 7.82.


reliance on numbers rather than on words is consistent withthe direct-translation strategy. In contrast, the successfulproblem solvers need to reexamine the problem less than dothe unsuccessful problem solvers, and when they do lookback to a previously read part of the problem they are lesslikely to look at a number than are the unsuccessful problemsolvers. This relatively lower reliance on numbers is con-sistent with the problem-model strategy.

It should be noted that contrary to our expectations, therewas a selection effect for the successful problem solvers,f(7) = 7.18, p < .01, in addition to an expected one for theunsuccessful problem solvers, f(7) = 6.79, p < .01 (seeFigure 2). That is, even successful problem solvers fixatednumbers more than variable names. This effect might reflectthe fact that the eye-fixation protocols cover both the com-prehension and planning stages of problem solution and thatit is not possible to separate these stages in the protocolsusing the available data. Regardless of what problem solversfixate during problem comprehension stages, their fixationsshould be focused primarily on the numbers during theplanning stage because the solution involves combining thenumbers.

Do successful and unsuccessful problem solvers learn atdifferent rates? In our study, successful and unsuccessfulproblem solvers examined four blocks of problems, so it ispossible to determine whether their comprehension strate-gies changed across the four blocks. Before considering thepattern of their eye fixations we first examined the rate ofreversal errors on consistent and inconsistent problems oneach block, as shown in Figure 3. We conducted an analysisof variance (ANOVA) on the error-rate data for unsuccess-ful problem solvers with block and consistency as factors.The unsuccessful problem solvers produce a strong consis-tency effect in each block, as indicated in an overall con-sistency effect, F(l, 7) = 32.41, p < .001, MSE = 0.52, andthe lack of a significant Consistency X Block interaction,F(3, 21) < 1, MSE = 0.52. Unsuccessful problem solversappear to remain unsuccessful throughout the entire session,as their error rate remains high across all four blocks, andthe effect of block is not significant for the unsuccessfulproblem solvers, F(3, 21) < 1, MSE = 0.40.

It was not valid to conduct an ANOVA on the data for thesuccessful problem solvers, because there was no variancein these participants' performance on the consistent prob-lems. An examination of Figure 3 indicates that, interest-ingly, a slight consistency effect is present for these partic-ipants for the first block, but any semblance of this patterndisappears on subsequent blocks. Thus, the successful prob-lem solvers might be characterized as somewhat successfulon the first block and highly successful on subsequentblocks.

If unsuccessful and successful problem solvers maintaintheir respective strategies across all four blocks, then theunsuccessful problem solvers should reexamine numbers

On the first block, each group produced 2.66 regressions perproblem, and overall, the successful problem solvers averaged 1.46regressions, whereas the unsuccessful problem solvers averaged2.11.

.20 r

Unsuccessful - Inconsistent

Unsuccessful - Consistent

• Successful - Inconsistent*4 Successful - Consistent*

Block

Figure 3. Proportion of errors on consistent and inconsistentproblems in each of four blocks by successful and unsuccessfulproblem solvers. An asterisk indicates that successful problemsolvers made no errors on consistent problems in any block and noerrors on inconsistent problems in Blocks 3 and 4.

and relational terms at a higher rate than the successfulproblem solvers on each of the four blocks. Figure 4 showsfor each block the mean number of times per problemthat unsuccessful and successful problem solvers lookedback at a number or relational term and the number oftimes for each block they looked back at a variable namein the problem. We conducted an ANOVA on the per-centage of regressions to numbers or relational terms,with the variables being group and block. The percentageof regressions that involved numbers and relational termswas similarly higher for the unsuccessful problem solversthan it was for the successful problem solvers on each ofthe blocks, as is indicated by a significant group effect,F(l, 14) = 21.03, p < .01, MSE = 35.9, and no signifi-cant interaction between group and block, F(3, 42) < 1,MSE = 19.9.

We also conducted an ANOVA on the number of regres-sions to numbers or relational terms, with group and blockas factors. The number of regressions to numbers and rela-tional terms differed across the four blocks, F(3, 42) =16.38, p < .001, MSE = 5.16, and the differences weresimilar for both groups as is indicated by a lack of interac-tion between group and block, F(3, 42) < 1, MSE = 5.16.An examination of Figure 4 reveals that for both groupsthere was a monotonic decrease in the number of regres-sions to both numbers and relational terms across the fourblocks.

Finally, we conducted an ANOVA on the number ofregressions to variable names, with group and block asvariables. The mean number of regressions to variablenames differed across the four blocks, F(3, 42) = 6.03,p < .01, MSE = 7.81, and the difference was similar forboth groups as is indicated by a lack of interaction betweengroup and block, F(3, 42) < 1, MSE = 7.81. Again, bothgroups showed a monotonic decrease in regressions acrossblocks.


1

&co

o

I§

25

20

15

10

Unsuccessful - Numbers & Relational Terms

Successful - Numbers & Relational Terms

Unsuccessful - Variable Names

Successful - Variables Names

1 2 3 4Block

Figure 4. Mean number of times per problem that successful and unsuccessful problem solverslooked back at numbers or relational terms and at variable names in each of four blocks.

These results are consistent with the idea that althoughunsuccessful problem solvers became somewhat more effi-cient in their processing of the problems over the course ofthe session (i.e., reduced regressions), they did not changetheir comprehension strategies to the problem-model ap-proach. A relatively large difference in their regressionsto numbers and relational terms compared to their regres-sions to variable names remained across all four blocks.Similarly, although the successful problem solvers alsobecame more efficient in their processing of the prob-lems, they appeared to maintain their problem-modelstrategy. The successful problem solvers showed a rela-tively small difference in the number of regressions tonumbers and relational terms versus variable namesacross all four blocks. In short, the successful and unsuc-cessful problem solvers displayed qualitatively differentcomprehension strategies that remained qualitatively dif-ferent with practice.

Examples of two approaches to comprehending mathe-matical problems. To illustrate the main results of thisstudy, in Appendixes C and D we present protocols of asuccessful and an unsuccessful problem solver solving aproblem with an inconsistent, marked relational term. Theprotocol presented in Appendix C was generated by anunsuccessful problem solver. The participant started byreading all the words in the problem. After this initialreading, the participant regressed to a previously read line ofthe problem 39 times, and these regressions were almostexclusively focused on the numbers in the problem and theunits of measurement accompanying these numbers. Forexample, the participant refixated numbers 35 times. In

contrast, the participant reread a variable name (Vons) onlytwice, when she reread the question. Therefore this protocolshows a strong selection effect. Consistent with our pre-dictions, the participant made a conversion error on thisproblem.

This protocol can be contrasted with the protocol of asuccessful problem solver solving the same problem, whichis presented in Appendix D. This participant regressed to apreviously read line of the problem fewer times (13), andthese regressions were not focused solely on the numbers inthe problem. The participant refixated a number 8 times andrefixated a variable name 6 times. Because this protocoldoes not show a strong selection effect, we propose that it ismore characteristic of the problem-model approach. Con-sistent with our predictions, this participant solved the prob-lem correctly.

Experiment 2

Our goal in Experiment 2 was to compare how successfuland unsuccessful problem-solvers remember story problemsthat they have solved. Our predictions are based on the ideathat students who tend not to make reversal errors (i.e.,successful problem solvers) are more likely to rememberthe situation described in the problem because they use aproblem-model strategy for encoding word problems,whereas students who tend to make reversal errors (i.e.,unsuccessful problem solvers) are more likely to rememberthe exact wording of the keyword (e.g., less or more)because they use a direct-translation strategy for encoding


word problems. In short, our predictions are based on theidea that successful problem solvers are more sensitive toessential meaning and less sensitive to exact wording incomprehending word problems than are unsuccessful prob-lem solvers. Therefore, we predict that successful problemsolvers will perform better than unsuccessful problem solv-ers on tests of memory for the essential meaning of wordproblems but worse than unsuccessful problem solvers ontests of memory for the exact wording of word problems.

Method

Participants and design. The participants were 37 college stu-dents recruited from the psychology subject pool at the Universityof California, Santa Barbara. Participants who committed no errorson the four target problems were classified as successful problemsolvers (N = 17); participants who committed at least one reversalerror on the four target problems were classified as unsuccessful(N — 13), and participants who did not commit at least onereversal error but did commit at least one other kind of error wereexcluded from the analysis (N = 7). The two groups did not differsignificantly on reported mean Scholastic Aptitude Test—Mathe-matics (SAT-Math) scores (565 and 574 for successful and un-successful groups respectively), /(28) < 1, nor did they differsignificantly on mean reported rating of mathematical ability on ascale ranging from very poor (1) to very good (5), 3.56 and 3.77for successful and unsuccessful problem solvers respectively,|/|(28) < I.3

Materials. The materials consisted of a participant question-naire, four sets of a 12-sheet problem-solving test, a 4-sheet recalltest, and a 4-sheet recognition test. The participant questionnairewas an 8.5 in. X 11 in. sheet that solicited basic informationincluding the student's SAT-Math score and that asked the student"to rate your mathematics ability" on a 5-point scale ranging fromvery poor (1) to very good (5). These measures were included toprovide preliminary information regarding the relation of perfor-mance on the target problems to more general measures of math-ematical achievement.

The problem-solving test consisted of 12 sheets, each 8.5 in. X5.5 in., with a problem on each sheet; 4 of the sheets contained thetarget problems from Experiment 1 (see Table 1) and 8 containedthe filler problems from Experiment 1 (see Appendix B). In eachof the four sets, the eight filler problems were included in the sameorder; in each of the four sets different versions of the gas problemoccurred as the 3rd problem, the butter problem as the 6th prob-lem, the package delivery problem as the 9th problem, and thefast-food restaurant problem as the 11th problem. In each set,each of the four target problems was presented in one of fourversions (consistent-more, consistent—less, inconsistent-more, andinconsistent-less) with presentation version for each problemcounterbalanced across sets. Table 1 shows the four versions ofeach of the four target problems. The recall test consisted of foursheets, each 8.5 in. X 5.5 in., in which the student was asked towrite down the problems about "gas," "butter," "package deliv-ery," and "workers in fast-food restaurants," respectively. Therecognition test consisted of four sheets, each 8.5 in. X 5.5 in.,respectively containing four versions of each of the four targetproblems (consistent-more, consistent-less, inconsistent-more,and inconsistent-less.

Procedure. Participants were tested in groups of 1 to 4 persession, with the different versions of the problem-solving testmixed randomly within each group. Each participant was seated atan individual work booth. First, he or she completed the participant

questionnaire. Second, the problem-solving test was presented,with test form completely randomized between subjects. Partici-pants were told to show their work on the sheet for each problemand to keep working on a problem until instructed to tum the pageto the next problem. Participants were given 1 min to solve eachproblem. Third, the problem-solving test was collected, and therecall test was presented. Participants were told to keep working ona sheet until instructed to turn the page to the next problem, andthey had 1.5 min to work on each sheet. Fourth, the recall test wascollected, and the recognition test was presented. Participants weretold to circle the problem on each sheet that corresponded to theproblem they had actually solved on the problem-solving test; theyworked at their own rates for a maximum of 2.5 min. At comple-tion of the recognition test, each participant was thanked anddismissed.

Results and Discussion

Scoring. The four target problems on the problem solv-ing test were scored as "correct" if the numerical answerwas correct or "incorrect" if the numerical answer wasincorrect. Incorrect problems were further classified as re-versal errors if the problem solver added when the correctoperation was subtraction or subtracted when the correctoperation was addition, as arithmetic errors if the problemsolver made a computational error or miscopied a number,and as a strategic error if the problem solver left out acomputational step.

We computed a semantic error score for each participantby tallying the number of semantic errors on the 4-itemrecall test and on the 4-item recognition test and by express-ing this as a proportion of the total number of problems (i.e.,dividing that number by 8). A semantic error occurred whenthe student produced a problem on the recall test or circleda problem on the recognition test in which the relationbetween the two terms was reversed (e.g., remembering thatgas at Chevron costs less than at ARCO when the problemhad stated that gas at Chevron costs more than gas atARCO). We computed a literal error score for each partic-ipant by tallying the number of literal errors on the 4-itemrecall test and on the 4-item recognition test and dividing by8 to express this as a proportion of problems solved. Aliteral error occurred when the student produced a problemon the recall test or circled a problem on the recognition testin which the relation between the two terms was consistentwith the presented problem, but the wording of the rela-tional term was changed (e.g., remembering "Gas at Chev-ron costs 5<t less . . . than gas at ARCO" when the problemhad stated that "This [gas at ARCO] is 50 more . . . than gasat Chevron"). Examples of each type of error are shown inTable 2.

Do successful and unsuccessful problem solvers differ inmemory for meaning and memory for exact wording of wordproblems? According to our predictions, successful prob-lem solvers are more sensitive to the essential meaning ofword problems (i.e., the described situation) than unsuc-

3 It was not possible to include SAT-Math information forExperiment 1 because too many students in that experiment did notrecall their scores.


Table 2Example of Scoring of Recall and Recognition Tests

Problem Scored as

PresentedAt ARCO, gas costs $1.13 per gallon. CorrectThis is 5 cents less per gallon than gas costs at Chevron.If you want to buy 5 gallons of gas.how much will you pay at Chevron?

Recognized or recalledAt ARCO, gas costs $1.13 per gallon. Literal errorGas at Chevron costs 5 cents more per gallon than gas at ARCO.If you want to buy 5 gallons of gas.how much will you pay at Chevron?

At ARCO, gas costs $1.13 per gallon. Semantic errorGas at Chevron costs 5 cents less per gallon than gas at ARCO.If you want to buy 5 gallons of gas.how much will you pay at Chevron?

At ARCO, gas costs $1.13 per gallon. Semantic errorThis is 5 cents more per gallon than gas costs at Chevron.If you want to buy 5 gallons of gas.how much will you pay at Chevron?

cessful problem solvers. In particular, we predicted thatsuccessful problem solvers would be more likely than un-successful problem solvers to remember the correct relationbetween the two variables in each target problem (e.g.,whether gas at Chevron costs more or less per gallon thangas at ARCO), but successful problem solvers would be lesslikely than unsuccessful problem solvers to remember theexact wording of the relation between the terms (e.g.,whether the problem contained the word "more" or "less").Figure 5 shows the proportion of semantic and literal errorson remembering (i.e., recalling or recognizing) the fourtarget problems by successful and unsuccessful problemsolvers. Consistent with the predictions, the successfulproblem solvers made significantly fewer semantic errors inremembering the problems than did unsuccessful problemsolvers, f(28) = 5.26, p < .0001, whereas the unsuccessfulproblem solvers made significantly fewer literal errors thansuccessful problem solvers, ?(28) = 2.72, p < 02. Thispattern is consistent with the idea that the successful prob-lem solvers were more likely than unsuccessful problemsolvers to construct a problem model while comprehendingthe word problems, whereas unsuccessful problem solverswere more likely than successful problem solvers to use adirect-translation strategy for encoding word problems—thus showing less sensitivity to the situation described in theproblem. In this analysis, the data were collapsed over thedifferent presentation orders for the problems, because thisvariable did not affect memory significantly or interact withproblem-solving success in a 2 (groups) X 4 (orders)ANOVA (p > .10 in both cases).

In a more specific analysis, we compared the performanceof the unsuccessful and successful groups separately for therecall and recognition tests. As predicted, the unsuccessfulgroup committed significantly more semantic errors thandid the successful group on recall (Ms = 2.39 and 1.35,

respectively), t(28) = 2.87, p < .01, and on recognition(Ms = 1.54 and 0.35, respectively), t(28) = 5.22, p <.0001, whereas the successful group committed signifi-cantly more literal errors than did the unsuccessful group onrecall (Ms = 1.18 and 0.69, respectively), f(28) = 2.12,p < .05, and on recognition (Ms = 1.77 and 0.77, respec-tively), /(28) = 2.57, p < .05. These data were collapsedover presentation order, because this variable did not showsignificant main effects or interactions with problem-solving success (p > .10 in all cases).

§

I

.50

.40

.30

.20

.10

Unsuccessful

Successful

Semantic errors Literal errors

Figure 5. Mean proportion of semantic and literal errors pro-duced on memory tests by successful and unsuccessful problemsolvers.


Discussion

These experiments provide converging evidence concern-ing the hypothesis that unsuccessful problem solvers aremore likely to comprehend by direct translation and thatsuccessful problem solvers are more likely to comprehendby building a problem model. First, unsuccessful problemsolvers devote a higher percentage of their fixations tonumbers and relational terms when they reread part of theproblem, as compared with successful problem solvers.Overall, the unsuccessful problem solvers looked back atparts of the problem more than did successful problemsolvers, thus suggesting that they are struggling to figure outhow to solve the problem. However, unsuccessful problemsolvers seem to use a comprehension strategy that empha-sizes looking at numbers and relational terms—as predictedby the direct translation approach. In contrast, successfulproblem solvers devote a greater percentage of their fixa-tions to variable names than do unsuccessful problem solv-ers, thus suggesting they are more likely to be using aproblem-model strategy. In this study, we specify the dif-ferences in the way successful and unsuccessful problemsolvers examine numbers and important words in a prob-lem. This work adds to the emerging literature on the use ofthe eye fixation methodology to study processing of math-ematical information (De Corte et al , 1990; Hegarty et al.,1992; Littlefield & Rieser, 1993; Mayer, Lewis, & Hegarty,1992; Verschaffel et al., 1992).

Second, the differences between successful and unsuc-cessful problem solvers both in the pattern of errors and inthe pattern of eye fixations remain stable over the course ofthe session. There is no evidence that practice causes un-successful problem solvers to change towards a problem-model strategy nor that practice causes successful problemsolvers to change towards a direct-translation strategy. Per-haps it is not surprising that unsuccessful problem solversdid not change to a problem-model strategy during thesession, because it appears that they did not develop thisstrategy during years of mathematics instruction.

Third, when successful problem solvers make errors inremembering word problems, they are more likely than areunsuccessful problem solvers to remember the situationdescribed in the problem (e.g., one variable is greater or lessthan another) but less likely than unsuccessful problemsolvers to forget the specific relational keyword used in theproblem (e.g., less or more). This pattern is consistent withthe idea that successful problem solvers are more likely toconstruct a meaningful representation of the situation de-scribed in the problem whereas unsuccessful problem solv-ers are more likely to focus on keywords and numbers.

Although these experiments provide converging evidencethat successful and unsuccessful problem solvers tend to usequalitatively different comprehension processes for wordproblems, it would be incorrect for us to conclude that allunsuccessful problem solvers use one strategy and that allsuccessful problem solvers use a different strategy on allstory problems. Strategy selection is likely to be based onboth individual and situational factors (Siegler & Jenkins,1989), and all we can claim from the current set of findings

is that successful and unsuccessful problem solvers dif-fer in their tendencies (or likelihood) to use one strategy oranother.

The eye-fixation data and memory data indicate thatsuccessful problem solvers construct a problem model inwhich the numbers in the problem statement are tied to theirappropriate variable names, which means that they payrelatively more attention to the variable names while reread-ing the problem. Beyond this, it is not possible to determinethe exact nature of the problem models of successful prob-lem solvers from the present data. Although we stated ourtheory in terms of a problem model with a number-lineformat, it is possible that the problem models constructed bysuccessful problem solvers have some other format, such asan array of objects (Riley & Greeno, 1988; Riley et al.,1983).

In this research, we have focused on the solution ofcompare problems, but the direct-translation and problem-model strategies documented in the context of these prob-lems have the potential of explaining individual differencesin the solution of other types of mathematics problems. Forexample, Reed (1993) found conversion errors in motionproblems in which college students were asked to comparetwo speeds (e.g., an athlete's running and biking speeds).Regardless of the correct solution, students tended to addnumbers from the problem when they were asked to find thefaster speed and to subtract when asked to find the slowerspeed. Reed's results suggest that the direct-translationstrategy might be even more prevalent among college stu-dents than indicated by performance on the simpler prob-lems used in this study. The direct-translation strategy canalso account for the finding that some students are lesssuccessful than others in differentiating relevant from irrel-evant information in word problems (Littlefield & Rieser,1993; Low & Over, 1989; 1990).

These results offer useful educational implications. Theyshow that less successful problem solvers do not switch toa more meaning-based strategy with brief practice alone. Ifunsuccessful problem solvers are prone to use a short-cutcomprehension strategy, they need a reason to change to amore meaning-based strategy. Unfortunately, a direct-trans-lation strategy may be effective for many of the wordproblems they are asked to solve within the context ofschool mathematics so that these students never developedthe problem-model approach in school and persist in usingthe direct-translation approach as adults. Thus, a first step incomprehension strategy instruction is to present studentswith problems that help them see that direct translation doesnot work well for some problems. A second step is toprovide instruction in a method that emphasizes understand-ing the situation described in the problem, such as theproblem-model strategy. Lewis (1989) successfully usednumber-line diagrams to help students acquire a problem-model strategy for comprehending word problems. In con-clusion, we have demonstrated the importance of addingdomain-specific comprehension strategies to the arsenal ofskills taught via cognitive strategy instruction (Pressley,1990).


References

Anderson, J. R. (1983). The architecture of cognition. Cambridge,MA: Harvard University Press.

Briars, D. J., & Larkin, J. H. (1984). An integrated model of skillin solving elementary word problems. Cognition and Instruc-tion, 1, 245-296.

Campbell, J. I. D. (Ed.). (1992). The nature and origins of math-ematical skills. Amsterdam: Elsevier Science Publishers.

Cardelle-Elawar, M. (1992). Effects of teaching metacognitiveskills to students with low mathematics ability. Teaching &Teacher Education, 8, 109-121.

Chi, M. T. H., Glaser, R., & Farr, M. J. (Eds.). (1988). The natureof expertise. Hillsdale, NJ: Erlbaum.

Clark, H. H. (1969). Linguistic processes in deductive reasoning.Psychological Review, 76, 387-404.

Cummins, D., Kintsch, W., Reusser, K., & Weimer, R. (1988). Therole of understanding in solving word problems. Cognitive Psy-chology, 20, 405-438.

De Corte, E., Verschaffel, L., & Pauwels, A. (1990). Influence ofthe semantic structure of word problems on second graders' eyemovements. Journal of Educational Psychology, 82, 359-365.

Dossey, J. A., Mullis, I. V., Lindquist, M. M., & Chambers, D. L.(1988). The mathematics report card: Are we measuring up?Princeton, NJ: Educational Testing Service.

Ehrlich, K., & Rayner, K. (1983). Pronoun assignment and seman-tic integration during reading: Eye movements and immediacyof processing. Journal of Verbal Learning and Verbal Behavior,22, 75-87.

Ericsson, K. A., & Smith, J. (Ed.). (1991). Toward a generaltheory of expertise. Cambridge, England: Cambridge UniversityPress.

Hegarty, M., Mayer, R. E., & Green, C. E. (1992). Comprehensionof arithmetic word problems: Evidence from students' eye fix-ations. Journal of Educational Psychology, 84, 76-84.

Hinsley, D., Hayes. J. R., & Simon, H. A. (1977). From words toequations. In P. Carpenter & M. Just (Eds.), Cognitive processesin comprehension (pp. 89-106). Hillsdale, NJ: Erlbaum.

Johnson-Laird, P. (1983). Mental models: Towards a cognitivescience of language, inference, and consciousness. Cambridge,MA: Harvard University Press.

Just, M. A., & Carpenter, P. A. (1987). The psychology of readingand language comprehension. Newton, MA: Allyn & Bacon.

Kintsch, W., & Greeno, J. G. (1985). Understanding and solv-ing word arithmetic problems. Psychological Review, 92, 109-129.

Lapointe, A. E., Mead, N. A., & Phillips, G. W. (1989). A worldof differences: An international assessment of mathematics andscience. Princeton, NJ: Educational Testing Service.

Lewis, A. B. (1989). Training students to represent arithmeticword problems. Journal of Educational Psychology, 81, 521-531.

Lewis, A. B. & Mayer, R. E. (1987). Students miscomprehensionof relational statements in arithmetic word problems. Journal ofEducational Psychology, 79, 363-371.

Littlefield, J., & Rieser, J. J. (1993). Semantic features of similarityand children's strategies for identification of relevant informa-tion in mathematical story problems. Cognition & Instruction,11, 133-188. ^

Low, R., & Over, R. (1989). Detection of missing and irrelevantinformation within algebraic story problems. British Journal ofEducational Psychology, 59, 296-305.

Low, R., & Over, R. (1990). Text editing of algebraic wordproblems. Australian Journal of Psychology, 42, 63-73.

Mayer, R. E. (1981). Frequency norms and structural analysis ofalgebra story problems. Instructional Science, 10, 135-175.

Mayer, R. E. (1982). Memory for algebra story problems. Journalof Educational Psychology, 74, 199-216.

Mayer, R. E. (1989). Introduction to special section on cognitionand instruction. Journal of Educational Psychology, 81, 452-556.

Mayer, R. E. (1992). Thinking, problem solving, cognition (2nded.). New York: Freeman.

Mayer, R. E., Lewis, A. B., & Hegarty, M. (1992). Mathematicalmisunderstandings: Qualitative reasoning about quantitativeproblems. In J. I. D. Campbell (Ed.), The nature and origins ofmathematical skills (pp. 137-154). Amsterdam: Elsevier Sci-ence Publishers.

McKnight, C. C, Crosswhite, F. J., Dossey, J. A., Kifer, E.,Swafford, J. O., Travers, K. J., & Cooney, T. J. (1987). Theunderachieving curriculum: Assessing U. S. school mathematicsfrom an international perspective. Champaign, IL: Stipes.

Nathan, M. J., Kintsch, W., & Young, E. (1992). A theory ofalgebra word problem comprehension and its implications forthe design of learning environments. Cognition and Instruction,4, 329-390.

Newell, A., & Simon, H. A. (1972). Human problem solving.Englewood Cliffs, NJ: Prentice-Hall.

Okamoto, Y. (1992). A developmental analysis of children'sknowledge and processing of arithmetic word problems. Unpub-lished doctoral dissertation, Stanford University.

Pressley, M. (1990). Cognitive strategy instruction that reallyimproves children's academic performance. Cambridge, MA:Brookline Books.

Reed, S. K. (1987). A structure-mapping model for word prob-lems. Journal of Experimental Psychology: Learning, Memory,and Cognition, 13, 124-139.

Reed, S. K. (1993). Imagery and discovery. In B. Roskos-Ewoldson, M. J. Intons-Peterson, & R. E. Anderson (Eds.),Imagery, creativity, and discovery (pp. 287-312). Amsterdam:North-Holland.

Riley, M. S., & Greeno, J. G. (1988). Developmental analysis ofunderstanding language about quantities and of solving prob-lems. Cognition and Instruction, 5, 49-101.

Riley, M. S., Greeno, J. G., & Heller, J. I. (1983). Development ofchildren's problem-solving ability in arithmetic. In H. P.Ginsburg (Ed.), The development of mathematical thinking (pp.153-196). San Diego, CA: Academic Press.

Robitaille, D. F., & Garden, R. A. (1989). The IEA study ofmathematics II: Contexts and outcomes of school mathematics.Oxford, England: Pergamon Press.

Schoenfeld, A. H. (1985). Mathematical problem solving.Orlando, FL: Academic Press.

Schoenfeld, A. H. (Ed.). (1987). Cognitive science and mathemat-ics education. Hillsdale, NJ: Erlbaum.

Siegler, R. S., & Jenkins, E. (1989). How children discover newstrategies. Hillsdale, NJ: Erlbaum.

Smith, M. U. (Ed.). (1991). Toward a unified theory of problemsolving: Views from the content domains. Hillsdale, NJ:Erlbaum.

Stern, E. (1993). What makes certain arithmetic word problemsinvolving the comparison of sets so difficult for children. Jour-nal of Educational Psychology, 85, 7-23.

Steinberg, R. J., & Frensch, P. A. (Ed.). (1991). Complex problemsolving: Principles and mechanisms. Hillsdale, NJ: Erlbaum.


Stevenson, H. W., & Stigler, J. W. (1992). The learning gap. NewYork: Summit Books.

Stigler, J. W., Lee, S-Y., & Stevenson, H. W. (1990). Mathemat-ical knowledge of Japanese, Chinese, and American elementaryschool children. Reston, VA: National Council of Teachers ofMathematics.

van Dijk, T. A., & Kintsch, W. (1983). Strategies of discoursecomprehension. New York: Academic Press.

Verschaffel, L., De Corte, E., & Pauwels, A. (1992). Solvingcompare problems: An eye movement test of Lewis and Mayer'sconsistency hypothesis. Journal of Educational Psychology, 84,85-94.

Appendix A

Solution Procedures of Successful and Unsuccessful Students on an Inconsistent Problem

Problem

At Lucky, butter costs 65 cents per stick.This is 2 cents less per stick than butter at Vons.If you need to buy 4 sticks of butter,how much will you pay at Vons?

Successful Procedure

.65 + .02 = .67 .67 X 4 = 2.58

Unsuccessful Procedure

.65 - .02 = .63 .63 X 4 = 2.52

Appendix B

Problem Solving Test

1. Mark drove for 2 hours at 35 mph, and for 3 hours at 45mph. How far did he drive?

2. At June Lake, Scott catches 3 fish every hour. His friendAnne catches 3 fish every hour. Together, how many fishwill they catch after 4 hours of fishing?

3. [Target Problem 1: Gas Problem.]4. Each month Marie pays $520 for her rent and $24 for her

electricity. How much is the combined cost of her rent andelectricity for one year?

5. In a 15-yard-long pool Mitchell swims one length in 20seconds. At this rate, how many yards does Mitchell swimin 1 minute?

6. [Target Problem 2: Butter Problem.]7. Right now Ricardo has 14 books checked out from the

library. If he returns 4 books per week and doesn't checkout any new books, how many books will he have left in 2weeks?

8. A housing complex that has 25 apartments charges $650monthly rent for each apartment. How much does the com-plex receive each month in rent?

9. [Target Problem 3: Package Delivery Problem.]10. Herman studies 2 hours every weekday (Monday through

Friday) and a total of 3 hours over the weekend. How manyhours does he spend studying during a 7-week period?

11. [Target Problem 4: Fast Food Restaurant Problem.]12. Greg's living room is 9 yards by 7 yards and he has selected

carpeting that sells for $15 per square yard. How much willit cost him to carpet his living room floor?

Appendix C

Protocol of an Unsuccessful Student Solving an Inconsistent Problem: Lucky (Less, Inconsistent)

1: At Lucky, butter costs 65 cents per stick2: This is 2 cents less per stick than butter at Vons/butter3: If you need to buy 4 sticks of butter/4 sticks of butter4: How much will you pay at Vons3: sticks/44: How much will you pay at Vons3: 4 sticks of butter/you1: 65 cents per2: 2 cents less per1: 652: cents3: 4 sticks2: 2 cents1: 65 cents3: sticks/buy2: cents

1: cents/65 cents2: stick/per/less/cents1: 65 cents2: 2 cents1: 65 cents3: 4 sticks1: 65 cents3: 4 sticks2: 2 cents1: 65 cents2: less3: 4 sticks2: cents less1: 65 cents3: 4 sticks2: cents

(Appendixes continue on next page)


1: 65 cents2: cents/21: 65 cents per/butter2: is 2 cents less per stick3: to buy 4 sticks2: cents

1: 65/butter costs 65 cents per stick2: 2 cents less per3: sticks/4 sticks of/4 sticks/4 sticks of butter/44: you pay/you/much will you pay at Vons/pay/you1: 65 cents/65

Appendix D

Protocol of a Successful Student Solving an Inconsistent Problem: Lucky (Less, Inconsistent)

1: At Lucky, butter costs 65 cents per stick/costs 65 cents per2: This is 2 cents less per stick than butter costs at Vons/than

butter costs at Vons/cents less per stick1: butter costs/Lucky, butter costs 65 cents per stick2: less3: to buy 4 sticks of butter/need to buy 4 sticks of butter4: you pay1: 65 cents2: less per stick than butter at Vons/less per3: buy 4 sticks of butter4: Vons2: less per stick than

1: 65 cents per2: than/less per3: 4 sticks of butter2: stick/less per3: buy 4 sticks of butter4: Vons/will you pay at Vons

Received July 9, 1993Revision received September 12, 1994

Accepted September 19, 1994

MEMBERS OF UNDERREPRESENTED GROUPS:REVIEWERS FOR JOURNAL MANUSCRIPTS WANTED

If you are interested in reviewing manuscripts for APA journals, the APA Publications and Commu-nications Board would like to invite your participation. Manuscript reviewers are vital to thepublication process. As areviewer, you would gain valuable experience in publishing. The P&C Boardis particularly interested in encouraging members of underrepresented groups to participate more in thethis process.

If you are interested in reviewing manuscripts, please write to Leslie Cameron at the address below.Please note the following important points:

• To be selected as a reviewer, you must have published articles in peer-reviewed journals. Theexperience of publishing provides areviewer with the basis for preparing a thorough, objectivereview.

• To select the appropriate reviewers for each manuscript, the editor needs detailed information.Please include with your letter your vita. In your letter, please identify which APA journal youare interested in and describe your area of expertise. Be as specific as possible. For example,"social psychology" is not sufficient—you would need to specify "social cognition" or "attitudechange" as well.

• Reviewing a manuscript takes time. If you are selected to review a manuscript, be prepared toinvest the necessary time to evaluate the manuscript thoroughly.

Write to Leslie Cameron, Journals Office, American Psychological Association, 750 First Street, NE,Washington, DC 20002-4242.

Documents

Comprehension of Arithmetic Word Problems: A Comparison of ... · The specific approach used in this study is to observe the comprehension processes of students whom we assume have