How much of symbolic manipulation is just symbol pushing?

David H. Landy (dhlandy@gmail.com) Psychology, 603 E. Daniel Street

University of Illinois, Urbana-Champaign, IL 61820

Robert L. Goldstone (rgoldsto@indiana.edu) Psychological and Brain Sciences, 1101 E. 10th Street

Indiana University, Bloomington, IN 47405

Abstract

This paper explores the hypothesis that schematic abstraction—rule following—is partially implemented through processes and knowledge used to understand motion. Two experiments explore the mechanisms used by reasoners solving simple linear equations with one variable. Participants solved problems displayed against a background that moved rightward or leftward. Solving was facilitated when the background motion moved in the direction of the numeric transposition required to solve for the unknown variable. Previous theorizing has usually assumed that such formal problems are solved through the repeated application of abstract transformation patterns (rules) to equations, replicating the steps produced in typical worked solutions. However, the current results suggest that in addition to such strategies, advanced reasoners often employ a mental motion strategy when manipulating algebraic forms: elements of the problem are “picked up” and “moved” across the equation line. This demonstration supports the suggestion that genuinely schematic reasoning could be implemented in perceptual-motor systems through the simulated transformation of referential (but physical) symbol systems.

Keywords: Symbolic reasoning; formal reasoning; high-level cognition; mathematical cognition

Introduction Reasoning over abstractions—schemas, rules with variables, or hard to perceive generalities—is a skill which seems on its face to require specialized cognitive structures (Anderson, 2005; Markman & Dietrich, 2000; Sloman, 1996). One common conception of this specialized architecture is that the human mind operates over a set of internal symbols and variables much like external formal languages (Anderson, 2007; Fodor, 1975; Marcus, 2001). On this perspective, mental structures that perform symbolic manipulations are precursors to and ingredients of cognition. A frequently articulated alternative to this view is that schematic rules may be implemented via non-symbolic or incompletely symbolic perceptual manipulations and simulations (Barsalou, 1999; Clark, 1998; Dennett, 1994).

Often, such perceptual symbol systems have been conceptualized as simulations of semantic situations picked out by a symbolic form or formalism (Barsalou 2008). Under this conceptualization, symbol systems such as natural or formal languages can play either of two roles. First, they might provide the seeds for a perceptual simulation of the situation referred to in an utterance (e.g.,

simulating an eagle when reading the word “eagle”, Zwaan et al, 2004). The present paper explores a second possible mechanism for implementing symbolic reasoning in perceptual-motor action that rely on simulating the perceptual-motor environment associated with the physical form of the notation itself. Recently, it has been suggested that formal languages, and mathematical languages in particular, often serve as diagrams whose analog physical structure relates systematically to mathematical or formal truths. Therefore, treating the formal notations as images directly suitable for perceptual-motor processing provides a way to implement abstract cognition in perceptual-motor systems (Dörfler, 2002; Landy & Goldstone, 2007a). Naturally, the results of such processes must then be translated into the appropriate referent situation. Furthermore, symbolic reasoning achieved via the perception and manipulation of physical notations must respect the constraints of systems which may differ substantially from those that would be expected from a Language of Thought (Endress, et al, 2005; Landy and Goldstone, 2007b; Novick and Catley, 2007; Pothos, 2006). In this work, we argue that proofs are often physically designed so that they appeal to processing systems typically used for dynamic events, and we evaluate one possible strategy for solving a standard class of algebra problems (“solve for x”) involving simulating the motion of the elements of the notation used to express these problems.

Ways to solve single-variable equations Table 1 displays a standard demonstration that the equation y*3+2=8 has solution y=2. The justification for lines 2 and 4 comes from the Euclidean notion that things done to like things yield like things, and therefore that if an equation

�

X =Y is valid, then the equation

�

X A =Y A is also valid, for any operation

�

or value

�

A . How are such processes carried out by human reasoners?

Conceptually, there are (at least) two good strategies for solving such problems. In an algebraic solution, a reasoner constructs the solved equation shown at the bottom of table 1, and then uses straightforward arithmetic to generate the answer. In the unwind strategy, one starts by finding the isolated constant, identifies the next available operation on the variable side (+2 in this case), inverts the operation, and solves the resulting problem (8-2). One then uses this number as the starting point, identifies the next available operations on the left, and repeats.

It is possible to capture both of these processes using rewrite rules—essentially an internal resource which mirrors our formal understanding of proof. Such a rule specifies that if the current expression matches a particular pattern, then a particular result may be produced. Such a rule might be written as

�

X =Y X + A =Y + A. Rewrite laws are highly general constructs, capable of representing any inferential system (including simple algebra), represented in any physical format. In such an approach to reasoning, learning how to solve problems is a largely matter of learning which rewrite transformations are legal and appropriate to a particular situation. Learning systems that depend on rewrite laws operate in a manner that does not depend much on the particular representation language a rule is expressed in. Such strategies have been employed in attempts to capture the axioms of legal mathematical syntax; a similar approach has successfully captured many interesting components of equation solution (Anderson et al. 2005; note that ACT-R uses a production system, rather than a rewrite system. Production systems are more general than rewrite systems; however, ACT-R models of algebra have employed rewrite-like productions, so the difference here is minimal).

A second way to implement an algebraic equation solution is to make use of the actual, analog, visual properties of the mathematical language rather than the abstract formalism (Dörfler, 2002; Landy & Goldstone, 2007b). Transformations, on this conceptualization, involve applying processes used to understand physical space to understand mathematical derivations. Learning the practice of mathematical proof is a matter, then, of learning physical constraints on the way parts of an equation can move and transform, akin to learning physical constraints on the motion of real objects. Such strategies will here be referred to as “flipbook strategies,” because they assume that the cognitive processes that connect and animate successive pages of a flipbook also connect successive proof lines. A natural flipbook strategy for accomplishing the isolation of the variable term in an equation is to treat terms as moving across the equation line to the other side. This strategy is suggested by the metaphorical contraction of Steps 2 and 3 in Table 1 as “moving the 2 from the left to right side of the equal sign.” We suggest that this strategic language reflects literal truth about the resources used to solve such problems; these resources might be identical to low-level perceptual or motor processes involved in perceiving, imagining, and

causing real motion in the world, or alternatively they might comprise the metaphorical application of high-level knowledge about motion events to notational mathematical forms (Casasanto and Boroditsky, 2008). These two possibilities will not be distinguished in this paper, but will both be captured under the general label, “motion strategy.” An alternative flipbook strategy is to consider lines as paired creation/destruction events. Euclid’s law, in this framing, demands that (top-level) creation events be paired across equation lines.

It is worth noticing that different operations create different burdens in the unwind strategy. In the equation in Table 1, one must remember to subtract 2 from 8 (and not 8 from 2). In the equation y-3=6, however, one must merely remember to add 3 and 6 (regardless of order). The need to maintain operand order on problems which display commutative operations (and therefore involve the computation of non-commutative operations) should make the shortcut strategy more difficult on such problems, encouraging motion and Euclidean strategies. The operation

�

is commutative if A

�

B always equals B

�

A. This is compatible with Barsalou’s argument that people only resort to perceptual simulations when they are given difficult problems that cannot be solved using simple cognitive shortcuts based on unstructured association. By both accounts, perceptually grounded simulations are employed when structure-sensitive responses are required. Perceptual processes, in this view, underlie sophisticated, not superficial, reasoning.

If reasoners using formal notations commonly use flipbook strategies, this has substantial practical implications for our conception of formal reasoning. The primary purpose of this paper is not to explore those implications in depth, but to lay the groundwork by investigating a prediction made by such transformative strategies. The question is not whether people metaphorically describe formal proofs as temporal events—they patently do; rather the question is whether that metaphorical language affects or reflects actual cognitive processes sometimes directly involved in equation solution. In this paper, we attempt to selectively interfere with (and facilitate) motion strategies by asking participants to solve problems presented against a moving background. There is no particular reason to expect either the Euclidean or the unwind shortcut to be affected by background motion.

Table 1: Using Euclid’s axioms of equality to isolate y Statement Reason

Given

Subtract 2 from both sides

Arithmetic Simplification

Divide both sides by 3

Arithmetic Simplification

Figure 1: Illustration of the motion strategy, in

which the objects in an equation are rearranged via continous movements, rather than the sequential application of rules.

However, given that motion strategies share processing resources with motion recognition, they should be selectively affected by actual perceived motion. Motion in the same direction as the motion of the variable would be expected to facilitate equation solving, while motion in the opposite direction should interfere with equation solving using the motion strategy. Thus, a moving background can be used to reveal the kinds of situations (if any) in which people utilize resources dedicated to processing motion, to manipulate mathematical expressions.

Experiment 1

Method Seventy-two undergraduates attending Indiana University received partial course credit in exchange for participation. One participant was removed from the analysis due to extremely poor performance.

Materials and Procedure: Stimuli consisted of single-operation algebra problems, expressed in standard mathematical notation. In each, one side consisted of a single number, between 1 and 36, selected so that there would be a wide and relatively uniform range, and so that the eventual solution would always be an integer. The other side consisted of an expression of the form

�

y M , where

�

was one of the four basic operations (“+”, “-“, “*”, or “/”), and M was an integer between 1 and 6. The problems were constructed so that the solutions were always positive integers. Each participant solved 140 such problems, equally divided across operations. In half of each set of problems, the variable appeared on the left side of the equation; in half, it appeared on the right. Finally, each of these conditions appeared equally often across 7 levels of background motion horizontal velocity.

The background was generated with 200 small black circles which were position above and below the equation

line. The circles moved with velocity v = X + C, where X was a uniformly distributed random variable in two dimensions (ranging from -10 to 10 cm/s), and C was a condition-dependent constant horizontal velocity. Seven uniformly spaced levels of C were chosen, ranging uniformly between strongly leftward (-12 cm/s) to strongly rightward (+12cm/s). The overall mean speed of the circle motions produced by this equation is not constant; the balls moved more quickly when C was very different from 0.

Participants were given brief written instructions. These instructions asked the participants to “solve for y” by moving symbols until y was isolated, and to respond by typing the resulting value of y into the keyboard. Motion language was deliberately used, both because this is a standard description of the problem solution, and because our goal here was to see whether people employed motion resources literally under any circumstances. Future research will explore whether instruction priming is necessary to induce motion affects.

Results The effect of compatibility of background motion on operations of differing commutativity was analyzed with a repeated-measures analysis of covariance (ANCOVA) with operation and variable side as independent variables and expected mean background velocity as the covariate. The analysis revealed a marginally significant interaction between background motion and variable side (F(1, 67)=3.98, p<=.05), such that more errors were made when the background moved toward the variable, rather than away from it. Furthermore, there was a significant three-way interaction between variable side, commutativity, and background motion, such that the effect of motion compatibility was greater for equations displaying commutative rather than non-commutative operations (F(1,67)=5.75, p<.05; see Figure 2).

Figure 2: Interaction between variable side and background motion, for equations displaying commutative (left) and non-commutative (right) operations. Lefthand panels display error proportions for each group; the righthand bars display 95% confidence intervals of the interaction contrast (calculated from the outer four points of each graph).

Figure 3: Results from Experiment 2. Error proportion is plotted across the interaction between variable side and background motion for equations displaying 2 (top), 1 (middle) or 0 commutative operations. Lefthand panels display error proportions for each group; the righthand bars display 95% oonfidence intervals of the within-participants interaction contrast (calculated from the outer four points of each graph).

Discussion Solutions to single operation arithmetic equations contained fewer errors when the perceived movement of the grating was congruent with the imagined motion specified by the motion strategy, particularly on problems requiring the participant to make non-commutative computations, suggesting that participants use the motion strategy particularly when the order of items matters.

Experiment 1 suggests that reasoners typically use imagery of or knowledge about dynamic, continuous motion to guide sequential computation; however, participants were tested only on very simple equations of only four types. Furthermore, participants’ mathematical expertise was not investigated, and so it is not clear whether motion strategies dominate among novices, mathematical experts, or both. Experiment 2 addresses both of these issues.

Experiment 2

Method Participants Fifty-eight undergraduates attending either the University of Illinois, or Indiana University received partial course credit in exchange for participation. Materials and Procedure Participants sat at a comfortable distance from a computer monitor (roughly 50cm). Stimuli consisted of 80 two-step algebra problems, expressed in standard mathematical notation. In each problem, one side consisted of a single number, between 1 and 63. The other side was an expression with the pattern

�

y M N , where

�

was either multiplication (“*”) or division (“/”),

�

was either addition or subtraction, and M and N were integers. M and N had values between 1 and 18, and were selected so that each step of the typical solution path of a problem involved only positive integers. The symbols were were displayed with a medium gray color. Participants solved each of the 80 equation three times: once with each background state. Thus, participants solved a total of 240 equations.

Each problem had two versions. In one, the variable appeared on the left; in the other, it appeared on the right; thus, there were 40 formally distinct problems. Problems were coded by whether they contained zero, one, or two commutative operations in their representation (remember that problems containing more commutative operations in their representation contain fewer commutative calculations in their solution). 10 problems contained no commutative operations, 20 contained one, and 10 contained two commutative operations.

Apparent background motion was produced using a moving sinusoidal grating. The grating occupied the entire screen, and had a spatial frequency of .53 cycles per centimeter. Gratings were oriented orthogonal to their movement, which was either leftward, rightward, or upward at a speed of 1.5Hz. Screen luminance was not controlled across monitors; the color of the symbols was medium gray; background color ranged from light gray to nearly white.

Information about participants’ level of mathematical background was collected after completion of the experiment. The least ambiguous criterion proved to be whether the student had taken a calculus course, so that measure was used to roughly partition the subjects into “mathematically experienced” and “mathematically inexperienced” groups. 41 participants had taken calculus, while 17 had not. Participant instructions were identical to those in Experiment 1.

Results As in Experiment 1, there was a significant three-way interaction between variable side, direction of background motion, and number of commutative operations, according to a 4-way ANOVA which included mathematical experience as a between-participants factor (F(1, 56)=6.03, p<.05, see Figure 3); as in Experiment 1, problems with two commutative operations were more affected by motion compatibility than those with one or zero commutative operations. Additionally, problems displaying fewer commutative operations were substantially more difficult for participants to solve (F(1, 56)=13.84, p<.001). The 2-way interaction between variable side and background motion was not significant (F(1,56)=0.124, p~.76).

Participants reporting taking a calculus course were more accurate across problem types (error rate for experienced participants, M = .1, for inexperienced M = .27, F(1, 56)=7.9, p<.01). Experienced participants were also marginally more affected by the compatibility of variable side and background motion (F(1,56)=3.91, p<=.05). Within problems featuring only commutative operations (those predicted to be most affected by background motion), motion compatible problems were solved more accurately than incompatible problems (F(1, 56)=4.78, p<.05); the 3-way interaction between math ability, background motion, and variable side was also significant (F(1, 56)=4.85, p<.05; see Figure 4). Thus, participants who had taken a calculus course were more affected by the compatibility of the

background than students without calculus experience. A planned comparison of the levels of operation commutability revealed that the 2-way effect of background motion and variable side differed significantly between problems of the y*N+M format and those with even a single non-commutative operation (F(1,56)=4.77, p<.05); while the two-way interaction of variable side and background motion was significant with all commutable-operation problems F(1,56)=8.0, p<.01), it was completely eliminated for problems containing just a single non-commutable operation (F(1,56)=0168, p=.89).

Discussion Experiment 2 replicated the major results of Experiment 1, in that formally irrelevant background motion affected error rates, but did not affect response times. As in Experiment 1, this was primarily true for problems involving additions and multiplications. Beyond showing that the influence of motion on algebraic problem solving generalizes to slightly more complex problems and to different kinds of backgrounds, Experiment 2 extends the results of Experiment 1 in two ways. First, it might have been expected from Experiment 1 that problems involving a single addition or multiplication operation would show some effect of background motion compatibility; instead, we found that only problems with all commutable operations showed such a compatibility effect.

Second, Experiment 2 revealed that participants with more experience with calculus were more affected by background motion than those without calculus experience. One might expect that students with higher mathematical skill would be more able to focus on mathematically relevant aspects of a problem, and ignore irrelevant components such as the moving background. Our interpretation is that the imagined motion strategy is an advanced strategy that students come to adopt through experience with formal notations, rather than a strategy that students initially use while learning, and then abandon as

Figure 4: Results from Experiment 2. The left figure displays the error rate across motion and equation types for students

who had taken at least one calculus course, the right figure for those who had not taken calculus.

their sophistication increases. This makes sense, under the flip-book hypothesis: the motion strategy results from perceived creation, destruction, and motion events in static mathematical forms, which becomes available as a result of repeated exposure to forms which afford that animation.

General Discussion Two experiments demonstrated that visual background motion interacts with mathematical operations in a rich and intricate manner. Reasoners solving a variety of one-operation and two-operation single-variable equations were affected by background motion presented simultaneously with equations. The effect was non-uniform: solving was facilitated when the motion was in the direction required by the imagined spatial transposition event, and impaired in the opposite direction. The effect was strongest when the operations to be performed were order-sensitive, and when the solver had a relatively high mathematical background.

The influence of real motion on the execution of algebraic transformations suggests that cognitive capacities related to real motion processing are being employed online to solve algebraic problems. One interpretation is that the construction of the appropriate algebraic solution is implemented via the continuous reconfiguring of the literal problem components. Another, less dramatic but still striking possibility is consistent with this evidence: the existence of the motion metaphor may prime sensitivity in rules which do not themselves derive from motion reasoning. That is, the rules may in some sense have started as abstractions, but acquired over use connections to physical motion. Such de-abstracted rewrite rules would have broadly the same properties as the application of motion-specific processes to mathematical symbols.

Flipbook strategies and abstract rewrite laws create similar mathematical systems from the formal perspective, but they differ in the way they individuate particulars in a proof; there is a gap between common language for solving for x and the formal rewrite laws that justify the steps. The latter do not specify token-token identity relationship between elements on successive proof lines. Consider the rewrite law

�

X + X =Y X + X =Y 2 + 2 . Rewrite rules cannot capture the intuition that the left X goes with the left X on successive lines; it treats all four as distinct tokens of the same type. Rewrite laws individuate particular symbols or expressions on the basis of their formal identicality rather than a history of transformations.

Flipbook strategies, on the other hand, naturally lend themselves to the historical individuation of particular elements.; two sub-expressions in successive equation lines are “the same thing” if one was built out of the other, regardless of whether they have the same literal form.

Most approaches to modeling mathematical reasoning presuppose a translation of the physical notation elements to a symbolic tree or proposition representation, and proceed to apply rules of transformation to those forms. Our results suggest that people frequently do what they intuitively think they do—move around notational elements in space.

Acknowledgments This research was funded by U.S. Department of Education, Institute of Education Sciences Grant R305H050116 National Science Foundation Grant 0527920, and AFOSR Grant # FA9550-07-1-0147.

References Anderson, J.R. (2005). Human symbol manipulation within

an Integrated Cognitive Architecture. Cognitive Science 29, 313-341

Anderson, J.R. (2007). How can the human mind occur in the physical universe? New York: Oxford University Press.

Barsalou, L. W. (1999). Perceptual symbol systems. Behavioral and Brain Sciences, 22, 577-660.

Barsalou, L.W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617-645.

Casasanto, D. & Boroditsky, L. (2008). Time in the Mind: Using space to think about time. Cognition, 106, 579-593. Clark, A. (1998). Being there: Putting brain, body, and world together again. Cambridge, MA: MIT Press.

Dennett, D.C. (1994). The role of language in intelligence, in Jean Khalfa (ed), What is Intelligence? Cambridge: Cambridge University Press.

Dorfler, W. (2002). Formation of mathematical objects as decision making. Mathematical Thinking and Learning,

4, 337-350. Endress, A. D., Scholl, B. J., & Mehler, J. (2005). The role

of salience in the extraction of algebraic rules. Journal of Experimental Psychology: General, 134, 406-419.

Fodor, J. A. (1975). The language of thought. NY: Crowell. Landy, D. & Goldstone, R. L. (2007A). Formal notations

are diagrams: Evidence from a production task. Memory & Cognition 35(8).

Landy, D. & Goldstone, R. L. (2007B). How abstract is symbolic thought?. Journal of Experimental Psychology: Learning, Memory, and Cognition 33(4), 720-733.

Marcus, G. F. (2001). The algebraic mind: Integrating connectionism and cognitive science. Cambridge, MA: MIT Press.

Markman, A. B. & E. Dietrich (2000b). Extending the Classical View of Representation. Trends in Cognitive Science, 4(12), 470-475.

Novick, L. R., & Catley, K. M. (2007). Understanding phylogenies in biology: The influence of a Gestalt perceptual principle. Journal of Experimental Psychology: Applied, 13, 197-223.

Pothos, E. M., Chater, N., & Ziori, E. (2006). Does stimulus appearance affect learning? American Journal of Psychology 119(2) pp. 277-301.

Sloman, S. A. (1996). "The Empirical Case for Two Systems of Reasoning." Psychological Bulletin, 119(1), 3-22.

Zwaan, R.A., Madden, C.J., Yaxley, R.H., & Aveyard, M.E. (2004). Moving words: Dynamic mental representations in language comprehension. Cognitive Science, 28, 611-619.