Cross-context look at upper-division student difficulties with integration

Bethany R. Wilcox and Giaco CorsigliaDepartment of Physics, University of Colorado Boulder, Boulder, Colorado 80309, USA

(Received 1 August 2019; published 10 October 2019)

We investigate upper-division student difficulties with direct integration in multiple contexts involvingthe calculation of a potential from a continuous distribution (e.g., mass, charge, or current). Integration is atool that has been historically studied at several different points in the curriculum including introductoryand upper-division levels. We build off of these prior studies and contribute additional data around studentdifficulties with multivariable integration at two new points in the curriculum: middle-division classicalmechanics and upper-division magnetostatics. To facilitate comparisons across prior studies as well as thecurrent work, we utilize an analytical framework that focuses on how students activate, construct, execute,and reflect on mathematical tools during physics problem solving (i.e., the ACER framework). Using amixed-methods approach involving coded exam solutions and student problem-solving interviews, weidentify and compare students’ difficulties in these two different contexts and relate them to what has beenfound previously in other levels and contexts. We find that some of the observed student difficulties werepersistent across all three contexts (e.g., identifying integration as the appropriate tool, and expressing thedifference vector), while other difficulties seemed to fade as students advanced through the curriculum(e.g., expressing differential line, area, and volume elements). We also identified new difficulties thatappear in different contexts (e.g., interpreting and expressing the current density). These findings representone of the first cross-context looks at students’ difficulties with integration across multiple courses in thephysic curriculum. These results can help instructors and researchers focus instruction and materialsdevelopment on the aspects of integration that are most challenging in a particular context or persistentacross multiple contexts.

DOI: 10.1103/PhysRevPhysEducRes.15.020136

I. INTRODUCTION

Physics education research (PER) as a field has a longhistory of conducting student difficulties research focused onidentifying the challenges and problems that students facewhen dealingwith specific physics concepts ormathematicaltools [1,2]. The findings from these studies have historicallybeen used to inform the development of curricular materials,pedagogical approaches, or classroom practice in order tospecifically target these challenging areas to directly addressand overcome known student difficulties [2,3]. One clearexample of this process was the creation of the “Tutorialsin Introductory Physics” [4], which were developed basedon extensive investigations of specific student difficultieswith introductory physics content and have been shown toimprove student learning gains [5].Historically, studies of student difficulties have been

performed in relation to a single concept, topical area, ormathematical tool and within the context of a single course.

However, the cyclic nature of the physics curriculum meansthat students often encounter individual concepts and math-ematical tools multiple times, and in multiple contexts, overthe course of their undergraduate career. It is reasonable toassume that over this time period, and with multiple expo-sures, students difficulties with these concepts and tools maynaturally evolve. For example, some difficulties faced earlyon may resolve themselves over time as students see theseideasmultiple times in the standard curriculumandgainmoreexperience, while other difficulties may be more persistent,reappearing despite repeated exposures. Alternatively, as thecomplexity of the physics content associatedwith these topicsand tools increases, new difficulties not present in earliercontexts may also begin to appear.Understanding the nature of how student difficulties with

common concepts andmathematical tools change as studentsare exposed to these ideas in newcontexts can have importantimplications for both instructors and researchers. By attend-ing to how students’ ideas evolve, we can avoid spendingtime and effort on developing new materials to targetdifficulties that are already being sufficiently addressed inthe standard curriculum in favor of focusing on difficultiesthat persist across multiple contexts and exposures. Yet,despite the potential advantages, investigations of studentdifficulties with specific topics or tools across multiple

Published by the American Physical Society under the terms ofthe Creative Commons Attribution 4.0 International license.Further distribution of this work must maintain attribution tothe author(s) and the published article’s title, journal citation,and DOI.

PHYSICAL REVIEW PHYSICS EDUCATION RESEARCH 15, 020136 (2019)

2469-9896=19=15(2)=020136(14) 020136-1 Published by the American Physical Society

exposures is rare within the PER literature. In lieu ofcomprehensive investigations looking at the evolution ofstudent difficulties over time, another option for investigatingthis dynamic would be to make explicit comparisonsbetween distinct studies investigating the same topic or toolat different levels of the curriculum. However, such cross-study comparisons would be difficult without a guidingframework to facilitate making connections across differentstudies. One framework that was created to provide just sucha guiding structure is the ACER framework [6], whichfocuses on how students activate, construct, execute, andreflect on mathematical tools in the context of physicsproblem solving [7].One example of student difficulties research that focuses

on a mathematical tool that appears multiple times through-out the undergraduate curriculum is direct integration.Here, we use “direct integration” to refer to the process ofcalculating a physical quantity (e.g., the gravitational poten-tial) by directly summing the contributions to that quantityfrom each “bit” of a physical distribution (e.g., a threedimensional mass distribution). We build on the existingresearch on students difficulties with direct integration toconstruct a pseudolongitudinal and cross-context view ofhow students’ difficulties evolve over time. To accomplishthis, we first review the existing literature on studentdifficulties with direct integration at all levels of the cur-riculum (Sec. II). We then build on this work by presentingmethods (Sec. III) and findings (Sec. IV) from a pseudo-longitudinal investigation of student difficulties with directintegration in middle-division classical mechanics (in thecontext of gravitation) and upper-division electricity andmagnetism (in the context of both the scalar and vectorpotential). Throughout this paper, the ACER frameworkprovides a consistent structure that facilitates making com-parisons across topical areas as well as to previous studies.

II. BACKGROUND: STUDENT DIFFICULTIESWITH DIRECT INTEGRATION

Here, we discuss prior work investigating student diffi-culties with direct integration. Throughout this section, theACER framework will be used to facilitate comparisons offindings across these diverse studies. As such, we beginwith a summary of our own prior work using the ACERframework to investigate students use of direct integra-tion in the context of upper-division electrostatics andthen continue to discuss comparisons with work in othercontexts and at other levels.

A. The ACER framework and an examplein upper-division electrostatics

The ACER framework is an analytical frameworkdeveloped to help structure and organize investigationsof students’ difficulties when utilizing sophisticated math-ematical tools in the context of physics problem solving [7].

The framework considers four main components present inthis type of mathematically involved problem solving:activation of the mathematical tool, construction of themathematical model, execution of the mathematics, andreflection on the results. These components were identifiedby studying the general structure of expert problem solvingusing modified task analysis [8]. The framework itself isgrounded in both a resources [9] and an epistemic framing[10] perspective on the nature of knowledge and the pro-cess of learning. While the broad structure of the ACERframework was designed to be applicable across contextsand mathematical tools, the framework was also designed tobe operationalized for use with specific topics and tools.This operationalization process is important to ensure thatthe framework captures the more tool- and context-specificaspects of students’ difficulties. The specific operationali-zation of the ACER framework for the contexts of gravi-tation and the vector potential will be discussed in greaterdetail in Sec. III B. For the purposes of cross-studycomparisons, we will focus categorizing students’ difficul-ties within the general components of the framework (i.e.,activation, construction, execution, and reflection).One of the investigations which prompted and informed

the development of the ACER framework was that of directintegration in the context of upper-division electrostatics,specifically the calculation of the electric potential via theintegral form of Coulomb’s law [7]. This initial study laid thefoundation for the work described in the remainder of thismanuscript and features very similar contexts and method-ologies including quantitative analysis of students’ examsolutions as well as qualitative analysis of interview data.From this analysis, we identified two broad categories ofdifficulties that the study population exhibited with integra-tion in the context of Coulomb’s law. The first was difficultyidentifying direct integration as the appropriate solutionmethod (i.e., difficulty with the activation component).This difficulty was exhibited by roughly a quarter of thestudent population [7]. The second was difficulty operation-alizing the Coulomb’s law integral for the specific physicalsituation given (i.e., difficulty with the construction compo-nent). The twomost common difficulties observedwerewithcorrectly expressing the differential charge element, dq,and the difference vector (i.e., Griffiths’ “script r”) in away consistent with the geometry of the given physicalsituation. These two difficulties were each exhibited byroughly half of the students [7]. Additionally, we found thatvery few students in our population showed explicit andspontaneous attempts to interpret or check their solutions(i.e., operating within the reflection component) [7].

B. Interpreting other prior work throughthe ACER framework

In addition to our own prior work investigating studentdifficulties with direct integration in the context of upper-division electrostatics, a number of others have investigated

WILCOX and CORSIGLIA PHYS. REV. PHYS. EDUC. RES. 15, 020136 (2019)

020136-2

student difficulties in introductory courses. Here, wesummarize this prior work from the perspective of theACER framework to facilitate cross-study comparisons.It is important to note that these studies were not designedor conducted using the ACER framework, and often hadgoals that went beyond pure investigations of students’difficulties (e.g., theoretical development such as under-standing students’ resource activation or conceptual blend-ing); however, for the purposes of this summary, we focuson the aspects of these findings that directly relate tostudent difficulties with integration as a mathematical toolin physics problem solving.Many of the existing studies on student difficulties with

integration were conducted in the context of introductoryphysics courses. For example Nguyen and Rebello [11]examined student difficulties when calculating the electricfield from a straight or curved line segment. They focusedexplicitly on understanding how students recognized theneed for integration and how they set up and then computedthe integrals [11]. From the point of view of ACER, thisframing aligns well with a focus on the activation andconstruction components of the framework as well as,potentially, the execution component. With respect toactivation, Nguyen and Rebello found that students oftenused simple recall of similar problems to activate integra-tion as the correct mathematical tool. In questions wherethey had less experience with similar problems, studentshad more difficulty identifying the need for a integral, andthose who did were most often cued by the presence of anonconstant value in the prompt. With respect to con-struction, the two primary difficulties focused around theinfinitesimal quantity and accounting for the vector natureof the electric field. Nguyen and Rebello noted that manystudents struggled to recognize the need for, and physicalmeaning of, the infinitesimal quantity as well as to expressit in a useful way (e.g., expressing dq as λdx). Finally,with respect to execution, Nguyen and Rebello documentedsome difficulties related to maintaining an awareness ofthe physical meaning of the symbols while performingthe integrals. They also noted some difficulties withrespect to the actual mechanics of computing the integrals(e.g., u substitutions).Meredith and Marrongelle [12] also conducted inter-

views with students solving several problems requiringintegration in the context of introductory electrostatics,including one asking for the electric field near a bar ofcharge. The focus of this work was on identifying students’mathematical resources when solving integrals; however,they also documented a number of difficulties that theirinterview students had working through these types ofquestions. In terms of activation, they found that recogniz-ing the need for an integral in the case of the bar of chargewas a challenge for as many as half of their students, withmany of these students instead treating the bar as a pointcharge at the rod’s center. For students who successfully

identified the need for integration, Meredith andMarrongelle identified three possible cues commonly used:recall of prior examples, identification of a term or variableon which the quantity in question depended, or theconceptualization of a building up a whole by summingup smaller constituent parts. In terms of construction,Meredith and Marrongelle also observed difficulties withthe infinitesimal, finding that students sometimes incor-rectly used the idea of dependence on a particular variableto simply assume that would be the variable of integrationwithout consideration of the physical meaning of theintegrand.Both of the two studies discussed above identified

difficulties with the infinitesimal or differential term inthe larger context of solving a physics problem involvingintegration, an element that appears in the constructioncomponent of the ACER framework. Others have targetedthis issue more directly. For example, Hu and Rebello [13]conducted an interview study with introductory physicsstudents in which they focused specifically on students’applications of differentials when calculating the electricfield from a charged bar. They identified different math-ematical resources and conceptual metaphors that studentsused when considering the need for, and physical meaningof, the differential term in these integrals. Similarly, Amosand Heckler [14] investigated the relationship betweenintroductory students’ understanding of differentials (e.g.,dx), differential products (e.g., λdx), and integrals. Theyfound that understanding of differentials alone did notrelate strongly to performance on integral problems, butrather required an additional step of explicitly incorporatingideas about differential products.Others, rather than focusing on the construction compo-

nent, focused instead on the activation component throughinvestigations of how students identify integration as thecorrect mathematical tool for a particular physics problem.To investigate this, Savelsbergh et al. [15] used problemsorting tasks with introductory physics students whichasked them to determine the correct approach to variousproblems relating to electricity and magnetism; several ofthese tasks included problems for which direct integrationvia Coulomb’s law or the Biot-Savart law were the correctapproach. They found that, despite knowing the possiblesolution types for these problems, students had difficultymapping the problem at hand onto one of the knownproblem types. In terms of the ACER framework, theseresults would indicate that even if a student can produc-tively recognize or replicate work in the construction orexecution components, they may still struggle to identifythe correct mathematical tool in the activation component.Taken together, the studies described above tell a

relatively coherent story with respect to the difficultiesstudents at the introductory level face with multivariableintegration in the context of, for example, calculating theelectric field from a charge distribution. These difficulties

CROSS-CONTEXT LOOK AT UPPER-DIVISION … PHYS. REV. PHYS. EDUC. RES. 15, 020136 (2019)

020136-3

are focused on the activation and construction componentsof the ACER framework and emphasize difficulties iden-tifying integration as necessary and manipulating thedifferential term. Both of these difficulties also appear inour own prior work investigating student difficulties withthese same types of calculations in the context of upper-division electrostatics, suggesting that these difficulties arerelatively persistent across several exposures to the use ofmultivariable integration in physics problems. However, asthe majority of the studies at the introductory level arebased primarily on student interview data, conclusionsabout potential changes in the frequency of these difficul-ties between the introductory and upper-division popula-tion are not possible. Missing from the majority of theseprior studies is explicit attention to students’ reflections,which is also consistent with our finding that studentsrarely exhibit reflective behaviors spontaneously.The current study builds on this prior work by contrib-

uting data on students difficulties at an additional twopoints within the undergraduate curriculum: middle-division classical mechanics (in the context of gravitation)and upper-division magnetostatics (in the context of thevector potential). Additionally, we will leverage bothquantitative exam data and qualitative interview data, alongwith comparison to our prior work, to make statementsabout how the frequency of these difficulties shifts (or not)over the course of these multiple exposures.

III. METHODS

A. Context

Data for this study were collected from two courses atthe University of Colorado Boulder (CU) over the course ofthree distinct semesters. Both courses in this study are thefirst in a two semester sequence—one targeting a combi-nation of classical mechanics and math methods (Class.Mech.), and the other targeting electricity and magnetism(E&M). Class. Mech., covers chapters 1–5 in Taylor’s text[16] as well as various chapters in Boas [17], and studentsare typical sophomores and juniors. E&M, covers the first 6chapters of Griffths’ text [18] (i.e., electrostatics andmagnetostatics), and students in this course are juniorsand seniors. For both courses, most students are physics,astrophysics, and engineering physics majors. During thesemesters of data collection, both courses were taught withvarying degrees of interactivity through the use of research-based teaching practices including Peer Instruction usingclickers [19] and tutorials [20].We collected data from two primary sources for this

investigation: student solutions to instructor-designed ques-tions on traditional midterm exams, and group think-aloud,problem-solving interviews. Our approach mirrors that ofour earlier investigations of students’ difficulties with directintegration described in the prior section (Sec. II A). In thisapproach, exams provided quantitative data identifying

common difficulties and interviews offered deeper insightinto the nature of those difficulties. At CU, Class. Mech. isa prerequisite for E&M, and, thus, students’ responses inthese two courses during the same semester represent apseudolongitudinal view of the evolution of students’reasoning over the course of several semesters. These dataare pseudolongitudinal, rather than truly longitudinal,because rather than tracking individual students and com-paring their performance at different points in time, we arecomparing the aggregate performance of two different setsof students at different points in the curriculum. Thepseudolongitudinal nature of our data has implicationsfor the interpretation of these data that will be discussed inmore detail in the following section (Sec. IV).Midterm exam data were collected from one semester in

Class. Mech. (N ¼ 77 students and 1 instructor) and twosemesters in E&M (N ¼ 163 distinct students and 3instructors). Of the four total instructors involved in thesecourses, two were traditional research faculty and two werephysics education researchers. One semester of the E&Mcourse was co-taught by one PER faculty member and oneof traditional research faculty with shared exams, activities,and homework. The exam questions used for the studywere developed via collaboration between the traditionalresearch faculty and the PER faculty. The exam questionswere all variations of calculating the potential from a shortline or strip segment (see Fig. 1). For example, in Class.Mech. the students were asked for the gravitational poten-tial energy from a rod of linear mass density. The E&Mstudents were asked two questions: one asking for thescalar potential from a short charged rod, and one askingfor the vector potential from a short line carrying constantcurrent or a short strip carrying uniform surface currentdensity. The most efficient solution approach in each case isto calculate the requested potential directly using

Uðr⃗Þ ¼ −GMZV

dmjr⃗ − r⃗0j ; ð1Þ

Vðr⃗Þ ¼ 1

4πϵ0

ZV

dqjr⃗ − r⃗0j ; ð2Þ

A⃗ðr⃗Þ ¼ μo4π

ZV

J⃗jr⃗ − r⃗0j dV

0: ð3Þ

Here G, ϵo, and μo are fundamental constants, M is themass of the test object, r⃗ is the vector from the origin tothe field point, and r⃗0 is the vector from the origin to thesource point.While all exam questions in the study were some

variation on calculating the potential from a short rod orstrip, there were several important differences between theprompts used in each case. All questions provided a set ofCartesian axes in the figure; however, the orientation of thedistribution (i.e., what axis it lay along) varied between

WILCOX and CORSIGLIA PHYS. REV. PHYS. EDUC. RES. 15, 020136 (2019)

020136-4

prompts. More significantly, one version of the two vectorpotential questions provided students with the correctmathematical tool to use [i.e., Eq. (3)], whereas in allother exam questions students had to decide for them-selves which tool to use. From the point of view of ACER,this means that this specific vector potential promptbypassed the activation component. Additionally, onlythe two scalar potential prompts resulted in integralexpressions that could actually be calculated by hand;in the other cases, students either were asked only to set upthe integral or were provided the result of the integral.With respect to ACER, this means that the questionstargeting gravitation and the vector potential all effec-tively bypassed the execution component.To address the lack of explicit reflection and other

limitations with the exam data, we also conducted think-aloud interviews with pairs of students. Four interviewswere conducted with students (N ¼ 8) who had previouslycompleted or were currently enrolled in the Class. Mech.course, and three interviews were conducted with students(N ¼ 6) who had previously completed, or were currentlyenrolled in, the E&M 1 course. In these interviews, studentswere asked to respond to a slightly more challengingversion of the exam questions, which asked them tocalculate the potential along an axis other than the centralaxis from a ring of uniform density (see Fig. 2). Theincrease in difficulty in the interviews was to ensure thequestion was sufficiently challenging for a pair of studentsworking together. The paired interview structure was

adopted to create a more authentic interview environmentin which interview participants would naturally encourageeach other to express their reasoning without promptingfrom the interviewer. In both interview sets, students werenot told which mathematical tool to use; however, a correct

FIG. 2. The vector potential version of the interview question.For the Class. Mech. students, the distribution was described as amass distribution with uniform λ (and the arrow indicatingdirection of current was removed), and students were asked tocalculate the gravitational potential energy for a small massplaced on the x axis.

(a) (b)

FIG. 1. Examples of the exam questions used in the study. Variations on (a) included asking instead for the scalar potential from a shortrod of charge or for the vector potential from a short line segment of current.

CROSS-CONTEXT LOOK AT UPPER-DIVISION … PHYS. REV. PHYS. EDUC. RES. 15, 020136 (2019)

020136-5

solution to the problem will produce an integral that is noteasily evaluated by hand. As such, interviewees whoreached this integral were asked to stop at that point.From the perspective of the ACER framework, theseinterview questions targeted activation, construction, andpotentially reflection, but did not capture the majority of theexecution component. This design was based on findingsfrom our prior work suggesting that pure executionerrors are rarely the primary difficulties encountered bystudents [7].The exam solutions contained little evidence of spon-

taneous reflection (see Sec. IV D). In one semester, theexam question asking for the scalar potential include twofollow-up questions asking students to discuss the limit-ing behavior of the potential in the large-r limit and toconfirm that their solution exhibited this behavior. Theinterviews also included an additional question designedto examine students’ ability to come up with meaningfulreflections when prompted to do so. Five of the sevenpairs (those with interview time remaining after finishingthe first question) completed this second question tar-geted directly at reflection. Students were asked to assessthe plausibility of three expressions for either gravita-tional potential energy or vector potential caused by anunspecified, but localized, mass or current distribution.Valid approaches included (but are not limited to)checking units or limits, but neither approach wassuggested explicitly.

B. The operationalized framework

The process of operationalizing ACER is presented indetail in Ref. [7]. Briefly, in order to operationalize theframework, a content expert utilizes a modified form oftask analysis [7,8] in which they work through theproblems of interest while carefully documenting theirsteps and mapping these steps onto the general compo-nents of the framework. Additional content expertsthen review and refine the resulting outline until con-sensus is reached that the key elements of the problemhave been accounted for. This expert-guided scheme thenserves as a preliminary coding structure for analysis ofstudent work. If necessary, the operationalization can befurther refined to accommodate aspects of student prob-lem solving that were not captured by the expert taskanalysis.In prior work, we operationalized the ACER framework

for the use of multivariable integration in the context ofcalculating the electric potential from a continuous chargedistribution [7]. Here, we modify this operationalizationsuch that it is appropriate for the use of multivariableintegration to calculate potentials in a wider range ofcontexts including gravitation and magnetostatics. Theelements of the modified operationalized ACER frameworkare detailed below. Element codes are for labeling purposesonly and are not meant to suggest a particular order, nor

are all elements always necessary for every problem.In particular, the elements of construction and executionare unlikely to occur in the specific order listed as expertscan, and often do, iterate back and forth between setting upand evaluating expressions.

1. Activation of the tool

The first component of the framework involves theselection of a solution method. The modified task analysisidentified four elements that are involved in the activationof resources identifying multivariable integration as theappropriate tool.

A1: The problem asks for a vector field or the associatedpotential.

A2: The problem provides an expression for, or descrip-tion of, the distribution that is the source of the field orpotential.

A3: The provided distribution does not have appropriatesymmetry to utilize mathematically simpler ap-proaches (e.g., variations of Gauss’ s law).

A4: Direct calculation of the potential is more efficientthan direct calculation of the associated field.

Elements A1–A3 are cues typically present in theproblem statement. Element A4 is specific to problemsasking for the potential and is included to account for thepossibility of solving for potential by first calculating theassociated field. This method is valid but often moredifficult.

2. Construction of the model

Here, mathematical resources are used to map thespecific physical situation onto the general integral expres-sion. The integral expression produced at the end of theconstruction component should be in a form that could, inprinciple, be solved with no knowledge of the physics ofthis specific problem. We identify four key elements thatmust be completed in this mapping.

C1: Use the geometry of the distribution to select acoordinate system.

C2: Express the differential element (e.g., dm or J⃗dV 0) inthe selected coordinates.

C3: Select integration limits consistent with the differ-ential charge element and the extent of the physicalsystem.

C4: Express the difference vector, jr⃗ − r⃗0j, in the selectedcoordinates.

Elements C2 and C4 can be accomplished in multipleways, often involving several smaller steps. In order toexpress the differential element, the student must combinethe charge density and differential to produce an expressionwith dimensions consistent with the physical quantity beingsummed over (e.g., J⃗dV). Construction of the differencevector often includes a diagram that identifies both thesource, r⃗0, and field point, r⃗, vectors.

WILCOX and CORSIGLIA PHYS. REV. PHYS. EDUC. RES. 15, 020136 (2019)

020136-6

3. Execution of the mathematics

This component of the framework deals with themathematics required to compute a final expression. Inorder to produce a formula describing the potential or field,it is necessary to

E1: Maintain an awareness of which variables are beingintegrated over (e.g., r0 vs r).

E2: Execute (potentially multivariable) integrals in theselected coordinate system.

E3: Manipulate the resulting algebraic expressions into aform that can be readily interpreted.

4. Reflection on the result

The final component of the framework involves verify-ing that the expression is consistent with expectations.While many different techniques can be used to reflect onthe result, these two checks are particularly common forproblems involving multivariable integration:

R1: Check the units of intermediate and final expressions.R2: Check the limiting behavior to ensure it is consistentwith the nature and geometry of the distribution.

Element R2 is especially useful when the student alreadyhas some intuition for how the potential or field shouldbehave in particular limits. However, if they do not come inwith this intuition, reflection on the results of this type ofproblem is a vital part of developing it.In the next section, we will apply this operationalization

of ACER to investigate and compare students’ reasoningand prevalent difficulties when solving the physics prob-lems described in the previous section (Sec. III A) involv-ing integration in the context of gravitation, electrostatics,and magnetostatics.

IV. RESULTS

Here we present results from our investigations ofstudents’ use of direct integration. Since our goal is tocompare changes in students’ reasoning across multiplepoints in the curriculum, we organize our results here bycomponent of the ACER framework and present data fromthe context of gravitation, electrostatics, and magnetostaticstogether. Throughout this section, we also compare to our

prior work investigating students’ use of integration inthe context of electrostatics to provide an additional datapoint to understand differences in students’ reasoningacross multiple contexts. This earlier work was describedin Sec. II A and also involved analysis of students’ examsolutions and interview responses to questions similar tothose considered here for the scalar potential, but involvingcalculating the electric potential from a line, surface, andvolume charge distributions.

A. Activation of the tool

One of the two vector potential exam question in theE&M course provided students with Eq. (3), effectivelybypassing the activation component of the problem-solvingprocess. Here, we will focus on data from the remainingcourses, representing N ¼ 325 exam solutions. Evidencefor the individual elements of activation within the examsolutions can be difficult to identify because students oftendo not articulate their thought process on the exams. Inparticular, there was rarely explicit evidence that studentsattended to elements A1 (i.e., the prompt asked for thepotential) or A2 (i.e., the prompt provided a distribution).These elements, while a critical part of fully justifyingdirect integration as the appropriate tool, are often a tacitstep in the problem-solving process.Elements A3 (i.e., recognizing the distribution does not

have sufficient symmetry) and A4 (i.e., recognizing directcalculation of the potential is most efficient) were mosteasily identified when students did not approach theproblem using the appropriate mathematical tool (seeTable I for a summary). For example, students whoattempt to calculate the potential by first calculating theassociated field or force by treating the distribution ashighly symmetric (e.g., treating it as a point source orusing Gauss’s law) have demonstrated a difficulty withelement A3. This difficulty was manifested by roughly atenth of the solutions in the contexts of both gravitation(7%, N ¼ 5 of 77) and the scalar potential (14%, N ¼ 23of 160). The increase in the fraction of students exhibitingdifficulties with element A3 between Class. Mech. andE&M was driven in part by students using Gauss’s law tocalculate the electric field. Difficulty with element A3 was

TABLE I. Difficulties activating direct integration via Eqs. (1)–(3) as the appropriate mathematical tool for exam questions featuring arod or strip distribution (e.g., Fig. 1). Each N column gives the number of students who were coded as having an error with each of thegiven elements of the framework. This is also given as a percentage of the overall students who were coded as having difficulty with theactivation component in the % column (N given in parentheses). For completeness, Nt represents the total number of solutions thatincluded at least one element of activation. Codes are not exhaustive or exclusive (so percents may not total to 100%) but represent themost common themes.

Gravitation (Nt ¼ 77) Electrostatics (Nt ¼ 160) Magnetostatics (Nt ¼ 50)

Element Difficulty N % N % N %

A3 Point source or Gauss 5 33% (of N ¼ 15) 23 49% (of N ¼ 47) 0 0% (of N ¼ 1)A4 Calculate of force or field 9 60% (of N ¼ 15) 23 49% (of N ¼ 47) 1 100% (of N ¼ 1)

CROSS-CONTEXT LOOK AT UPPER-DIVISION … PHYS. REV. PHYS. EDUC. RES. 15, 020136 (2019)

020136-7

observed in none of the N ¼ 50 vector potential solutions,possibly because no “point-source” approximation existsfor the vector potential due to the lack of magneticmonopoles. However, we also observed no studentattempting to approach the rod of the current questionusing Ampere’s law.Another observed difficulty with activation was attempt-

ing to calculate the potential by first calculating theassociated field or force via direct integration and thentaking the line integral or, in the case of gravitation, usingU ¼ mgh. The former approach is valid in the case ofgravitation and the scalar potential, though significantlymore difficult, and represents a difficulty with element A4.The latter approach represents a difficulty with both A4 andA3, where the student has taken a harder route by opting tocalculate the gravitational field g⃗, and also overgeneralizedthe equationU ¼ mgh, which is only valid near the surfaceof Earth. Students with difficulties with element A4represented just over 10% of students in the context ofboth gravitation (12%,N ¼ 9 of 77) and the scalar potential(14%, N ¼ 23 of 160). In the case of the vector potential,the approach of first calculating the magnetic field is not avalid approach as the relation between the vector potentialand magnetic field is more complex (i.e., B⃗ ¼ ∇ × A⃗), andonly one of theN ¼ 50 vector potential solutions attemptedthis approach.Overall, roughly three-quarters of students (80%,

N ¼ 62 of 77 in gravitation; 70%, N ¼ 113 of 160 inelectrostatics) correctly activated direct integration viaEqs. (1)–(2) as the correct approach to the exam questions(Fig. 1). These results are consistent with what was foundin our earlier investigation in electrostatics where just underthree-quarters (73% [7]) of students correctly used Eq. (2)to calculate the electric potential in problems involvingvarious charge distributions with the most common alter-native approach being to attempt the problem by firstcalculating the electric field via either Coulomb’s law orGauss’s law. This suggests that, while Eq. (2) is at least thesecond time in the curriculum that students have encoun-tered the direct integration approach, they are not showing asignificant increase in success rates for activation of thistool, and potentially showing some increased difficulties.In both of these cases, students typically enter the coursealready familiar with the potential or vector field inquestion in the context of simple and symmetric distribu-tions. Class. Mech. students have seen the gravitationalfield from a point mass or near the surface of Earth (whereU ¼ mgh is a valid expression), and E&M students havecalculated the electric field from highly symmetric chargedistributions using Gauss’s law.Together these results suggest in all contexts the chal-

lenge for students with respect to activation may beeliminating inappropriate but simpler and/or more familiarmethods as viable solution paths. This difficulty appears in20%–30% of solutions in both contexts where such simpler

or more familiar methods exists, though the pseudolon-gitudinal nature of our data makes it impossible todetermine if this is because the students who strugglein the context of gravitation continue to struggle later inelectrostatics, or if some students who did not encounterthis difficulty in gravitation later encounter it in electro-statics and vice versa.The interviews provide additional insight into the issue

of activation. Neither the gravitation nor the vector poten-tial interview questions prompted students to utilizeEqs. (1) or (3), and difficulties with elements A3 andA4 were observed in both the sets of interviews. One of thefour pairs of gravitation interviewees treated the given ring(see Fig. 2) as a point mass, representing a failure in A3.One student argued that any rigid body could be treated as apoint at its center of mass; the other was less confident inthis approach, but said that they did not know how toaccount for the details of this particular mass distribution.When prompted to consider approaching the problem viadirect integration, both students expressed concerns aboutthe potential for “horrible algebra.”These latter two sentiments also arose in the vector

potential interviews wherein two of three pairs attempted tocalculate the magnetic field via Ampere’s law, planning touse the result to “solve” for the vector potential by inverting

the relationship ∇⃗ × A⃗ ¼ B⃗ (i.e., a difficulty with elementA3). This approach is ultimately not possible, and bothpairs struggled to find a useful Amperian loop. Whendirectly prompted by the interviewer to consider Eq. (3),both pairs struggled to use it (discussed in Sec. IV B), andsubsequently returned to the Ampere’s law strategy despitetheir earlier trouble. This difficulty, while not observed inthe exam solutions, is consistent with our hypothesis thatdifficulties with activation are linked primarily to elimi-nating simpler or more familiar approaches. The interviewsprovide additional detail to this interpretation by suggestingthat both perceived and actual difficulties with the directintegration approach may impede activation of this tool andmake students less likely to eliminate what they perceive aseasier approaches even if they doubt their applicability.The choice to calculate the magnetic field via Ampere’s

law reflects difficulties with element A3. None of the vectorpotential interview students attempted to calculate themagnetic field via direct integration as a step towardfinding the vector potential (i.e., element A4), thoughone pair spontaneously considered it, but preferred notto attempt it. Alternatively, in the gravitation context,difficulties with element A4 were observed independentof difficulty with A3; two of the four pairs of gravitationinterviewees calculated the gravitational force instead ofpotential energy. This includes the pair who first answeredwith the point mass expression for potential energy, but,when prompted by the interviewer to attempt integration,switched to calculating the force. However, only one ofthese two pairs could name the correct relationship between

WILCOX and CORSIGLIA PHYS. REV. PHYS. EDUC. RES. 15, 020136 (2019)

020136-8

force and potential energy that would allow them to answerthe question fully.

B. Construction

In all three contexts, the largest number of studentdifficulties were observed in the construction componentwhere students are operationalizing the general integralforms in Eqs. (1)–(3) for the specific physical situationgiven in the problem. Overall, N ¼ 72 (out of 77) of thegravitation solutions, N ¼ 130 (out of 160) of the scalarpotential solutions, and N ¼ 135 (out of 138) of the vectorpotential solutions included at least one element of con-struction in their solution (i.e., they used direct integrationand did more than just write an equation or final answer).With respect to element C1 (i.e., selecting an appropriatecoordinate system), all exam prompts provided a figurewhich included a preselected origin and labeled Cartesianaxes (effectively bypassing this element), and only 3students in the entire dataset utilized anything other thanCartesian variables. Similarly, all but one pair of inter-viewees chose polar coordinates—the simplest option—with the final pair opting for Cartesian coordinates.One E&M interviewee did, however, express concernsabout the use of curvilinear coordinates, worried that theinseparability of Poisson’s equation for A⃗ in this systemmay also invalidate Eq. (3).Element C2 proved more challenging for students with

roughly a fifth of the gravitation solutions (17%, N ¼ 12of 72), just over a tenth of the scalar potential solutions(14%, N ¼ 18 of 130), and a third of the vector potentialsolutions (30%, N ¼ 41 of 135) providing responsesreflecting difficulties expressing the differential element(see Table II). For the questions considered here, thesedifficulties could manifest in two primary ways: difficultiesexpressing the density (e.g., λ, J⃗) or difficulties expressingthe differential line or area element (dl or dA).For the students in the context of gravitation, difficulty

expressing the differential line element was most common(83%, N ¼ 10 of 12) and included expressing dl as afunction of the field variables (i.e., r vs r0, N ¼ 5) and

expressing the differential element as an area or volumeelement (N ¼ 2). Similar difficulties with the differentialline element were observed in the E&M students’ solutions,though representing a smaller fraction of the difficultieswith element C2 (61%, N ¼ 11 of 18 in electrostatics;51%, N ¼ 21 of 41 in magnetostatics). Moreover, focusingonly on the exam questions featuring line distributions, theoverall fraction of students exhibiting difficulties with thedifferential line element decreases from 14% (N ¼ 10 of72) to 9% (N ¼ 20 of 218) from Class. Mech. to E&M.This downward trend across consecutive exposures sug-gests that difficulty with element C2 may be one that fadesover time as students gain more experience.None of the interviewed Class. Mech. pairs had trouble

writing the differential line element in polar coordinates.It is possible that some of these students may have hadmore difficulty on their own—one, for example, firstsuggested dl ¼ Rdθdϕ—but as pairs, the studentswere able to correct each others mistakes. The E&Minterviewees were equally successful, with the exceptionof the pair working in Cartesian coordinates, who wrotedl ¼

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffidx2 þ dy2

p. However, this pair did generate the

correct form once prompted to switch to polar coordinates.Difficulties expressing the linear mass density λ were

less common (42%,N ¼ 5 of 12) for students in the contextof gravitation and limited almost exclusively (N ¼ 4) toincorrectly substituting the total mass of the rod for λ. Thisdifficulty was observed with similar frequency in thecontext of the scalar potential (44%, N ¼ 8 of 18).At first glance, this trend would also seem to suggest thatstudent difficulties around the density stay relatively con-stant over time; however, this pattern does not continue inthe context of magnetostatics. One version of the vectorpotential exam question provided students with the expres-sion for the surface current density K⃗ [see Fig. 1(b)],effectively bypassing the need for students to express thisquantity. Alternatively, the other version of the questionrequired students to express (or translate) the volumecurrent density J⃗ for the line of current on their own.In this case, difficulties around the current density

TABLE II. Difficulties expressing the differential element (i.e., dm, dq, or JdV 0). Each N column gives the number of students whowere coded as having a given error. This is also given as a percentage of the overall students who were coded as having difficultywith element C2% column (N given in parentheses). For completeness, Nt represents the total number of solutions that included atleast one element of construction. Codes are not exhaustive or exclusive (so percents may not total to 100%) but represent the mostcommon themes.

Gravitation (Nt ¼ 72) Electrostatics (Nt ¼ 130) Magnetostatics (Nt ¼ 135)

Difficulty N % N % N %

Incorrect differential line or area element,e.g., dl ¼ dx rather than dx0 or dl ¼ da

10 83% (of N ¼ 12) 11 61% (of N ¼ 18) 21 51% (of N ¼ 41)

Incorrect expression for density,e.g., λ ¼ M or I ¼ J

5 42% (of N ¼ 12) 8 44% (of N ¼ 18) 27 66% (of N ¼ 41)

CROSS-CONTEXT LOOK AT UPPER-DIVISION … PHYS. REV. PHYS. EDUC. RES. 15, 020136 (2019)

020136-9

represented the majority of the difficulties with elementC2 (90%, N ¼ 27 of 30 solutions exhibiting difficultieswith C2). This difficulty most often (63%, N ¼ 17 of 27)manifested as students simply leaving J in their final ex-pression without transitioning to the current I carried bythe wire segment (i.e., incorrectly replacing JdV 0 → Jdz0)and resulting in a final expression with the wrong units.Consistent with the results of the exam coding, no issues

with the linear mass density arose in the Class. Mech.interviews, but all three E&M pairs experienced somedegree of difficulty with the current density. Two pairs did,after unguided discussion, generate a valid expression forthe magnitude of J⃗ using delta functions, but both omittedthe direction (although one pair caught this error much lateron). Both pairs used the units of current density to guidetheir choice of delta functions. The third pair also refer-enced units, but argued that the units of J⃗ were current pervolume. They were unable to find an expression for thecurrent density and were eventually provided with the one-dimensional form of Eq. (3).One interpretation of this sudden increase in the fre-

quency of a difficulty that otherwise seemed to be holdingsteady as students progressed may be that it is, in fact, atotally new difficulty. The volume current density, J⃗, is aconceptually and mathematically more challenging quan-tity than either mass or charge density. Moreover, while thequantity ρdV 0 (where ρ is volume mass density) has a clearphysical interpretation as the total mass on each differentialchunk of the object (i.e., dm), the quantity J⃗dV 0 has no suchclear physical interpretation. One exchange between E&Minterviewees discussing the meaning of the integral inEq. (3) demonstrates this difficulty. First, they associated“the current” with the entire ring of current, making thechoice of a specific r⃗0 that represents the entire circleambiguous. One student then had the insight that r⃗0 pointsto “a current density at a given location along the loop,” andused the phrase “for each J” to describe this. Their partnerobjected to this language as there is only one function J⃗,and asks if they meant “differential J.” While over thecourse of this exchange the pair made progress towardsunderstanding Eq. (3), their interpretation of the differentialelement was still dimensionally incorrect (they had notaccounted for the volume element). Moreover, their diffi-culties were exacerbated by the lack of any commonterminology with which to refer to the quantity beingsummed over.After constructing an expression for the differential

element, students must select limits of integration thatare consistent with this element and the physical extentof the system (element C3). Difficulties with the limitsappeared in less than a fifth of the gravitation solutions(19%, N ¼ 14 of 72), roughly a quarter of the scalarpotential solutions (28%, N ¼ 36 of 130), and roughly atenth of the vector potential solutions (12%, N ¼ 16 of

135). In all cases, the two most common issues were notintegrating over the actual extent of the rod (N ¼ 11 of 14in gravitation, N ¼ 26 of 36 in electrostatics, and N ¼ 7 of16 in magnetostatics), and not including limits at all (N ¼ 1of 14 in gravitation, N ¼ 8 of 36 in electrostatics, andN ¼ 9 of 16 in magnetostatics). Combined with findingsfrom our previous work in the context of electrostaticsshowing difficulties with limits in only 14% of solutions[7], these data suggest that difficulties with element C3 mayfluctuate somewhat over students’ first exposures to directintegration in these contexts, but stay present to at leastsome extent even after multiple exposures. In the inter-views, none of the six pairs who made it to this point in theinterview struggled to choose limits of 0 to 2π for the ring.This includes the pair working in Cartesian coordinateswho still articulated this choice, suggesting that for somethe limits may have been an automatic response to thecircular geometry rather than a considered decision.The fourth element in construction (C4) deals with the

difference vector (r⃗ − r⃗0) which points from the sourcepoint to the field point. In our prior investigation, nearlyhalf of students encountered difficulties when attempting toexpress this vector [7] for a disk-shaped charge distribution.The distributions provided in this study result in a simplerexpression for difference vector, and, unsurprisingly, stu-dents had somewhat less difficulty producing a correctexpression with 30%–45% of students unable to do so(30%, N ¼ 22 of 72 in gravitation; 45%, N ¼ 45 of 130 inelectrostatics; 35%, N ¼ 47 of 135 in magnetostatics).However, the fact that the fraction of students exhibitingdifficulties with the difference vector remains high, andperhaps even increases slightly, suggests that this difficultyis persistent across multiple exposures.Lending detail to the results from the exams, two types of

difficulties related to the difference vector arose in theinterviews. In two of the four Class. Mech. interviews, thestudents generated incorrect expressions for the magnitudeof the difference vector. When prompted to consider morecarefully or to draw the relevant vectors, however, bothpairs came to a valid expression. These students appearedto understand the meaning of the difference vector termin the integrand (i.e., the distance from source to test), buthad made geometric errors in calculating its magnitude.Alternatively, some of the E&M pairs struggled with themeaning of the r⃗ − r⃗0 term. This difficulty appeared to berelated to Griffiths’ “script-r” notation (script-r ¼ r⃗ − r⃗0)[18]). A total of four pairs of students used this notationeither spontaneously or when prompted. At least onestudent in each pair expressed doubts about the relationshipbetween script r, r⃗, and r⃗0 or the meaning of each vector(e.g., switching the primed and unprimed variables). Twoof these pairs had sufficient difficulty remembering orgenerating these ideas that they never attempted to write anexpression for the difference vector or its magnitude. Whilethe convention of using script r, r⃗, and r⃗0 permits concise

WILCOX and CORSIGLIA PHYS. REV. PHYS. EDUC. RES. 15, 020136 (2019)

020136-10

formulas, it requires students to parse or, in the case of theinterviewees in this study, simply try to remember themeaning of these symbols before they can proceed toanalyzing the geometry of the problem at hand. Thisfinding is consistent with our previous findings in thecontext of electrostatics [7].

C. Execution

The execution component of the framework deals withthe procedural aspects of working with mathematical toolsin physics. Overall, N ¼ 66 of the gravitation solutions,N ¼ 128 of the electrostatics solutions, and N ¼ 77 of themagnetostatics solutions included at least one element ofexecution. On one of the vector potential exam questions,students were only asked to set up the integral and thus didnot include execution. In the remaining exam questions, itis not possible to know for certain from an exam solutionwhether students keep track of which variables are beingintegrated over (i.e., source vs field variables—elementE1); however, we can identify cases where studentsexplicitly integrated over the wrong variables (i.e., inte-grating over the field variables). Roughly a tenth of thestudents in both courses (N ¼ 5 of 66 in gravitation,N ¼ 20 of 128 in electrostatics, and N ¼ 7 of 77 inmagnetostatics) explicitly integrated over variables in theirexpression which represented the field point.Only the scalar potential questions resulted in integrals

that could be reasonably calculated by hand, and just over athird (37%, N ¼ 47 of 128) of solutions exhibited one ormore mistakes in doing so. The other questions resulted inintegrals for which the general solution was provided tothe students (effectively bypassing element E2). However,some students still attempted to perform integrals, typicallybecause a mistake in an earlier step of the solution resultedin a solvable integral. Of the N ¼ 23 Class. Mech. studentswho attempted to perform integrals just under half(43%, N ¼ 10 of 23) made significant errors in the process.This fraction dropped to only 17% (N ¼ 3 of 18) in thecontext of the vector potential students who attempted toperform an integral. Note that these students had, bynecessity, already made one or more errors in the activationand construction components of the framework beforereaching execution.Given the high pressure and individual nature of both

exams, we expected that many students would makemathematical errors particularly with element E3 (algebraicmanipulation). To account for this, we distinguish in ouranalysis between small math errors (e.g., dropping a factorof 2 or minus sign) and more fundamental mathematicalerrors (e.g., dropping the bottom limit or executing inte-grals incorrectly). In the context of the scalar potential, justover half of the students who made mathematical errorsmade more fundamental errors (60%, N ¼ 28 of 47);however, for roughly three-quarters of these students(71%, N ¼ 20 of 28), the mistake made in the execution

stage was preceded by a significant difficulty in theactivation or construction components. This trend is con-sistent with the findings of our prior work that suggesteddifficulty performing integrals was rarely the primarybarrier to students’ success when using integrals in thecontext of physics problems [7].In interviews, students were not asked to evaluate the

integral expression they derived in the interviews, althoughone pair did switch from integrating over the sourcevariables to over the field variables when consideringhow they might do so. Otherwise the interviews providedlittle insight into the execution component.

D. Reflection

The reflection component deals with the process ofchecking and/or interpreting the final expression. It is oftenthe case that mistakes in the construction or executioncomponents resulted in an expression for the potential thathad the wrong units or did not have the correct behavior inparticular limits (i.e., elements R1 and R2, respectively).Overall, we found that very few of our students (N ¼ 20 ofthe 329 students in any of the three contexts to complete thequestion) made explicit, spontaneous attempts to reflect ontheir solution using either of these checks on exams. Thisnumber should be interpreted as a lower bound on thefrequency of spontaneous reflections, as it is possible thatmore of the exam students made one of these checks andsimply did not write it down explicitly on their examsolution; however, the interview results suggest this is lesslikely. Although all three E&M interview pairs consideredunits when writing J⃗, and one pair checked extreme casesfor their difference vector expression, none of the inter-viewees spontaneously reflected on their final integralexpression. Some explained that they used these tools onlywhen particularly worried about a result, and only if theyfelt they had time.Another strategy for understanding reflection involves

looking at the number of solutions where the final expres-sion included an error that would have been detected by oneor more of these checks. Table III lists this along with thenumber of solutions that explicitly included each reflectivecheck. Overall, these results suggest that an explicit check

TABLE III. Number of exam students who explicitly utilizedeach of the two common reflective checks (Nexplicit) along withthe number of solutions that included an error that would havebeen detected by this check (Nincorrect). Data from all threecontexts have been combined here due to low Nexplicit. The totalnumber of students who progressed far enough in their solution toutilize one of these checks was N ¼ 329.

Reflective check Nincorrect Nexplicit

Units (R1) 78 3Limits (R2) 62 10

CROSS-CONTEXT LOOK AT UPPER-DIVISION … PHYS. REV. PHYS. EDUC. RES. 15, 020136 (2019)

020136-11

of units would likely be the most effective reflectivepractice for students in terms of detecting errors, but thatour students are rarely executing this (or other) checkspontaneously. This finding is consistent with our priorinvestigation [7] and implies that students do not growmore likely to spontaneously execute these kinds ofreflective behaviors as they progress further through thecurriculum.We also investigated whether students can perform these

reflective checks when asked to do so. In one of the twoscalar potential exam questions, two follow-up questionsasked students to discuss how the potential should behaveas you got far from the charge distribution and to confirmthat their expression was consistent with this expectation.Most students (72%, N ¼ 47 of 65) correctly articulatedthat the potential should fall off as 1=r in the limit oflarge r. The most common alternative answer was simplythat the potential “goes to zero” with no discussion of howit goes to zero (17%, N ¼ 11 of 65). When asked to showthat their answer matched this behavior, nearly half (44%,N ¼ 24) of the 55 students who responded executed anappropriate Taylor expansion of their prior solution, and afurther quarter (25%, N ¼ 14) had made prior mistakes intheir solution such that their solution already had a pure 1=rdependence. The remaining quarter of students (24%,N ¼ 13) simply plugged in r ¼ ∞ into their expressionand showed that their solution went to zero for large r.The interviews also provided insight into students

prompted reflection. Five of the seven interviews includeda second question targeted at reflection. All five pairssuggested and were able to carry out a units check, althoughonly one pair did so without first expanding the relevantconstant (G or μ0) in fundamental units. All five alsochecked that the expressions vanished in the limit r → ∞.On the other hand, three pairs expected the expressions toblow up in the r → 0 limit, but this criterion incorrectlyeliminated one of the proposed answers. Moreover, only onepair successfully determined the large-r functional form ofthis answer, and only when prompted to do so.Interviewees were also asked to reflect on their final

answers for the potential from the ring given in the firstinterview question. Their strategies here matched thoseemployed in the second question, but it is worth notingthat two students expressed uncertainty as to the effect of theintegral on units or limits. Combined with the exam results,this suggests that students are capable of checking units andlimiting values (e.g., that a function goes to zero) but are bothless inclined towards, and have more difficulty, checkinglimiting forms (e.g., how the potential goes to zero).

V. SUMMARY AND IMPLICATIONS

Here, we build on prior work investigating students’ useof direct integration as a mathematical tool in physicsproblem solving. We extend this prior work, which focusedon students’ difficulties in the context of junior-level

electrostatics, by investigating students’ use of integrationin two additional points in the undergraduate curriculum:in the context of gravitation at the sophomore level, andmagnetostatics at the junior level. With the goal of makingpseudolongitudinal comparisons across these three differ-ent content areas, we again utilized the ACER framework tostructure the design and analysis of our investigations. Thishas allowed us to directly compare the reasoning anddifficulties students exhibit at key points in the problem-solving process when utilizing direct integration, anddetermine whether and how these difficulties evolve orshift as students encounter the same mathematical toolmultiple times in different contexts.With respect to activation, we found that across all three

contexts some students (roughly a quarter) showed diffi-culties in identifying direct integration as the correctapproach. In nearly all cases, students who did not usedirect integration for the potential instead tried to approachthe problem by first calculating the associated field byanother method. Moreover, in interviews, students oftenexpressed concern about direct integration being toochallenging, instead opting for more familiar approachesor approaches perceived as being easier. This tendency toavoid particular approaches because they are “too hard”may well be something these students have learnedimplicitly in their courses. Physicists tend to give studentsquestions that can be solved in a relatively straightforwardmanner and often employ tricks to simplify the mathemat-ics of a problem. This tendency may have the unintendedconsequence of making students less willing to attemptapproaches they consider to be mathematically complex inthe belief that there must be an easier approach. Moreover,this difficulty does not appear to fade significantly overmultiple exposures. This suggests that instructors may needto place greater emphasis on explicit discussions of howto identify the correct approach to a problem and how toeliminate mathematically simpler approaches when theyare not applicable.In terms of construction, we found a greater degree of

variation in students’ difficulties in the different contexts.In all contexts, one of the primary construction difficultiesencountered related to expressing the differential element;however, the nature of this difficulty varied significantly.Difficulties expressing the differential line element weremost prevalent in the context of gravitation, and graduallybecame less frequent in the context of electrostatics andthen magnetostatics. This may suggest that this difficultyfades somewhat over the course of the current curriculum.Alternatively, the primary difficulties in expressing thedifferential element in the context of magnetostatics relatedto the current density, while difficulties with the mass orcharge density were less common in the context ofgravitation and electrostatics and occurred with roughlythe same frequency. We argue that this difficulty, whilemanifest in the mathematics of the problem, actually

WILCOX and CORSIGLIA PHYS. REV. PHYS. EDUC. RES. 15, 020136 (2019)

020136-12

reflects a conceptual difficulty relating to interpretingthe current density itself and the physical meaning of theJ⃗dV 0 term. This represents an example of a situation wherea new difficulty not observed in early exposures to directintegration arose specifically as a consequence of a changein the physical context of the problem. One other elementof construction which saw significant difficulties wasexpressing the difference vector. However, the frequencyof this difficulty stayed more consistent across the contextof three contexts, and perhaps even got worse after theintroduction of script r, with between a third and a halfof students displaying difficulties in both cases, suggestingthat this difficulty is also persistent over multipleexposures.Based on the findings of the earlier study of students’ use

of direct integration in electrostatics, which found thatdifficulties around the execution were rarely the primarybarrier to student success, the current study included only afew questions asking students to work through the mechan-ics of actually performing integrations. However, consistentwith this previous finding, the majority of execution errorsobserved in this study were made by students who hadalready made one or more significant mistakes in theactivation or construction components of the their solution.This result, once again, suggests that the proceduralmistakes made during the integration process were notthe most significant issues encountered by these students.Finally, with respect to reflection, we found that in all

three contexts spontaneous bids for reflection were veryrare in both the exams and interviews. This suggests thatmultiple exposures to direct integration in physics problemsolving do not appear to encourage students to executereflective checks on their solutions when not promptedto do so. Exams and interviews further suggested whenprompted to come up with possible reflective checks,students in all three contexts are able to identify appropriatepossibilities (e.g., checking units or limits). However,interviews also suggested that actually executing these

checks, particularly in cases where doing so requires extrasteps such as executing a Taylor expansion, may be morechallenging for students, particularly after only earlyexposure to direct integration in the context of gravitation.Overall, the results of this study, combined with our

earlier investigation, provide insight into students’ use ofdirect integration in multiple contexts and provide exam-ples of cases where observed students’ difficulties getbetter, change, and remain the same over the course ofthese multiple exposures. The results can be used to helpinstructors identify areas where additional effort is neededto address persistent issues (e.g., identifying integrationas the appropriate tool, and expressing the differencevector), versus areas where difficulties fade with experience(e.g., expressing differential line, area, and volume ele-ments), as well as to anticipate new difficulties that appearin different contexts (e.g., interpreting and expressing thecurrent density).This work represents a novel example of how the ACER

framework can serve as a standardized tool to facilitatecross-study comparisons of students’ use of mathematicaltools in physics problem solving. Future work will involvecross-context comparisons of students use of other math-ematical tools in other physics contexts. For example,building on prior work with the Dirac delta function inthe context of electrostatics could be extended to includemore mathematically complex uses in quantum mechanicsor Fourier transforms. Similarly, investigations of students’use of separation of variables in electrostatics could formthe basis of understanding changes in students’ difficultieswhen they encounter this tool again in quantum mechanics.

ACKNOWLEDGMENTS

This work was supported by the University of ColoradoPhysics department. Particular thanks to the members ofPER@C for all their help and feedback, as well as thestudents and faculty who participated in the study.

[1] L. Hsu, E. Brewe, T. Foster, and K. Harper, Resource letterrps-1: Research in problem solving, Am. J. Phys. 72, 1147(2004).

[2] D. E. Meltzer and R. K. Thornton, Resource letter alip–1:Active-learning instruction in physics, Am. J. Phys. 80,478 (2012).

[3] S. V. Chasteen, B. Wilcox, M. D. Caballero, K. K. Perkins,S. J. Pollock, and C. E. Wieman, Educational transforma-tion in upper-division physics: The science educationinitiative model, outcomes, and lessons learned, Phys.Rev. ST Phys. Educ. Res. 11, 020110 (2015).

[4] L. C. McDermott and P. S. Shaffer, Tutorials in Introduc-tory Physics and Homework Package (Prentice Hall,Englewood Cliffs, NJ, 2001).

[5] N. D. Finkelstein and S. J. Pollock, Replicating and under-standing successful innovations: Implementing tutorials inintroductory physics, Phys. Rev. ST Phys. Educ. Res. 1,010101 (2005).

[6] M. D. Caballero, B. R. Wilcox, L. Doughty, and S. J.Pollock, Unpacking students use of mathematics inupper-division physics: where do we go from here?,Eur. J. Phys. 36, 065004 (2015).

CROSS-CONTEXT LOOK AT UPPER-DIVISION … PHYS. REV. PHYS. EDUC. RES. 15, 020136 (2019)

020136-13

[7] B. R. Wilcox, M. D. Caballero, D. A. Rehn, and S. J.Pollock, Analytic framework for students use of math-ematics in upper-division physics, Phys. Rev. ST Phys.Educ. Res. 9, 020119 (2013).

[8] R. Catrambone, Task analysis by problem solving (taps):Uncovering expert knowledge to develop high-qualityinstructional materials and training, in 2011 Learningand Technology Symposium (Columbus, Georgia, 2011).

[9] D. Hammer, Student resources for learning introductoryphysics, Am. J. Phys. 68, S52 (2000).

[10] T. J. Bing, An epistemic framing analysis of upper-levelphysics students’ use of mathematics, Ph.D. thesis,University of Maryland, 2008.

[11] D.-H. Nguyen and N. S. Rebello, Students’ difficulties withintegration in electricity, Phys. Rev. ST Phys. Educ. Res. 7,010113 (2011).

[12] D. C. Meredith and K. A. Marrongelle, How students usemathematical resources in an electrostatics context, Am. J.Phys. 76, 570 (2008).

[13] D. Hu and N. S. Rebello, Understanding student use ofdifferentials in physics integration problems, Phys. Rev. STPhys. Educ. Res. 9, 020108 (2013).

[14] N. Amos and A. F. Heckler, Mediating relationship ofdifferential products in understanding integration in in-troductory physics, Phys. Rev. Phys. Educ. Res. 14,010105 (2018).

[15] E. R. Savelsbergh, T. de Jong, and M. G. M. Ferguson-Hessler, Choosing the right solution approach: Thecrucial role of situational knowledge in electricity andmagnetism, Phys. Rev. ST Phys. Educ. Res. 7, 010103(2011).

[16] J. R. Taylor, Classical Mechanics (University ScienceBooks, Sausalito, CA, 2005).

[17] M. Boas, Mathematical Methods in the Physical Sciences(John Wiley & Sons, New York, 2006).

[18] D. J. Griffiths, Introduction to Electrodynamics (PrenticeHall, Englewood Cliffs, NJ, 1999).

[19] E. Mazur, Peer Instruction: A User’s Manual, Series inEducational Innovation (Prentice Hall, Upper Saddle River,NJ, 1997).

[20] S. V. Chasteen, S. J. Pollock, R. E. Pepper, and K. K.Perkins, Transforming the junior level: Outcomes frominstruction and research in E&M, Phys. Rev. ST Phys.Educ. Res. 8, 020107 (2012).

WILCOX and CORSIGLIA PHYS. REV. PHYS. EDUC. RES. 15, 020136 (2019)

020136-14