Research and Practice in Technology Enhanced Learning ......laboratory, (c) develop tools for assessing learners’ prior knowledge and skills in collab-orative inquiry learning, general

February 16, 2011 14:46 WSPC/S1793-2068/RPTEL S1793206810000918

Research and Practice in Technology Enhanced LearningVol. 5, No. 3 (2010) 205–244c© World Scientific Publishing Company &

Asia-Pacific Society for Computers in EducationDOI: 10.1142/S1793206810000918

FACILITATING LEARNING FROM COMPUTER-SUPPORTEDCOLLABORATIVE INQUIRY: THE CHALLENGEOF DIRECTING LEARNERS’ INTERACTIONS

TO USEFUL ENDS

ISABEL BRAUN

Department of Educational Science, University of Freiburg

Rempartstraße 11, 79098 Freiburg, [email protected]

http://www.ezw.uni-freiburg.de/unterrichtsforschung

NIKOL RUMMEL

Institute of Education, Ruhr-Universitat BochumUniversitatsstraße 150, 44801 Bochum, Germany

[email protected]

Chemistry problems can be tough nuts to crack for students, particularly when the prob-lems are of a mathematical type. Collaborative inquiry activities that are embedded incomputer-supported learning environments can provide students with opportunities toconstruct the kind of knowledge required to solve complex chemistry problems. Yet,directing learners’ collaborative inquiry in computer-supported learning environmentsto such useful ends remains a challenge. In this article, we introduce a new approach

to facilitating learning from computer-supported collaborative inquiry. The approachwe present is a collaboration script that guides pairs of learners through a sequence ofinquiry activities and prompts effective interactions in an adaptive fashion. We devel-oped an initial version of a learning environment that included the collaboration script, asimulated chemistry laboratory and a note-taking and mind-mapping tool. In an experi-mental study involving two conditions (scripted collaboration, unscripted collaboration),we had pairs of students work on a stoichiometry problem using the learning environ-ment. We compared the learning outcomes and learning experiences of the two con-ditions, and explored learner-learner and learner-system interactions employing a casestudy methodology. Based on the quantitative and qualitative results, we derived a setof design recommendations for the next development cycle. Our main recommendationsare to (a) inform learners about the value of the script support, (b) add algebra practiceitems and some general chemistry instruction to the learning environment as well as sup-port for interrelating the multiple external representations in the simulated chemistrylaboratory, (c) develop tools for assessing learners’ prior knowledge and skills in collab-orative inquiry learning, general chemistry and basic algebra, (d) increase the degree towhich the learning environment can adapt to different learner needs, and (e) improvethe interface and interaction design of the simulated chemistry laboratory.

Keywords: Collaborative inquiry learning; computer support; scripting.

205

http://dx.doi.org/10.1142/S1793206810000918


206 I. Braun & N. Rummel

1. Introduction

How much hydrogen and how much nitrogen are needed to produce two tons ofammonia? What is the maximum mass of iron that can be obtained from two tonsof iron ore? When asked to solve chemistry problems, more precisely, stoichiometryproblems, similar to these (let alone more complex ones) high school and collegestudents across the world have considerable difficulty (e.g. BouJaoude & Barakat,2000; Gabel et al., 1984; Schmidt, 1990, 1992). Yet, the ability to solve stoichiometryproblems is of high relevance in chemistry research and practice, including manymedical fields and branches of the industry. Researchers found that deficiencies inconceptual knowledge, that is, knowledge of the theories, principles and chemicalprocesses underlying the problems, account for most of the difficulties students havein solving stoichiometry problems (e.g. BouJaoude & Barakat, 2003; Schmidt, 1990,1992). This finding gives rise to the question of what can be done to help studentsacquire deeper understanding of stoichiometric concepts.

Empirical evidence indicates that inquiry learning can promote the acquisitionof conceptual knowledge. Numerous computer-based learning environments havebeen developed to enable and support students’ inquiry activities, for example,the Web-based science inquiry environment (WISE, http://wise.berkeley.edu/) orSimQuest (http://www.simquest.nl/learn.htm). Yet, simulations and cognitive toolsappear to be insufficient to secure the benefits of inquiry learning (e.g. de Jong &van Joolingen, 1998). Therefore, the design of several such environments (e.g. Co-Lab, http://www.co-lab.nl/) was extended to include collaborative learning, anotherinstructional approach that can help learners to acquire deep understanding of thetheories, concepts and principles underlying complex problems. CoChemEx (Tso-valtzi et al., 2008) constitutes an example of a computer-based learning environmentthat enables collaborative inquiry learning. Against the background of research onunsupported collaborative inquiry learning, a collaboration script was implementedin CoChemEx. The script constitutes a novel approach to facilitating collaborativeinquiry in that it combines structural and adaptive support. The implementationof the script represented the first step of a design cycle geared towards support forcollaborative inquiry learning that automatically adapts to learner needs. On-goingdesign efforts have focussed on the conceptual development of the collaborationscript and on the development of automated assessment facilities within CoChemEx.

In this article, we describe the conceptual development of the collaboration scriptand report a full test run of a learning environment that included the collabora-tion script. While building on knowledge gained through our involvement in thedevelopment of CoChemEx and using some of the same components as CoChemEx,the present learning environment constitutes a prototype as it has been developedspecifically for use with high school students in Germany. In the initial exploratorystudy we report in this paper, we had pairs of students work on a stoichiome-try problem using the learning environment and analyzed their learning outcomes,collaboration and inquiry, interactions with the system and learning experiences.


Facilitating Learning from Computer-Supported Collaborative Inquiry 207

Following the principles of design-oriented research (e.g. Brown, 1992; Cobb et al.,2003; Schoenfeld, 2006), we derived a set of design recommendations from the testrun that will feed into another cycle of development and testing. These design rec-ommendations are presented at the end of this paper.

2. Background

In the following, we provide a review of the theoretical and empirical backgroundthat informed the design of the stoichiometry learning environment and shapedour research. It needs to be pointed out that our design efforts and research pro-gram are situated in a European tradition of research on inquiry learning, whichmanifests itself in projects such as Co-Lab (van Joolingen et al., 2005). In thistradition, inquiry activities (generating a hypothesis, manipulating variables, inter-preting results, etc.) are set apart and often arranged in sequence either throughexplicit instruction or through system affordances. Structuring students’ inquiryin such a way is regarded necessary against the background of learners’ scienceprocess skills and limited prior knowledge in the domain under study. Althoughthe notion of open-ended, project-based inquiry has replaced this understanding ofinquiry learning in a large part of the learning sciences community many studiesfollowing up on research in the former tradition continue to be conducted in Europe.Hence, the following review focuses on a particular strand of research on inquirylearning. The reader is referred to the works of other authors, particularly thoseof John D. Bransford, Marcia C. Linn, Brian J. Reiser, James D. Slotta, Nancy B.Songer, Daniel D. Suthers, Robert F. Tinker and Barbara Y. White for a broaderperspective on technology-enhanced inquiry learning in science.

2.1. Experimentation in chemistry education: The benefits

and challenges of inquiry learning

Experimentation plays a major role in chemistry education, taking various shapesdepending on the pedagogical philosophies drawn on by teachers and instructionaldesigners. The more traditional approaches call for experiments to be planned andperformed by the teacher. Other approaches encourage student laboratory work,the implementation of which ranges from reproductive experimentation, which lim-its the students’ task to carrying out a specified procedure and verifying a givenchemical reaction, to free experimentation, which leaves it up to the students whatexperiments to design and conduct in order to solve a given task. Free experimen-tation can be regarded as an instance of (scientific) inquiry or discovery learning,a pedagogical philosophy that emphasizes the active role of the learners in instruc-tional contexts. In inquiry learning environments, the learners do not receive directinstruction on the target concepts. Instead, they are encouraged to construct knowl-edge from the experiences and examples they obtain through experimentation ormodelling (de Jong & van Joolingen, 1998).



Researchers have highlighted the similarities between the learning processes thatcharacterize inquiry learning in scientific disciplines and the activities and cogni-tive processes leading to scientific discovery: “In scientific discovery learning learn-ers more or less take the role of scientists who want to design theory based onempirical observations” (de Jong, 2005, p. 215). Because of the parallels betweeninquiry learning and scientific discovery the former is typically modeled on the lat-ter. A well-known model of inquiry learning in scientific disciplines, which draws onmodels of scientific discovery, is that by de Jong and his colleagues. They refinedtheir model through more than a decade and a half of research. In a recent article,de Jong (2006) distinguished five learning processes that make up what he termedthe inquiry cycle: During (1) orientation, the learner coarsely analyzes the problemthat is to be solved. This is followed by the processes of (2) hypothesis generationand (3) experimentation with the latter comprising the subprocesses of experimentdesign, prediction and data interpretation. The remaining two learning processesare (4) conclusion and (5) evaluation. During conclusion, the learner decides onthe validity of the hypothesis tested by means of experimentation. During evalua-tion, the learner engages in “a reflection on the learning process and the domainknowledge acquired” (p. 109). de Jong emphasized that the learning processes mayoccur in an iterative fashion, with one process being initiated before the previousone has been completed. He also pointed out that metacognitive processes, that is,monitoring and planning, complement the inquiry cycle.

Drawing on models of inquiry learning such as that of de Jong (2006), it canbe assumed that free experimentation yields learning when the learners “take therole of scientists” (de Jong, 2005, p. 215), engaging in all of the learning processesthat make up the inquiry cycle. Yet, when inquiry learning is put into practice,be it in chemistry or in any other scientific discipline, this is where the challengebegins. Inquiry learning has to be supported in an appropriate manner for thelearning processes to take effect because the cycle of orientation, hypothesis gen-eration, experimentation, conclusion and evaluation rarely occurs spontaneously ininquiry learning environments (de Jong & van Joolingen, 1998). Instead, the learn-ers often show unproductive or even counterproductive behavior during inquiryactivities (such as experimentation and modelling), which ultimately leads to situa-tions in which the potential benefits of inquiry learning do not unfold (Hmelo-Silveret al., 2007). For example, learners tend to generate incomplete hypotheses, ignoreconflicting evidence or seek confirming evidence only (de Jong & van Joolingen,1998).

2.2. Computer-based learning environments for inquiry

learning in chemistry

Technology can serve various functions in inquiry learning settings. In chem-istry education, the most obvious function is that of enabling inquiry activities.Computer-based learning environments offer opportunities for students to engage



in experimentation and modeling activities that could not be realized without com-puters. Several constraints apply to the types of real chemistry experiments thatcan be planned and carried out by the students themselves, including safety issues,the complexity of the setup and procedures, and the cost of chemicals, glasswareand measuring equipment. Computer-based learning environments, more preciselysimulations, are not affected by these constraints. Using a simulation, learners canplan and carry out an unlimited number of experiments of varying degrees of com-plexity on their own. The Molecular Workbench (http://workbench.concord.org/)can be considered an excellent example of a simulation environment that enablesinquiry learning in chemistry. Students can use the Molecular Workbench to experi-ment with various chemical reagents, observe reactions at the microscopic level, testhypotheses and explore relationships among reagents and products. Neither couldthese experiments be carried out nor would observations at the microscopic levelbe feasible in a student laboratory (The Concord Consortium, n.d.).

As pointed out in the preceding section, inquiry learning has to be supported forits potential benefits to unfold. De Jong and his colleagues have proposed to provideinstructional support for inquiry learning through cognitive scaffolds (e.g. de Jong,2006). The majority of cognitive scaffolds to be found in the research literatureon inquiry learning take the form of tools that are integrated into computer-basedlearning environments, an example of which is Co-Lab (van Joolingen et al., 2005).

Furthermore, technology can be employed to create new opportunities for learn-ing from inquiry. Consider the following example, which extends the framework of“barriers, biases and opportunities” (Bromme et al., 2005) to inquiry learning inchemistry: A simulation of a chemistry laboratory has the advantages of reduc-ing the time, effort and materials required to carry out experiments as well as thehazards involved in them. The downsides are the lack of faithfulness to the realexperimental setup and procedures, which may be a barrier to learning, and theabsence of visual and auditory sensations, which may augment certain biases ininformation gathering and processing. But the fact that chemical reactions occur-ring in a simulated laboratory are typically not realistic with regard to some aspectscan be utilized to shift the learners’ attention from laboratory techniques to chem-ical concepts (Nakhleh et al., 2002). In the same vein, the absence of visual andauditory sensations can be turned into a new opportunity for learning by shifting thefocus from surface features of the experiments (fulmination, vapor, color changes)to the underlying chemical processes. Thus, computer-based learning environmentscan help to promote the acquisition of conceptual knowledge by enhancing inquirylearning: When they are designed accordingly they help learners to focus theirknowledge construction activities on the theories, principles and chemical processesunderlying the chemistry problems they are trying to solve.

An example of a computer-based learning environment for inquiry learning inchemistry that enables and supports inquiry learning is CoChemEx (Tsovaltzi et al.,2008). CoChemEx is a learning environment that aims at facilitating conceptualchemistry learning through structured collaborative inquiry activities and adaptive



Workbench

Stockroom

Informationwindow

Figure 1. Annotated screenshot of the interface of the simulated chemistry laboratory VLab(http://www.chemcollective.org/applets/vlab.php).

scaffolds. The central component of CoChemEx is the simulated chemistry labora-tory VLab (http://www.chemcollective.org/applets/vlab.php), an educational soft-ware environment designed to facilitate learning in various branches of chemistry,including stoichiometry (Evans et al., 2008). VLab simulates a physical chemistrylaboratory and has three main components (Figure 1): Experiments are conductedon the workbench where a number of tools are available to the learners. The stock-room is located to the left of the workbench. It holds an unlimited supply of thechemical reagents needed to solve a given task and distilled water. The informationwindow is located to the right of the workbench. It displays various types of dataregarding the contents of the selected piece of glassware, including volume, stateof matter, molarity and temperature. VLab is the simulated chemistry laboratorythat we started from when we designed our stoichiometry learning environment.

2.3. Collaborative inquiry learning

Before the turn of the last century, studies on inquiry learning focused almost exclu-sively on individual learning. Those researchers who recently turned their attentionto collaborative inquiry learning (e.g. Chang et al., 2003; Okada & Simon, 1997;van Joolingen et al., 2005) mostly investigated the effectiveness of a computer-based learning environment or the effectiveness of some kind of cognitive scaffoldthat had been added to the environment. Their claim was that collaboration, thatis, two or more learners solving a problem or studying a text together, enhances theeffectiveness of inquiry learning with simulations and cognitive tools.



There is a general lack of empirical studies on the effects of collaborative learningin stoichiometry and in every other branch of chemistry. We were able to identifyfew studies that investigated collaborative learning in chemistry (Basili & Sanford,1991; Chiu & Linn, 2008; Fasching & Erickson, 1985; Kaartinen & Kumpulainen,2002; Tingle & Good, 1990; Towns & Grant, 1997). The studies by Chiu and Linn(2008) and Kaartinen and Kumpulainen (2002) were the only ones we were able toidentify that dealt with collaborative inquiry learning in chemistry. Chiu and Linn(2008) had high school students study a unit on chemical reactions collaborativelyusing the Web-based Inquiry Science Environment (WISE). They found that thestudents gained a more adequate understanding of chemical reactions at all threelevels of representation (macroscopic, microscopic and symbolic) from engaging incollaborative inquiry. The study also suggested that explanation prompts facilitatemetacognitive monitoring and encourage subsequent metacognitive regulation withstudents learning collaboratively using a simulation environment. Kaartinen andKumpulainen (2002) had 18 university students work in small groups on a learn-ing task requiring them to engage in collaborative experimentation. Instructionalsupport for collaboration and experimentation was not provided. Kaartinen andKumpulainen found that the number of explicatory answers given by the studentsin response to a set of questions on the target concept increased from pretest toposttest whereas the number of descriptive and practical answers decreased, thussuggesting a positive influence of collaborative experimentation on the students’conceptual knowledge. But the study suffered from a number of methodologicalweaknesses, particularly the lack of a control condition. Hence, definite conclusionswith regard to the effectiveness of collaborative inquiry learning in chemistry cannotbe drawn from the results.

Of the remaining studies we identified that dealt with collaborative learning inchemistry, two also used a one-group design (Fasching & Erickson, 1985; Towns &Grant, 1997), thus limiting conclusions with regard to the effectiveness of theapproach under investigation. Hence, only two studies remain to be reviewed here.Basili and Sanford (1991) compared the number of misconceptions about conserva-tion laws held by college students who had worked in small groups to the numberof misconceptions held by students who had received traditional instruction for thesame number of class periods. Their main finding was a lower proportion of miscon-ceptions about conservation laws among the students in the treatment condition.The study thus showed that college students can benefit from collaboration in stoi-chiometry at the level of conceptual knowledge. Conflicting evidence was presentedby Tingle and Good (1990). They did not find differences in problem-solving perfor-mance and conceptual understanding between a sample of high school students whohad solved stoichiometry problems individually and a sample of students who hadworked on the problems collaboratively. However, the study suffered from severaldrawbacks. Hence, the conclusions that can be drawn are subject to limitations.

Despite the scarcity and inconsistency of empirical findings there are reasonsto assume that collaborative inquiry learning in chemistry can be effective with



regard to the acquisition of conceptual knowledge. First of all, evidence from studiesin mathematics (e.g. Moschkovich, 1996), physics (Ploetzner et al., 1999), biology(Okada & Simon, 1997) and scientific experimentation (Teasley, 1995) indicates thatcollaborative learning (in face-to-face as well as in computer-mediated settings) canpromote the acquisition of conceptual knowledge.

There also is a rich body of research on the mechanisms of learning during collab-oration. Studies have shown that giving explanations is an important process withregard to learning during collaboration (Hausmann et al., 2004; Okada & Simon,1997; Webb, 1989). In a meta-analysis, Webb (1989) found that giving explana-tions has positive effects on learning and performance when the explanations arehighly elaborate, the reason being that such explanations require active processing.Another mechanism that has been discussed in the literature is co-construction orjoint production of knowledge (Berg, 1993, 1994; Hausmann et al., 2004; Jeong &Chi, 1997). Berg (1993, 1994), for example, observed a significant positive corre-lation between joint production of knowledge during collaboration and subsequentalgebra performance. Metacognitive activities such as planning, monitoring andreflection have also been found to contribute to learning during collaboration (e.g.Bielaczyc et al., 1994).

While engaging in explanatory, co-constructive and metacognitive activities hasproven to be important for knowledge acquisition and subsequent problem-solvingperformance they are not the only processes that constitute “good” collaboration.To create conditions under which mechanisms such as explaining can take effectthe learners have to manage their interactions and organize themselves. The coreprocess enabling effective interactions among the collaborating learners is coordina-tion. It comprises aspects such as coordinating communication and interaction atthe task level, the content level and the process level (Dillenbourg et al., 1995; Meieret al., 2007) and coordinating the collaborative activities with regard to attention,motivation, time and technology (Barron, 2000; Meier et al., 2007).

Although there is empirical evidence of the effectiveness of collaborative learn-ing in scientific disciplines and the mechanisms of learning during collaboration arewell-understood, the implementation of collaborative (inquiry) learning in instruc-tional contexts, be it a face-to-face or a computer-mediated one, does not guaranteelearning. Left to their own devices, learners seldom engage in effective collaboration.Rather, they tend to deal with problems at the surface level only (e.g. Salomon &Globerson, 1989), arrive quickly at a shared solution instead of engaging in deepand meaningful interactions (e.g. Kerr et al., 1996), insufficiently coordinate theirinteractions (e.g. Baker & Bielaczyc, 1995; Barron, 2000) and often fail to reachand maintain shared understanding (Baker & Bielaczyc, 1995).

Altogether, the claim that collaboration enhances the effectiveness of inquirylearning with computer simulations is justifiable as collaborative learning can pro-mote the acquisition of conceptual knowledge through specific mechanisms andunder certain conditions. Like individual inquiry learning, however, collaborative(inquiry) learning requires instructional support to unfold its potential benefits.



Collaboration scripts represent a means of instructional support that can enable andencourage the learners to engage in effective collaboration. Therefore, the followingsection deals with collaboration scripts.

2.4. Instructional support for collaboration: Collaboration scripts

Starting with the work of Dansereau (1988) and O’Donnell and Dansereau (1992),evidence has accumulated that collaboration scripts are an effective means ofinstructional support for collaboration. What are collaboration scripts? In the mostgeneral terms, collaboration scripts are sequences of activities arranged to promptinteractions among the learners that might not occur spontaneously or take anunproductive or even counterproductive form if unsupported (Kollar et al., 2006).In her definition, King (2007) emphasizes the instructional nature of collaborationscripts: Collaboration scripts

“describe how collaborative learning can be externally structuredor scaffolded for the purpose of prompting group interaction thatpromotes learning. Scripting of the interaction during collaborationis designed so that the roles of participants, actions engaged in, andthe sequence of events, prompt specific cognitive, socio-cognitive,and metacognitive processes, thus ensuring that the intended learn-ing takes place.” (p. 25)

The prompting of cognitive, socio-cognitive and metacognitive processes referred toby King in her above definition is often realized with conversation starters or pre-structured questions, which the learners are instructed to use during collaboration.The conversation starters are designed to promote those processes known to underlyeffective collaborative learning, in particular elaborated explanations, knowledge co-construction and joint reflection (e.g. King, 1994; Teasley, 1997; also see Section 2.3).

Collaboration scripts have been studied extensively in face-to-face learningsettings (for a comprehensive review, see Kollar et al., 2006). Among the earli-est examples of script-like instructional techniques developed and implemented inthe classroom are the jigsaw approach by Aronson (1978) and the equally well-known reciprocal teaching approach by Palinscar and Brown (1984). Collaborationscripts for computer-supported (including computer-mediated) settings entered theresearch agenda towards the end of the past decade (cf. Kollar et al., 2006). Oneof the first studies on scripting collaboration in computer-supported settings wasthat by Hron et al. (1997). Recent years have seen a surge in the number of studiespublished on collaboration scripts for computer-supported settings (e.g. Rummel &Spada, 2005b; Weinberger et al., 2007).

Yet, very few studies among the large body of research on collaborationscripts are from the domain of chemistry. And as far as we are aware, noneof these studies dealt with scripting collaboration in stoichiometry. One team ofresearchers has implemented and tested collaboration scripts for general chemistry.



Sumfleth et al. (2004a, 2004b) conducted a study involving 7th grade students.Small groups of students were given a number of materials, including a collabo-ration script that provided guidance on role distribution and turn taking amongthe collaborators. Compared to the students under the control condition, who wereexposed to teacher-centered instruction during the six-lesson treatment phase, thestudents under the treatment condition acquired significantly more knowledge ofchemistry. The difference in knowledge was yet stronger at follow-up, six monthsafter the treatment. Walpuski (2006), who was a member of the Sumfleth team atthat time, tested the effectiveness of two means of instructional support — struc-tural support and error feedback. The structural support was designed to guide thecollaborating learners through a sequence of activities similar to the process of sci-entific discovery. Walpuski’s sample consisted of 320 German high school students.The results were not consistent with the findings reported by Sumfleth et al. (2004a,2004b). Walpuski (2006) identified a significant main effect of error feedback but noeffect of structural support, that is, the collaboration script. Altogether, the evidenceavailable to date does not allow definite conclusions with regard to the effective-ness of collaboration scripts as a means of instructional support for collaborationin chemistry. There is a clear need for further research on scripting collaboration inchemistry, particularly in stoichiometry.

Evidence of the effectiveness of collaboration scripts comes, however, from otherscientific disciplines (e.g. King, 1994) and from the domain of mathematics (Berg,1993, 1994; Kramarski, 2004). Berg (1993, 1994), for example, developed a scriptto support collaborating learners in the domain of algebra. In her field study sheasked pairs of high school students to interact as instructed by the collaborationscript. The students of the control condition received teacher-centered instructionduring the treatment phase. Berg found that the collaboration script had a pos-itive effect on performance at posttest and at follow-up. Berg’s findings are cor-roborated by Kramarski (2004). She found that learners who worked through asequence of metacognitive activities gained more from solving math problems col-laboratively than learners who did not receive that kind of instructional supportfor collaboration.

Since the publication of the first studies on collaboration scripts for computer-supported settings researchers have been voicing their concerns about the restric-tiveness of collaboration scripts (see e.g. Hron et al., 1997). In his frequently citedarticle, Dillenbourg (2002) argued that a balance needs to be obtained betweenstructuring the interactions among the learners and allowing them to adapt thecollaboration script to their needs. Dillenbourg introduced the term over-scriptingto refer to the negative effects of imposing “one size fits all” structures on thelearners’ natural interactions and problem solving. The problem parallels the assis-tance dilemma (Koedinger & Aleven, 2007), which continues to be discussed inthe literature on individual learning — whether to provide instructional supportand, if so, when and to what extent. With regard to collaboration scripts as ameans of instructional support for collaboration, adaptation may be a way out ofthe dilemma (Rummel & Weinberger, 2008). Adaptive collaboration scripts can



meet the learners’ needs without the risk of constraining their interactions in anon-beneficial way or impeding their problem solving.

So far little research has been conducted on adaptively scripting collaboration(Rummel & Weinberger, 2008). Gweon et al. (2006) investigated the effects of anadaptive collaboration script implemented in a computer-mediated learning envi-ronment in which pairs of students worked through complex math problems. Theadaptive component of the script consisted of prompts that aimed at explanation,reflection and balanced participation. The results of the Gweon et al. study showedthat the adaptive collaboration script influenced positively the quality of the stu-dents’ collaboration and their individual learning outcomes. While the results werepromising, the study also highlighted the need for further research into adaptive col-laboration scripts. Our hope is that our work will contribute to closing this currentgap in research on collaboration scripts.

3. Our Approach to Facilitating Learning fromComputer-Supported Collaborative Inquiry

Research has shown that instructional support is necessary to ensure learning frominquiry in computer-based learning environments (e.g. de Jong & van Joolingen,1998). Based on the empirical evidence available to date, collaborative learningconstitutes a reasonable extension of computer-based inquiry learning environ-ments. But collaborative learning has to be scaffolded for its mechanisms to takeeffect. Thus, when inquiry learning is extended to computer-supported collabora-tive inquiry learning the instructional support provided to the learners should bedesigned in such a way that (a) the learners engage in the central activities of theinquiry cycle (de Jong, 2005), (b) the learners show “good” collaboration, that is,engage in fruitful collaborative activities and (c) the learners’ interactions (partic-ularly their explanatory, co-constructive and metacognitive activities) are directedat their inquiry activities and at productive use of the simulations and cognitivetools offered to them.

The present state of research suggests that collaboration scripts can meet thesedemands (cf. Kollar et al., 2006). We hypothesize that a promising approach toscripting computer-supported collaborative inquiry learning is to integrate struc-tural and adaptive support. We developed such a collaboration script and partlyincorporated it into the computer-based learning environment, which the learnersused to carry out their collaborative inquiry activities.

3.1. Structural support components

A sequence of activities modeled on de Jong’s (2006) inquiry cycle formed thebackbone of the collaboration script. Hence, the script prompted the processes whichare known to be important for learning from inquiry. We made sure, however, tophrase the script instructions (presented to the learners in the form of a booklet) ina way that emphasized the collaborative aspects of the activities, thus encouraging



the processes responsible for learning during collaboration. In addition, the scriptinstructions assisted the learners in using the computer-based learning environmentproductively by pointing out to them which component of the learning environmentto use for which inquiry activity.

The inquiry activities were grouped into five phases (see Table 1): (1) brain-storming, (2) developing a strategy, (3) conducting experiments, (4) analyzing theexperiments and (5) checking the solution. Apart from the first phase, all of thephases were designated as collaborative phases. We decided to place an individualphase at the beginning of the sequence of activities as research has shown thatadding some individual activities to the joint activities instructed by collabora-tion scripts enhances the quality of interactions (Hermann et al., 2001; Rummel &Spada, 2005a). Moreover, the central activity of the first phase was brainstorming.Empirical evidence indicates that collaborative brainstorming is less effective thanindividual brainstorming (Diehl & Stroebe, 1991), which provides further supportfor our decision to designate the first phase as an individual one.

In order to structure the learners’ collaborative inquiry to a sufficient extentwhile at the same time being minimally coercive, the script instructions asked thelearners to work through the phases once only, and left it up to them to decide onthe order of activities within each phase.

At two points of the sequence of activities, the phases were supplemented byconversation starters that aimed at promoting co-construction and explaining —two of the mechanisms which have been found to contribute substantially to learn-ing during collaboration (e.g. Hausmann et al., 2004; King, 1994; Teasley, 1997).We assumed that those points, namely the beginning of the second phase and thebeginning of the fourth phase, marked critical moments with regard to the learn-ers’ collaborative inquiry and required additional structural support. Therefore, theconversation starter at the beginning of the second phase encouraged learners toco-construct knowledge: “I think our ideas of how to solve the chemistry problemdiffer the most with regard to. . .” At the beginning of the fourth phase the learnerswere presented with a conversation starter prompting them to engage in explaining:“Looking at the results of our experiments I do not understand why. . .” The con-versation starters were integrated into the computer-based learning environment.

3.2. Adaptive support components

The main adaptive support component of our collaboration script consisted of34 prompts aimed at promoting effective collaboration and preventing ineffectiveinteractions. We took both a top-down and a bottom-up perspective for the devel-opment of the adaptive collaboration prompts. First, we created a multi-level taxon-omy of the major aspects of effective and ineffective collaboration to be found in theliterature and constructed generic collaboration prompts based on the taxonomy.Second, we created a multi-level taxonomy comprising behaviors that we expectedto occur frequently with high school students engaging in collaborative inquiry in



Table

1.O

ver

vie

wofth

ephase

softh

eco

llabora

tion

scri

pt.

Seq

uen

ceofA

ctiv

itie

sC

om

ponen

tsofth

eC

om

pute

r-B

ase

dP

roduct

tobe

Cre

ate

dT

ime

Lea

rnin

gE

nvir

onm

ent

tobe

Use

dby

the

Lea

rner

sR

ecom

men

dati

on

Phase

1:B

rain

storm

ing

(1)

Rea

dth

epro

ble

mdes

crip

tion

(2)

Bra

inst

orm

about

chem

icalco

nce

pts

and

pri

nci

ple

sre

leva

nt

toth

est

oic

hio

met

ric

pro

ble

m(3

)Take

note

son

main

idea

s(4

)In

form

collabora

tor

when

done

Cognit

ive

tool(n

ote

-takin

gand

min

d-m

appin

g)

Sum

mary

ofid

eas

7m

in

Phase

2:D

evelo

pin

ga

strate

gy

(1)

Rea

dth

eco

nver

sati

on

start

er(2

)D

iscu

ssdiff

eren

ces

and

sim

ilari

ties

bet

wee

nid

eas

(3)

Dev

elop

ast

rate

gy

for

solv

ing

the

pro

ble

mand

pla

nsu

itable

exper

imen

ts(4

)Take

note

son

stra

tegy

Cognit

ive

tool(n

ote

-takin

gand

min

d-m

appin

g)

Sum

mary

ofst

rate

gy

7m

in

Phase

3:C

onducti

ng

experim

ents

(1)

Conduct

thre

e(m

ax.)

exper

imen

tsand

consi

der

poss

ible

resu

lts

(2)

Take

note

son

main

exper

imen

talre

sult

s

Sim

ula

tion

(vir

tualch

emis

try

labora

tory

),co

gnit

ive

tool

(note

-takin

gand

min

d-m

appin

g)

Note

son

exper

imen

tal

resu

lts

7m

in

Phase

4:A

naly

zin

gth

eexperim

ents

(1)

Rea

dth

eco

nver

sati

on

start

er(2

)D

iscu

ssch

emic

alco

nce

pts

and

pri

nci

ple

sw

ith

regard

toth

eex

per

imen

talre

sult

s(3

)C

onsi

der

aso

luti

on

toth

epro

ble

m(4

)Take

note

son

the

analy

sis

and

inte

rpre

tati

on

ofth

eex

per

imen

talre

sult

s

Sim

ula

tion

(vir

tualch

emis

try

labora

tory

),co

gnit

ive

tool

(note

-takin

gand

min

d-m

appin

g),

calc

ula

tor

Sum

mary

ofre

sult

s10

min

Phase

5:C

heckin

gth

eso

luti

on

(1)

Wri

tedow

nth

eso

luti

on

and

hav

eth

eex

per

imen

ter

chec

kit

Cognit

ive

tool(n

ote

-takin

gand

min

d-m

appin

g)

3m

in



the stoichiometry learning environment. In developing this taxonomy, we drew onthe comments of chemistry teachers who had tested the learning environment, pilottests involving high school students and observations we made while watching anumber of learners work with the simulated chemistry laboratory VLab (the corecomponent of the learning environment) during a previous study (Tsovaltzi et al.,2008). Finally, we matched the generic collaboration prompts of the top-down tax-onomy to the lowest-level categories of the bottom-up taxonomy and rephrased theprompts to maximize their relevance and clarity in the context of the stoichiometrylearning environment. We had not expected the taxonomies to match perfectly asevery system constitutes a specific learning setting, thus affording specific behav-iors. Therefore, when a generic collaboration prompt did not match a category ofthe bottom-up taxonomy it was removed. When a category of the bottom-up tax-onomy did not line up with any of the generic collaboration prompts we created anew prompt.

An example of a collaboration prompt, together with the branches of the twotaxonomies linked to this prompt, is displayed in Figure 2 (please contact authorsfor a full list of the prompts and for the two taxonomies). This is how the exam-ple is to be decoded from the right: The research literature indicates that givingjustifications, which is very similar to giving explanations (see Section 2.3), is oneof the processes responsible for learning during collaboration (Okada & Simon,1997). Therefore, the learners should be encouraged to explain to one another theirrationale for proposing a particular experiment. And this is how the example is tobe read from the left: The learners are engaging in a collaborative inquiry activ-ity and focusing on the same section of the learning environment or on the samematerial. Next, it is observed that the learners are talking to one another. At acertain moment, one of the learners proposes to carry out an experiment. How-ever, he does not explain or justify, for example by referring to a chemical concept,why they should carry out the experiment. Instead, he might say something like,“Let’s mix some of the blue solution and some of the yellow solution and see whathappens. How about 50ml each?” The learners will then receive the collaborationprompt displayed at the intersection of the two taxonomies: “Before you conductan experiment in the virtual laboratory explain to each other why you decided onthis experiment.” However, the prompt will be delayed until the collaborators plana second experiment without a theoretical rationale. With the exception of somesevere instances, one-time observations do not trigger prompts immediately so thatthe learners remain in charge of their collaboration and are allowed time to monitorand regulate their behavior on their own.

The second adaptive support component of our collaboration script consisted ofsix prompts that were to safeguard fidelity to the script, or more precisely, fidelityto the sequence of activities described in the preceding section. The overall purposeof the script was to provide instructional support to the learners while minimizingthe risk of over-scripting. Therefore, fidelity to the script was not enforced but



Fig

ure

2.E

xam

ple

ofadapti

ve

collabora

tion

pro

mpt.



prompted in an adaptive fashion when the learners lingered on an activity, skippedan important activity altogether or showed trial-and-error behavior.

The third adaptive support component of the script was a set of hints designedto provide support for experimentation at the strategic and conceptual levels. Weincorporated this minimum amount of domain-specific support into the script inorder to ensure that the collaborating learners did not fail to learn because of a dead-end problem-solving path, severe errors in their experimental design or continuoustrial-and-error behavior.

All of the adaptive support components were implemented in a Wizard of Ozfashion in this first exploratory step of our work. The collaborating learners weremade to believe that their actions in the computer-based learning environmentand their verbal interactions were being monitored by a computer program. Butin fact, a confederate to the experimenter (wizard) worked behind the scenes. Thewizard observed the learners’ collaborative inquiry activities in the computer-basedlearning environment. When she observed a lack of effective collaboration or theoccurrence of ineffective interactions, as specified in the bottom-up taxonomy, shesent a prompt or hint that appeared on the learners’ screen.

4. Research Questions

The collaboration script described in the previous section was designed to enhancecollaborative inquiry learning when using the simulated chemistry laboratory VLab(http://www.chemcollective.org/applets/vlab.php). The collaboration script, thesimulation and a cognitive tool (see Section 5.3 for details) formed a prototypelearning environment. The prototype was computer-based with the exception of onecomponent of the collaboration script (i.e. the sequence of inquiry activities). In theexploratory study reported in this paper, we conducted a full empirical test run withthe prototype comparing a scripted collaboration condition to an unscripted one.The results of the study will guide our efforts of improving the collaboration scriptand inform the next iteration of our learning environment. Towards this end, weaimed at assessing the benefits of the learning environment and the added value ofthe collaboration script, understanding which factors contribute to the effectivenessor ineffectiveness of our approach to scripting, and identifying features of the learn-ing environment that might hinder productive collaboration and inquiry. The studyalso served the purpose of exploring the particular needs of the target population(German high school students) with respect to stoichiometry learning. We tried toanswer the following research questions:

• Can learners benefit from working collaboratively on a stoichiometry problemusing the learning environment with regard to procedural and conceptual knowl-edge of stoichiometry?

• Which features of the learning environment enable or hinder productive collabo-ration and inquiry?



• Do learners who are supported by a collaboration script, which combines struc-tural and adaptive support (as described above), benefit more from using thelearning environment than learners who are not supported by such a script?

• How does the collaboration script influence learners’ collaborative inquiry andproblem-solving efforts?

• Which script components, features of the simulation and learner characteristicsenhance or impede the effectiveness of the collaboration script?

• Is the current implementation of the collaboration script (non-adaptive struc-tural support, low degree of coerciveness of both structural and adaptive support,computer-based with one paper-based component) conducive to learning?

• Does the support provided by the collaboration script meet the needs of the targetpopulation?

5. Method

5.1. Participants

A total of 54 students from 12 high schools1 in the south-west of Germany, aged15 to 18 years (M = 16.60, SD = 0.74) participated in the study. Twenty-nine ofthe participants were female. Participation in the study was voluntary. The studentswere allowed to name a friend or classmate to be paired up with for the study. Thosewho did not name a friend or classmate were paired up with another student bythe experimenter. Every participant received 20 Euros for taking part in the study.

We excluded three of the 27 dyads that participated in the study from the maindata analysis. Two of the dyads stood out as outliers with regard to the numberof chemistry courses they had taken and their learning outcomes respectively. Thethird dyad was excluded because of irregularities in the experimental proceduresand because one of the two students dropped out before the end of the experiment.Thus the final sample included 24 dyads.

5.2. Design

The dyads (n = 24) were randomly assigned to two conditions. Under the scriptedcollaboration condition (n = 12 dyads), the dyads were assisted by the collaborationscript. Under the unscripted collaboration condition (n = 12 dyads) the dyads werenot assisted by the collaboration script. To ensure equal opportunities for learningunder both conditions, the dyads of the unscripted collaboration condition weregiven descriptions of the computer equipment, programs and files available to themduring the learning phase as well as a set of general experimentation hints (e.g.“Were the samples of zinc you used in your experiments large in enough for a reac-tion to occur?”). Both of these collaboration script substitutes were non-adaptive.

1More precisely, all of the students attended a Gymnasium at the time of the study. In the Germaneducation system, the Gymnasium is the type of high school that prepares students for college.



The duration and procedures of the experiment were identical for both condi-tions. Every participant took an individual knowledge posttest. We also collecteddata on the participants’ prior knowledge of stoichiometry, their attitudes towardcollaboration, their academic achievement in chemistry and mathematics and thenumber of chemistry courses they had taken.

5.3. Learning environment

During the learning phase, the dyads of both conditions worked on a stoichiome-try problem (see Section 5.4) in a computer-based learning environment. The maincomponent of the learning environment was the simulated chemistry laboratoryVLab described in the background chapter (see Figure 1). Its interface includes aworkbench, the stockroom and the information window. The information windowdisplays, for example, volume and molarity (see Figure 3).The other componentsof the learning environment were Microsoft Office PowerPoint, which we set up tofunction as a simple cognitive tool providing note-taking and mind-mapping func-tionality, and an on-board calculator. The components of the learning environmentoperated independent of one another. A dual-monitor setup allowed the dyads tosimultaneously view the virtual chemistry laboratory and the other tools.

Figure 3. Annotated screenshots of the information window of the simulated chemistry laboratoryVLab (http://www.chemcollective.org/applets/vlab.php).



5.4. Task for collaborative inquiry

The stoichiometry problem the dyads worked on during the learning phase askedthem to identify the chemical reaction of copper sulphate and zinc, to formulatethe chemical equation and to produce 20 grams of copper. The sample of copperwas to be free of zinc residues. The stockroom of the virtual chemistry laboratorycontained copper sulphate solution and zinc powder for the dyads to use in theirexperiments. The dyads were given information about the molarity of the coppersulphate solution, the ionic charge of copper sulphate and the molar mass of copperand zinc. They were also provided with a small chemistry glossary, which contained,for example, an entry on molar mass.

5.5. Experimental setup and procedures

The setup included both a collaborative workstation and an individual workstation.The wizard was seated at a separate workstation in the laboratory. Under theunscripted collaboration condition the dyads were not required to use the individualworkstation and the wizard’s computer was turned off.

All of the experiments were carried out by the same experimenter. Also, one per-son (a graduate student with substantial experience in scaffolding student learningin the domain of chemistry) served as the wizard in all of the experiments. Followingthe instructions about the experiment, the participants engaged in three prepara-tion activities: They (1) individually read a text about key stoichiometric concepts,(2) they watched two short videos on the computer-based learning environmentand (3) they collaboratively worked on two tasks to familiarize themselves withthe virtual chemistry laboratory and the note-taking and mind-mapping tool. Afterthe preparation phase, participants took an individual knowledge pretest (10 min-utes, time on task controlled by the experimenter). After a short break, the dyadswere given detailed instructions about the learning phase. They were told they had50 minutes to work on the stoichiometry problem and three chances of submittingthe correct solution. Each of the learners was given a description of the stoichiometryproblem. When the learners had read the description the experimenter announcedthe beginning of the learning phase and handed each of the learners a booklet thatcontained instructions on the phases of the collaboration script (scripted collab-oration condition) or descriptions of the computer equipment, programs and filesavailable during the learning phase (unscripted collaboration condition). The book-lets also contained a small chemistry glossary and a set of hints on how to use thesoftware, more specifically the virtual chemistry laboratory.

Under the scripted collaboration condition, the procedures for the learning phasewere as follows: During the first cycle, participants followed the collaboration script(see Table 1). After the first cycle, they were free to choose which activities toengage in. At the beginning of the first cycle, each learner was seated at one ofthe workstations. After the learners had completed the first phase of the script,they received technical support from the experimenter in initiating the transitional



phase. This also functioned as an activation signal to the wizard. After 40 minutes,the experimenter announced the time remaining of the learning phase. The dyadswere free to decide when they wanted to submit a solution. When they indicatedthey had solved the stoichiometry problem the experimenter checked the dyad’ssolution and gave standardized feedback. Under the unscripted collaboration con-dition, learners jointly decided whether and how to use the two workstations. Theywere given technical support if needed. After 25 minutes, the learners were handedthe experimentation hints (see Section 5.1). The procedures otherwise followed thoseunder the scripted collaboration condition.

After a short break, participants individually filled in the remaining question-naires (attitude questionnaire, evaluation questionnaire) and took the individualknowledge posttest (30 minutes, time on task controlled by the experimenter). Thenthey were interviewed about their experience during the study, debriefed and paidfor their participation. The total duration of the experiment was three hours.

5.6. Instruments

We developed a test to assess participants’ knowledge of stoichiometry at posttest.Participants were given a calculator together with the test and were allowed towork on the items in the order indicated for 30minutes. The knowledge posttestconsisted of eight free-response problem-solving items, which were subsumed undertwo scales: Four of the problem-solving items made up the procedural scale; fouritems made up the conceptual scale. The conceptual problem-solving items coveredall of the concepts that we expected participants’ to learn about from workingon the stoichiometry problem. The procedural problem-solving items covered basicstoichiometric procedures such as calculating mass and amount of substance. Whilethese items did not require conceptual knowledge, participants holding correct andadequate conceptual knowledge were likely to be more successful in solving them.

We also assessed participants’ prior knowledge of stoichiometry. The knowl-edge pretest was made up of three free-response problem-solving items (one con-ceptual, two procedural), which paralleled the first three items of the knowledgeposttest. The time that participants could spend working on the knowledge pretestwas limited to 10minutes.

The procedural problem-solving items and the conceptual problem-solving itemsof the knowledge tests were analyzed separately. Performance on the proceduralproblem-solving items was rated on a three-point scale, with two points indicatinga correct solution, one point indicating an incorrect solution and zero points indi-cating the item was not answered at all or the answer was unrelated to the problemdescription. Performance on the conceptual problem-solving items was rated on afour-point scale to allow for discrimination of answers that consisted of a partiallycorrect solution and answers that were incorrect and/or marked by severe con-ceptual errors, that is, errors indicating insufficient knowledge of the theories andprinciples underlying stoichiometry. Thus, correct solutions received three points,



partially correct solutions received two points, incorrect solutions and answers thatcontained two or more conceptual errors received one point. Again, blanks andanswers that did not relate to the problem description resulted in zero-point ratings.

Socio-demographic data and information on their academic achievement and thenumber of chemistry courses they had taken were provided by the participants priorto their participation in the study.

The attitude questionnaire that we used to assess participants’ attitudes towardcollaboration was a translated and slightly modified version of the Social Inter-dependence Scales (cooperative interdependence, individualistic interdependence,competitive interdependence) by Johnson and Norem-Hebeisen (1979). The atti-tude questionnaire was preceded by a questionnaire assessing the participants’ eval-uation of several aspects of the study. It combined one global (overall evaluationof the experiment) and three specific scales (collaboration, learning environment,instructional support). Participants were asked to provide their answers to the atti-tude questionnaire and the evaluation questionnaire on a six-point scale, rangingfrom not at all true to completely true.

Furthermore, we took screen and audio recordings during the learning phaseof the experiments and interviewed participants about their experience during thestudy at the very end of the experiments.

6. Results and Discussion

6.1. Learning outcomes — Implications for the design of the

stoichiometry learning environment and the collaboration script

The participants of both conditions showed weak performance on the conceptualproblem-solving items of the posttest, scoring less than half of the maximum totalon average (see Table 2). Despite stronger average performance on the proceduralproblem-solving items, the learning outcomes achieved in our study have to be con-sidered low. It could be argued that the knowledge posttest went beyond the level ofunderstanding that could be achieved from working on a single stoichiometry prob-lem in the learning environment when starting from the participants’ level of priorknowledge and general academic achievement. However, participants’ average per-formance on the knowledge pretest did not indicate a floor effect and, hence, a com-plete lack of prior knowledge in stoichiometry. Many participants failed to solve the

Table 2. Mean scores on the knowledge posttest scales by condition controlling for priorknowledge of stoichiometry.

Scripted Collaboration Unscripted Collaboration

M SD M SD

Conceptual problem-solving items 5.07 0.48 4.84 0.48Procedural problem-solving items 5.83 0.29 6.13 0.29

Note: The maximum total for the conceptual problem-solving items was 12 points. Themaximum total for the procedural problem-solving items was 8 points.



conceptual problem-solving items and arrived at partially correct solutions to theprocedural problem-solving items (at pretest as well as at posttest) because of whatseemed to be deficiencies in basic algebra skills (e.g. transforming fractions). Thehigh to medium correlations among posttest performance and academic achieve-ment in mathematics (conceptual problem-solving items: r = 0.57, p < 0.001;procedural problem-solving items: r = 0.38, p < 0.01) provide support to this con-clusion. As we had assumed that students from the target population possessedsufficient knowledge of algebra and as we had designed the stoichiometry problemaccordingly, participants’ low-level algebra skills might have impeded their learn-ing. Hence, our findings suggest that success in solving stoichiometric problemsdoes not depend only on conceptual knowledge of chemistry, as concluded fromprevious studies (e.g. BouJaoude & Barakat, 2003; Schmidt, 1990, 1992), but alsoon basic algebra skills. This is in line with Gabel and Bunce’s (1994) conceptual-ization of stoichiometric problem solving as solving “chemistry problems involvingmathematical reasoning skills” (p. 301). Hence, in future iterations of our stoichiom-etry learning environment we have to take into account students’ prior knowledgebeyond the focal learning domain and accommodate deficiencies in the adjacentdomain of algebra. This could be realized, for example, by including instructionalunits on the computational aspects involved in stoichiometric problem solving. VanMerrienboer et al.’s (2002) model of instructional design could then form the frame-work for the design of the learning environment. The model specifies how learningenvironments should be designed to facilitate the acquisition of skills that haveto be coordinated when solving complex problems. Stoichiometry problems consti-tute complex problems and solving them requires skills such as identifying relevantchemical concepts, determining the quantities consumed and produced by chemicalreactions, formulating equations and carrying out algebraic procedures. Accordingto the Van Merrienboer et al. model, learning environments used to train such com-plex problem-solving skills should be made up of four components: (1) a variety ofauthentic learning tasks grouped into task classes and ordered in simple-to-complexsequences; (2) part-task practice for low-level procedural skills, that is, sets of prac-tice items that are interspersed among the learning tasks; (3) continuous availabilityof supportive information, that is, resources (e.g. cognitive strategies) for solvingclasses of learning tasks; (4) just-in-time (JIT) information (e.g. information dis-plays), that is, information relevant to a set of practice items which is presented tolearners before they work on the items. When applying the Van Merrienboer et al.model to our stoichiometry learning environment we would maintain the sequenceof learning activities specified by the collaboration script but supplement it withpart-task practice and JIT information.

The participants of the two conditions did not differ significantly with regard toprior knowledge of stoichiometry, attitudes toward collaboration, academic achieve-ment in chemistry and mathematics and number of chemistry courses taken. Hence,the subsamples can be considered comparable. A one-way multivariate analy-sis of covariance (MANCOVA) with procedural and conceptual problem-solving



performance at posttest as the dependent variables and prior knowledge of sto-ichiometry (measured by the pretest) as the covariate did not yield significantdifferences between the scripted collaboration condition and the unscripted col-laboration condition, F (2, 20) = 0.465, n.s., η2 = 0.04. Thus, it seems that thedyads of the scripted collaboration condition did not benefit more from solvingthe stoichiometry problem than the dyads of the unscripted collaboration conditionthat did not receive instructional support for collaborative inquiry learning. Thisresult suggests that scripting collaborative inquiry does not enhance stoichiometrylearning — a finding contrary to studies reporting positive effects of collaborationscripts on learning in science and mathematics (e.g. Berg, 1993, 1994; King, 1994;Kramarski, 2004). Several factors might have contributed to the ineffectiveness ofthe collaboration script in our study, all of which implicate modifications of ourapproach to scripting and improvements of the design of the stoichiometry learningenvironment.

First, the participants had never encountered a collaboration script. Thus, apply-ing the script might have increased cognitive load during the learning phase (cf. Dil-lenbourg, 2002; Gweon et al., 2006). Training students on how to use the collabora-tion script prior to the learning phase might prove effective (cf. Rummel & Spada,2005). However, we consider turning the paper-based component of the collabo-ration script into a technology-enhanced component a more promising approach.If the sequence of inquiry activities, which was instructed on paper in our initialprototype, was implemented via the interface of the virtual chemistry laboratorythe amount of structural support provided could be adapted to the collaborators’changing needs (cf. Diziol et al., 2010). Furthermore, the support could then befaded out as the collaborators internalize the script instructions (cf. Wecker et al.,2010). This would move the stoichiometry learning environment further along theline toward a system providing adaptive support for collaborative inquiry learning.

Second, we observed that most of the dyads spent more time on calculationsthan on discussing chemical concepts and stoichiometric principles during Phase 4.Hence, although the collaboration script targeted explanatory activities, partici-pants tended to focus on carrying out procedures. This suggests that the collabo-ration script has to be modified to focus students on the concepts and principlesunderlying the stoichiometry problem. Phase 4 of the collaboration script could, forexample, be split and specify two roles. In Phase 4a, one of the collaborators wouldbe assigned the task of explaining and drawing conclusions. The other would beassigned the task of monitoring the explanations for adequacy and completeness. InPhase 4b, one of the collaborators would perform the calculations required to solvethe stoichiometry problem. The second role would involve monitoring the calcula-tions and checking the solution against the explanations constructed in Phase 4a.

Third, the demands involved in relating the multiple external representations inthe virtual chemistry laboratory might have rendered the collaboration script inef-fective. The information window of the virtual chemistry laboratory holds multipleexternal representations (Figure 3) that differ in their form (text, graphics) and in



the level at which they represent matter and the chemical processes it is subject to(macroscopic level, microscopic level, symbolic level). Coordinating and integratingthe information from separate representations can be considered one of the mostdifficult tasks when learning with multiple external representations in any knowl-edge domain (Ainsworth, 2006). In the case of the virtual chemistry laboratory, thedifficulty of relating the representations is increased by the fact that understandingthe interrelatedness of the macroscopic level, the microscopic level and the symboliclevel is essential to stoichiometric problem solving but a challenge in itself, regardlessof the design and functions of the representations. As Gabel (1998) pointed out, oneof the chief reasons why many high school and college students fail to solve chemistryproblems is the difficulty of understanding the interrelatedness of the three levelsof representation. In fact, participants’ answers in the interviews and the chem-istry teachers’ comments on the virtual chemistry laboratory pointed toward thecomplexity of the data displays. With our collaboration script, the challenge wasexacerbated by the absence of instructional support for relating representations.When redesigning the collaboration script we will, therefore, include scaffoldingfor interrelating the different representations in the simulation. The literature onsupporting connection-making between multiple representations demonstrates thatpromoting active integration activities on the part of the students is crucial forlearning (e.g. Bodemer & Faust, 2006; Bodemer et al., 2004). This means, thatthe redesigned collaboration script should prompt learners to explain the differentrepresentations and their interrelations to one another. Prompts for connection-making alone, however, are not sufficient. The system should support students bydynamically linking representations and thus highlighting relevant relations (Vander Meij, 2007). Furthermore, it was found that support for interrelating repre-sentations should focus on deep, structural relations rather than surface features(Seufert & Brunken, 2006).

6.2. Learning processes — Instructional design, interface design

and interaction design implications

In addition to our quantitative data analyses, we carried out a detailed qualita-tive analysis of two selected cases from the scripted collaboration condition. Byanalyzing the screen and audio recordings taken during the learning phase of theexperiments we gained insights into the processes that occurred during the learningphase. This allowed us to extend and refine the design implications drawn fromthe quantitative results. The qualitative analysis was exploratory and guided bya set of questions. The questions focused on the following aspects: (1) The qual-ity of the learners’ collaboration, (2) the effects of the collaboration script on thelearners’ interactions and their joint attempts at solving the stoichiometry problemand (3) the learners’ reactions and fidelity to the collaboration script. In particular,we were interested to see whether the dyads accepted the adaptive support andwhether their collaboration improved following the prompts and hints.



We selected one dyad (Sio and Adi2) for the qualitative analysis that had shownbelow-average performance at pretest but above-average performance at posttest,both on the conceptual problem-solving items and on the procedural problem-solving items. In the second dyad (Sun and Mia), the pattern was reversed: Theyhad achieved above-average scores at pretest but showed below-average performanceat posttest. These patterns of prior knowledge and learning outcomes suggested tous that Sun and Mia had learned less from working on the stoichiometry problemdespite script support than Sio and Adi.

6.2.1. Sio and Adi: Successful learning despite difficulties in understanding

Table 3 shows how Sio and Adi structured their learning phase. The table alsodisplays the episodes central to their collaborative inquiry and problem solving aswell as the prompts and hints given to them by the wizard.

To reveal the outcome of the dyad’s efforts up front: Sio and Adi were not suc-cessful at solving the stoichiometry problem even though they conducted a numberof crucial experiments and drew some correct conclusions. The problem was thatthey often failed to take into account all of the information available in the virtualchemistry laboratory when analyzing the results of their experiments. During theepisode beginning at 39:29 minutes, for example, they did not take into consider-ation the information on the solid species. Adi even uttered his incomprehensiongiven the “disappearance” of the solid species they had added to a beaker: “I can’tbelieve there’s nothing in it!” (translation by the authors) In fact, we observedthe same difficulty with several dyads. In the virtual chemistry laboratory, aqueousspecies and solid species are displayed on different tabs. But these tabs were easyto miss and several dyads seemed to consider one of the tabs only. Thus, we assumethe design of the virtual chemistry laboratory posed an unnecessary challenge to thestudents. This makes improvements of the interface and interaction design of thevirtual chemistry laboratory paramount.

However, Sio and Adi experienced difficulties beyond usability issues. Both stu-dents showed deficiencies in their understanding of the chemical composition andthe role of one of the substances involved in the chemical reaction. A similar lackof understanding of general chemistry concepts emerged with many dyads, indicat-ing that at least some students require more domain-related instructional supportthan we had provided. As with the deficiencies in basic algebra skills, we suggestinstructional units be added to the stoichiometry learning environment. If the gen-eral design of the learning environment was refined based on the Van Merrienboeret al. (2002) model of instructional design, as suggested in Section 6.1, this could berealized by supportive information that students can access when needed. In addi-tion, we plan to implement a macro-assessment of students’ knowledge of generalchemistry concepts prior to their collaborative inquiry in the stoichiometry learning

2The participants are given fake names to increase the readability of the following text.



Table

3.Sio

and

Adi:

Over

vie

wofti

me

spen

ton

phase

s,im

port

ant

epis

odes

,pro

mpts

and

hin

tsgiv

enby

the

wiz

ard

.

Phase

Fro

m–to

(min

)O

nse

tofE

pis

ode

Short

Des

crip

tion

ofE

pis

ode

Pro

mpt

Giv

enby

Wiz

ard

Phase

1(b

rain

storm

ing)

(17:0

3m

in)

(indiv

idualw

ork

)

Tra

nsi

tionalphase

(exch

angin

gid

eas)

17:0

4–19:1

0(2

:06

min

)(i

ndiv

idualw

ork

)

Phase

2(d

evel

opin

g

ast

rate

gy)

19:1

1–29:5

3(1

0:4

2m

in)

25:5

9T

he

wiz

ard

gav

ea

collabora

tion

pro

mpt:

The

dyad

then

com

pare

dth

eir

sum

mari

esofid

eas

and

consi

der

edboth

inth

eir

join

tst

rate

gy.

“It

isim

port

ant

that

each

of

you

contr

ibute

sso

me

ofth

eid

eas

you

cam

eup

wit

hduri

ng

the

firs

tphase

toyour

join

tst

rate

gy.

”P

hase

3(c

onduct

ing

exper

imen

ts)

29:5

4–en

dofle

arn

ing

phase

(appro

x.

20

min

)

30:4

8T

he

dyad

conduct

eda

cruci

alex

per

imen

tand

made

ajo

int

att

empt

at

under

standin

gth

ere

sult

.H

owev

er,th

eyfa

iled

todra

wa

defi

nit

eco

ncl

usi

on.

39:0

9E

xper

imen

tati

on

hin

t:It

cannot

be

evalu

ate

dw

het

her

the

dyad

pro

cess

edth

ishin

t.

“D

iscu

ssw

ith

your

part

ner

whet

her

the

sam

ple

ofzi

nc

you

use

din

your

exper

imen

tw

as

larg

een

ough

for

are

act

ion

toocc

ur.

”39:2

9T

he

dyad

conduct

eda

cruci

alex

per

imen

tbut

failed

todra

wan

adeq

uate

concl

usi

on.T

hey

had

not

taken

into

consi

der

ati

on

the

info

rmati

on

on

the

solidsp

ecie

s.42:2

9T

he

dyad

com

pare

dtw

osa

mple

sobta

ined

from

the

sam

eex

per

imen

tand

dre

wa

corr

ect

concl

usi

on

wit

hre

spec

tto

the

solid

spec

ies.

43:2

4Fid

elity-t

o-s

crip

tpro

mpt:

The

dyad

ignore

dth

epro

mpt.

“R

emem

ber

tota

ke

note

son

the

resu

lts

ofyour

exper

imen

ts.”

46:3

0E

xper

imen

tati

on

hin

t:T

he

dyad

reso

rted

totr

ial-and-e

rror

exper

imen

tati

on.

Sam

eas

at

39:0

9.



environment. Based on their performance on the macro-assessment, the collaborat-ing learners’ attention will then be directed to specific supportive information units.Such a macro-assessment would take us yet another step closer toward a systemproviding adaptive support for collaborative inquiry learning in stoichiometry. Asimilar approach was taken by Walker et al. (2010): In their computer-supportedcollaborative learning system, help resources were made available to learners on thebasis of an on-line assessment of their behaviors.

Concerning their collaboration, Sio and Adi showed effective behavior for themost part of the learning phase. Each of the two gave many explanations to his col-laborator, they co-constructed some explanations and — in general — they coor-dinated their interactions well. During the second phase (developing a strategy),for example, Sio explained to Adi why some of his ideas from the first phase wereincorrect; Adi in turn explained why his calculations were correct; then the collab-orators co-constructed a chemical equation. However, it has to be pointed out thatmost of the explanations that Sio and Adi gave or co-constructed did not refer totheoretical concepts, stoichiometric principles or processes, therefore having to bedescribed as shallow. This observation reinforces our previous suggestion of splittingsome of the phases of the collaboration script and assigning roles to the collabora-tors to promote certain cognitive and metacognitive activities. By redesigning ourcollaboration script to separate conceptual activities from procedural activities (inthe case of Phase 2, discussing concepts and planning experiments), we hope to beable to increase the number of deep explanations constructed by the collaboratinglearners.

Yet, the quality of this dyad’s collaboration is reflected in the low number ofprompts they received from the wizard. The only collaboration prompt they receivedseems to have influenced their collaboration in a positive way: At 25:59 minutes, Sioand Adi were reminded to integrate their ideas from the previous phase while devel-oping a joint strategy. They studied this prompt and responded to it by comparingtheir notes and writing down a joint strategy using ideas from both summaries.

While they readily took up the adaptive support provided through the collab-oration prompt, Sio and Adi showed low fidelity to the structural components ofthe collaboration script: They spent approximately 17 minutes on the first phase,which is more than twice the time that was suggested in the instructions. Theynever clearly transitioned from the third phase (conducting experiments) to thefourth phase (analyzing the experiments). And they did not take any notes beyondthe first ones on their joint strategy. On the one hand, this shows that Sio andAdi benefited from the low degree of coerciveness of the collaboration script as itallowed them to adapt the structural support to their needs. On the other hand, theymissed the opportunities for learning offered in the fourth phase by not observingthe sequence of inquiry activities and neglecting the conversation starters. In par-ticular, taking notes on the results of their experiments (as instructed by the script)might have helped them in planning a more thought-out series of experiments, thusincreasing the effectiveness and efficiency of their inquiry.



In conclusion, the design of the collaboration script has to strike a balancebetween imposing structure on the learners, which seems necessary given the typi-cally low quality of their collaborative inquiry, and allowing the learners to adaptthe support to their needs. Turning the structural support components into adap-tive support components, hence creating a completely adaptive collaboration script,might solve the “scripting dilemma”. We suggest to macro-adapt the structural sup-port, for example based on the learners’ prior experience with collaborative inquirylearning, as it ensures that the learners’ collaborative inquiry activities are effectivefrom the very start.

6.2.2. Sun and Mia: Failing in learning despite adaptive support

Table 4 shows how Sun and Mia structured their learning phase. The table alsodisplays the episodes central to their collaborative inquiry and problem solving aswell as the prompts and hints given to them by the wizard.

Like Sio and Adi, Sun and Mia were not successful at solving the stoichiometryproblem. They conducted crucial experiments but failed to interpret the resultscorrectly, particularly the data displayed in the information window of the virtualchemistry laboratory. Moreover, it was striking that they continuously used distilledwater and the Bunsen burner despite receiving experimentation hints from thewizard encouraging them to reconsider their experimental strategy.

Concerning their collaboration, Sun and Mia more or less coordinated their inter-actions well. But their behavior was largely ineffective with regard to the processesaccounting for learning during collaboration. At the beginning of the second phase(developing a strategy), for example, Mia correctly argued that heating might notbe a correct procedure; Sun did not take up this objection but suggested to move onand carry out the procedure in a trial-and-error fashion. This episode is an exam-ple of the many similar episodes that we observed with this dyad. For the mostpart of the learning phase, Sun did not acknowledge and join in Mia’s attemptsat constructing explanations. Even when Sun and Mia engaged in co-construction,their reasoning and consensus building remained superficial.

Sun and Mia received two collaboration prompts from the wizard. At the begin-ning of the third phase (conducting experiments), the wizard sent a collaborationprompt because Sun demonstrated ineffective behavior by ignoring Mia’s attemptat constructing an explanation. Sun and Mia studied the prompt and responded toit by explaining to one another. However, the explanations were shallow. In addi-tion, Mia did not take up Sun’s objections which led the wizard to send yet anothercollaboration prompt. Sun and Mia ignored this prompt — just like they ignoredthe four experimentation hints they were given.

The dyad’s fidelity to the structural components of the collaboration script canbe described as moderate. Sun and Mia engaged in the inquiry activities of the firstthree phases as instructed and then departed from the sequence of activities andthe corresponding instructions.



Table

4.Sun

and

Mia

:O

ver

vie

wofti

me

spen

ton

phase

s,im

port

ant

epis

odes

,pro

mpts

and

hin

tsgiv

enby

the

wiz

ard

.

Phase

Fro

m–to

(min

)O

nse

tof

Short

Des

crip

tion

ofE

pis

ode

Pro

mpt

Giv

enby

Wiz

ard

Epis

ode

Phase

1(8

:54

min

)(i

ndiv

idualw

ork

)

Tra

nsi

tional

phase

8:5

5–10:5

4(1

:59

min

)(i

ndiv

idualw

ork

)

Phase

210:5

5–17:3

6(6

:41

min

)13:4

9T

he

wiz

ard

gav

ea

fidel

ity-t

o-s

crip

tpro

mpt:

The

dyad

then

took

note

son

thei

rst

rate

gy.

But

thei

rco

nse

nsu

sbuildin

gw

as

super

fici

al.

“It

isim

port

ant

that

you

dev

elop

ajo

int

stra

tegy

and

take

note

son

itbef

ore

you

mov

eon

tophase

3(c

onduct

ing

exper

imen

ts).

”

Phase

317:3

7–24:5

3(7

:16

min

)21:4

8C

ollabora

tion

pro

mpt:

The

dyad

then

engaged

insh

allow

expla

nato

rybeh

avio

r.“E

xpla

inin

gto

one

anoth

erth

ere

sult

sof

your

exper

imen

tsw

illhel

pyou

inso

lvin

gth

est

oic

hio

met

rypro

ble

m.”

22:5

0C

ollabora

tion

pro

mpt

ignore

d.

“It

isim

port

ant

that

you

take

up

your

part

ner

’sques

tions

and

obje

ctio

ns.

”

Phase

424:5

3–26:4

3(1

:50

min

)



Table

4.(C

ontinued

)

Phase

Fro

m–to

(min

)O

nse

tof

Short

Des

crip

tion

ofE

pis

ode

Pro

mpt

Giv

enby

Wiz

ard

Epis

ode

Fre

ech

oic

eof

act

ivit

ies

26:4

4–en

dof

learn

ing

phase

(appro

x.24

min

)

29:3

7E

xper

imen

tati

on

hin

tig

nore

d.

“E

xpla

into

one

anoth

erw

hy

you

use

ddis

tilled

wate

rin

your

exper

imen

tsand

what

happen

edw

hen

you

added

dis

tilled

wate

r.”

30:2

9E

xper

imen

tati

on

hin

tig

nore

d.

“T

hin

kabout

the

tools

you

hav

ebee

nusi

ng

inyour

exper

imen

tsand

expla

inyour

reaso

nin

gto

your

part

ner

.”

31:1

9T

he

dyad

conduct

eda

cruci

alex

per

imen

tbut

failed

toco

rrec

tly

inte

rpre

tth

ere

sult

and

dra

wan

adeq

uate

concl

usi

on.

37:4

6E

xper

imen

tati

on

hin

tig

nore

d.

“R

emem

ber

that

you

are

not

goin

gto

learn

ing

much

from

tria

land

erro

r.It

isim

port

ant

that

you

thin

kabout

what

the

resu

lts

ofyour

exper

imen

tm

ight

be

bef

ore

you

carr

yit

out.

”

43:3

0E

xper

imen

tati

on

hin

tig

nore

d.

Sam

eas

at

29:3

7.

43:2

3T

he

dyad

conduct

eda

cruci

alex

per

imen

tand

made

an

import

ant

obse

rvati

on

regard

ing

the

pro

duct

sobta

ined

.H

owev

er,th

eydre

win

corr

ect

concl

usi

ons

from

thei

robse

rvati

on.



The case study of Sun and Mia highlights the importance of ensuring fidelityto both the structural and the adaptive components of the collaboration script.Sun and Mia missed the opportunities for learning offered in the fourth phase. Itseems possible that they were not aware of their need for structural support ordid not perceive the activities specified by the collaboration script to be of use intheir joint efforts of solving the stoichiometry problem. They also ignored almostall of the prompts and hints they were given. As a consequence, the quality oftheir collaborative inquiry hardly improved during the learning phase. Two addi-tions to the collaboration script could increase learners’ awareness of their needfor instructional support and help them understand the benefits of scripted col-laboration. First, building on our previous suggestion of assessing prior experiencewith collaborative inquiry learning before the learning phase and macro-adaptingthe structural support to learners’ needs, we propose to feed the results of theassessment back to the learners. Second, we suggest that informed training (Brownet al., 1981) precedes the learning phase. The informed training could take theform of an instructional unit placed after the initial assessment of collaborationand inquiry skills or the form of links to explanations. The links would be providedwith the prompts and hints. Although the option of following the links bears littlerisk of decreasing motivation it will probably be less effective than a mandatoryinstructional unit as learners are likely to ignore the links together with the adap-tive prompts, particularly as they move towards the middle of the learning phase.Another option would be to block the interface of the learning environment for acertain time after a prompt or hint was given. However, this is likely to decreasemotivation and result in behavior similar to what is known as gaming the systemin research on intelligent tutoring systems (e.g. Baker et al., 2008): Some learnerswill simply wait until the system restrictions are released instead of processing theprompts and hints. Therefore, we advocate feedback on collaboration and inquiryskills and informed training.

6.3. Evaluation of the experiment — Implications for the

implementation of the collaboration script

The participants gave positive ratings to their experience during the study. For allof the four scales (overall experience, learning environment, instructional support,collaboration) the average ratings were above the mean of the rating scale. Descrip-tively, the dyads of the unscripted collaboration condition rated the overall experi-ence, the instructional support they received and their collaboration more positivethan the dyads of the scripted collaboration condition. However, these differencesdid not reach statistical significance (see Table 5). This pattern was reversed inthe ratings of the learning environment, but again only descriptively (see Table 5).The somewhat higher ratings of the learning environment under the scripted col-laboration condition probably constitute a novelty effect — the participants seemedutterly impressed with the adaptive prompts which they believed were generated



Table 5. Mean evaluation of the experiment by condition.

Scripted Collaboration Unscripted Collaboration

M SD M SD t(22) p

Overall experience 4.58 0.29 4.78 0.46 −1.32 n.s.Computer-supported

learning environment4.62 0.59 4.72 0.77 −0.36 n.s.

Instructional support(collaboration script orsubstitute)

4.00 0.73 3.88 0.99 0.34 n.s.

Collaboration 3.83 0.80 4.35 0.68 −1.70 n.s.

Note: Participants provided their answers on a six-point scale ranging from not true at all (1)to completely true (6).

based on an innovative computer algorithm. Based on comments made in the end-of-session interviews and on our observations during the learning phase, we assumethat the dyads of the scripted collaboration condition tended towards lower ratingsof the instructional support because they did not perceive the collaboration scriptto be useful, particularly as they progressed in the inquiry cycle. This could alsoexplain why they were slightly less satisfied with their collaboration and enjoyedthe experiment a bit less than the dyads of the unscripted collaboration condition.We assume that the perceived value of scripted collaboration influences learners’acceptance of script support, which in turn affects learning processes and outcomes(as seen in the two case studies). Therefore, we will need to ensure that learnersrecognize the usefulness of the collaboration script in the next iteration of the stoi-chiometry learning environment. We believe this can be achieved by improving theimplementation of the collaboration script. Therefore, we would like to reinforcesome of our previous suggestions: Inform learners about their need for instructionalsupport and the value of the collaboration script, fade out structural support basedon an on-line assessment of the quality of collaborative inquiry, and provide struc-tural support via the interface of the learning environment.

7. Conclusions

We developed a prototype of a stoichiometry learning environment that includeda collaboration script, a simulated chemistry laboratory called VLab (http://www.chemcollective.org/applets/vlab.php) and a cognitive tool, which provided note-taking and mind-mapping functionality. The prototype was computer-based withthe exception of one structural component of the collaboration script, namely thesequence of inquiry activities, which was instructed on paper. The environment pro-vided feedback to learners based on an on-line assessment of the quality of theircollaboration and inquiry activities. In this paper, we reported an initial exploratorystudy in which we had pairs of students work on a stoichiometry problem using thelearning environment, with half of the dyads being supported by the collaborationscript and half of the dyads not receiving instructional support for collaborative



inquiry learning. Our quantitative analyses consisted of comparisons of the learningoutcomes and learning experiences under the two conditions. Using a qualitativeapproach, we also analyzed the learning processes of two dyads from the scriptedcollaboration condition. The qualitative analysis focused on the effects of scriptsupport on collaborative inquiry learning, the actual implementation of the collab-oration script and learners’ interactions with the computer-based learning environ-ment. We drew a number of conclusions from the findings which yielded implicationswith regard to improvements of the collaboration script as well as the instructionaldesign, interface design and interaction design of the learning environment. As ourlarger research program aims at a learning environment that integrates the variouscomponents and provides automated adaptive support for collaborative inquiry,these design implications will feed into another cycle of development and testing. Inthe following, we summarize our plans for improving the design of the stoichiometrylearning environment. The summary moves from the overall design of the learningenvironment to more specific aspects.

7.1. Designing for stoichiometry learning

None of the components of the stoichiometry learning environment provided instruc-tional support for interrelating the multiple representations of the virtual chemistrylaboratory, although this is known to be a very challenging task for chemistrylearners (Ainsworth, 2006; Gabel, 1998). In the next design cycle we will, there-fore, develop and implement instructional support for relating representations.We plan to combine prompts for active connection-making with dynamic sys-tem features that support interrelating representations by automatically updatingrepresentations, thus highlighting relevant information. The focus of the connection-making will be on deep structural relations, rather than surface features.

A need for basic algebra skills training and instruction on general chemistry con-cepts emerged from our study. Therefore, the stoichiometry learning environmentwill be redesigned following the instructional design model of Van Merrienboer et al.(2002). More specifically, (1) the collaborative inquiry activities will be interspersedwith practice items (e.g. transforming fractions) and (2) explanations of key chem-istry concepts (e.g. endothermic/exothermic reactions) will be made available tothe learners via the interface of the learning environment. Learners’ attention willbe directed to specific explanations depending on their prior knowledge in order toensure high relevance of the information to their learning and, thus, increase theprobability of deep cognitive processing.

7.2. Designing for deep learning

Despite being instructed to engage in conceptual reasoning by the collaborationscript, the participants of our study tended to focus on carrying out procedures.Furthermore, shallow reasoning seemed to prevail. Thus, the collaboration script hasto be improved to bring to bear the benefits of collaborative learning. We plan to



redesign Phase 2 (Developing a strategy) and Phase 4 (Analyzing the experiments)of the collaboration script. Each of the two phases will be split in such a waythat procedural activities follow conceptual activities. For example, learners willbe instructed to interpret and discuss the results of their experiments based onraw values or very rough calculations before they move on to fill in equations andcalculate exact solutions. In addition, monitoring will be facilitated by assigning anexplanatory role to one of the learners and a metacognitive role to the other learner.

7.3. Designing for productive collaborative inquiry

The results of the two case studies suggest that learners may master some of thedemands of collaborative inquiry without instructional support. But they tend tomiss opportunities for learning from collaborative inquiry when they do not makeuse of the instructional support provided to them. We also saw that the perceivedvalue of the collaboration script as a means of instructional support points towardnecessary improvements. Therefore, the collaboration script will be developed fur-ther: (1) The next version of the collaboration script will see the paper-based struc-tural support component, meaning the sequence of inquiry activities, implementedvia the interface of the stoichiometry learning environment. That way we hope thestructural support will not be perceived as extraneous material or an additionalresource but a core feature of the learning environment. (2) Informed training onthe sequence of inquiry activities will precede the learning phase. The training willinclude feedback on the learners’ collaboration and inquiry skills. (3) Brief explana-tions of what triggered an adaptive prompt will be added to the pop-up messages.Students will be able to access these explanations at any time during the learningphase.

7.4. Designing for different learner needs

The two case studies also highlighted strengths of our approach to scripting: Thelow degree of coerciveness of the structural support components allowed the learnersto adapt the collaboration script to their needs. And learners were given adaptivesupport (by the Wizard of Oz) only when the quality of their collaborative inquiryindicated a need for assistance. As our collaboration script seems a very promisingapproach we plan to turn the structural support components into adaptive supportcomponents in the next design cycles. For this to be accomplished, we first have todevelop an assessment tool. The tool will be used to assess the learners’ collaborationand inquiry skills at the outset of the learning phase. Then, the interface of thestoichiometry learning environment has to be redesigned to allow for adaptation.Alongside the development of the assessment tool and the adaptive interface, we willcontinue to develop the original adaptive support components. We will develop anassessment tool for evaluating the quality of collaborative inquiry. As automatedadaptive support for stoichiometry learning is the ultimate goal of our researchprogram design efforts will also be made in that direction. Furthermore, the practice



items and explanations of key concepts (see Section 7.1) will also be macro-adaptedto the collaborating learners’ needs.

7.5. Designing for productive learner-system interactions

We pointed out difficulties associated with the use of the virtual chemistry labora-tory to the participants during the preparation phase of our experiment. As seen inthe first case study (Sio and Adi), the instructions were insufficient: The dyad didnot switch between the aqueous species tab and the solid species tab although thatwould have been necessary to solve the stoichiometry problem. Hence, we stronglyrecommend to improve the interface design and interaction design of the virtualchemistry laboratory. In addition, we would like to make a plea for redesigning thesimulation to provide for more authentic learning experiences. We do not elabo-rate on our recommendations in this paper as research on the design of the virtualchemistry laboratory has been reported elsewhere (e.g. Evans et al., 2008).

Feasibility issues will certainly limit the number of planned developments thatwe will be able to realize in the upcoming design cycle. Yet, we believe that afterjust a few additional design cycles and test runs the learning environment can beused productively for collaborative inquiry and, hence, to improve stoichiometrylearning.

Acknowledgments

We wish to express our sincere appreciation to Katharina Westermann for serving asthe Wizard of Oz in our experiments. Our sincere thanks also go to the members ofthe CoChemEx project team, Bruce McLaren, Dimitra Tsovaltzi, Andreas Harrer,Niels Pinkwart and Oliver Scheuer. We gratefully acknowledge the support of DavidYaron and his team at Carnegie Mellon University, specifically Michael Karabinos,who granted us permission to use the virtual chemistry laboratory in our study andassisted us in creating a German version of it.

References

Ainsworth, S. (2006). DeFT: A conceptual framework for considering learning with mul-tiple representations. Learning and Instruction, 16(3), 183–198.

Aronson, E. (1978). The jigsaw classroom. Oxford, England: Sage.Baker, M., & Bielaczyc, K. (1995). Missed opportunities for learning in collaborative

problem-solving interactions. In J. Greer (Ed.), Proceedings of AI-ED ’95: WorldConference on Artificial Intelligence in Education (pp. 210–220). Washington, D.C.:Association for the Advancement of Computing in Education (AACE).

Baker, R., Walonoski, J., Heffernan, N., Roll, I., Corbett, A., & Koedinger, K. (2008).Why students engage in “gaming the system” behavior in interactive learning envi-ronments. Journal of Interactive Learning Research, 19(2), 185–224.

Barron, B. (2000). Achieving coordination in collaborative problem-solving groups. Journalof the Learning Sciences, 9(4), 403–436.



Basili, P. A., & Sanford, J. P. (1991). Conceptual change strategies and cooperative groupwork in chemistry. Journal of Research in Science Teaching, 28(4), 293–304.

Berg, K. F. (1993, April). Structured cooperative learning and achievement in a high schoolmathematics class. Paper presented at the Annual Meeting of the American Educa-tional Research Association, Atlanta, GA.

Berg, K. F. (1994, April). Scripted cooperation in high school mathematics: Peer inter-action and achievement. Paper presented at the Annual Meeting of the AmericanEducational Research Association, New Orleans, LA.

Bielaczyc, K., Pirolli, P. L., & Brown, A. L. (1994). Collaborative explanations andmetacognition: Identifying successful learning activities in the acquisition of cog-nitive skills. In A. Ram, & K. Eiselt (Eds.), Proceedings of the Sixteenth AnnualConference of the Cognitive Science Society (pp. 39–44). Hillsdale, NJ: Erlbaum.

Bodemer, D., Ploetzner, R., Feuerlein, I., & Spada, H. (2004). The active integration ofinformation during learning with dynamic and interactive visualisations. Learningand Instruction, 14(3), 325–341.

Bodemer, D., & Faust, U. (2006). External and mental referencing of multiple representa-tions. Computers in Human Behavior, 22(1), 27–42.

BouJaoude, S., & Barakat, H. (2000). Secondary school students’ difficulties with stoi-chiometry. School Science Review, 81(296), 91–98.

BouJaoude, S., & Barakat, H. (2003). Students’ problem solving strategies in stoichiom-etry and their relationships to conceptual understanding and learning approaches.Electronic Journal of Science Education, 7(3). Retrieved December 28, 2007, fromhttp://ejse.southwestern.edu.

Bromme, R., Hesse, F. W., & Spada, H. (2005). Barriers, biases and opportunities ofcommunication and cooperation with computers: Introduction and overview. In R.Bromme, F. W. Hesse & H. Spada (Eds.), Barriers and biases in computer-mediatedknowledge communication and how they may be overcome (pp. 243–264). New York:Springer.

Brown, A. L., Campione, J. C., & Day, J. D. (1981). Learning to learn: On training studentsto learn from texts. Educational Researcher, 10(2), 14–21.

Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges increating complex interventions in classroom settings. The Journal of the LearningSciences, 2(2), 141–178.

Chang, K.-E., Sung, Y.-T., & Lee, C.-L. (2003). Web-based collaborative inquiry learning.Journal of Computer Assisted Learning, 19, 56–69.

Chiu, J., & Linn, M. C. (2008). Self-assessment and self-explanation for learning chem-istry using dynamic molecular visualizations. In International Perspectives in theLearning Sciences: Cre8ting a Learning World. Proceedings of the 8th InternationalConference of the Learning Sciences (Vol. 3, pp. 16–17). Utrecht, The Netherlands:International Society of the Learning Sciences, Inc.

Cobb, P., Confrey, J., DiSessa, A., Lehrer, R., & Schauble, L. (2003). Design experimentsin educational research. Educational Researcher, 32, 9–13.

Dansereau, D. F. (1988). Cooperative learning strategies. In C. E. Weinstein, E. T. Goetz &P. A. Alexander (Eds.), Learning and study strategies: Issues in assessment, instruc-tion, and evaluation (pp. 103–120). Orlando, FL: Academic Press.

de Jong, T. (2005). The guided discovery principle in multimedia learning. In R. E.Mayer (Ed.), Cambridge handbook of multimedia learning (pp. 215–228). Cambridge,England: Cambridge University Press.

de Jong, T. (2006). Scaffolds for scientific discovery learning. In J. Elen & R. E. Clark(Eds.), Handling complexity in learning environments: Theory and research (pp. 107–128). London: Elsevier.



de Jong, T., & van Joolingen, W. R. (1998). Scientific discovery learning with computersimulations of conceptual domains. Review of Educational Research, 68(2), 179–201.

Diehl, M., & Stroebe, W. (1991). Productivity loss in idea-generating groups: Trackingdown the blocking effect. Journal of Personality and Social Psychology, 61, 392–403.

Dillenbourg, P. (1999). What do you mean by collaborative learning? In P. Dillenbourg(Ed.), Collaborative learning: Cognitive and computational approaches (pp. 1–19).Oxford, England: Elsevier.

Dillenbourg, P. (2002). Over-scripting CSCL: The risks of blending collaborative learningwith instructional design. In P. A. Kirschner (Ed.), Three worlds of CSCL. Can wesupport CSCL? (pp. 61–91). Herleen, The Netherlands: Open Universiteit Nederland.

Dillenbourg, P., Baker, M., Blaye, A., & O’Malley, C. (1995). The evolution of researchon collaborative learning. In E. Spada & P. Reiman (Eds.), Learning in humansand machines: Towards an interdisciplinary learning science (pp. 189–211). Oxford,England: Elsevier.

Diziol, D., Walker, E., Rummel, N., & Koedinger, K. (2010). Using intelligent tutor tech-nology to implement adaptive support for student collaboration. Educational Psy-chology Review, 22(1), 89–102.

Evans, K. L., Yaron, D., & Leinhardt, G. (2008). Learning stoichiometry: A comparisonof text and multimedia formats. Chemistry Education Research and Practice, 9,208–218.

Fasching, J. L., & Erickson, B. L. (1985). Group discussions in the chemistry classroom andthe problem-solving skills of students. Journal of Chemical Education, 62, 842–846.

Gabel, D. L., Sherwood, R. D., & Enochs, L. (1984). Problem-solving skills of high schoolchemistry students. Journal of Research in Science Teaching, 21(2), 221–233.

Gabel, D. L., & Bunce, D. M. (1994). Research on problem solving: Chemistry. In D. L.Gabel (Ed.), Handbook of research on science teaching and learning (pp. 301–326).New York: Macmillan.

Gabel, D. L. (1998). The complexity of chemistry and implications for teaching. In B. J.Fraser & K. G. Tobin (Eds.), International handbook of science education. Dordrecht,The Netherlands: Kluwer Academic Publishers.

Gweon, G., Rose, C., Carey, R., & Zaiss, Z. S. (2006, April). Providing support for adaptivescripting in an on-line collaborative learning environment. Paper presented at theSIGCHI conference on human factors in computing systems, Montreal, Canada.

Hausmann, R. G. M., Chi, M. T. H., & Roy, M. (2004). Learning from collaborativeproblem solving: An analysis of three hypothesized mechanisms. In K. D. Forbus,D. Gentner & T. Regier (Eds.), 26th Annual Conference of the Cognitive ScienceSociety (pp. 547–552). Mahwah, NJ: Erlbaum.

Hermann, F., Rummel, N., & Spada, H. (2001). Solving the case together: The chal-lenge of net-based interdisciplinary collaboration. In P. Dillenbourg, A. Eurelings &K. Hakkarainen (Eds.), European perspectives on computer-supported collaborativelearning. Proceedings of the European Conference on Computer-Supported Collabo-rative Learning (pp. 293–300). Maastricht, NL: McLuhan Institute.

Hmelo-Silver, C. E., Duncan, R. G., & Chinn, C. A. (2007). Scaffolding and achievementin problem-based and inquiry learning: A response to Kirschner, Sweller, and Clark(2006). Educational Psychologist, 42(2), 99–107.

Hron, A., Hesse, F. W., Reinhard, P., & Picard, E. (1997). Strukturierte Kooperation beimcomputerunterstutzten kollaborativen Lernen [Structured cooperation in computerbased collaborative learning]. Unterrichtswissenschaft, 25(1), 56–69.

Jeong, H., & Chi, M. T. H. (1997). Construction of shared knowledge during collaborativelearning. In R. Hall, N. Miyake & N. Enyedy (Eds.), Proceedings of Computer-Supported Collaborative Learning (pp. 124–128). Hillsdale: Erlbaum.



Johnson, D. W., & Norem-Hebeisen, A. A. (1979). A measure of cooperative, competitive,and individualistic attitudes. Journal of Social Psychology, 109, 253–261.

Kaartinen, S., & Kumpulainen, K. (2002). Collaborative inquiry and the construction ofexplanations in the learning of science. Learning and Instruction, 12, 189–212.

Kerr, N. L., MacCoun, R. J., & Kramer, G. P. (1996). “When are N heads better (orworse) than one?”: Biased judgment in individuals versus groups. In J. H. Davis &E. Witte (Eds.), Understanding group behavior: Consensual action by small groups(Vol. 1). Hillsdale, NJ: Erlbaum.

King, A. (1994). Guiding knowledge construction in the classroom: Effects of teaching chil-dren how to question and how to explain. American Educational Research Journal,30, 338–368.

King, A. (2007). Scripting collaborative learning processes: A cognitive perspective. In F.Fischer, I. Kollar, H. Mandl & J. M. Haake (Eds.), Scripting computer-supportedcollaborative learning: Cognitive, computational, and educational perspectives. NewYork: Springer.

Koedinger, K. R., & Aleven, V. (2007). Exploring the assistance dilemma in experimentswith cognitive tutors. Educational Psychology Review, 19(3), 239–264.

Kollar, I., Fischer, F., & Hesse, F. W. (2006). Collaboration scripts — A conceptualanalysis. Educational Psychology Review, 18(2), 159–185.

Kramarski, B. (2004). Making sense of graphs: Does metacognitive instruction make a dif-ference on students’ mathematical conceptions and alternative conceptions? Learningand Instruction, 14(6), 593–619.

Meier, A., Spada, H., & Rummel, N. (2007). A rating scheme for assessing the qualityof computer-supported collaboration processes. International Journal of Computer-Supported Collaborative Learning, 2(1), 63–86.

Moschkovich, J. N. (1996). Moving up and getting steeper: Negotiating shared descriptionsof linear graphs. Journal of the Learning Sciences, 5(3), 239–277.

Nakhleh, M. B., Polles, J., & Malina, E. (2002). Learning chemistry in a laboratory environ-ment. In J. K. Gilbert, O. De Jong, R. Justi, D. F. Treagust & J. H. Van Driel (Eds.),Chemistry education: Toward research-based practice. Dordrecht, The Netherlands:Kluwer Academic Publishers.

O’Donnell, A. M., & Dansereau, D. F. (1992). Scripted cooperation in student dyads:A method for analyzing and enhancing academic learning and performance. In R.Hertz-Lazarowitz & N. Miller (Eds.), Interaction in cooperative groups: The theo-retical anatomy of group learning (pp. 120–141). Cambridge, England: CambridgeUniversity Press.

Okada, T., & Simon, H. A. (1997). Collaborative discovery in a scientific domain. CognitiveScience, 21(2), 109–146.

Palinscar, A. S., & Brown, A. L. (1984). Reciprocal teaching of comprehension-fosteringand comprehension-monitoring activities. Cognition and Instruction, 1(2), 117–175.

Ploetzner, R., Fehse, E., Kneser, C., & Spada, H. (1999). Learning to relate qualitativeand quantitative problem representations in a model-based setting for collaborativeproblem solving. Journal of the Learning Sciences, 8(2), 177–214.

Rummel, N., & Spada, H. (2005a). Can people learn computer-mediated collaboration byfollowing a script? In F. Fischer, H. Mandl, J. Haake & I. Kollar (Eds.), Script-ing computer-supported communication of knowledge. Cognitive, computational, andeducational perspectives. Dordrecht, The Netherlands: Kluwer.

Rummel, N., & Spada, H. (2005b). Learning to collaborate: An instructional approach topromoting collaborative problem solving in computer-mediated settings. Journal ofthe Learning Sciences, 14(2), 201–241.



Rummel, N., & Weinberger, A. (2008). New challenges in CSCL: Towards adaptive scriptsupport. In G. Kanselaar, V. Jonker, P. A. Kirschner & F. Prins (Eds.), Interna-tional perspectives of the learning sciences: Creating a learning world. Proceedings ofthe Eighth International Conference of the Learning Sciences (ICLS 2008) (Vol. 3,pp. 338–345). Retrieved March 8, 2008, from http://isls.org/icls2008.

Salomon, G., & Globerson, T. (1989). When teams do not function the way they oughtto. International Journal of Educational Research, 13, 89–100.

Schmidt, H.-J. (1990). Stolpersteine im Chemieunterricht. Empirische Untersuchungenuber Schulerfehler beim Stochiometrischen Rechnen [Stumbling blocks in chemistryeducation. Empirical studies on student errors in stoichiometric calculating]. Frank-furt am Main/Aarau, Germany: Diesterweg/Sauerlander.

Schmidt, H.-J. (1992). Harte Nusse im Chemieunterricht. Empirische Untersuchungenuber Schulervorstellungen [Tough nuts to crack in chemistry education. Empiri-cal studies on student conceptions]. Frankfurt am Main/Aarau, Germany: Diester-weg/Sauerlander.

Schoenfeld, A. H. (2006). Design experiments. In Green, J. L., Camilli, G., & Elmore, P.B. (Eds.), Handbook of complementary methods in education research (pp. 193–205).Mahwah, NJ: Erlbaum.

Seufert, T., & Brunken, R. (2006). Cognitive load and the format of instructional aids forcoherence formation. Applied Cognitive Psychology, 20, 321–331.

Sumfleth, E., Rumann, S., & Nicolai, N. (2004a). Kooperatives Arbeiten im Chemieunter-richt [Cooperative work in the chemistry classroom]. In Essener Unikate (Vol. 24,pp. 74–85).

Sumfleth, E., Rumann, S., & Nicolai, N. (2004b). Schulische und hausliche Kooperationim Chemieanfangsunterricht. In Doll, J., & Prenzel, M. (Eds.), Bildungsqualitat vonSchule: Lehrerprofessionalisierung, Unterrichtsentwicklung und Schulerforderung alsStrategien der Qualitatsverbesserung (pp. 284–302). Munster, New York, Munchen,Berlin: Waxmann.

Teasley, S. D. (1995). The role of talk in children’s peer collaborations. DevelopmentalPsychology, 31(2), 207–220.

Teasley, S. D. (1997). Talking about reasoning: How important is the peer in peer collab-oration? In L. B. Resnick, R. Saljo, C. Pontecorvo & B. Burge (Eds.), Discourse,tools and reasoning: Essays on situated cognition (pp. 361–384). Berlin: Springer.

The Concord Consortium. (n.d.). The Molecular Workbench. Retrieved November 21, 2010from http://mw.concord.org/modeler/articles/brochure.pdf.

Tingle, J. B., & Good, R. (1990). Effects of cooperative grouping on stoichiometric problemsolving in high school chemistry. Journal of Research in Science Teaching, 27(7),671–683.

Towns, M. H., & Grant, E. R. (1997). “I believe I will go out of this class actually know-ing something”: Cooperative learning activities in physical chemistry. Journal ofResearch in Science Teaching, 34(8), 819–835.

Tsovaltzi, D., Rummel, N., Pinkwart, N., Scheuer, O., Harrer, A., Braun, I., et al. (2008).CoChemEx: Supporting conceptual chemistry learning via computer-mediated col-laboration scripts. In P. Dillenbourg & M. Specht (Eds.), Proceedings of the ThirdEuropean Conference on Technology Enhanced Learning (EC-TEL 2008), LectureNotes in Computer Science 5192. (pp. 437–448). Berlin: Springer.

Van der Meij, J. (2007). Support for learning with multiple representations designingsimulation-based learning environments. Universiteit Twente, Twente, Netherlands.

van Joolingen, W. R., de Jong, T., Lazonder, A. W., Savelsbergh, E. R., & Manlove, S.(2005). Co-Lab: Research and development of an online learning environment for



collaborative scientific discovery learning. Computers in Human Behavior, 21(4),671–688.

Van Merrienboer, J. J. G., Clark, R. E., & de Croock, M. B. M. (2002). Blueprints for com-plex learning: The 4C/ID-model. Educational Technology Research and Development,50, 39–64.

Walker, E., Rummel, N., & Koedinger, K. (accepted). Designing automated adaptive sup-port to improve student helping behaviors in a peer tutoring activity, InternationalJournal of Computer Supported Collaborative Learning.

Walpuski, M. (2006). Optimierung von experimenteller Kleingruppenarbeit durch Struk-turierungshilfen und Feedback: Eine empirische Studie [Optimization of experimen-tal work in small groups by means of structural support and feedback: An empiricalstudy]. Berlin, Germany: Logos.

Webb, N. M. (1989). Peer interaction and learning in small groups. International Journalof Education Research, 13, 21–39.

Wecker, C., Kollar, I., Fischer, F., & Prechtl, H. (2010). Fostering online search compe-tence and domain-specific knowledge in inquiry classrooms: Effects of continuous andfading collaboration scripts. In K. Gomez, L. Lyons & J. Radinsky (Eds.), Learningin the Disciplines. Proceedings of the 9th International Conference of the LearningSciences (ICLS 2010) (Vol. 1, pp. 810–817). International Society of the LearningSciences, Inc, Chicago, IL.

Weinberger, A., Stegmann, K., Fischer, F., & Mandl, H. (2007). Scripting argumentativeknowledge construction in computer-supported learning environments. In F. Fischer,I. Kollar, H. Mandl, & J. M. Haake (Eds.), Scripting computer-supported collaborativelearning: Cognitive, computational, and educational perspectives (pp. 188–208). NewYork: Springer.

Documents

Research and Practice in Technology Enhanced Learning ......laboratory, (c) develop tools for assessing learners’ prior knowledge and skills in collab-orative inquiry learning, general