Computers in Human Behavior 27 (2011) 94–98

Contents lists available at ScienceDirect

Computers in Human Behavior

journal homepage: www.elsevier .com/locate /comphumbeh

Examining different types of collaborative learning in a complexcomputer-based environment: A cognitive load approach

Liming Zhang a,*, Paul Ayres b, KaKin Chan a

a Faculty of Education, University of Macau, Macau, Chinab University of New South Wales, Sydney, Australia

a r t i c l e i n f o

Article history:Available online 16 June 2010

Keywords:Collaborative learningIndividual learningComputer-based learning environmentCognitive load theoryCognitive load measurementQuasi-experiment

0747-5632/$ - see front matter � 2010 Elsevier Ltd. Adoi:10.1016/j.chb.2010.03.038

* Corresponding author.E-mail address: lmzhang@umac.mo (L. Zhang).

a b s t r a c t

This study compared the effects of two collaborative learning strategies (Open-ended and Task-based)with an individualized learning strategy on individual learning in a computer-based environment. Theexperiment sought ecological validity by conducting it under real teaching and homework conditions.Ninety-four students from grade 9 participated in a webpage design task. Cognitive load theory was usedto predict that the collaborative approaches would outperform the individualized approach due toreduced cognitive load. This hypothesis was confirmed by performance scores and cognitive load onlyin the case of the Open-ended collaborative learning condition. Evidence was also found that theOpen-ended collaborative learning condition outperformed the Task-based collaborative one. It was con-cluded that in collaborative learning a more Open-ended task design together with moderate indepen-dent sub-task requirements leads to more effective learning.

� 2010 Elsevier Ltd. All rights reserved.

1. Introduction 2009b; Laughlin, Bonner, & Miner, 2002; Kirschner, Paas, & Kirsch-

The main aim of this study was to compare two different condi-tions of collaborative learning with an individual learning condi-tion on the complex task of webpage design. Collaborativelearning has been widely studied and advocated throughout theprofessional literature. It refers to an instruction method in whichstudents work together in small groups toward a common goal.The students are responsible for one another’s learning as well astheir own. Thus, the success of one student may help other stu-dents to be successful (Gokhale, 1995). Collaborative learning isbased on the premise that the collaboration process will includediscussion, argumentation, and reflection upon the task at hand,thus leading to deeper processing of the information and richerand more meaningful learning (Kirschner, Paas, & Kirschner,2009a). Collaborative learning is considered to be effective if thelearning outcomes of the n members of a group are higher thanthe sum of the learning outcomes of n comparable individuallearners (Kirschner et al., 2009a).

Studies that have compared individual performance with groupperformance have found that for simple tasks, such as recallinginformation, group performance is inferior to that of the individu-als (Andersson & Ronnberg, 1995; Kirschner, Paas, & Kirschner,2009b). For more complex tasks, such as problem solving, evidencesuggests that groups outperform individuals (Kirschner et al.,

ll rights reserved.

ner, this issue). Based on the above results, the type of task seemsto be an important factor in determining whether collaborativelearning is beneficial or not. To explain why collaborative strate-gies can be effective on complex tasks Kirschner et al. (2009b) usedCognitive load theory (CLT; see Sweller (1999), Sweller, van Mer-riënboer, and Paas (1998) and Van Merriënboer and Ayres(2005)) to argue that groups potentially have a larger workingmemory capacity than individuals.

In CLT the capacity of working memory (WM) is argued to playa crucial role in learning. WM is extremely limited in terms of howmuch information can be retained (Miller, 1956) or processed(Cowan, 2001). The cognitive load which is unnecessary and inter-feres with schema acquisition and automation is referred to as anextraneous cognitive load (Sweller et al., 1998). On complex taskssuch as problem solving much of the WM resources are devotedto solving the problem and not learning about the problem. Basedon CLT the problem solving methodology creates extraneous cogni-tive load, as effort may not be directly connected to learning, thusnot generating germane cognitive load – the load devoted to sche-ma acquisition (Sweller et al., 1998). In addition, if the materials tobe learned are complex, then the task is high in intrinsic cognitiveload. High levels of intrinsic and/or extraneous load interfere withlearning by taking up the precious resources of a very limitedcapacity WM, leaving little available processing for germane load.Hence, Kirschner et al. (2009b) argue that on complex tasks theextraneous/intrinsic cognitive load can be shared amongst themembers of the group, freeing up processing capacity at the indi-vidual level for germane load.

L. Zhang et al. / Computers in Human Behavior 27 (2011) 94–98 95

1.1. Performance hypotheses

In this study, three teaching strategies were compared witheach other in learning about webpage design on a computer. Twoof the strategies were based on collaborative learning principles,whereas one was based on individual learning. Learning how toconstruct a webpage, particularly for novices, is a fairly complextask, and therefore it was predicted that a collaborative approachwould be more effective than an individualized approach (as ar-gued by Kirschner et al. (2009b)). However, group-work is effectiveprovided that information is meaningfully shared and communi-cated within the group. To ensure productive collaboration, inde-pendent sub-tasks were created that required the web pages tohave a number of consistent features including background colorand graphic design. This requirement dictated that individualswithin the groups had to communicate and discuss amongst them-selves these features to reach a common agreement.

There was also a key difference between the two collaborativegroups. One group, the Open-ended collaborative condition (OC)had a more Open-ended task that allowed groups to choose theirown webpage design. In contrast, the second group, the Task-basedcollaborative condition (TC) had less choice, as all subgroups wererequired to deign a personal homepage. According to Wageman(1995) reducing the interdependency by dividing tasks into inde-pendent sub-tasks negatively influences team performance. There-fore, we hypothesized that higher dependency sub-tasks amongstgroup members would support effective group learning results,however, at the cost of the quality of the individual group mem-ber’s learning. On the other hand moderate independent sub-tasksfor each group member in collaborating learning could facilitatethe freed WM capacity to be used more effectively towards ger-mane load. Consequently, it was hypothesized that moderate inde-pendent sub-task requirement to each individual group memberwould facilitate the individual group member’s learning moreeffectively. Thus we expected the OC group to outperform the TCgroup. Furthermore, it was also anticipated that collaborativegroups that had more Open-ended design would be more moti-vated to make better designs and perform accordingly.

1.2. Design principle for assessing learning

In designing this study an important principle was adhered to.Kirschner et al. (2009a) point out that there are often two draw-backs in the research comparing individual performance withgroup performance. One is that the effects on learning were oftenonly indirectly tested by measuring performance and/or group pro-cesses during the learning phase (e.g., number of contributions,moves, types of contributions, etc.). They often did not directlymeasure actual learning achievement through tests that were spe-cifically designed to assess post-acquisition learning (Kester &Paas, 2005; Kirschner et al., 2009a). Because of this indirect testing,measures are a determination of the quality of the group productor group process rather than of individual learning (Kirschneret al., 2009b). The second drawback is that the studies usually fo-cused on group performance instead of on the contribution of eachgroup member (Kerr & Tindale, 2004; Kirschner et al., 2009a). Thebasic teaching purpose is to improve the student’s performanceindividually. The quality of group processes or products does notnecessarily reflect the quality of learning of the individual groupmembers, as the group product might, for example, be the resultof the input of the most knowledgeable or diligent group member(Kirschner et al., 2009a). Consequently, the guiding assessmentprinciple of this study was to measure actual learning individually.So within the collaborative learning environment, the individualmember’s performance instead of the group performance wasmeasured.

1.3. Choice of learning content and ecological validity

A further purpose of this study was to explore effective teachingstrategies for learning about computer applications. Computertechnology is one of the curriculum subjects in the secondary edu-cation in Macao. It aims at equipping the students with the con-temporary computer technology to support their further tertiaryeducation and professional applications. The contents usually cov-er computer architecture and software applications, includingMicrosoft Office, VB, Flash, and Website Development etc. Theusual teaching method is that the teacher demonstrates the soft-ware operations step-by-step with the computer, and then the stu-dents practice the operations in class followed by individualhomework or group projects. Computer technology is not consid-ered a major subject in secondary education in Macao as thereare only one or two lectures assigned to it per week. Consequently,this study hopes to show how a collaborative methodology can im-prove student outcomes in this learning domain. Furthermore, thestudy aimed for a high degree of ecological validity in the sensethat it was conducted in the normal school setting of class teachingand homework assignments.

2. Method

2.1. Participants

The participants in this study were 94 grade 9 students from asecondary school in Macao, with 33 students in Class A, 28 in ClassB, and 33 in Class C. A quasi-experimental design was used in thestudy as student cohorts in Classes A, B and C were assignedrespectively to the groups Individual assignment, Task-based col-laborative, and Open-ended collaborative. In Task-based andOpen-ended collaborative learning classes, the students weregrouped with 4 or 5 members, arranged by the teacher based onthe students’ computer course marks in the previous semester.The selection principle was that each group should contain stu-dents with a range of abilities, enabling group members are ex-pected to share their strengths and also develop their weaker skills.

2.2. Teaching materials and procedure

The topic to be learned was website development using Micro-soft Frontpage. The study lasted 5 weeks and involved two distinctphases corresponding to class lectures and homework study.

2.3. Class lectures

Over 5 weeks students were given one lecture a week, lasting40 min. The content consisted of 9 sections in website design,including different text style editing, picture, video, animation, ta-ble, frame, background, music, and hyperlink. The lectures for thethree classes were designed and delivered by the same teacher,using material he had developed and taught over 3 years. In thelectures, the teacher demonstrated the software operations step-by-step on a computer, and then the students practiced the opera-tions in class. In these class lectures all three groups experiencedidentical materials, lectures and activities.

2.4. Homework study

In this phase students in the three groups were asked to com-plete a major homework task of a website design, which was as-signed in the first lecture of the study. However, each group wasrequired to complete different tasks to reach the end product asdescribed below.

96 L. Zhang et al. / Computers in Human Behavior 27 (2011) 94–98

Individual assignment. For this condition the students were re-quired to complete their homework individually with the assignedtheme of a personal homepage design. Each student needed to de-velop at least 5 web pages to be submitted at the end of the 5-weekperiod. The visual appearance, computer technique used and thecontent design in the personal homepage were decided and de-signed entirely by individual students.

Task-based collaborative project. For this condition each memberwas required to develop their own personal homepage, which in-cluded at least 5 web pages. In addition students were requiredto work collaboratively to link all the group members’ personalhomepages together. In order to achieve this collaboration the vi-sual appearance, content design and arrangement of each groupmembers’ personal homepages were required to be consistent.The group members had to communicate and discuss with eachother to reach the common understanding.

To create a group atmosphere students were also required togive a presentation of their project in front of the whole classafter the study was completed, where both the teacher and stu-dents exchanged comments and provided feedback on the pre-sentations. These task requirements were designed to promotethe group members’ responsibility for the whole group in orderto engage them in communication and collaboration for thistask.

Open-ended collaborative project. For this condition the home-page design task was more Open-ended. In contrast to the otherconditions, it was not limited to a personal homepage. It couldbe any kind of homepage (e.g. after school activity homepage,their favorite animal’s homepage, etc.). Consequently, groupmembers needed to communicate and discuss with each other,in order to make decisions on the theme of the group home-page. However, to achieve equivalence with the other two clas-ses there was the requirement that each group member developat least 5 web pages by themselves. Furthermore, the sub-theme of each member’s work should have a connection withthe group theme, including the unified visual appearance, con-tent design and arrangement. Similar to the Task-based collab-orative condition, group presentation was also arranged at theend of the study.

2.5. Scoring the common performance task (webpage design)

Each of the three conditions was required to submit individualweb pages, but designed under different circumstances. The Indi-vidual assignment condition was required to work by themselvesin designing at least 5 web pages around the theme of a personalwebsite, whereas the two other conditions had collaborative tasks.In the case of the Task-based collaborative strategy, group mem-bers had to agree on common features to the personal website de-signs. In contrast, the Open-ended collaborative strategy requiredgroup members to design their own theme and not be restrictedto a personal website design. To score the common tasks submittedindividually a rubric was adopted that could take account of con-tent differences by focusing on visual appeal, the computer tech-niques used, and content design. It was revised based on theWebQuest Rubric (Dodge, 2001).

Table 1Adjusted group means (Std. Error) for performance scores on the webpage design.

Group Scores

Individual 67.8 (2.76)Task-based 70.4 (2.93)Open-ended 79.1 (2.72)

2.6. Cognitive load measures

Cognitive load was measured by the NASA-TLX 6-dimensionalquestionnaire (Hart & Staveland, 1988), which was designed tomeasure mental demand, physical demand, temporal demand, ef-fort, frustration tolerance, and performance. The rating scale was5-point, ranging from ‘‘1 = very much disagree” to ‘‘5 = very muchagree”.

2.7. Pre-test

Prior to the lectures starting all students were pre-tested ontheir knowledge of webpage design. Students were required to de-velop one webpage using Microsoft Frontpage. It was markedbased on an evaluation rubric.

2.8. Post-test

The post-test included student’s achievement evaluation onwebsite design, and cognitive load measurement using NASA-TLXquestionnaire.

2.9. Procedure

In summary students were initially pre-tested on their knowl-edge of webpage design. They then embarked on a series of lec-tures and differentiated homework assignments lasting 5 weeks.At the end of this period all students were required to submit indi-vidually designed web pages. At this point they were administeredthe cognitive load self-rating measure. Following this formal phaseof the study, the two classes that worked collaboratively presentedtheir assignments to their peers and received feedback. It should benoted that these group presentations had no influence on the stu-dents’ final marks. Their purpose was to promote groupresponsibility.

3. Results

3.1. Pre-test measures

Group mean scores on the prior knowledge test indicated verylow levels of prior knowledge of webpage design. A one-way ANO-VA revealed no significant difference between the three groups(F < 1).

3.2. Performance measures

The adjusted group scores on the individual performance mea-sures are shown in Table 1. A one-way ANCOVA revealed signifi-cant differences between the groups F(2, 90) = 4.59, MSE = 238.3,p < .02, g2

p = .09. Bonferroni adjusted pair-wise one-way compari-sons found that the Open-ended collaborative group significantlyoutperformed the Individual assignment group (p < .0.01), andmarginally outperformed the Task-based collaborative group(p < .05). The comparison between the Task-based collaborativegroup and the Individual assignment group was non-significant.

3.3. Cognitive load measures

The adjusted group scores of each subscale of the NASA-TLXinstrument are shown in Table 2. The 6 scales were used as depen-dent measures in a one-way MANCOVA, which was significant: Pil-lai’s Trace (12, 172) = 5.79, p < .001, g2

p = .29. Univariate tests (seeTable 3 for a summary) revealed that five scales (mental demand,temporal demand, effort, frustration of tolerance and performance)were significantly different, but one was not (physical demand).

Table 2Adjusted group means (Std. Error) for cognitive load subscales.

Group Factor 1 Factor 2 Factor 3 Factor 4 Factor 5 Factor 6

Individual 6.9 (0.28) 20.4 (0.91) 7.2 (0.30) 11.6 (0.35) 19.1 (0.65) 11.1 (0.65)Task-based 5.8 (0.30) 20.0 (0.97) 7.1 (0.32) 11.2 (0.37) 17.8 (0.69) 12.8 (0.70)Open-ended 5.2 (0.28) 19.8 (0.90) 5.9 (0.29) 8.1 (0.34) 15.8 (0.64) 15.4 (0.64)

Note: Factor 1 = mental demand, Factor 2 = physical demand, Factor 3 = temporal demand, Factor 4 = effort, Factor 5 = frustration tolerance, and Factor 6 = performance.

Table 3Univariate tests for cognitive load subscales.

CL subscales F(2, 90) p Value Partial g2 aPair-wise comparisons

Mental demand 9.05 .000 0.17 3 < 1, 2 < 1Physical demand 0.09 .917 0.00 NoneTemporal demand 5.79 .004 0.11 3 < 1, 3 < 2Effort 29.6 .000 0.40 3 < 1, 3 < 2Frustration tolerance 6.15 .003 0.12 3 < 1Performance 10.9 .000 0.19 3 > 1, 3 > 2

a Note: 1 = Individual assignment, 2 = Task-based collaborative, and 3 = Open-ended collaborative.

L. Zhang et al. / Computers in Human Behavior 27 (2011) 94–98 97

Bonferroni adjusted pair-wise comparisons revealed that theOpen-ended collaborative group experienced less mental demand,temporal demand, effort, frustration tolerance, and had greaterperformance than the Individual assignment group. Further thisgroup reported less temporal demand and effort than the Task-based collaborative group, and greater performance. In comparingthe Task-based collaborative group with the Individual assignmentgroup, only the mental demand subscale produced a significant dif-ference (the Task-based group had the lower mental demand).These results suggest that the Open-ended collaborative learningstrategy was the most effective method in reducing the various as-pects of self-reported cognitive load in this computer environment.

4. Discussion

There were two main hypotheses in this study. First, it washypothesized that in performing complex tasks, individual groupmembers would perform better than learners working individu-ally. The results indicated that this was true for the Open-endedcollaborative group, but not for the Task-based collaborativegroup.

In the introduction it was argued that the reason why the col-laborative groups would be superior was that on complex tasksthe difficulties associated with extraneous and intrinsic cognitiveload could be shared amongst the members of the group, freeingup processing capacity at the individual level for germane load. Inother words, more processing time can be assigned directly tolearning. The results from the NASA-TLX instrument supportedthis argument. In the case of the comparisons between theOpen-ended group and the Individual group the former had sig-nificantly lower ratings on the subscales that link directly to cog-nitive load (mental demand, temporal demand and effort). Incomparing these two groups, performance scores and cognitiveload measures are in complete accordance. In comparing theTask-based group and the Individual assignment group, onlyone of the cognitive load measures was significantly different(mental demand). Again this much weaker result matched thenon-significant performance results.

The second hypothesis predicted that the Open-ended groupwouldoutperform the Task-based collaborative group.Thispredictionwas supported. Further evidence for this effect also came from twosignificantly lower load ratings by the Open-ended group on two ofthe cognitive load subscales (less temporal demand and effort). Againlower cognitive load measures match higher performance.

Overall the cognitive measures collected consistently concurredwith a CLT explanation for the effects. In conclusion, we argue thata collaborative approach can be more effective on a complex com-puter-based task; however, the conditions of collaboration areimportant and moderate the impact of the strategy.

This study set out to achieve strong ecological validity by repro-ducing the exact conditions that students learn under while atschool. However, it lacked many controls of the laboratory due tothe 5 weeks experiment period that involved compulsory out-of-class activities. The students’ learning activities could not beenmonitored after school and consequently there may have beensome violations to the conditions. Nevertheless it was the samefor all groups in terms of, for example, any additional help theymight receive, but the exact scale of such events cannot be calcu-lated. Consequently the results must be treated with some cautionbecause of these lack of controls and its quasi-experimental nat-ure; however, the findings are consistent with the theoreticalunderpinnings of the paper. Future research could address theseproblems by repeating this promising study within the laboratoryunder much tighter controls.

References

Andersson, J., & Ronnberg, J. (1995). Recall suffers from collaboration: Joint recalleffects of friendship and task complexity. Applied Cognitive Psychology, 9,199–211.

Cowan, N. (2001). The magical number 4 in short-term memory: A reconsiderationof mental storage capacity. Behavioral and Brain Sciences, 24, 87–114.

Dodge, B. (2001). A rubric for evaluating WebQuests. Available from: <http://webquest.sdsu.edu/webquestrubric.html> Retrieved 01.09.09.

Gokhale, A. A. (1995). Collaborative learning enhances critical thinking. Journal ofTechnology Education, 7, 22–30.

Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (task loadindex): Results of empirical and theoretical research. In P. A. Hancock & N.Mesh Kati (Eds.), Human mental workload (pp. 139–178). Amsterdam:Elsevier Science.

Kerr, N. L., & Tindale, R. S. (2004). Group performance and decision making. AnnualReview of Psychology, 56, 623–655.

Kester, L., & Paas, F. (2005). Instructional interventions to enhance collaborationin powerful learning environments. Computers in Human Behavior, 21,689–696.

Kirschner, F., Paas, F., & Kirschner, P. A. (2009a). A cognitive load approach tocollaborative learning: United brains for complex tasks. Educational PsychologyReview, 21, 31–42.

Kirschner, F., Paas, F., & Kirschner, P. A. (2009b). Individual and group-basedlearning from complex cognitive tasks: Effects on retention and transferefficiency. Computers in Human Behavior, 25, 306–314.

Laughlin, P. R., Bonner, B. L., & Miner, A. G. (2002). Groups perform better than thebest individuals on letters-to-numbers problems. Organizational Behavior andHuman Decision Processes, 88, 602–605.

98 L. Zhang et al. / Computers in Human Behavior 27 (2011) 94–98

Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits onour capacity for processing information. Psychological Review, 63, 81–97.

Sweller, J. (1999). Instructional design in technical areas. Melbourne: ACER Press.Sweller, J., van Merriënboer, J. J. G., & Paas, F. (1998). Cognitive architecture and

instructional design. Educational Psychology Review, 10, 251–296.

Van Merriënboer, J. J. G., & Ayres, P. (2005). Research on cognitive load theory andits design implications for E-learning. Educational Technology, Research andDevelopment, 53, 5–13.

Wageman, R. (1995). Interdependence and group effectiveness. AdministrativeScience Quarterly, 40, 145–180.