Using a wiki to evaluate individual contributionto a collaborative learning projectG. TrentinIstituto Tecnologie Didattiche, Consiglio Nazionale delle Ricerche, Genova, Italy
Abstract One critical issue arising in the educational use of collaborative learning concerns the teachersdifficulty in evaluating the contribution and participation of each student in group-work. Thisarticle aims to illustrate and discuss a methodology that enables evaluation of the collaborativelearning process based on co-writing in a wiki environment.After considering the effectivenessof co-writing as a strategy of collaborative learning, the article will highlight issues regardingmethods for evaluating each students contribution to the collaborative process and to thegroups overall action. A solution will be proposed to address the problem. It is based upon theelaboration of information traced automatically by wiki, employing survey grids and formulaedeveloped ad hoc to calculate participation and contribution indexes. These tools will be illus-trated together with their application in two university courses. Results demonstrate the addedvalue given by the proposed approach to the evaluation process of co-writing. However, thesefindings also highlight critical issues and some possible remedies for the lack of specific wikifunctions to automatically extract information required for quantitative analysis of the actionstaken by members of the learning group.
Keywords collaborative learning, co-writing, evaluation, networked learning, social software, universityteaching.
Traditional educational environments are often charac-terised by a process whereby the teacher assigns a learn-ing activity that is generally carried out autonomouslyby the student. However, this strips the learning processof a fair amount of its social dimension (Bornstein &Bruner 1989; Sullivan 1994). So the idea of fosteringcollaborative learning strategies presents itself as ameans of strengthening this dimension by creatingthe conditions for individual cognitive developmentas a result of group interaction (Treleaven & Cecez-Kecmanovic 2001; Garrison 2003).
In the specific area of networked collaborative learn-ing (Haughey & Anderson 1998; Trentin 2006), thesestrategies are often implemented by assigning a groupof students with the task of collaboratively discoveringthe solution to a given problem (collaborative problem-solving) or developing a written text (co-writing) basedon a given argument (Trentin 2004).
Online activities now can benefit greatly from theenormous possibilities offered by social software(Malloch 2005; Alexander 2006). These include wikis,which are characterized by a variety of unique and pow-erful information-sharing and collaboration featuresthat offer key advantages, such as allowing learners tobe actively involved in their own knowledge construc-tion (Boulos et al. 2006), as well as improvingco-writing processes (Parker & Chao 2007) and facili-tating their monitoring. For example, some of theseaffordances include the possibility
Accepted: 16 January 2008Correspondence: Guglielmo Trentin, Istituto Tecnologie Didattiche,Consiglio Nazionale delle Ricerche, Via De Marini 6, 16149 Genova,Italy. Email:firstname.lastname@example.org
SPECIAL ISSUEOriginal article
2008 The Author. Journal compilation 2008 Blackwell Publishing Ltd Journal of Computer Assisted Learning (2009), 25, 4355 43
to implement distributed collaborative writing(Lowry et al. 2004a); and
to exploit some embedded wiki functions (versioning,tags, comments, linkers) to support the monitoring ofboth the students activities and their level of contri-bution to the collaborative work.
This latter affordance is significant, as teachers oftenexperience difficulty in evaluation when proposingco-writing activities to their students (Shen et al. 2004;Swan et al. 2006). The problem lies not only in evaluat-ing the level of learning produced by the process itself,but also in gauging the actual degree to which the indi-vidual has actively participated in and contributed to theshared written work (Macdonald 2003; Collazos et al.2004).
This paper aims to provide a solution to this problemby presenting and analysing a methodological approachfor organizing co-writing based on the use of the wiki asa means for managing the evaluation of collaborativelearning processes.
Co-writing and collaborative learning
Collaborative development of a written text transformsthe students ordinary, solitary written work into a col-lective process, yielding strong benefits on a social andcognitive level (Clifford 1992; Sullivan 1994). Indeed,co-writing processes (Hale & Wyche-Smith 1988;Guerrero et al. 2003) offer an excellent opportunity notonly to practise reading and writing skills, but also tostimulate reflection, knowledge sharing and criticalthinking (Brown & Palincsar 1989; Scardamalia &Bereiter 2003). In short, they provide an opportunity toenhance knowledge and skills through a process ofstrong social connotation (Cooper et al. 1994; Picciano2002; Stahl 2006).
Furthermore, co-writing that is conducted online isalmost always done so asynchronously, and is mediatedand indirect (Weng & Gennari 2004). Therefore, stu-dents have greater opportunities to reflect deeply onwhat they read and write when replying to their remoteinterlocutors, besides practising their language skills(Flower 1996).
This can amplify the studentssense that there may bemultiple interpretations of the same topic of study ordiscussion point (Cunningham 1991). It also underlinesthe fact that interpretations may converge or diverge,
highlighting the natural complexity of interrelationswithin the realms of knowledge.
Besides the cognitive aspects, it is also worth consid-ering the importance of mastering co-writing tech-niques, which are increasingly being required in theworld of work. In many professions, documents,reports, guidelines, project proposals and the like arewritten collaboratively using network technologies(Lowry et al. 2004b).
However, while co-writing offers clear advantagesfor the learning process, it also presents obstacles in theevaluation of each students
contribution to the development of the artefact pro-duced by the group; and
level of progress in reaching the educational objec-tives of the course.
These matters represent the key research issues thathave stimulated the study and experimentation reportedin this paper.
Wikis, co-writing and evaluation
The literature reports many experiences in the educa-tional use of wikis (Byron 2005; Notari 2006; Parker &Chao 2007). Several of these have addressed theproblem of evaluating the contents that students havedeveloped and the level of learning/competencesreached in developing them (Bruns & Humphreys 2005;Hamer 2006). On the other hand, it would seem that thearea regarding evaluation of the collaborative processcarried out by students has not yet been fully dealt with.The aim of this research has therefore been to define andtest a new methodological approach to the organizationof co-writing via wiki, which enables evaluation andmonitoring of collaborative learning. The research hascentred on two successive editions (20052006 and20062007) of an online course on Network Technology& Human Resources Development (NT&HRD) at thePolitical Science Faculty of the University of Turin, andhas involved around 30 students.
Why choose wikis for co-writing?
One of the NT&HRD modules envisages the collabora-tive development of a short thesis. In previous years,this activity was carried out using the traditional method
44 G. Trentin
2008 The Author. Journal compilation 2008 Blackwell Publishing Ltd
of interacting via computer conference and sharingindividual contributions as attached files. This processrequires a central editor willing to undertake thetask of collecting the contributions and shaping thefinal document according to the groups indications.However, this posed three main disadvantages:
excessive overhead for one group member, namelythe editor;
the danger that each group member merely concen-trates on one branch of knowledge covered in the finalcollaborative work; and
difficulty in gauging the extent to which each groupmember had critically examined the overall work,besides performing his or her individually assignedtask.
It was subsequently decided in later editions to tryusing a wiki as a co-writing environment, exploiting thepotential it offered to
redistribute responsibility for editing the overalldocument to all group members;
spur each participant, through specific group workorganization, to collaborate in the various stages inproducing the overall work; and
establish an evaluation mechanism based on analysisof the interactions among participants, on evaluationon each individuals productions and on the reticularstructuring of the final work tasks performed usingdata from wiki default traces (comments, linkers,tags, versioning).
PBWiki (http://www.pbwiki.com) was adopted forthe experimentation, a choice made solely on the basisthat this application is free of charge; it allows passwordaccess and both a classic and WYSIWYG editor.
From centralized to distributed editing
Using hypertext approaches for collaborative writingcan almost entirely avoid the need to burden a sole editorwith the task of managing the different versions of thedeveloping written text. Compared with other standal-one hypertext applications such as ToolBook, HTMLeditors, PowerPoint, etc., wikis offer special affor-dances, above all the possibility of distributed writing(Hart-Davidson et al. 2006). As well as writing and
seeing their own pages in real time, students using a wikican see the pages that others have published and hyper-textually linked, without having to wait for an editor toassemble the various parts developed individually ondifferent personal computers. Furthermore, being ableto constantly check the works state of progress encour-ages students to find other hypertext links and ideas fordeveloping their own part of the work.
General rules for distributed editing
Co-writing calls for general rules to be defined for draft-ing the shared document (Lowry et al., 2004b). Thepurpose of this is not only to ensure the stylistic homo-geneity of the final document, but also to define effec-tive co-writing strategies for reaching the learningobjectives that one intends to pursue.
Style-wise, students are asked to agree on typo-graphical rules, such as the formats to be used for char-acters and paragraphs, names of recurring hot-wordlinks (returns to the general index, to the head of thesection managed by each student and so on) and theirposition in the text.
As to co-writing strategies, these are generallydefined by the teacher because there is an educationalobjective involved (Cohen 1994; Felder & Brent 2001).
In NT&HRD, for example, the objective is to developthe students ability to summarize the subjects beingstudied and to identify as many conceptual links amongthem as possible. Therefore, students are advised to usea sort of top-down strategy and write the summary ofeach subject in no more than 20 lines per page. If theywish to write an exhaustive description and find there isinsufficient space, they are to highlight hot-words in thetext that link to further pages with a detailed examina-tion of the corresponding concepts. This process may berepeated to no further than three levels of depth from thehome page.
The co-writing methodology for development of theshared document
To fully benefit from the possibilities offered by wiki forco-writing and collaborative learning evaluation, thestudents work should be organized so that everyone ismotivated to play a part in each development stage ofthe shared script. The methodology adopted in theNT&HRD course is illustrated here point by point:
Wiki use in a collaborative learning project 45
2008 The Author. Journal compilation 2008 Blackwell Publishing Ltd
1 Individual study of recommended materials Havingbeen given the theme for the short thesis, students areprovided with a list of recommended study materials.Some of these may be found in the courses onlinerepository (articles, book chapters, etc.) while otherscan be retrieved directly on the web using a set of keywords provided by the teacher.
2 Co-planning of the hypertexts general structure anddivision of work Having studied the materials, thegroup is required to draw up collaboratively (in aforum) the hypertexts general structure (sections andfirst level subsections) and define the layout of thewiki home page. Then the work is divided among thegroup members.
3 Development of the various parts of the wiki Working individually, the group members developthe section of the text assigned to them and in thismanner create a branching hypertext document fol-lowing the above-mentioned top-down approach. Inwriting each page, they are advised to proceed step-by-step (from substance to form): write outthe summary; mark the hot-words to be linked to thepages with detailed examinations; and format thepage.
4 Links to pages created by others To prevent stu-dents concentrating exclusively on their part of thetext, they are required to browse the whole hypertext
to search for pages compiled by others which may beconceptually linked to one or more pages in their ownpage cluster (Fig 1). Clearly, this activity gets thestudents to examine the conceptual links throughoutthe work and fosters a more complete overall visionof the subject. The students are encouraged toperform the task while they are actually developingtheir pages and not merely leave it for last as finalrefinement. Reading the pages of co-authors as theyevolve not only sparks new ideas and suggestsimprovements for the students own text, but alsohelps to avoid duplications especially when two ormore students work on conceptually close subjectmatters. This also leads to a gradual transformation inthe hypertext structure from hierarchical (Fig 1) toreticular (Fig 2).
5 Peer review Once the different sections of theshared document have been written, the students areasked to peer-review all the pages and suggest to theircolleagues how to integrate and improve their respec-tive texts. In this case, the aim besides that men-tioned in point 4 is to encourage interactionbetween the author (the student who generated thepage) and the users (all the other students accessingit) on the chosen subject (Thompson 1988). Thisinteraction is facilitated by the comments functionassociated to each wiki page, through which short
Fig 1 Development of clusters of pagesassociated with each section of the text.
46 G. Trentin
2008 The Author. Journal compilation 2008 Blackwell Publishing Ltd
dialogues can take place among the differentcontributors/users of the hypertext.
The evaluation of collaborative learning
In the NT&HRD course, evaluation of collaborativelearning is based on three key elements:
The level of learning (achievement o...