Supporting small-group learning using multiple Web 2.0 tools: A case study in the higher education context

ACCEPTED MANUSCRIPT Jari Laru, Piia Näykki, Sanna Järvelä, Supporting small-group learning using multiple Web 2.0 tools: A case study in the higher education context, The Internet and Higher Education, Available online 28 August 2011, ISSN 1096-7516, 10.1016/j.iheduc.2011.08.004.(http://www.sciencedirect.com/science/article/pii/S1096751611000546)Keywords: Case study; Cloud-based social software; Explorative analysis; Higher education; Small-group collaboration

Supporting small-group learning using multiple Web 2.0 tools: A case study in the higher education context

JARI LARU*Faculty of Education, University of Oulu, Snellmania,Oulu, P.O. Box 2000, 90014 University of Oulu, [email protected]+358-40-5118478http://www.claimid.com/jarilaru

PIIA NÄYKKI, SANNA JÄRVELÄFaculty of Education, University of Oulu, Snellmania,Oulu, P.O. Box 2000, 90014 University of Oulu, [email protected], [email protected]

Abstract: In this single-case study, small groups of learners were supported by use of multiple social software tools and face-to-face activities in the context of higher education. The aim of the study was to explore how designed learning activities contribute to students’ learning outcomes by studying probabilistic dependencies between the variables. Explorative Bayesian classification analysis revealed that the best predictors of good learning outcomes were wiki-related activities. According to the Bayesian dependency model, students who were active in conceptualizing issues by taking photos were also active blog reflectors and collaborative knowledge builders in their group. In general, the results indicated that interaction between individual and collective actions likely increased individual knowledge acquisition during the course.

Keywords: Case study, Cloud-based social software, Explorative analysis, Higher education, Small-group collaboration

., 11/08/11,

Pls. note that this document was created using UK English; thus, it has been edited using UK spelling/grammar rules.

., 11/08/11,

An asterisk is typically used to indicate the corresponding author.

mailto:[email protected]




1. Introduction

Technology is one of the most significant mechanisms currently transforming the

learning process. Over the course of history, a range of artefacts has been produced (e.g.,

invention of the chart) that has modified the way in which people learn in various situated

practices (Pea, 1993). In particular, representational tools such as calculators and mind maps

have dramatically changed our daily practices in many spheres of life (Säljö, 2003). New

technologies provide opportunities for creating learning environments that extend the

possibilities of old technologies (e.g., books, blackboards, television, radio) and offer new

prospects for multiple social interactions (Bransford, Brown, & Cocking, 1999).

In recent years, a plethora of digital and networking tools has been established on the

Internet. These digital applications—which enable interaction, collaboration and sharing

among users—are frequently referred to as Web 2.0 (Birdsall, 2007) or social software tools

(Kesim & Agaoglu, 2007). These applications are further assumed to be a step change in the

evolution of Internet technology in higher education (Wheeler, 2009), which has evolved from

being primarily used to distribute course materials, communicate and evaluate to being used

to enhance educational processes that support collaborative learning and knowledge building

(Collins & Halverson, 2010; Cress & Kimmerle, 2008; Schroeder, Minocha, & Schneider, 2010).

Much has been written on the benefits of blogs (Halic, Lee, Paulus, & Spence, 2010; Hemmi,

Bayne, & Land, 2009; Wheeler, 2009; Xie, Ke, & Sharma, 2008),wikis (Cress & Kimmerle,

2008; Hemmi et al., 2009; Wheeler, 2009) and social networking sites (Arnold & Paulus,

2010) in education. However, very little formal research focusing on the integration of

multiple social software tools in higher education pedagogy has been published as of yet

(Uzunboylu, Bicen, & Cavus, 2011; Wheeler, 2009).

., 11/08/11,

Breaking up this lengthy sentence will improve the clarity of this paragraph

., 11/08/11,

Wordy - reduced - OK?

., 11/08/11,

Revised to improve flow of text


Crook (2008) and Meyer (2010) have argued a need for more empirical research on

the educational use of Web 2.0, its adoption and its impact on higher education. In this single-

case study, small groups of learners were supported by multiple social software tools and

face-to-face activities in the context of higher education. The purpose of this study was to

explore how designed learning activities contribute to students’ learning outcomes by

studying the probabilistic dependencies between the variables.

2. Theoretical background

2.1. Social software to support individual reflection

One activity that can promote the use of blogs in education is self-reflective practise

(Sharma & Fiedler, 2007; Xie et al., 2008). Self-reflecting is a central concept in metacognitive

learning in which students are encouraged to construct explanations, pose questions and

provide further information to each other (Cohen & Scardamalia, 1998). While constructing

explanations, the students become aware of their thought processes, gaps in knowledge and

lack of understanding (Webb, 1989). Through contributing their ideas and making their

thought processes visible, the students are able to reflect on their cognitive processes and

discuss with others what they do or do not know and understand.

Previous research (Xie et al., 2008) has shown that reflection is effortful action that

requires external support in order to engage students for extended periods of time. For

example, Xie et al. (2008) have introduced various strategies for encouraging reflection, and

they have concluded that blog-writing activities, journaling and peer feedback are all

appropriate reflection strategies.

Weblogs are popular journaling tools that offer students a means of externalising their

reasoning and reflecting on their experiences (Xie et al., 2008). Hence, Weblogs can be used as

., 11/08/11,

As meant?

., 11/08/11,

As meant?


‘learning logs’ that capture the cumulative history of a learning project in action and record

personally meaningful material that can foster and facilitate reflective practices such as

conversations with oneself and others (Halic et al., 2010; Hemmi et al., 2009; Sharma &

Fiedler, 2007; Xie et al., 2008). The main idea of blogging is similar to that of network

discussions: The students make their thinking visible and externalize their thinking by

periodically posting journal entries to their personal or collaborative blogs, allowing other

learners to comment on their learning blogs (Xie et al., 2008).

Second, in addition to self-reflective blog writing, peer feedback can provide a different

perspective and help students to assimilate and accommodate their thinking. Blogs can

facilitate reflective thinking, because people can easily access different points of view by

looking at peers’ blogs or comments (Xie et al., 2008). Furthermore, Really Simple Syndication

(RSS) offers novel ways to increase access to different points of view by enabling various

contributions to be aggregated, even though they may have originated from diverse sources

(e.g., blogs, file-sharing tools, and wikis) (Crook, 2008; Lee, Miller, & Newnham, 2008).

2.2. Social software to support collaborative learning

The potential of collaborative learning groups has been strongly supported by the

literature, which emphasizes students’ possibilities for constructing knowledge and

experiencing shared understanding through these groups (Dillenbourg, Baker, Blaye, &

Malley, 1996; Dillenbourg, 1999).

Social software applications (e.g., wikis) provide new opportunities for collaborative

learning and knowledge building (Cress & Kimmerle, 2008; Dohn, 2009). Moreover, they

present significant challenges to the views of knowledge (Cress & Kimmerle, 2008; Dohn,

2009), learning (Crook, 2008; Ravenscroft, 2009) and goals of the procedures implicit in Web

., 11/08/11,



2.0 practises on the one hand (Collins & Halverson, 2010; Crook, 2008; Dohn, 2009) and the

educational system on the other (Collins & Halverson, 2010; Dohn, 2009).

Dohn (2009) has stressed that Web 2.0 and/or educational practises must be reshaped

to fit each other, given that they originate in different contexts. From the perspective of

collaboration within Web 2.0 tools, who contributes is less important than the fact that

contributions are made and that they stand a chance of being revised by adding, deleting or

changing their components until the outcome corresponds to group direction and consensus

(Dohn, 2009).

Alternatively, Cress and Kimmerle (2008) see an imminent connection between

collaborative knowledge building in wikis and learning; from their perspective, one person’s

individual knowledge can serve as a resource for the learning of others. In their seminal paper

on knowledge building with wikis, they describe how people make use of each other’s

knowledge through collaborative knowledge building with artefacts. When interacting with a

wiki, individuals can learn as a result of either externalization or internalization. This learning

can take place by assimilation (extending knowledge by simply adding new information) or by

accommodation (modifying and creating new knowledge).

In this study, the pedagogical ideas behind the design are grounded in collaborative

learning, and special effort has been placed on enhancing and supporting collaborative

learning as a cognitive and social activity (Teasley, 1997). The students’ learning tasks,

including social and individual activities, were supported by designing learning assignments

that consisted of recurrent individual and collective phases in which students used Web 2.0

tools in concert to perform the designed tasks.

., 11/08/11,



In sum, all these activities to be undertaken with social software tools were also

aligned in such way that Web 2.0 characteristics (Dohn, 2009) were taken into account. For

example, Web-mediated resources were largely utilised; all created content was open, and

wiki pages had distributed authorship; different materials were reproduced and transformed

from multiple individual or collaborative learning spaces; and open-endedness and lack of

finality were actively promoted to all participating students.

3. Aims of the study

In this single-case study, small groups of learners were supported using multiple social

software tools and face-to-face activities in the context of higher education. The aim of the

study was to explore how designed learning activities contribute to students’ learning

outcomes by studying the probabilistic dependencies between the variables. The research

questions are as follows: 1) How much did students learn during the course? 2) Which social

software and face-to-face variables were the best predictors for identifying differences

between high- and low-performing groups of students? 3) What was the impact of social

software and face-to-face sessions on individual students’ learning gain?

4. Methods

This study followed the principles of the case study method. A case study is defined as

an empirical study that investigates a contemporary phenomenon within its real-life context,

especially when the boundaries between the phenomenon and the context are not evident

(Yin, 2003).

In practise, the research design of the current study employed a single-case study with

embedded multiple units of analysis. As multiple social software tools and face-to-face

activities were used to support learning in a higher education course, the behaviour of


students within phases of the learning design and students’ learning outcomes were

considered as the embedded units.

These units were analysed using quantitative techniques as the primary approach. In

order to return to larger units of analysis, Bayesian methods (Jensen, 2001) were used to

classify and model the complex dependencies between the different variables.

4.1. Participants and the research setting

The research participants were 21 undergraduate students in a five-year teacher

education programme in the Faculty of Education at the University of Finland. All of the

students were enrolled in a required course titled Future Scenarios and Technologies in

Learning during the spring semester of 2009. The 21 participants included 16 females (76%)

and 5 males (24%). The prevalence of females reflects the gender ratio of education majors at

the university.

4.1.1. The task

The participants worked in groups of four to five students for 12 weeks. Groups were

required to complete a wiki project by the end of the semester. In order to complete the wiki

project, students needed to participate in recurrent solo and collective phases mediated by

the use of social software tools and face-to-face meetings in their respective sessions (see

Figure 1).

On the first day of the course, in a campus computer lab, the instructor gave all

participating students pre-configured accounts to social software services and mobile devices

needed for photo-taking activities (see Section 4.1.2).

After ensuring that the students in their respective groups understood the instructions

provided, no further support was provided during the tasks. In other words, the assignments

., 11/08/11,

Revised to improve flow of text.

., 11/08/11,


., 11/08/11,

As meant?

., 11/08/11,

Keep formatting of sub-headers consistent.


required the students not only to learn and apply content knowledge, but also to generate

their own learning objectives and to determine what information to include in their final

contribution in their group wiki to be presented to the class.

--- Insert Figure 1 about here ---

The pedagogical design of this course was as follows:

A. Ground [Lecture] (weeks 1-3 and 6-8): Each of six one-week working periods started with

a lecture in which students were grounded in main theoretical concepts. The specific

themes were in the following order: 1. Learning infrastructure, 2. Learning communities,

3. Metacognition, 4. Self-regulated learning, 5. Learning design, and 6. Social Web as a

learning environment.

B. Reflect [Discussion] (weeks 1-3 and 6-8): The purpose of this collaborative phase was to

reflect on the lecture topic in groups and to formulate a problem to be solved based on the

group members’ shared interests during the following solo learning phases. Groups were

advised to set their own learning objectives based on the topic and to write down these

objectives in their personal blogs for further reflection.

C. Conceptualize [Photo-taking] (weeks 1-3 and 6-8): In this solo phase, individual students

were required to conceptualize their group members’ shared interests. In order to do so,

they were required to identify and capture situated pictorial metaphors describing their

shared interests. In practise, their tasks were to explore their everyday working and living

environments and take photos with a camera phone.

D. Reflect and elaborate [Blogging] (weeks 1-3 and 6-8): The task of this phase was to further

reflect and elaborate on photos in the students’ personal blogs. First, they were required

to analyse collected visual representations in order to discard ideas that were not


relevant to their groups’ shared learning objectives. Second, they were required to write

blog entries about chosen photos in which they further elaborated associations between

photos, group-level objectives and students’ everyday situated practises.

E. Review and evaluate [Discussion] (weeks 4 and 9): The first task of this collaborative face-

to-face activity was to review group members’ Weblogs from the previous three-week

period. The second activity was to evaluate the usefulness of blog entries in the context of

their shared learning objectives and to discard irrelevant ideas. The outcome of this phase

was used as material for co-construction of knowledge in the groups’ wikis.

F. Co-construct knowledge [Wiki work] (weeks 4-12): The task in this collaborative

assignment was focused on integrating each group’s chosen blog entries and visual

representations into a cohesive and comprehensive product of all course topics. In other

words, the given goal was to formulate what they had learnt ‘in their own words’ and

produce it as uniform material that could be put to authentic use.

G. Monitor peer students’ contributions [Monitor] (whole course): This was not an assignment

per se, but it enabled students to obtain different perspectives by seeing what others were

doing with social software tools, and it helped students to assimilate and accommodate

their thinking. In practise, monitoring activities were done by using cloud-based

syndication tools (RSS).

4.1.2. Tools

The idea of making use of each other’s knowledge was operationalized in a socio-

technical design. It consisted of recurrent individual and collective phases in which students

used multiple Web 2.0 tools and mobile phones in concert to perform designed tasks (Figure

2).


First, all students received a personal mobile multimedia computer, which was

integrated with features including a 3.2 megapixel digital camera, 3G connectivity and an

Internet browser. The mobile device was the main tool for the students in Phase C, who were

required to identify and capture situated pictorial metaphors describing their group’s shared

interests.

The device was equipped with a ShoZu cloud-based file-sharing tool, which was used as

a bridge to connect mobile phones to the Flickr cloud-based file-sharing service for photos.

ShoZu offered functions to add tags, titles and descriptions before putting photos on the Flickr

photostream. In addition, the phone’s Web browser was configured to show students’

accounts on the Google Reader Mobile cloud-based RSS aggregator. This service was used to

show all of the course-related content on the mobile phones at the students’ disposal (Figure

2).

Second, an individual Wordpress.com account was created for each student. This

blogging service was used as a personal learning diary for the students in which they

individually reflected further on their ideas by writing journal entries regarding the

respective pictures/videos sent to blogs via the Flickr file-sharing service (Phase C). The

students’ blogs were used as a storage facility for their group’s shared working problems

(Phase B) and as an anchor resource in the review and evaluate phase (Phase E). In addition,

the blogging service was the platform for course-level activities, a place for course-related

announcements.

The cloud-based Wikispaces wiki service was also used for two purposes: First, it

offered collaboration tools for the groups to use (i.e., empty wiki page and discussion tool) in

order to support their collaborative knowledge co-construction (Phase F). Second, it was used


at the course level for distributing resources (i.e., course curricula, lecture slides, hyperlinks

and how-to guides) and displaying syndicated content from Flickr (student accounts) and

WordPress (course blog, student blogs).

In addition, the FeedBlendr and FeedBurner RSS services were used to merge

individual, group and class-level feeds from multiple Flickr, WordPress and Wikispaces

accounts. In practise, these merged feeds were included as RSS widgets in a sidebar of the

respective blog or wiki. These tools enabled the students to combine social software tools, and

they may be seen as additional collaborative tools that facilitated relationships between

different task phases, the students, the content they produced and the tools used in this study

(See Lee et al., 2008).


4.2. Data collection

The data was composed of video recordings, social software usage activity and pre-

and post-tests of students’ conceptual understanding. Respective data variables are stored in

parentheses embedded into the descriptions below (see also Appendix I).

4.2.1. Conceptual knowledge test

To assess their conceptual understanding, the students completed identical paper-and-

pencil pre- and post-tests with a pre-test/post-test quasi-experimental design. Specifically,

the conceptual-knowledge measure consisted of six constructed-response questions that were

developed based on the key concepts of the course. Students were asked to write definitions

of the lecture themes, meaning that each theme was also connected to the learning design

described in Section 4.1.1. and was thus used for measuring the students’ learning outcomes

(gain) in a particular week of the course.


4.2.2 Video data

Video recordings captured each group’s six collaborative reflection sessions

(B.discussion) and two collaborative reviewing and evaluation sessions (E.discussion) (42

hours of video data). The duration of those sessions was determined by each group, and the

average duration of one session was 44 minutes (where the duration ranged from 13 minutes

to 86 minutes).

4.2.3 Social software activity data

Social software usage activity data was collected at the student level through multiple

sources. First, the total number of Flickr photos per weekly topic and the average number of

photos for all topics (C.photo) were calculated.

Second, the total number of words in each blog entry and the number of blog entries

were measured for each weekly topic. Then, the average values of these were calculated for all

topics (D.blog.posts; D.blog.words/post) to be used in the Bayesian multivariate analysis.

Third, activity measures of the students’ wiki usage were calculated by using adds and

deletes as coding categories for cumulative history data. A measure of student cumulative

involvement in the wiki was given by the sum Activity(u) = add(u) + delete(u), called the edit

activity of author u, providing the total number of words (F.wiki.wc.activity) or edits

(F.wiki.edits.activity) that u touched by adding or deleting them. This value was used to

calculate students’ active use of their respective group wikis and their interactions in the wiki

discussion forum and embedded comments in the wiki (F.wiki.edits.comments;

F.wiki.wc.comments). A further characterization of how an author u contributed to the group

wiki was given by the difference Net added (u) = add(u) – delete(u), called the net number of

words added or edits performed, providing the total number of words or edits by which u

., 11/08/11,

As meant?

., 11/08/11,

Paragraphs separated - OK?

., 11/08/11,

Paragraphs combined - OK?


increased the length of the text (F.wiki.wc.net) or the number of edits (F.wiki.edits.net) when

words or edits that u deleted were deducted. This value was used to calculate the amount of

new content students contributed to the wiki.

Finally, the total number of read RSS items was measured by using statistics collected

automatically by Google Reader (G.rss.monitor).

4.3 Data analysis

Data was analysed using a quantitative paired samples t-test for the conceptual

knowledge tests, qualitative on-task analysis for video recordings and multivariate Bayesian

methods for the dependencies between social software usage, face-to-face activities and

learning gain.

4.3.1. Quantitative analysis of conceptual knowledge tests

In the first stage of analysis, a conceptual knowledge test was analysed in order to

answer the first research question: How much did students learn during the course?

Three independent researchers (including the first and second authors of this paper)

developed the criteria and marked the learning tests (points 0-3). The criteria were as

follows: 0 points represented low understanding (the student had no understanding of the

concept). One point represented some level of understanding (the student had some

understanding (i.e., knew what the concept was connected to) but no detailed knowledge of

it). Two points represented a basic level of understanding (the student understood what the

concept was connected to and knew some details about the concept). Finally, 3 points

represented the highest level of understanding (the student had a deep understanding of the

concept and knew very specific details about the concept).


The tests were analysed by marking points from 0 to 3 for individual answers. This was

done by three researchers who first independently marked the tests and then compared the

results and negotiated possible differences. According to the test results, all of the students’

understanding of the main concepts increased during the course. However, there were

differences between their levels of understanding of the different concepts.

To analyse the learning outcomes through the pre-test/post-test scores, a paired

samples t-test was conducted, and a normalized learning gain was calculated (Hake, 1998).

Next, the average normalised gain score was used to identify high-performing and low-

performing students for further explorative Bayesian analysis. Note that contrasting the

activity and artefacts of high performers to those of low performers is intuitively appealing

(Jonassen, Tessmer, & Hannum, 1999) and has been shown to reveal important

characteristics and aspects that are not uncovered using other approaches (Wyman & Randel,

1998).

4.3.2. Qualitative analysis of videotaped face-to-face sessions

In the second stage, video data transcripts were analysed in order to clarify individual

students’ activity levels in collaborative face-to-face assignments. Results of this analysis were

used as an activity measure of face-to-face activities for descriptive analysis of learning

phases and explorative Bayesian analysis (research questions 2 and 3).

This analysis was adapted from the method that focuses on the duration of on-task and

off-task episodes (for further details of the method, see Järvelä, Veermans, & Leinonen, 2008).

In this analysis, the focus was placed on the number of task-related utterances, which were

used as a measure of on-task activities, while off-task activities, such as discussions about

their evening plans, were coded in an independent off-task category.


4.3.3. Descriptive analysis of social software and face-to-face activity variables

In the third phase, a descriptive analysis was carried out for all the variables in the

course design. First, the average values of an individual student’s face-to-face and social

software activities were calculated for Bayesian analysis (research questions 2 and 3). Second,

the mean, standard deviation and max-min values for all students (both high- and low-

performing students) were calculated in order to assist in the interpretation of the results of

Bayesian classification modelling and to provide an overview of the students’ activities during

the course (See Appendix).

4.3.4. Bayesian multivariate analysis of the impact of social software and face-to-face sessions

on learning outcome

In the fourth phase, Bayesian analysis (Jensen, 2001) was conducted to study the

probabilistic dependencies between the variables (research questions 2 and 3) described in

Section 4.2. In practise, the analysis was conducted with the Web-based online data analysis

tool B-Course1, which allowed users to analyse their data using two different techniques:

Bayesian dependency and classification modelling.

In general, Bayesian methods have many benefits for explorative analysis, as

summarized in Congdon (2003). For this study, the most relevant benefits were as follows: 1)

The theoretical minimum for the sample is zero, 2) Different kinds of multivariate variables

and distributions are accepted, and 3) It gives statistically robust tools to visualize and

categorize complex dependencies between variables. In short, Bayesian methods enabled us

to conduct statistical analyses of learning phases in our learning design.

1 http://b-course.cs.helsinki.fi/obc/


The first stage of Bayesian analysis involved conducting classification modelling

(Silander & Tirri, 1999) in order to answer the second research question: Which social

software and face-to-face variables were the best predictors for determining differences

between high- and low-performing groups of students? In the classification process, the

automatic search looked for the best set of variables to predict the class variable for each data

item. This procedure is akin to the stepwise selection procedure in traditional linear

discriminant analysis (Huberty, 1994).

The second stage of Bayesian analysis involved building a Bayesian network (Jensen,

2001) in order to answer the third research question: What was the impact of social software

and face-to-face sessions on individual students’ normalized learning gain? Such a Bayesian

network was the visualised result of Bayesian dependency modelling, in which the most

probable statistical dependency structure between variables was calculated.

A graphical visualization of a Bayesian network given by the B-Course program

(Myllymäki, Silander, Tirri, & Uronen, 2002) contains three components (See Figure 3 and

Table 3): 1) collected data as ellipses, 2) dependencies visualised as lines between nodes and

3) strength of each dependency as a ratio value in the table (see Table 3) and as a colour in the

network. The darker the line, the stronger the statistical dependency between the two

variables and the more important (higher ratio value) the dependency. A variable is

considered independent of all other variables if there is no line attached to it.

5. Results

First, results of the paired samples t-test will be presented to show how much students

learned during the course. Second, the best predictors for pointing out differences between

high- and low-performing groups will be explored using Bayesian classification analysis.


Third, the results of Bayesian dependency modelling showing probability dependencies

between the social software, face-to-face sessions and individual students’ normalized

learning gain shall be presented.

5.1 How much did students learn during the course?

A paired samples t-test was conducted to compare pre-test and post-test means.

Results showed that students gained higher scores in the post-test (M=7.95) than in the pre-

test (M=3.95), t(21)=8.33, p<.000. The effect size (Cohen’s d) was 1.69.

--- Insert Table 1 about here ---

Table 1 presents the mean values for pre-test and post-test raw scores and pre-post

normalized gain scores. Using the average normalized gain score (M=0.29; SD=0.16), high-

performing and low-performing students were identified for explorative Bayesian

classification analysis.

5.2. Which social software and face-to-face variables were the best predictors for determining

differences between high- and low-performing groups of students?

The second analysis explored which variables measuring social software usage and

face-to-face activities were the best predictors for pointing out differences between high- and

low-performing students. The model for classifying data contained items according to the

class variable level of the normalized learning gain (low performers and high performers)

with 12 variables of learning activities (descriptive values are shown in Appendix I, and items

are described in Section 4.2). The estimated classification accuracy for the model was 81.82%.

Table 2 lists the variables ordered by their estimated classification in the model. The

strongest variables—that is, those that best discriminate the independent variables—are


listed first. The percentage values attached to each variable indicate the predicted decrease in

the classification performance if the variable were to be dropped from the model. The table

shows that all variables in the model are equally important; that is, if we were to remove any

of the variables from the model, it would weaken the performance by 90.91%.


Results from the classification analysis showed that the best predictors of higher

learning gains were wiki-related activities.

First, the mean number of wiki edits (F.wiki.edits.activity; M=68.64; SD=77.90) was

two times higher among high performers than low performers (M=34.55; SD=21.16). Second,

the high performers were 1.5 times more involved in the wiki editing activities (M=3427.73;

SD=3810.10) than the low performers (M=2151.10; SD=2074.12) when the number of words

(F.wiki.wc.activity) that they touched by adding or deleting was taken into account. Third,

high-performing students increased the length of the text (F.wiki.wc.net) in their groups’

wikis about 1.4 times more often on average (M=1173.91; SD=444.70) than low-performing

students (M=856.45; SD=507.49).

In short, the descriptive analysis above shows that high performers were more active

in organizing wiki content in a new way and in adding new information. The latter of these

contribution categories is an example of assimilation, a process in which information coming

from the wiki is perceived and modified in a way that makes it fit into the individual’s

knowledge. The former category is an example of an activity in which students do not simply

assimilate new information into existing knowledge but actually change knowledge in order

to better understand the wiki and its information (Cress & Kimmerle, 2008).


5.3. What was the impact of social software and face-to-face sessions on individual students’

normalized learning gain?

The next stage of the analysis involved building a Bayesian network out of the 12 items

measuring students’ learning activities during the course (descriptive values are shown in

Appendix I, and items are described in Section 4.2). The rationale for this procedure was to

examine dependencies between variables by both their visual representation and the

probability ratio of each dependency in order to answer the third research question.

A Bayesian search algorithm evaluated the dataset in order to find the model with the

highest probability. During the extensive search, 174,987 models were evaluated. Figure 3

shows a visualization of the network, which contains two components: 1) collected data as

ellipses and 2) dependencies visualised as lines between nodes. As mentioned, the darker the

line, the stronger the statistical dependency between the two variables and the more

important the dependency. Table 3 shows the strength of each dependency as ratio values in

the probability table.

In practise, if one removes the arc from the model with the high probability ratio, it

decreases the probability of the model by the same amount. However, in many dependencies

in the model, removing the arc between nodes would not change the probability of the final

model (listed at the bottom of the probability table).



The Bayesian dependency model shows 7 strong (probability ratio >1,000,000) and 25

weaker relationships between variables. However, based on the analysis, only one strong

dependency between activities and learning gain was found: the connection between

., 11/08/11,


., 11/08/11,

Mentioned on p. 15

., 11/08/11,

Somewhat redundant, as it is the title of this section - deleted - OK?


assimilative wiki editing activities (F.wiki.wc.activity) and learning gain (gain), which

triangulates with the results in the Bayesian classification model presented above.

Furthermore, there was one weak dependency, the one between monitoring other students’

work via syndication services (G.rss.monitor) and learning gain (gain). Additionally, there

were two other connections between other variables (B.discussion, C.photo) and normalized

learning gain (gain) included in the visual network model, but their probabilities were so low

that they were dropped from the dependency table automatically. It is worth noting that the

wiki activities described above were strongly related to commenting on wiki content.

When the Bayesian model is further explored, it reveals that the average number of

blog posts (D.blog.posts) is the central variable in the model, as it has strong statistical

relationships to both assimilative (F.wiki.wc.net; F.wiki.edits.net) and accommodative wiki

activities (F.wiki.wc.activity; F.wiki.edits.activity). In practice, it can be said that students who

were actively reflecting and elaborating were also active in inserting and modifying

information in the wikis. This variable (D.blog.posts) also has a central role in the chain of

strong relationships, including all virtual activities in the study design (see Figure 1.): C.

Conceptualize, (C.photos), D. Reflect and elaborate (D.blog.posts), F. Co-construct knowledge

(F.wiki.wc.activity), and learning gain (Gain). This result demonstrates the successful use of

Web 2.0 characteristics in this study, an example of a series of activities in which intermediate

learning products were reproduced and transformed. Furthermore, it shows how higher

education course students can make use of each other’s knowledge through collaborative

knowledge building (Cress & Kimmerle, 2008).

There were also several weaker dependencies in the Bayesian model. First, results

showed that active following of RSS feeds was slightly related to an increased number of

., 11/08/11,

As you are referring to the results of your own study, there is no need for a reference here.


situated visual representations (C.photos), an increased number of wiki editing activities

(F.wiki.*) and learning gain (gain). However, no connection was found between usage of RSS

feeds and blogging. Second, both collaborative face-to-face phases (B.discuss, D.discuss) were

slightly related to social software usage (D.blog.*; F.wiki.*) except the phase in which students

had to take photos.

6. Discussion

In our case, we found that using social software tools together to perform multiple

tasks likely increased individual knowledge acquisition during the course. Multivariate

Bayesian classification analysis revealed that the best predictors of good learning outcomes

were wiki-related activities. In addition, according to the Bayesian dependency model,

students who monitored their peers’ work via syndication services and who were active by

adding, modifying or deleting text in their group’s wiki obtained higher scores. The model also

shows that many other learning activities were indirectly related to learning outcome.

First, learning scores from pre-test to post-test were statistically significant with high

learning effect, indicating a substantial gain in conceptual knowledge test scores from pre-test

to post-test. This finding provides support for the learning design used in this study and for

the use of multiple cloud-based social software tools in a higher education context, and it was

further used to contrast high performers and low performers in the following explorative

Bayesian analysis.

Second, results from the Bayesian classification analysis revealed differences between

high performers and low performers and showed that the best predictors of higher learning

gain were wiki-related activities. Descriptive analysis of chosen predictor variables showed

that high performers were more active in organizing wiki content in a new way (mean


number of wiki edits was two times higher and mean word length of edited content was two

times higher when compared to low performers) and in adding new information (mean length

of inserted words was 1.4 times higher than that of low-performers). The latter of these

contribution categories is an example of assimilation, a process in which information coming

from the wiki is perceived and modified in a way that makes it fit into an individual’s

knowledge. The former category is an example of an activity in which students do not simply

assimilate new information into existing knowledge but actually change knowledge in order

to better understand the wiki and its information (Cress & Kimmerle, 2008).

After 174,987 models were calculated, the final Bayesian dependency model included

7 strong relationships and 25 weaker relationships between variables. Interestingly, the only

strong dependency between activities and learning outcome was found between assimilative

wiki editing activities and learning gain, which triangulates with results in Bayesian

classification modelling. Furthermore, there was one weak dependency, between monitoring

other students’ work via syndication services and learning outcome. There were two other

connections between other variables and learning gain included in the network model, but

their probabilities were so low that removing them would not change the probability of the

final model, and therefore, those were dropped automatically from the final model during the

analysis. It is also worth noting that the wiki activities described above were strongly related

to commenting on wiki content.

When the Bayesian model is further explored, it reveals that the average number of

blog posts per student is the central variable in the model, as it has strong statistical

relationships to both assimilative and accommodative wiki activities. In practise, it can be said

that students who were actively reflecting and elaborating on visual representations in their

., 11/08/11,

This section of your discussion is strikingly similar to your results section - consider revising by interpreting the results rather than merely restating them.


own blogs were also active in inserting and modifying knowledge in the wikis. This can be

considered an example of learning that is both reflective and collaborative at the juxtaposition

of community and personal spaces (Wheeler, 2009).

This blog post variable also has a central role in the chain of strong relationships,

including almost all social software-related tasks in this study: average number of photos

taken and shared by each student, average number of blog posts, total sum of wiki activity,

and learning gain. This chain of activities demonstrates the successful use of Web 2.0

characteristics in this study, an example of a series of activities in which intermediate learning

products were reproduced and transformed by performing structured collaborative

assignments using Web 2.0 tools. It also shows how higher education course students can

make use of each other’s knowledge through collaborative knowledge building (Cress &

Kimmerle, 2008).

The remaining variables were weaker than those presented above. First, the results

showed that monitoring who does what (implicit peer feedback for individual reflection)

using syndication tools (RSS) was slightly related to an increased number of situated visual

representations (photos), number of wiki editing activities and learning gain. However, the

model did not show connections between blog and syndication variables. Therefore, it can be

argued that different perspectives on the form of syndicated content did not contribute to

reflective blog-writing activities. Instead, the results showed that active monitoring of the

activities of others using different social software tools increased students’ number of wiki

activities. Generally, these results further reinforced the findings of Jermann and Dillenbourg

(2008), who determined that the tools can provide information to foster group members’

reflections of their contributions: ‘what to do’ and ‘who does what’. Second, the results

., 11/08/11,

Avoid beginning sentences with “so” in academic writing.

., 11/08/11,

Again, this section of your discussion is very similar to your results section - consider revising.


revealed that the explicit peer feedback that students received by participating in

collaborative face-to-face sessions (sense-making session and meaning-making session)

slightly increased social software usage activities.

7. Conclusion

It can be concluded that the carefully crafted pedagogical activities and Web 2.0 tools

used together to perform designed tasks likely increased students’ individual knowledge

acquisition during the course. This is in accordance with Meyer’s (2010) claim regarding how

assignments should be structured and orchestrated to encourage learning to occur. It also

reinforces findings of Halic et al. that a “technological tool works better when it’s coupled with

compatible pedagogical conceptions,” and yet “interaction is insufficient to achieve cognitive

engagement. Some type of facilitation in online environments may be necessary” (2010, p.

211).

The findings of our case study, together with the described socio-technical design,

illustrate practical implications for designing the use of multiple social software tools to

support collaborative learning in higher education. Therefore, by providing an explicit socio-

technical example, this study can contribute to pedagogical practices when educators are

considering how they should use cloud-based social software as a learning platform

(Schroeder et al., 2010; Wheeler, 2009). First, the findings from this study contribute to the

emerging body of studies surrounding the empirical research regarding the educational use of

Web 2.0 and its adoption and impact (Crook, 2008). Second, this article is also a timely and

rare contribution to the emerging discussions on how to design and integrate the use of

multiple Web 2.0 tools in higher education contexts in a pedagogically meaningful way


instead of using legacy virtual learning environments (Hemmi et al., 2009; Schroeder et al.,

2010; Uzunboylu et al., 2011; Wheeler, 2009).

This case study was limited by the single-case design and the lack of other student

groups completing the same tasks with the same socio-technical design. The rationale for the

single-case design is that it is a revelatory case (Yin, 2003). In practise, this study is a rare

contribution to the empirical analysis of integrating face-to-face situations and social software

in higher education. In addition, the course in which the data collection was conducted was

the first implementation of the described socio-technical design at the university.

Furthermore, this study used embedded multiple units of analysis in order to

qualitatively collect and analyse complex dependencies between different learning phases and

students’ learning outcome, which raises concerns of a small sample size within subunits (Yin,

2003). To overcome the problems raised by the relatively small sample size, data was

analysed using Bayesian methods, which do not have theoretical minimums for sample sizes

and offer other benefits for explorative data analysis (Congdon, 2003; Jensen, 2001).

It also has been argued that research designs in authentic contexts inevitably provide

principles that can be localised for others to apply to new settings and to produce

explanations of innovative practises (Fishman, Marx, Blumenfeld, Krajcik, & Soloway, 2004).

Therefore, research investigations conducted in authentic contexts are still needed as a first

step to understand these new opportunities in terms of learning interaction and collaboration

that social software can provide.


Acknowledgements

This research was supported by the Doctoral Programme for Multidisciplinary Research on

Learning Environments, Finland, and a grant from the Finnish Cultural Foundation.


References

Arnold, N., & Paulus, T. (2010). Using a social networking site for experiential learning:

Appropriating, lurking, modeling and community building. The Internet and Higher

Education, 13(4), 188-196. doi:10.1016/j.iheduc.2010.04.002

Birdsall, W. F. (2007). Web 2.0 as a social movement. Webology, 4(2). Retrieved from

http://www.webology.ir/2007/v4n2/a40.html

Bransford, J., Brown, A. L., & Cocking, R. R. (1999). How people learn: Brain, mind, experience,

and school. Washington, DC: National Academy Press. Retrieved from

http://www.nap.edu/catalog.php?record_id=9853

Cohen, A., & Scardamalia, M. (1998). Discourse about ideas: Monitoring and regulation in face-

to-face and computer-mediated environments. Interactive Learning Environments, 6(1-

2), 93-113. doi:10.1076/ilee.6.1.93.3610

Collins, A., & Halverson, R. (2010). The second educational revolution: Rethinking education

in the age of technology. Journal of Computer Assisted Learning, 26(1), 18-27.

doi:10.1111/j.1365-2729.2009.00339.x

Congdon, P. (2003). Applied Bayesian modelling. Chichester: Wiley.

Cress, U., & Kimmerle, J. (2008). A systemic and cognitive view on collaborative knowledge

building with wikis. International Journal of Computer-Supported Collaborative

Learning, 3(2), 105-122. doi:10.1007/s11412-007-9035-z

Crook, C. (2008). Web 2.0 technologies for learning: The current landscape - opportunities,

challenges and tensions (British Educational Communications and Technology Agency

(BECTA) Report: Web 2.0 technologies for learning at Key Stages 3 and 4). Retrieved

from http://dera.ioe.ac.uk/1474/


Dillenbourg, P. (1999). What do you mean by collaborative learning? In P. Dillenbourg (Ed.),

Collaborative-learning: Cognitive and computational approaches (pp. 1-19). Oxford:

Elsevier.

Dillenbourg, P., Baker, M., Blaye, A., & Malley, C. O. (1996). The evolution of research on

collaborative learning. In E. Spada & P. Reiman (Eds.), Learning in humans and

machines: Towards an interdisciplinary learning science (pp. 189-211). Oxford: Elsevier.

Dohn, N. (2009). Web 2.0: Inherent tensions and evident challenges for education.

International Journal of Computer-Supported Collaborative Learning, 4(3), 343-363.

doi:10.1007/s11412-009-9066-8

Fishman, B., Marx, R., Blumenfeld, P., Krajcik, J., & Soloway, E. (2004). Creating a framework

for research on systemic technology innovations. Journal of the Learning Sciences,

13(1), 43-76. Retrieved from http://www.jstor.org/stable/1466932

Hake, R. (1998). Interactive-engagement versus traditional methods: A six-thousand-student

survey of mechanics test data for introductory physics courses. American Journal of

Physics, 66(1), 64-74. doi:10.1119/1.18809

Halic, O., Lee, D., Paulus, T., & Spence, M. (2010). To blog or not to blog: Student perceptions of

blog effectiveness for learning in a college-level course. The Internet and Higher

Education, 13(4), 206-213. doi:10.1016/j.iheduc.2010.04.001

Hemmi, A., Bayne, S., & Land, R. (2009). The appropriation and repurposing of social

technologies in higher education. Journal of Computer Assisted Learning, 25(1), 19-30.

doi:10.1111/j.1365-2729.2008.00306.x

Huberty, C. J. (1994). Applied discriminant analysis. New York: Wiley.

., 11/08/11,

As meant?


Järvelä, S., Veermans, M., & Leinonen, P. (2008). Investigating student engagement in

computer-supported inquiry: A process-oriented analysis. Social Psychology in

Education, 11(3), 299-322. doi:10.1007/s11218-007-9047-6

Jensen, F. V. (2001). Bayesian networks and decision graphs. New York: Springer.

Jermann, P., & Dillenbourg, P. (2008). Group mirrors to support interaction regulation in

collaborative problem solving. Computers & Education, 51(1), 279-296.

doi:10.1016/j.compedu.2007.05.012

Jonassen, D. H., Tessmer, M., & Hannum, W. H. (1999). Task analysis methods for

instructional design. Mahwah, NJ: L. Erlbaum Associates.

Kesim, E., & Agaoglu, E. (2007). A paradigm shift in distance education: Web 2.0 and social

software. Turkish Online Journal of Distance Education, 8(3), 66-75. Retrieved from

http://tojde.anadolu.edu.tr/tojde27/

Lee, M. J. W., Miller, C., & Newnham, L. (2008). RSS and content syndication in higher

education: Subscribing to a new model of teaching and learning. Educational Media

International, 45(4), 311-322. doi:10.1080/09523980802573255

Meyer, K. A. (2010). Web 2.0 research: Introduction to the special issue. The Internet and

Higher Education, 13(4), 177-178. doi:10.1016/j.iheduc.2010.07.004

Myllymäki, P., Silander, T., Tirri, H., & Uronen, P. (2002). B-course: A web-based tool for

Bayesian and causal data analysis. International Journal on Artificial Intelligence Tools,

11(3), 369-387. doi:10.1142/S0218213002000940

Pea, R. D. (1993). Practises of distributed intelligence and designs for education. In G. Salomon

(Ed.), Distributed cognitions: Psychological and educational considerations (pp. 47-87).

New York, NY: Cambridge University Press.


Ravenscroft, A. (2009). Social software, web 2.0 and learning: Status and implications of an

evolving paradigm. Journal of Computer Assisted Learning, 25(1), 1-5.

doi:10.1111/j.1365-2729.2008.00308.x

Schroeder, A., Minocha, S., & Schneider, C. (2010). The strengths, weaknesses, opportunities

and threats of using social software in higher and further education teaching and

learning. Journal of Computer Assisted Learning, 26(3), 159-174. doi:10.1111/j.1365-

2729.2010.00347.x

Sharma, P., & Fiedler, S. (2007). Supporting self-organized learning with personal web

publishing technologies and practices. Journal of Computing in Higher Education, 18(2),

3-24. doi:10.1007/BF03033411

Silander, T., & Tirri, H. (1999). Bayesian classification. In P. Ruohotie, H. Tirri, P. Nokelainen, &

T. Silander (Eds.), Modern modeling of professional growth (pp. 61-84). Hämeenlinna:

RCVE.

Säljö, R. (2003). Representational tools and the transformation of learning. In B. Wasson, U.

Hoppe, & S. Ludvigsen (Eds.), Designing for change in networked learning environments

(pp. 1-2). Dordrecht, The Netherlands: Kluwer Academic Publishers.

Teasley, S. D. (1997). Talking about reasoning: How important is the peer in peer

collaborations? In L. B. Resnick, R. Saljo, C. Pontecorvo, & B. Burge (Eds.), Discourse,

tools, and reasoning: Situated cognition and technologically supported environments (pp.

361-384). Heidelberg, Germany: Springer-Verlag.

Uzunboylu, H., Bicen, H., & Cavus, N. (2011). The efficient virtual learning environment: A case

study of web 2.0 tools and windows live spaces. Computers & Education, 56(3), 720-

726. doi:10.1016/j.compedu.2010.10.014


Webb, M. N. (1989). Peer interaction and learning in small groups. International Journal of

Educational Research, 13, 21-40. doi:10.1016/0883-0355(89)90014-1

Wheeler, S. (2009). Learning space mashups: Combining web 2.0 tools to create collaborative

and reflective learning spaces. Future Internet, 1(1), 3-13. doi:10.3390/fi1010003

Wyman, B. G., & Randel, J. M. (1998). The relation of knowledge organization to performance

of a complex cognitive task. Applied Cognitive Psychology, 12(3), 251-264.

doi:10.1002/(SICI)1099-0720(199806)12:3<251::AID-ACP510>3.0.CO;2-F

Xie, Y., Ke, F., & Sharma, P. (2008). The effect of peer feedback for blogging on college

students’ reflective learning processes. The Internet and Higher Education, 11(1), 18-

25. doi:10.1016/j.iheduc.2007.11.001

Yin, R. K. (2003). Case study research: Design and methods (3rd ed.). Thousand Oaks, CA: Sage

Publications.

., 11/08/11,

Pls. note: All straight quotes have been replaced with smart (curly) quotes.


Table 1. Pre-test and post-test raw scores and normalized gain scores

Table 2. Importance ranking of the social software usage and learning activity variables by the level of normalized gain score

Class variable: The level of normalized gain scoreDropa

low-performers < 0.29

high-performers > 0.29

Predictor variablesb % M SD M SD

F.wiki.wc.activity90.91 2151.09

2074.12 3427.73

3810.10

F.wiki.wc.net

90.91 855.45 507.49 1173.91 444.70

F.wiki.edits.activity90.91 34.55 21.16 68.64 77.90

Note. In the classification modelling process (Silander & Tirri, 1999), the automatic search looked for the best set of variables to predict the class variable for each data item.a. Decrease in predictive classification if item is dropped from the classification model.b. Classification accuracy is 81.82%.


Table 3

DependencyProbability ratio

D.blog.posts -> F.wiki.wc.activity 1:1.000.000.000D.blog.posts -> F.wiki.wc.netD.blog.posts -> F.wiki.edits.activityF.wiki.edits.activity -> F.wiki.edits.comments 1:1.000.000D.blog.posts -> F.wiki.edits.netF.wiki.wc.activity -> GainGain -> F.wiki.wc.commentsC.photos -> D.blog.posts 1:2254G.rss.monitor -> F.wiki.wc.activity 1:975G.rss.monitor -> F.wiki.wc.net 1:975G.rss.monitor -> F.wiki.edits.activity 1:931G.rss.monitor -> F.wiki.wc.comments 1:880G.rss.monitor -> F.wiki.edits.net 1:798D.blog.words/post -> E.discussion 1:797G.rss.monitor -> C.photos 1:72E.discussion -> F.wiki.wc.activity 1:44E.discussion -> F.wiki.wc.net 1:44B.discussion -> F.wiki.wc.activity 1:44B.discussion -> F.wiki.wc.net 1:44E.discussion -> F.wiki.edits.net 1:44B.discussion -> F.wiki.edits.net 1:44E.discussion -> F.wiki.edits.activity 1:44B.discussion -> F.wiki.edits.activity 1:44B.discussion -> F.wiki.wc.comments 1:44B.discussion -> D.blog.posts 1:31C.photos -> F.wiki.wc.comments 1:26G.rss.monitor -> Gain 1:17G.rss.monitor -> F.wiki.edits.comments 1:14G.rss.monitor -> E.discussion 1:4.91D.blog.words/post -> C.photos 1:3.62G.rss.monitor -> B.discussion 1:2.69Note. The probability ratio describes the strength of statistical dependency between the two variables and the importance of the dependency for the model.

., 11/08/11,

Missing table title


Appendix

Table 1. Descriptive statistics of students’ activities during the course

Descriptive statistics for face-to-face, social software activity and learning gain variables

All students (n=21) High-performers (n=10) Low-performers (n=11)Unit Mean Stdev Max Min Mean Stdev Max Min Mean Stdev Max Min

B. Reflect [discussion]B.discussion utterances 74.27 28.17 118 23 78.18 23.48 107 49 70.36 32.89 118 23

C. Conceptualize [photo-taking]C.photos photos 3.86 1.25 6 2 3.73 1.10 5 2 4.00 1.41 6 2

D. Reflect and elaborate [blogging]D.blog.posts posts 3.99 1.25 6 1.8 4.05 1.03 5.3 1.8 3.93 1.48 6 1.8

D.blog.words/postwords/post 88.09 37.76 153 9 101.27 40.11 153 30 74.91 31.67 128 9

E. Review and evaluate [discussion]E.discussion utterances 219.86 80.44 390 74 202.64 69.47 327 81 237.09 90.06 390 74

F. Co-construct knowledge [wiki-work]

F.wiki.edits.activity edits 51.59 58.37 271 4 68.64 77.90 271 5 34.55 21.16 72 4F.wiki.edits.net edits 16.86 14.71 59 2 19.91 17.47 59 3 13.82 11.36 42 2

F.wiki.wc.activity words2789.41 3064.02 12830 320 3427.73 3810.10 12830 355 2151.09 2074.12 6654 320

F.wiki.wc.net words1014.68 493.33 2067 122 1173.91 444.70 1854 353 855.45 507.49 2067 122

F.wiki.edits.comments edits 14.09 9.72 34 2 15.82 11.76 34 2 12.36 7.31 26 2F.wiki.wc.comments words 277.08 235.46 841 0 252.46 220.18 701 0 301.70 258.10 841 0

G. Monitor peer students’ contributions [monitor]G.rss.monitor read items 120.09 199.83 701 0 76.09 124.81 428 0 164.09 253.03 701 0

., 11/08/11,

Note: Font was made smaller to fit page.


Normalized learning gainGain pre-post

gain0.29 0.16 0.60 0.00 0.42 0.08 0.6 0.31 0.16 0.08 0.27 0

Note. Mean, standard deviation and max-min values for all students (both high- and low-performing students) were calculated in order help interpret the results of Bayesian classification modelling and to provide an overview of the students’ activities during the course.


., 11/08/11,

Pls. note that the following two pages have been left unedited, as they are not included in your stated word count.
