Unesco Paper Final

8/8/2019 Unesco Paper Final

http://slidepdf.com/reader/full/unesco-paper-final 1/20

Repurposing social networking technologies toencourage preservice teacher collaboration inonline communities: A mixed methods study

Michael MoroneyUniversiti Brunei Darussalam

The purpose of this study was to determine the extent to whichtrainee teachers were able to use social networking technology for online collaboration in a learning community. Communities of practice are at the forefront of inservice teacher professionaldevelopment and their usage in preservice courses may have valuealso, if for no other reason than to provide opportunities for participants to learn how to collaborate effectively online. Thisstudy used a mixed method approach to determine the extent to

which participants were totally committed to online collaborativelearning. Findings indicate that participants generally supported the concept of an online community but that they found it difficult to develop and sustain one. There were also indications that the perceived value of online collaborative learning was in many casesnot able to overcome long held practices and beliefs that valuecompetition and individual effort. Thus the concept of onlinecommunities of practice may more easily be introduced to inserviceteachers who already have working experience of collaboration and cooperation upon which they can draw or when the high stakesexamination system, in which participates operated, is replaced by

one that measures the transaction of learning. Directions for further research are discussed.

Introduction

There is considerable interest in collaborative learning, both amongstteachers and amongst students in classrooms. There is support for thismodel if for no other reason than because a team working together mayattain greater overall achievement than that achieved throughindividualized learning (Garrison & Anderson, 2005; Soller & Lesgold,

2007). Garrison et al. (2005) stated “it is a practical necessity” thatstudents be independent thinkers, interdependent collaborative learners,and that these characteristics are “…core values in any educationalexperience” (p. 22). Carugati et al. (2008) further highlighted theimportance of collaborative learning when they described web groups andsocial networks as two key concepts that best depict online e-learning. Ina web group environment participants use an Internet site as a repositoryfor shared resources. Social networks, on the other hand, may beconsidered Web 2.0 technologies that afford participants the opportunityto participate in joint knowledge creation through a process of negotiationwith members of an online community. Web groups are often associatedwith “Web 1.0” technologies and social networks are often associated withWeb 2.0 technologies. Online facilities such as Facebook, MySpace,Secondlife and Ning are examples of Web 2.0 technologies.

1



It is recognised that teachers benefit from collaborative learning justas students do. For example, those teachers engaged in professionaldevelopment learn best when they are able to work in teams on commonproblems over extended periods of time (Loucks-Horsley, Love, Stiles,Mundry, & Hewson, 2003; Loucks-Horsley & Matsumoto, 1999; Timperley,Wilson, Barrer, & Fung, 2007). Technology, in this situation, can be usedto add value to the quality of the professional development experience byfacilitating more and different opportunities to share ideas and to enhancecollaboration. In some education systems, such as the secondary schoolsystem in Brunei, it is often the case that a school may only have one ortwo specialist teachers. For example, in Brunei it is uncommon for anysecondary school to have more than one chemistry teacher. Furthermore,schools are geographically far enough apart that it is not feasible forteachers to gather together regular professional development sessions.

Therefore it is of benefit to teachers if online team collaboration can takeplace to supplement infrequent face-to-face meetings and improve the

effectiveness and relevance of teacher professional development.

However, this said, online CSCL environments are complex placesand online communities may not always meet the expectations of theirmembers (Salmon, 2005). Therefore there is the need to identify thefactors that do help to ensure online communities are successful.

Community members of any online learning group may be broadlycategorized as being: non-compliant, procedurally compliant or totallycommitted to online collaborative learning. For example a non-compliantteam member may choose not to participate in community activities inany meaningful way. Similarly, a procedurally compliant member is also

not especially useful as he will only do what is required to maintain theappearance of being a good member, while actually contributing verylittle. For example a procedurally compliant student may contribute therequired number of postings to an online forum to avoid being in default of course requirements whilst at the same time making little effort to providecontent that advances the collective knowledge of the community. Thebest environment for all concerned is one in which all members are totallycommitted to CSCL, for that is when a group will have the best chance of meeting the expectations of its members. Therefore it is worth the effortto find out what the factors are that will maximize communityperformance.

Bearing in mind the above qualifying comments it is arguablyimportant therefore to explore CSCL communities from two relatedperspectives; by seeking to understand how CSCL can be implementedand by exploring the means by which we might tell that collaborativelearning is taking place.

Research into computer supported collaborative learning (CSCL) isoften explored from the theoretical perspective of social constructivism.

This epistemological position favours the use of discourse by members of a team to construct individual meaning through the construction of sharedartifacts, for example through the development of online forum

discussions and wikis (Dougimas & Taylor, 2002). From the socialconstructivist perspective a group of learners will naturally gravitate

2



towards other like-minded colleagues to form a community of practice(COP) (Wenger, 1998; Wenger, McDermott, & Snyder, 2002). Forexample, a group of chemistry teachers may perceive the need to createnew and different learning opportunities for students that have difficultygrasping complex concepts. By working together the teachers are morelikely to mitigate the complexity of creating technology rich lessons fortheir students than if they worked alone (American Association of Collegesfor Teacher Education & Committee on Innovation and Technology, 2008).

Newcomers to a community, such as the chemistry teacherscommunity described in the example above, go through a learning andacceptance process in which they must master the sociocultural practicesof the COP they are joining (Lave & Wenger, 1991). Furthermore Lave etal. (1991) reported that it is only after this “apprenticeship” period thatnewcomers can begin to reap the benefits of belonging to a COP. Lave &Wenger describe the apprenticeship process newcomers go through asdevelopment of legitimate peripheral practice (Lave & Wenger, 1991).

Wenger et al. (2002) reported that it is important to adopt practicalmodels for COP development if communities of practice are to reach theirfull potential, it is not enough to leave development of a community tochance.

…communities typically undergo several changes in their focus,relationships and practice. They commonly shift from sharing ideasand tips to stewarding their practice – building refining, andexpanding the domain and its relationship to other domains. Theymove from a lose network of personal relationships to a group witha common sense of identity, combining intimate knowledge of eachothers’ approach with a sense of collective responsibility for the

domain.(Wenger et al., 2002, p. 111)

Several researchers have developed checklists that can be used to

guide the development of a COP. For example, Kopp & Mandl (2008)reported four common characteristics found in effective (CSCL-COP)groups: (1) goal orientation e.g. “I indentified myself with the group goal”,(2) task completion “the priority was task solving”, (3) cohesion e.g. “wecommunicate freely”, and (4) taking responsibility e.g. “I tried to completeassigned tasks”.

Additionally, Soller & Lesgold (2007) described four strategies foreffective group collaboration: (1) construction, (2) criticism, (3)accumulation, and (4) motivation. From the construction perspectivecommunity participants make their own new knowledge by observinggroup action (that imparts new knowledge). From the criticismperspective, individuals help each other overcome the discomfort of cognitive dissonance through exploring the reasons behind alternatebeliefs and attitudes. From the accumulation perspective, a broadknowledge base can be quickly developed and sustained when groupmembers contribute concurrently. And fourthly, from the motivationperspective students engage in social comparison in which they seek tomeasure their own actions against the actions of others (Soller & Lesgold,

2007).

3



The checklists just described are useful for developing communitiesof practice and do in deed complement each other. However, these 8learning environment characteristics are insufficient to guaranteesuccessful online collaboration. For example, Salmon (2005) suggests thatany successful online collaborative learning community will likely have askilled e-moderator to manage it, the implication for preservice teacherprofessional development being that it is sensible to ensure that groupsare able to acquire or develop a skilled moderator.

Aim of this research

The particular study outlined below in this paper is part of ongoingresearch into CSCL being conducted with preservice and inserviceteachers in Brunei Darussalam. Much of this research relates to the use of computer supported collaborative learning environments for theprofessional development of inservice and preservice secondary schoolteachers. There are many questions that the researcher is trying toanswer but the major question addressed in this paper is whether or notpreservice teachers totally commit to computer supported collaborativelearning or whether the context in which their learning takes place inhibitsparticipation beyond “procedural compliance”.

Significance of the research This research should add to an understanding of how preservice teacherscan be encouraged to adopt CSCL by identifying the factors that eitherafforded or constrained CSCL community development in this particularsetting.

Method

The paper reflects a pragmatic worldview that focuses on practicaloutcomes that might lead to change in practice (Creswell & Plano Clark,2007). At the same time it acknowledges the importance and utility of theconstructivist, advocacy/participatory and postpositivism worldviewsdescribed by Creswell and others (Creswell & Plano Clark, 2007;

Tashakkori & Teddlie, 2009).

To structure this research a modified parallel mixed model design

was chosen as the best fit (Bryman, 2008; Creswell & Plano Clark, 2007; Tashakkori & Teddlie, 2009). This model requires qualitative methodsand quantitative methods to be collated and discussed separately inrelation to findings before being drawn together in an overall meta-inference (Tashakkori & Teddlie, 2009). As shown in Figure 1, overallresults of the study were informed by the separate quantitative andqualitative strand results. The qualitative strand involved analysis of forumand weblog transcripts. The quantitative strand involved analysis of survey results and web server records as well as quantitative analysis of Ning and wiki sites.

4



Figure 1: Modified parallel mixed methods design

Participants

68 participants contributed to the study. All were enrolled in thesame educational technology course that ran for 14 weeks at the

University of Brunei (UBD) in 2009. 18 (26%) participants were male and50 (74%) were female. 61 (89.7%) students were in their fourth year of study and 7 (10.3%) were in their third year. Students came fromdifferent disciplines including 10 (14.7%) majoring in Biology, 10 (14.7%)in Chemistry, 13 (19.1%) in Economics, 11 (16.2%) in English Literature,10 (14.7%) in Geography, 8 (11.8%) in History, 5 (7.4%) in Mathematics,and 1 (1.5%) in Physics. All students were undergraduates enrolled ineither B.Sc. Education or B.A. Education programmes.

Participant CSCL activities

As a part of their coursework participants in the study were required

to work in teams to build and manage an online community of practicepresence. For example, a group of 10 preservice chemistry teachersformed a team to build a site that was used by “O’ and ‘A’ level chemistryteachers.

Participants formed 19 small groups responsible for repurposing anonline social networking site called a Ning. Additionally, groups decidedthey needed a way to organise online content that was easily navigableand accessible to online visitors, and for this purpose most groups choseto link their Ning to a wiki. The combination of a Ning and a wiki allowedgroups to manage the social collaborative aspects of their community onthe Ning site while the wiki site was used to manage content.

Data Collection

5



Qualitative and quantitative data was collected as a part of a parallelmixed methods design. Several data sources were used:

• Participant weblogs and online forum transcript files were usedas data for the study.

• A survey tool called the Web-based Learning EnvironmentInstrument (WEBLEI) was administered to gather quantitativedata about student perceptions of their CSCL environment.

• Web server logs were used to gather data about bothindividual and group online practice.

Qualitative data collection

Online forum contributions

Participants were asked to contribute to online forum discussions

about their CSCL experiences by addressing specific questions posed ontheir LMS site.

Weblog contributions

Students were required to maintain an online journal in which theywere to write down their thoughts about the affordances and constraintsof their CSCL environment.

Participant weblogs and online forum transcripts were collected froma web server and directly imported into NVIVO 8 for content analysis.

Quantitative data collection

WEBLEI

The WEBLEI is a 31 Likert item survey instrument that has foursubscales (see Table 1). It was used to collect data about studentperceptions of their CSCL learning environment. The instrument wasadministered online using a survey tool called LimeSurvey (seehttp://www.limesurvey.com). Data was automatically collected inelectronic form and exported into an SPSS 15 data file for statisticalanalysis.

Table 1

WEBLEI scales

Scale Scale meaning and example

Access scale Measures the extent to which students believe they can use

technology as and when they want to.

I can access the learning environment at times convenient tome.

Interaction scale Measures the extent to which students believe that the CSCL

environment facilitates collaborative learning

Other students respond promptly to my queries.

6



Response scale Measures the extent to which students experience a sense of

achievement and satisfaction in the CSCL environment.

I enjoy participating in discussions in this learning environment.

Results scale Measures the extent to which students believe the CSCLenvironment is structured to facilitate achievement of course

learning objectives.

The expectations of tasks and roles are clearly stated in the

online environment.

Web analytics data

Web analytics is the examination of web server data logs. Web logs provided a variety

of information about participant online behavior including the types of activities participants

engaged in as well as data about the frequency and timing of online interaction. In this study

it was possible to use very detailed LMS (Moodle) logs and less detailed, but never-the-less

useful Ning data. Ning data included information about the number of and the timing of

online discussion contributions as well as details about changes to site content (such as the

addition of videos, new blog postings and online polls).

Ethical considerations

Much data for the study was available in the public domain. That isto say anyone could enter the correct URL into their browser and go and

explore the data generated by study participants.

There is an argument that as long as participants have beeninformed that they make contributions to a public space when they goonline then there is less obligation on the part of researchers to “protectthe anonymity of individuals using the venue, or to seek their informedconsent” (Bryman, 2008, p. 654). Easily accessible electronic data isgenerated, whether planned for or not, whenever anyone goes online.

The ethical question a researcher faces is therefore not whether datashould be collected, but what to do with data that will inevitably end up inthe logs files of an Internet web server.

Data Analysis

Analysis of Web logs and online discussion transcripts

Web logs and online discussion contributions were analysed qualitativelywith NVivo 8, following the procedures outlined in Lincoln and Guba (1985)and Glaser and Strauss (1967). That is to say participant responses wereseparated into meaningful units where a unit was an idea, observation orthought. Responses were organised under theme headings. Themeswere grouped into subcategories in an iterative and cyclical processduring and after initial and repeated reading of transcripts. Participantcontributions were gradually reorganized into a logical thematic taxonomy

that could be collated and reported.

7



Analysis of the WEBLEI

Internal consistency and discriminant validity were calculated. Averagesfor Likert scale responses were calculated for WEBLEI subscales and theseformed the basis for further discussion and comparison with the results of other research strands.

Web analytics

Analysis of web server logs revealed participant usage patterns, includingdata about the number of postings individuals contributed to onlinediscussions, whether to a group Ning or to an LMS discussion thread.Analysis of Nings revealed the levels of participation of COP members andthe number and type of resources placed online. Analysis of logs alsorevealed the ratio of discussion thread postings to views.

Limitations

The study used a convenience sample and therefore results cannotreasonably be generalised to other populations. Furthermore, the studyis a snapshot of participant actions in week 10 of a 14-week semester.

Therefore the study cannot be used to report, for example, whether or notthe quality of collaboration in groups was improving over time. TheWEBLEI is a learning environment that was administered online for thefirst time. The implications of conducting the survey online indicate thatthe WEBLEI might be improved. Only one researcher carried out contentanalysis and therefore the findings were subjectively interpreted.

Findings and Discussion

In this section the findings for each research strand are reportedand inferences made. Concluding remarks bring together theindividual findings for each strand in a meta-inference section.

Forums and Blogs

Content analysis revealed that participants identified 53 factors thatimpact on the development of online communities of practice. Thesefactors were collapsed into three major categories: (1) reasons to connect,(2) enablers, and (3) limiting factors. A reason to connect is necessary if widespread adoption of CSCL is to take place and an online COP is to besustained. That is to say an online site needs to contain relevant content

that will help participants to solve the educational problems thatparticipants visit the site to find solutions to. Secondly, enablers wereidentified that could facilitate participation in an online community of practice. Thirdly, participants identified limiting factors that they feltprevented (or at least inhibited) active online learning in a community of practice.

Reasons to connect

Table 2 lists the main reasons why participants wanted to connect totheir online social network. Participants found that there were manyreasons for wanting to go online but that the most powerful motivation

was to exchange ideas with other group members. They exchanged ideasfor two reasons: (1) to learn new things, and (2) to confirm what they

8



already knew. Participants also reported that they valued active learningrather than being passive receivers of content and that they understoodthat the actual process of contributing to online discussions waseducationally valuable.

As the following quotation shows, participants often enjoyed

engaging in online discussions, even when they were exposed to ideasand viewpoints that challenged their own understanding.

Participant 5 “-online community makes exchangingideas a breeze (huge collection of tasty ideas… yummy)”

However, despite claims of enjoying CSCL content analysis revealedthat much of the time the online COP facility was used as a convenientmeeting place where operational strategies unrelated to CSCL were

discussed. For example, a discussion by an Economics group extended to52 replies as they grappled with the operational aspects of getting a grouptask completed on time:

Participant 1: lets just meet on saturday!!! =)

Participant 2: Okay, we have decided yesterday on what task we aregoing to give for each one of us... so I just updated it hereFor Compiling & Powerpoint: Khatib, Farhan & NohBarter Trade: Qieqie & IffahMoney: Zai, Rina & Qilah

Classroom Management & Assessment: Liyana & Salina”

This particular discussion thread quotation highlights anotherinteresting factor – that measuring the quality of discussions by countingthe length of posts or the number of posts in a discussion thread may notalways be an accurate way to measure the effectiveness of groupcollaboration.

Table 2: Reasons to connect

Motivation

1. Exchange of ideas

2. Active learning

3. Challenging ideas

4. Knowledge expression

5. Clarification and confirmation

6. Control over pace of learning

7. Critical thinking and related higher orderthinking skills

8. Interpersonal skills development

9



9. Reflection

10. Convenience of online communication

Participants not only enjoyed listening to what others had to say,they also liked to use the forum to confirm their own understanding.When other group members questioned the posting then it may haveindicated that the understanding behind the original posting neededfurther development. Further individual reflection might take place, andas understanding evolved then this was reflected in the transcripts…before a revised version of the original posting is published

Enablers

Participants described 6 essential community of practice enabling factors:(1) desirable membership characteristics, (2) high quality online content,

(3) members that participate regularly, (4) a code of practice (explicit orimplicit), (5) members can participate easily, and (6) members play anactive part in supporting the community (see Table 3). Open mindednesswas rated as the most important characteristic for members to have in anonline community. The remainder of this section summarises the viewsparticipants held about COP enablers.

Table 3: Enablers of successful CSCL networks

Enablers

1. Member characteristics

o Open minded

o Members are a part of a community

o Creative contributors

o Well mannered

2. Content quality

o Fresh attractive and interesting

content

o Use of different types of media

o High quality relevant content

3. Participation level

o Active participation is required

o Continuity of participation is

necessary

o Forced participation may be a good

thing

4. Code of practice

10



o Members are encouraged to contribute

o Members have equal rights

o Members stay on the topic

o Different viewpoints are respected

o Members may contribute anonymously

5. Connectivity

o Can connect anytime and from

anywhere

o Bandwidth is appropriate

o The technology is easy to use

6. Member responsibilities

o All have a responsibility to support

the community

o Participants promote new

membership and engagement

o Old hands help newcomers

Team members felt that they needed their online community to bemoderated by an administrator, and that one of the administrator’s main

jobs was to coordinate quality control activities. The moderator wastherefore “powerful” but also under constant “surveillance” by groupmembers to ensure they were doing their job properly. Participantsmentioned that the needed a commitment to continual improvement anda willingness to change content when understanding of the affordances of technology changed. For example, as newcomers to a COP became moreknowledgeable about the technologies they were learning to use theymade quite radical changes to the appearance and navigation structuresof their group wiki and Ning. This process of going through contentrevisions appears to be a natural event, and might be a useful predictor of how developed an online COP has become. In the start groups were keen

to upload as much content as possible and place it online without muchthought for accessibility of quality control. As the collective knowledge of their COP developed they learned how to control quality better and toensure that their site design was user friendly and focused towards theneeds of a particular audience.

All groups decided that they would like to upload material into a“knowledge base”. They felt that there needed to be a collection of onlineresources available for COP members and visitors to download. Eventually,all groups converged towards a common solution to their COP and CSCLneeds, a solution that put the social aspects of their community with theNing and a knowledge base on a separated wiki site.

Limiting factors

11



This section describes the perceived barriers COP developmentidentified by participants.

Although all participants felt that it was worthwhile to participateonline there were barriers that prevented full member participation. Fivegeneral themes related to barriers emerged from the data: (1) technical

difficulties, (2) learning preferences, (3) social network dysfunction, (4)participant competencies, and (5) member perceptions. Table 4 lists themajor concerns participants mentioned in their blogs and online forumdiscussion threads.

Table 4: Factors that prevented full membership in a COP

Limiting factors1. Technical difficulties

o Limited Internet access

o Insufficient member ICT skills

2. Social network dysfunction

o Members did not know social network etiquette

o The aims of the social network were too broad and

general

o Social network administration was ineffective

o Poor content quality control

3. Member perceptions

o Unequal participation levels

o Cultural conformance

o Lack of interest

o Mistrust

o Procedural conformance

o Irregular engagement

4. Participant competencies

o Undeveloped elaboration skills

o Lack of ICT skills and knowledge

5. Learning preferences

o Prefer face to face (f2f)

o Prefer to learn individually

Technical difficulties

12



Somewhat surprisingly, given the perceived widespread availabilityof Internet access in Brunei, 12% of participants reported technicaldifficulties that prevented adequate access to online facilities. Theyreported that they either did not have ready access to the Internet outsideUniversity or that quality of Internet bandwidth was unfit for purpose. Forexample, several students mentioned that it was good to be able to vieweducational videos and animations, but that it was an unreliable activitywhen bandwidth could not be guaranteed.

Social network dysfunction

Student familiarity with social networking tools such as Facebookmay have hindered attempts by group members to develop groupcohesion and to define community purpose. Some students talked aboutthings that had nothing to do with the task at hand, and seemed to beunaware of group focus, purpose, or the idea of a COP. In some instancesit was as if the COP was more or less an extension of Facebook withstudents making use of the extensive range of design options for the sakeof design rather than to solve an educational problem. Organizationalissues, especially poor moderator performance, also prevented realizationof COP potential. It is, for example, important that moderators ensure thattheir online facility is easily navigable and clearly focused. It may well bethat social networks are reasonably easy to set up and moderate.However, it is as well to remember that the nature of social interactionneeds to change when the technology is repurposed for use in a COP, andfor this to happen the moderator needs to show considerable knowledge,skill and leadership.

Member perceptions

Participants raised a number of concerns that perhaps relate directlyto the higher education learning environment rather than to theaffordances and constraints of online collaborative per se. Participantsresented the fact that some group members did not contribute,particularly when members had a stake in seeing that their COP wassuccessful. One participant was eventually ejected from his group (by hisgroup) because he didn’t contribute meaningfully online. Two onlinediscussions highlighted a lack of trust as being behind reticence tocontribute. Participants did not want to “risk” publishing their thoughts.

Interestingly, one group reported that cultural conformance was a

stumbling block that hindered COP development. Brunei participants wereself-described as non-confrontational, empathetic and polite to a fault. Itwas felt that participants were so well mannered that they would notcontribute a view that was contrary to a view already published. Acharacteristic such as this is clearly going to limit development of anonline COP.

Participant competencies

Several knowledgeable participants reported being somewhat fed-upwith having their time wasted by a minority of people in large groupforums that didn’t make meaningful contributions to discussions.

Newcomers to discussions often repeated what had already been said,contributed information that was off the topic or otherwise didn’tcontribute to the collective knowledge of participants. One participant

13



correctly suggested that this problem was because inexperiencedcontributors didn’t know better and weren’t being shown how to change.

Some participants reported that they were not confident about theirelaboration skills. Although all participants had proven English languageproficiency none could claim English as their home language. As one

would expect, English literature majors were considerably more fluentdefending a position online than participants training to be science,mathematics or geography teachers. Clearly this situation will alwayshave a negative impact on a COP, as it will reduce the variety views andthe quality of discussions upon which CSCL is based and which generatesthe collective knowledgebase at the heart of a community. Thereforeovercoming this barrier should be a high priority within a COP communityand every effort should be made to identify and support COP membersthat may feel this way.

Learning preferences

An unexpected finding was that some participants simply didn’t likecomputers and didn’t want to use them, and given the opportunity theywould prefer to work face to face to complete project work. This reticenceto contribute impacts on the individual concerned and also the othermembers of his team and so it will be worthwhile to find ways to changethe perception of preservice teachers that resist the use of technology.

Another learning preference perception revealed in forum discussionswas that some participants preferred to work individually and not in agroup. Such participants were resistant to CSCL (or collaboration of anykind) and therefore may be more difficult to convince that CSCL isworthwhile. There are two concerns here: (1) these participants will beteaching in schools in an environment in which they are expected tosupport collaborative learning pedagogy (something they do not believein), and (2) these participants may prefer not to contribute meaningfully toprofessional development teams. These are important issues to address,the consequences of impact negatively on the future learning of studentsand staff in schools (professional practice and professional development)as well as current preservice teachers.

Quantitative findings and discussion

WEBLEI

Table 5 summarises the findings of statistical analysis of WEBELIdata.

Table 5WEBLEI subscale descriptive statistics and reliability scores

Subscale

Number of Items

AlphaReliabilit

yScalemean

ScaleSD N

Meancorrelationwith other

scalesAccess 7 0.70 3.77 0.28 63 0.35

Interaction 8 0.66 3.08 0.32 64 0.22

14



Response 8 0.64 3.71 0.24 65 0.34

Results 8 0.86 3.82 0.19 65 0.34

The WEBLEI was administered online with the participation of 65(95.6%) respondents. Internal consistency (alpha reliability) was obtainedfrom the sample as indices of scale reliability. The alpha coefficient scoresranged from 0.64 to 0.86 in this study as compared to 0.68 to 0.87 in asimilar type of study carried out with the same instrument in Singapore(Chang & Fisher, 2003b). On the whole the values for this study wereacceptable and comparable to the previous study.

The mean correlation of each individual scale with the results of theother three scales was used for discriminant validity. WEBLEI discriminantvalues ranged from 0.22 to 0.35 in this study, as compared to 0.37 to 0.49in Chang and Fraser (2003a). On the whole the values for this study were

comparable to the previous study.

The first sub-scale, access, measured participant perceptions of howeasily they could physically connect to their social network site and theextent to which they could make their own decisions about the extent towhich they participated in online activities. The average participant scalevalue for access was 3.77, which is near to “often” on the WEBLEI Likertscale. This finding confirmed that participants were mostly able toconnect as and when they wanted to. This was an important finding asthe access scale result may well have a bearing on the remaining threescales in the sense that if access responses were poor then it is likely thatinteraction, response and results scale responses would be poor also.

The second subscale, interaction, measured participant perceptionsof the quality of online discussions. The average participant scale valuewas 3.08, which is close to “sometimes” and noticeably lower than theaverage for the access subscale. This was an interesting result because itmay mean that participants were able to get online easily, but once there,didn’t know how to collaborate effectively. This result further highlightsthe need for preservice teacher educators to be proactive and systematicabout developing and integrating online collaborative learningenvironments. Although participants may have had considerable socialnetworking experience it appears that this alone was insufficient for themto be able to successfully repurpose a social networking facility (Ning) foronline collaboration. Wenger et al. (2002) describe the challenges of facilitating threaded discussions:

…when a Web-based conversation goes through naturalcycles of low activity, it may look like the community is dead.When members think the community is waning, it starts avicious cycle. Who wants to contribute if no one is listening?The community coordinator can stimulate these discussionsby occasionally stirring the pot with an intriguing question or a provocative statement. Such interventions can return thecommunity to the awareness of members.

(Wenger et al., 2002, p. 131)

15



The third subscale, response, measured participant perceptions of their overall level of satisfaction with their online CSCL experience. Theaverage participant scale value was 3.71, which is close to “often” on theWEBLEI Likert scale. This indicated that students mostly enjoyed workingin the online environment. However, analysis of individual items in thissubscale revealed that most participants reported enjoying participation inonline discussions only “sometimes”. Furthermore, they also reportedthat they only enjoyed working in groups “sometimes as well. There aretwo item questions that arise out of these two particular item responses:(1) if students were taught how to collaborate online effectively would thisincrease their enjoyment when they work online, and (2) given that allparticipants consider a language other than English as their homelanguage would the enjoyment result have been different if students hadbeen able to write in the mixture of Malay and English they prefer to usewhen conversing online at social networking sites like Facebook.

The last sub-scale, results, measured participant perceptions of

whether or not the CSCL environment was structured in a way that wouldhelp them to achieve personal learning goals. The average participantscale value was 3.82, which is reasonably close to “often” on the WEBLEILikert scale, an unexpected finding given the lower scores for items thatrelated to satisfaction with the quality of online discourse in forums.Furthermore, the result seems to contradict the web analysis findings thatreport relatively low group forum participation, an indication of onlylimited COP cohesion.

Web analytics

Analysis of forum activities revealed that the 68 participants viewed

discussion postings a total of 8,812 (92.69%) times and contributed toonline forums 695 (7.31%) times. On average, each participant made10.2 individual postings to discussion threads and viewed 130.0 onlineforum discussion threads. The most active online participant contributed61 postings and viewed 445 contributions during the study. In this casethe participants’ percentage ratio of postings to views was 13.12%.However, at the other extreme there were 4 participants that made nocontribution to either a Ning or LMS forum (although the average numberof views for these four participants was 47.5).

The results from paired sample t-test analysis indicate that there is asignificant difference between the number of contributions made to team

discussions and the number of contributions made to whole groupdiscussions t (67)=4.17. p<0.05 with the mean total number of postingsfor small teams and all 68 participants together being 4.06 (SD=4.22) and6.16 (SD=5.73) respectively. This results indicates that participantscontributed significantly less often to their groups than they did to thecollective forum. These finding somewhat supported the notion that manyparticipants did not fully commit to CSCL and that there was even lesscommitment in the small group communities that did not have the samelevel of community moderation as present in the LMS general forum. Thisdoes not mean that participants think CSCL is a bad idea; in deed contentanalysis findings and WEBLEI findings support the view that participantshave a high regard for CSCL. What was apparent however was that

participants did not need the “computer supported” element of CSCLbecause they had ample opportunity for face-to-face team collaboration.

16



Therefore the COP side of participant websites remained largelyundeveloped while at the same time the online content (wiki) aspect of group sites developed at a rapid rate.

Conclusion

Three different approaches were used to address the same researchquestion, a survey, content analysis and web analytics. Each has itsown affordances and constraints, which have been briefly outlinedabove. In particular, content analysis of weblogs and forumtranscripts revealed the best indication of student commitment toCSCL. However, content analysis is notoriously difficult to performand the reliability of findings is often open to debate with generalagreement that the technique is “difficult, frustrating and timeconsuming” (Garrison & Anderson, 2005, p. 132). All the same, thefindings of content analysis provided a rich array of information that

can help to inform future development of online teacher teams(communities of practice). Perhaps the key observations identifiedin the qualitative strand of the study were that online communitiesof practice amongst UBD students need to be well structured,moderated, focused and that COPs take time and moderator skill todevelop. This was to be expected and is supported by COP literature(Lave & Wenger, 1991; Wenger, 1998; Wenger et al., 2002). Takingthis into account it may be that there is insufficient time in a one-semester course to develop the skills and knowledge students musthave if they are to be expected to commit to CSCL. This studyseems to support this view. However, this problem can be

somewhat mitigated by ensuring that there is faculty wide supportfor CSCL rather than limiting development of COPs to educationaltechnology courses. If CSCL is considered important andworthwhile, because it is inevitably going to be a standard feature of ongoing professional development of teachers and because there isincreasing focus on the need for teachers to work in teams, thenthere is a better chance that students will get a consistent messageabout the affordances of technologies within all the courses theytake.

The WEBLEI analysis provided a snapshot of participant

perceptions of their CSCL environment. The WEBLEI findings werelargely in agreement with content analysis findings. However,quantitative analysis of web server logs revealed that there was adifference between what participants believed and the actions theytook. On the one hand participants indicated that they weresatisfied with the results of online participation in an onlinecommunity and yet, for many participants, there is little evidence inserver logs to indicate that they were participating fully in theironline communities of practice. These contradictory findings maybe explained by the nature of the environment in which participantshad to work. They clearly valued participation and wanted tocollaborate online, but had insufficient time, skills or knowledge to

17



develop the social/communication aspects of an online communityof practice.

If CSCL is to work best then it requires participation from allmembers of a team of teachers. Therefore it remains a concern

that levels of commitment were not uniform and that someparticipants clearly did not commit to CSCL. Thus, it is important totry to understand those factors that prevented some participantsfrom forming groups that collaborated successfully while othersfoundered.

In summary, based on the amount of activity in sites, it ispossible to say that approximately 40% of participants were able togenerate some data to support the notion that they were fullycommitted to CSCL. Of course others were also committed but wereunable to provide evidence of this because of the vagaries of the

environment in which they were operating. Still, there were othersthat appear to have not been committed to CSCL, and the barrierspreventing full participation in CSCL need to be identified andmitigated.

Recommendations

This research highlighted the need to learn more about the commoncharacteristics of successful CSCL teams, and the factors that preventedsome teams from being successful in a COP–CSCL environment. The study

showed CSCL environments to be complex with many interrelated factorscontributing to COP effectiveness, making them somewhat problematic toimplement. Therefore, having identified the need to develop a wellstructured CSCL community development programme it remains to defineexactly what this will contain. The study results revealed that therewasn’t sufficient time in a single semester course to develop the skills,knowledge and propensities that all participants will need in their futurecareer and that a possible remedy to this problem would be to generatefaculty wide support for CSCL at all levels of preservice training.

References

American Association of Colleges for Teacher Education, & Committee onInnovation and Technology. (2008). Handbook of technological

pedagogical content knowledge for educators. New York: Routledge.Angeli, C. (2008). Distributed cognition: A framework for understanding

the role of computers in classroom teaching and learning. Journal of Research on Technology in Education, 40(3), 271–279.

Bryman, A. (2008). Social research methods. Oxford: Oxford UniversityPress.

Carugati, F., Selleri, P., Matteucci, M. C., Tomasetto, C., Mazzoni, E., &Gaffuri, P. (2008, September 1-2). Monitoring and analyzingcollaboration in e-learning environment: Two case studies applied to vocational training. Paper presented at the Conference onKnowledge Construction in E-learning Context: CSCL, ODL, ICT andSNA in Education, Cesena, Italy

18



Chan, J. C. C., Hew, K. F., & Cheung, W. S. (2009). Asynchronous onlinediscussion thread development: Examining growth patterns andpeer-facilitation techniques. Journal of Computer Assisted Learning,25(5), 438-452.

Chang, V., & Fisher, D. L. (2003a). The validation and application of a newlearning environment instrument for online learning in highereducation. In M. S. Khine & D. L. Fisher (Eds.), Technology-richlearning environments: A future perspective (pp. 1-20). Singapore:World Scientific Publishing Co. Pte. Ltd.

Chang, V., & Fisher, D. L. (Eds.). (2003b). The validation and application of a new learning environment instrument for online learning in higher education Singapore: World Scientific.

Creswell, J. W., & Plano Clark, V. L. (2007). Designing and conductingmixed methods research. Thousand Oaks, California: SagePublications, Inc.

Dougimas, M., & Taylor, P. (2002). Interpretive analysis of an internet-based course constructed using a new courseware tool called

Moodle. Paper presented at the 25th HERDSA Annual Conference.Garrison, D. R., & Anderson, T. (2005). E-Learning in the 21st Century: A

Framework for Research and Practice. New York, NY:RoutledgeFalmer.

Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory:Strategies for qualitative research. Chigago: Aldine PublishingCompany.

Hughes, J. (2005). The role of teacher knowledge and learning experiencesin forming technology-integrated pedagogy. Journal of Technology and Teacher Education, 13(2), 277-303.

Kopp, B., & Mandl, H. (2008, September 1-2). Collaborative learning: Aheterogeneous phenomenon. Paper presented at the Conference on

Knowledge Construction in E-learning Context: CSCL, ODL, ICT andSNA in Education, Cesena, Italy

Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. New York: Cambridge University Press.

Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry . Newbury Park, CA:Sage.

Loucks-Horsley, S., Love, N., Stiles, K. E., Mundry, S., & Hewson, P. W.(2003). Designing professional development for teachers of scienceand mathematics. London: Corwin Press Inc.

Loucks-Horsley, S., & Matsumoto, C. (1999). Research in professionaldevelopment for teachers of mathematics and science: Research

into practice. School Science and Mathematics, 99(5), 258-271.Mishra, P., & Koehler, M. J. (2006). Technological pedagogical contentknowledge: A framework for teacher knowledge. Teaches CollegeRecord, 108(6), 1017-1054.

Moroney, M., Koay, T., & Boorer, D. (2009). Actual and preferredperceptions of students engaged in collaborative online learning.

Journal of Applied Research in Education (In Press).Salmon, G. (2005). E-Moderating: The key to teaching & learning (2nd

ed.). London: RoutledgeFalmer.Soller, A., & Lesgold, A. (2007). Modeling the process of collaborative

learning. In H. Ulrich Hoppe, H. Ogata & A. Soller (Eds.), Computer-supported collaborative learning: Studies in technology enhanced

collaborative learning New York: Springer Science + BusinessMedia.

19



Songer, N. (2007). Digital resources versus cognitive tools: A discussion of learning science with technology. In S. K. Abell & N. G. Lederman(Eds.), Handbook of research on science education (pp. 471-491).New York: Routledge.

Stahl, G. (2004). Building collaborative knowing: Elements of a socialtheory of CSCL. In J. Strijbos, P. Kirschner & R. Martens (Eds.), What we know about CSCL and implementation in higher education (pp.53-85). Boston: Kluwer Academic Publishers.

Tashakkori, A., & Teddlie, C. (2009). Foundations of mixed methodsresearch: Integrating quantitative and qualitative approaches in thesocial and behavioral sciences. Thousand Oaks: Sage PublicationsInc.

Timperley, H., Wilson, A., Barrer, H., & Fung, I. (2007). Teacher professional learning and development: Best evidence synthesisiteration (BES). Auckland: Auckland University.

Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. . New York: Cambridge University Press.

Wenger, E., McDermott, R., & Snyder, W. (2002). Cultivating communitiesof practice: A guide to managing knowledge. . Boston: HarvardBusiness School.

About the author

Mike Moroney is an ICT/Educational Technology lecturer at the Universityof Brunei. His research interest is in repurposing Internet moderatedcommunities of practice for inservice teacher professional development.He is a PhD candidate at Auckland University.

Presented at ….

The 13th UNESCO-APEID

International Conference on Education

and World Bank-KERIS High Level Seminar

on ICT in Education

15-17 November 2009

Hangzhou, People’s Republic of China

20

Documents

Unesco Paper Final