Building knowledge networks through project-based online learning: A study of developing critical thinking skills via reusable learning objects

Computers in

Computers in Human Behavior 23 (2007) 2668–2695

Human Behavior

www.elsevier.com/locate/comphumbeh

Building knowledge networks throughproject-based online learning: A study

of developing critical thinking skills viareusable learning objects

Gulsun Kurubacak *

Anadolu University, College of Open Education, Department of Distance Education, Eskisehir 26470, Turkey

Available online 2 October 2006

Abstract

The main purpose of this paper is to explore and discuss the main characteristics of project-basedonline learning (PBOL) to promote learners’ critical learning skills through reusable learning objects(RLOs) from global online resources. Based on the main purpose of this study and the concerns, thekey research inquiries are: (1) How does a PBOL environment improve the learners’ critical thinkingskills via RLOs? (2) What are the patterns of the learners’ critical thinking skills in a PBOLenvironment when they focus on building and delivering RLOs? and (3) What are the learningimpacts and communicational outcomes of a PBOL environment on the learners’ critical thinkingskills when they work with RLOs? This is a qualitative case study that utilizes both qualitativeand quantitative data to provide detailed information to the researcher for the data collection.The research site was the Designing and Delivering an E-Learning Environment graduate course atthe Department of Distance Education of Social Sciences Institute in the large urban andgovernment-based University in the Spring Semester 2004. There were totally 11 learners (three ofthem were men and the rest of them were women) in this course.� 2006 Elsevier Ltd. All rights reserved.

Keywords: Reusable learning objects; Project-based online learning; Critical thinking; Knowledge networks

0747-5632/$ - see front matter � 2006 Elsevier Ltd. All rights reserved.

doi:10.1016/j.chb.2006.08.003

* Tel.: +90 222 335 0580/2573; fax: +90 222 335 0580.E-mail address: [email protected]

mailto:[email protected]

G. Kurubacak / Computers in Human Behavior 23 (2007) 2668–2695 2669

1. Introduction

The world is rapidly becoming more technologically complex. As a result, learners needto be taught and communicated differently than they have been in the past. Universitiesmust prepare learners to become active members of this changing society to adapt tothese transformations as they occur. Also, they must build rich online programs thatfacilitate and promote multi-way knowledge transfers by having various interactivecommunication methods, techniques and activities. To save labor, time, energy andmoney in programs, online workers (communication designers, online educators,technology staff, online learners, stakeholders, etc.) should share their knowledge andexperiences with each other to easily modify and powerfully reuse resources, which arecalled as reusable learning objects (RLOs). They consist of applets, graphics, templates,backgrounds, texts, animations, streaming videos, and other such data and elements(Edmonds & Barron, 2002; Rumetshofer & Woß, 2003). RLOs are extremely importantfor online workers to produce diverse educational contexts and built knowledgenetworks via new communication technologies.

The problem is how online learners design and ensure maximum RLOs. The producingprocess, therefore, should merge the new learning theories and cutting-edgecommunication technologies to decrease the risk of making large investments in onlinelearning (Duval, 2001; IEEE, 2002; Wiley, 2000). Besides, the best uses of RLOs will notoccur until they adopt innovative models to locate educational contents from multipleexperts, build multicultural digital content libraries, and also exchange learners ownthinking projects as well as evaluate, track and report learner cognitive progress andperformance online. Providing the guidelines for online learners in the development,delivery, and evaluation of RLOs can be able to promote interoperability standards thatthey can use across multiple disciplines.

Project-based online learning (PBOL) is capable of supporting learners’ learningprocess and enhancing their communicational experiences via RLOs. Besides, PBOLcan provide online learners with powerful digital reusable learning resources, and alsoengage them complex reusable activities by thinking and solving problems critically.Pedagogically, PBOL can encourage, guide and reinforce independence learning byinvolving learners in interactive communications via RLOs. As indicated by Rudinowand Barry (1994), when diverse learners understand and evaluate RLOs profoundly,they can promote their higher-order thinking skills and intellectual autonomiespowerfully. In the PBOL environments, therefore, these learners can discover newlogical strategies and specific procedures for producing RLOs. Besides, they can equipthemselves with the critical thinking skills that they need to deal with unexpectedcommunicational pitfalls in practical. To sum up, PBOL is one of the best methods todevelop powerful RLOs by providing their durability, interoperability, accessibility andreusability.

2. Purpose

The main purpose of this paper is to explore and discuss the main characteristics ofPBOL to promote learners’ critical learning skills through RLOs. As mentioned byFaro, Giordano, and Mineo (2005), therefore, RLOs in the context of PBOL canprovide learners with sharing knowledge across different knowledge societies. Also, the

2670 G. Kurubacak / Computers in Human Behavior 23 (2007) 2668–2695

researcher is concerned about providing powerful insights into how RLOs are to breakeducational contents down into small chunks. This can promote various effectivecommunicational opportunities and learning strategies for PBOL. Furthermore, a clearapproach makes it possible not only to process and transmit combinations of video,images, HTML pages, texts, sounds and graphics, but also to interact with these dataeffectively.

In this study, the RLOs-based process focuses on measurable educational objectivesand a variety of communicational activities. On the other hand, PBOL is a pragmaticlearning method to engage learners in projects designed to be realistic, intriguing andrelevant to real-life experiences, and model how theory can be transferred into practicein higher education. To integrate authentic contexts by clearly stating the pedagogicalguidelines to RLOs can able to encourage learners to take ownership and responsibilityfor their learning and communication processes. Improving learners’ independentlearning strategies by utilizing PBOL-based thinking help these learners become thefuture leaders we need in the field of information, communication, and learningtechnologies. RLOs, therefore, should have highly complex structures includedinteractive communication opportunities.

Needles to say, in a virtual society, PBOL can help online learners become engagedcitizens, informed individuals and dynamic members to be a part of practical relevantexperiences with RLOs. However, there are limited researches about building RLOsand designing these objects for different cognitive abilities to improve learners’ criticalthinking skills via PBOL. This paper makes a unique contribution to provide onlinedesigners with valuable insights about how to produce heterogeneous educationalcontents and objects, and use them for different cognitive reasons and purposes. Tosum up, this research is bringing a new ground by addressing key questions aboutdeveloping learners’ critical thinking skills through PBOL and RLOs. Based on themain purpose of this study and the concerns discussed above, the key research inquiriesare:

1. How does a PBOL environment improve the learners’ critical thinking skills via RLOs?2. What are the patterns of the learners’ critical thinking skills in a PBOL environment

when they focus on building and delivering RLOs?3. What are the learning impacts and communicational outcomes of a PBOL environment

on the learners’ critical thinking skills when they work with RLOs?

Online learners should be able to critically consider their communicational activities inrelation to RLOs. They should discuss the pros and cons of improving their RLOs-basedprojects to build knowledge networks. Besides, as noted by Sicilia and Garcıa-Barriocanal(2003), these RLOs-based projects should be required to satisfy concerns about theirgranularity and their independence of concrete contexts of use. These projects must bedescribed as the possibility and adequacy for the object to be usable in potential onlinelearning milieus.

3. Theoretical framework

The theoretical framework of this graduate course in the Department of DistanceEducation allowed the course owner and learners to produce an online milieu through


the strategies and principles of PBOL, and also to obtain the background ideas of buildingand utilizing RLOs based on the problems, perspectives and needs of their community toassist lifelong learning. The course owner and learners in this course engaged theopen-ending meanings of their learning designs from pedagogical and communicationalperspectives. There were not any prescribed approaches and/or solutions for complexproblems related to the real-life experiences.

The tasks and duties they performed helped them to build knowledge networks byobtaining the educational knowledge for RLOs. This course, therefore, provided thelearners with the active learning and communication processes so that they produced,gained and evaluated knowledge by producing and using RLOs. Also, theseparticipants tried to meet the demands of evidence-based inquiry scientifically, as adynamic activity and social goal-oriented procedure. The all course participants, thecourse owner and learners, worked together in a PBOL milieu. They embraced theprinciples and standards of a constructivist PBOL approach to build and deliver RLOseffectively.

To read, listen, understand, and remember knowledge successfully via PBOL wereextremely crucial for the learners to promote their critical thinking developments withRLOs. Moreover, these participants involved the different critical thinking activities in aconstructivist PBOL perspective. This approach helped them become active members intheir community. RLOs related to both visual metaphors and narratives in thisconstructivist class encouraged the participants to produce their own knowledge aboutdiverse problems of their society. Also, in this milieu, they worked on the variouscomplex problems to find powerful and innovative solutions and perspectives to buildtheir own understandings about RLOs. These helped them produce the standards andauthentic accountability to build a knowledge network. Therefore, not only did thesedigital complex activities in a PBOL milieu highlight the design guidelines andtechniques, but also RLOs described diverse educational assumptions and criticalanalysis of the communication perspective. This PBOL milieu was challenged enough todemand teamwork. Furthermore, the participants needed to logic that they weredependent upon one another in order to be successful. As highlighted by Barrit (2002),Jonassen (1999), and Robbins and Finley (1995), they were more likely to makecomprehensive arguments and decisions to freely contribute their ideas and feelings toteam members, an open and honest evaluation of team performance, and a readiness tocelebrate accomplishments.

As stated by Sicilia and Garcıa-Barriocanal (2003) and Williams (2000), theRLOs-based projects in this course referred durability, interoperability, accessibility andreusability. The first three characteristics were essentially of a technical nature whereasthe fourth one, reusability, was related mainly to educational design, and neither digitalformats nor content structure that were the main concern of interoperability andaccessibility.

Finally, in this course, the learners tried to build a knowledge network. As defined byPolsani (2003), they enabled and encouraged online knowledge transfer depending on astructured data storage system and indexing ensures that the content could be searchedin meaningful ways. This milieu provided the learners with reusing and easily modifyingresources, and helped them save time in content developments. Through mixing andmatching RLOs, a PBOL environment eliminated the redundancy of recreating


courseware that the learners involved in just the right amount of learning and at the rightlevel of communications.

4. Method

This is a qualitative case study. It aims to investigate the experiences on building andlearning with the RLOs in a PBOL constructivist milieu to improve the learners’ criticalthinking skills. Furthermore, this study provides the participants with engaging inPBOL-based thinking, and finding multi-way solutions via RLOs to the complexproblems of people and natural sources. For these reasons, this research utilized bothqualitative and quantitative data to provide the information in detail to the researcherfor the data collection. Moreover, the combination of these methods helped the authorgenerate new perspectives and stimulate new directions in the data analysis. Themethodology combinations provided data triangulation from a variety of data sources,and also methodological triangulation from multiple methods (Patton, 1990). Therefore,the researcher overcame the natural prejudices that derived from a single research method.

4.1. Research site

The research site was the Designing and Delivering an E-Learning Environment graduatecourse at the Department of Distance Education of the large urban and government-basedUniversity in the Spring Semester 2004. This course was an upper-level graduate course inthe Distance Education Department of Social Sciences Institute, had both face-to-face andonline classes. Also, the Designing and Delivering an E-Learning Environment courseutilized a constructivist PBOL approach and its blended learning strategies thatimproved the learner critical thinking skills to build and learning with RLOs. Besides,the researcher was the course owner, who performed as an online educator and acognitive coach when the learners needed to encourage their independence criticalthinking activities.

The learners in this course attended the regular face-to-face classes during the fourteenweeks in the Spring Semester 2004. Each traditional class lasted a three-hour session.Besides, they collaborated with each other synchronously and asynchronously viaWebCT. They posted all their works and papers online before and/or after the regularclass meetings. In the regular class sessions, the learners achieved their own learninggoals and objectives based on the philosophical approach of the course, discussed thecurrent social and political pressures on building RLOs, and discovered the aims oftheir course projects to meet the demands of scientific evidence-based inquiries.

The online class had nine important stages: The first one was the Syllabus, whichincluded the course goals and objectives, and also the course agreement. The course goals

and objectives identified educational concerns, issues and problems and specified goalsfor designing an educational learning setting. The course agreement was fundamental inthis course. It can be represented by a critical contract to the roles and responsibilitiesof learners and course owners, the learning methods and activities, and also theevaluation process. Besides, the special characteristics of the culturally diverse, disabledand adult learners were recognized and considered during the course agreement planning.

The second stage was the Bookmarks that highlighted the weekly course contents andcomplex reusable activities. The weekly course contents were a dynamic framework,


which defined academic records reveal the extent and quality of PBOL. By being aware ofindividual learning and communication styles, the weekly course contents were designedbased on the ideas about learning and communication situations of the learners,cognitive learning and communication styles of the learners, alternative learning andcommunication methods of the learners, and a variety of learning and communicationresources. The complex reusable activities included a number of learning andcommunication activities substituted for PBOL in the literature and in practice. Theseactivities followed a systematic procedure with special details that enhanced learningand communication at a satisfactory level in diverse planning results by PBOL.

The Self-Packages, as the third stage, identified subject content, and analyzed taskcomponents related to stated goal and purposes. This section covered the introduction,readings and links, the cases related to reusable learning objects, discussion questions, andalso self-test and comments. The subsections of the Self-Packages included specificstatements of major goals and objectives to be accomplished, suggested online activities,list of readings and other online resources for possible use, interactive cases for RLOs,and sample evaluation and self-assessment questions.

The fourth stage was the Projects. The learners’ projects built and uploaded theirreusable learning objects related to the course contents. The course participantsdesigned the specific RLOs-based projects in the groups. They tried to produceaccessible, reusable and interoperable RLOs during the semester. As noted by Polsani(2003), these objects, therefore, must have been tagged with metadata so that theycould be stored and referenced in a database; functioned in different educationalcontexts, and also independent of both the delivery media and knowledge managementsystems. In this stage, the RLOs were reusable learning objects represented analternative approach to content development. The learners, therefore, tried to developthe educational content (text, HTML Web pages, images, sounds and videos), glossaryof terms (terms and definitions), learner logs (individually written online documents),learner homepages (the learners’ free-speech pages), quizzes and assessments (questionsand answers) as their RLOs-based projects.

The fifth stage, the Communication Tools, contained email to communicate with theeducator and/or other learners privately, five chat rooms for the structured synchronouscommunications and/or group works, and a bulletin board for semi-structured andstructured e-activities. The Communication Tools were the generation of meaningthrough exchanges using a range of Internet-based tools, transmissions, and processesin this study.

The sixth one was the Announcements, which were sequences of related events inchronological order during the course. Besides, the Announcements defined PBOL-basedtasks on time. The Learner Logs was the seventh stage. The course learners individuallykept their online logs, which were written documents preserving knowledge of facts orevents about their weekly experiences toward PBOL, critical thinking and reusablelearning objects.

The Glossary as the eighth stage was featured an alphabetical listing of RLOs-relatedterms, covering the critical thinking and PBOL by the online learners. The ninth andfinal stage was Learner Homepages. The purpose of these Web pages was to reflect thelearners’ views and insights which sought to explain how they lived within onlinesocieties and cultures.


When sharing knowledge in the face-to-face and online classes, the participants in thisstudy faced with their culturally, ethnically, and racially mixed heritages to deal with reuseorientation and technological issues. Besides, they learnt about how to break the coursecontent down into meaningful pieces, and how to be reused these pieces in variouslearning milieus. They recognized diverse stereotypes, biases and misconceptions onrace, ethnicity, gender, social class, sexual orientation, religion, and disability in thisprocess. On the other hand, there were some challenges for the learners in thisconstructivist PBOL-based course. The learners had some difficulties to cope with thebarriers of how to build and learn with the RLOs. For examples, it was not easy toconfront the conflicts positively by understanding others’ perspectives that they hardlyappreciated their needs and expectations and realized their strengths and weaknesseswhen producing the RLO. These interactive communicational activities promotedonline collaborations between the learners and the course owners. However, they couldalmost not establish their roles, responsibilities, goals, and deadlines by themselves dueto the lack of utilizing critical communicative efforts and arguing several options toproduce their RLOs. Although the course owner provided a various feedback andguidelines during their RLO production process, it was very complex for these learnersto make critical decisions according to their current needs and priorities. Therefore,these learners barely understood how to build a knowledge society via the RLOs; toprovide with various equal access opportunities via the RLOs; and to share power andculture in the PBOL milieus. As discussed by Kemp, Morrison, and Ross (1994) andVockell and Deusen (1989), the learners in this course included members of ethniccultures with backgrounds and behaviors that differed markedly from those of themajority of learners whereas the course owner differed in ethnic background frommembers of the learner group.

4.2. Participants

The findings of this study represent the learners in the 2003–2004 school year at theDepartment of Distance Education of Social Sciences Institute. There were totally 11learners (three of them were men and the rest of them were women) in this course. Alllearners were asked individually to read and sign the informed consents form, whichdescribed the research in detail. They participated voluntarily in this research. When the

Table 1The backgrounds of the participants

The nick namesof the participants

Educational background Age Gender Took any PBOL-basedonline course before

Raymond Educational Sciences 37 Male NoNoah Educational Sciences 26 Male NoJoe Educational Sciences 24 Male NoClaire Educational Sciences 22 Female NoMonika Educational Sciences 23 Female NoJennifer Educational Sciences 21 Female NoPhoebe Educational Sciences 22 Female NoRose Mary Educational Sciences 22 Female NoSue Science 26 Female NoDaphne Social Sciences 27 Female NoGrace Social Sciences 25 Female No


study started, these learners were between 21 and 37 years old, and also they had differenteducational backgrounds (Table 1).

4.3. Data sources

The course delivery required active collaborations and interactions among the graduatelearners that they could able to complete course requirements perfectly. Therefore, thisgraduate course was designed based on the blending constructivist learning theory. Asmentioned by Bonk, Kim, and Zeng (2006), and Magoulas, Papanikolaou, andGrigoriadou (2003) therefore, the participants could facilitate online communicationalactivities to make right decisions toward their RLOs-based projects. This approachhelped the researcher collect the data from both the traditional class sessions and onlinemilieus.

The data from the traditional class sessions came from seven different sources: The firstones were a pre-survey and a post-test, which were about the course contents and thephilosophy of RLOs. These surveys were handed out at the beginning of the SpringSemester 2004 and at the end of the same semester. The classroom observations were thesecond data sources from the traditional class sessions. The researcher collected thedata from the face-to-face classes during the fourteen weeks of the Semester Spring2004. The third ones were three 30 min individual interviews and two focus groupinterviews, which were hold at the fourth, 7th and 11th weeks of the Spring Semester2004. The learners had two mid-term projects as the fourth data source. These projectswere concerned with understanding, designing, evaluating, and implementing RLOs foreasy to learn and communicate online. These learners uploaded their digitalRLOs-based projects at the thirteenth of the Spring Semester 2004. The learnerspresented their projects in the traditional classroom at the fourteenth week that thesepresentations were their final exam, which were the fifth data source in this study.Finally, these learners wrote articles about their works and sent them to one of theeducational technology journals; and also submitted their papers to one of theeducational technology conferences. Therefore, the publishing and submitting of theirprojects were the last data sources of this paper.

The data from the online class sessions were collected from six different sources. As thefirst data sources, the pre-survey and post survey were handed out at the second andthirteenth weeks of the Spring Semester 2004. These surveys were about building andlearning with in a PBOL environment, and constructing knowledge networks via RLOs.The online chats were the second online data source. The formal and/or informal chatswere done at least once a week during the Spring Semester 2004. The third online sourceswere the bulletin board, which provided the learners with semi-structured formalasynchronous communication opportunities to post their assignments and RLOs-basedprojects at least once a week during the Spring Semester 2004. The fourth online datasources were the learners’ logs. The learners kept individual logs to explain theirRLOs-based experiences in a constructivist PBOL milieu each week of the sameSemester. The observations were the fifth online data sources. The researcher collectedthese observational data from online communication milieus during the fourteen weeksof the Semester Spring 2004. The attitude survey was the last online data sources. It was afive point Likert-type scale included twenty structured questions, and two open-ended


and also semi-structured questions. These questions were about producing and learningwith RLOs in the PBOL-based communicational activities.

The researcher kept paper–pencil journals regularly to understand the learnerreflections, experiences and opinions toward constructing and learning with RLOs in aPBOL setting. Also, the researcher tried to find out their achievement challenges aboutthe discourses and investments for the communication-oriented resource designs. Finally,the participants in this course worked together that they completed their projects in ateam, and also conducted these projects through solving complex problems by developingtheir critical thinking skills effectively. Therefore, their RLOs-based projects were stronglyrelated to their real-life dilemmas about how to integrate knowledge from differentdisciplines.

4.4. Data analysis

The researcher analyzed and triangulated the collected data. It took totally one year tocomplete the data analysis and triangulation steps entirely; because the collected data fromthe different sources were very rich and dense. However, the multiple outcomes weretracked over time which included the learners’ critical thinking developments, thepatterns and experiences of PBOL, and also the pedagogical requirements and existingopportunities of designing RLOs. Moreover, the researcher focused on the ethics andvalues of online societies to design RLOs for diverse learner groups, especially thosefrom underserved communities. Furthermore, the researcher defined and analyzed thecommunicational activities among the participants in this study to learn about how tobuild online knowledge networks. Also, she tried to expose the design strategies andprinciples of RLOs in a constructivist PBOL setting, and to highlight the diversecognitive abilities of the research participants that increased their understandingstoward RLOs in the public arena.

The analysis of building knowledge networks through PBOL to develop critical thinking

skills via RLOs began at the beginning of the Spring 2004 Semester and continued throughthe final written report at the end of the Fall 2005 Semester. The data analysis process inthis study was extremely analytic and recursive to inform further decisions. It also wasstructured flexible and open to consider with the stakeholders and the reviews of relatedliteratures. During the data analysis procedure, the researcher briefly followed thesesteps given in a logical order: First of all, the researcher, negotiated with two otherpeople to analyze the qualitative and quantitative research data that prompted theresearch credibility and transferability, and then listened each tape cassette two times.After listening tape cassette, the researcher transcribed all of them. Besides, theresearcher wrote the field notes from both the traditional and the online classes that shecould identify patterns in the face-to-face and WebCT milieus. After these activities, theresearcher analyzed the contents of the interviews, the contents of the face-to-face andonline class observations, the qualitative data from the pre-survey and also post-survey,and the qualitative data from the attitude survey. She also triangulated all quantitativeand qualitative data by collaborated with other two researchers. Finally, the researcherconducted the content analysis to identify code and categorize the fundamental patternsand themes in the data collected from the different sources. Finally, she interpreted andsummarized the collected data that integrated the broader areas of the main research


inquiries; reported the research results in using a descriptive and narrative form, and wrotethe research report.

4.5. Credibility

To provide the credibility in this qualitative case study, the researcher adopted a stanceof neutrality with regard to the research phenomenon under study (Patton, 1990). In otherwords, she did not try to prove a specific perspective and manipulate the data from thedifferent sources. Utilizing both qualitative and quantitative methods in this study wasto examine more complementary and different aspects of the complex phenomenon,building knowledge networks through project-based online learning to develop critical

thinking skills via RLOs (Greene & Gilbert, 1989). The combination of this methodhelped the researcher to generate new perspectives and stimulate new directions in dataanalysis. A combination of the qualitative and quantitative methodologies provideddata triangulation from a variety of data sources and method triangulation from theconsistency of different data sources (Patton, 1990). Therefore, the researcher overcamethe intrinsic bias that could come from single methods. Finally, the researcher was acenter of the analytic process and had a very crucial role on the credibility issues of thisstudy. Also, she closely followed the design and methods of qualitative research bycollaborating with her peers and colleagues, and also the experts from the related fields,educational technology and research methodology. In other words, she did not try toprove a specific perspective and manipulate the data from the different sources (Table 2).

Table 2The credibility issue in this researcha

The research credibility

Credibility issues The case study tactics To provide Used tactics

Reliability � use case study protocol� develop case study data base

� internal reliability� external reliability

� data collection

Validity � use multiple sources evidence� establish chain of evidence� have key persons review draft

study report� do pattern matching, expla-

nation-building & time-seriesanalysis use replication logicwithin multiple-case Studies

� construct validity� internal validity� external validity

� data collection� composition� data analysis� research design

Triangulation � use variety of data sources� use multiple perspectives to

interpret the data collected� use multiple methods to the

focus of the study� check out the consistency of

different data sources

� data triangulation� theory triangulation� methodology triangulation� source triangulation

� data collection� data analysis

a This table was adopted and modified based on Case Study Tactics for four Design Text in the book, CaseStudy Research (Yin, 1994, p. 33).


4.6. The limitations and strengths of the study

This is a qualitative case study that contained similar characteristics in most qualitativeand quantitative research. First and foremost, this study allowed the researcher to focuson the research phenomena that existed and happened in its natural environment. Theresearcher did not control or manipulate any patterns and themes in this study. As aresult, generalization might not be possible whereas transferability could be doable.This study was to aim how to build knowledge networks through PBOL to develop

critical thinking skills via RLOs. The participants, therefore, were asked their ideas,opinions and beliefs toward these practices based on their experiences. The informationprovided necessary knowledge to online workers, as online communication designers,who made the critical decisions regarding the use of pedagogical and communicationaltechniques and methods.

The researcher was aware of the limitations of this study that she utilized the confidentmixed research methods and the data collected from the different sources. Furthermore,the researcher regularly communicated with the experts from the related fields,Educational Technology and Research Methods. Finally, the researcher in this studyregularly and systemically recorded all data collected from different sources in eachstep. Additionally, she carefully and securely stored all collected data by indexing,labeling, coding and categorizing into the private digital folders on the Internet, andCD-ROMs, and also by printing the hard-copies of the necessary documents. Allqualitative and quantitative research data were kept in the password-protectedelectronic folders and regular files. To sum up, the gathered information from thelearners from the Designing and Delivering an E-Learning Environment course was valid,useful and enlightening.

5. Findings and discussion

The present study focused on building knowledge networks through project-based onlinelearning (PBOL) to develop critical thinking skills via Reusable Learning Online Objects

(RLOs) in the Designing and Delivering an E-Learning Environment course at theDepartment of Distance Education of the large urban and government-based Universityin the Spring Semester 2004. This paper also discusses on the different factors that werenecessary to take into account in attempts to promote the learners’ critical thinkingskills in an interactive online milieu.

This section reports the findings emerged during the data analysis. It embraces variousinsights and answers related to the research inquire. These data collected from the differentsources reflect exactly the participants’ ideas, opinions and beliefs as well as theirexperiences toward producing RLOs in a PBOL environment. These helped theresearcher present the authentic findings of this study in-depth. The findings of thestudy provide exhaustively descriptive analysis and discussions of three main areas: (1)the learners’ critical thinking improvements, (2) critical thinking patterns, and (3) learning

impacts and communicational outcomes. Furthermore, the overall findings of this studyare briefly summarized and discussed at the end of this section.

The learners’ critical thinking improvements and patterns, and the learning impacts andcommunicational outcomes in the constructivist PBOL milieus are extremely importanttopics of how these learners are generating new skills to construct new meanings


beyond what is directly given (Anderson and Garrison, 1998; Bonk and Cunningham,1998; Bowers, 2000; Cecez-Kecmanovic and Webb, 2000; Clark, 1998; Cristea, 2003;Fosnot, 1996; Halpern, 1996; Hong and Kinshuk, 2004; Jonassen, 2000; Kemery, 2000;Levy, 1997; Moore and Tait, 2002; Nichols, 2003; Norton and Wiburg, 1998; Paul,1995; Palloff and Pratt, 1999; Patel and Kinshuk, 1997; Picciano, 2001; Resta, 2002).Theoretically, RLOs in these academic settings are rule-following tools. Therefore,RLOs can guide learners apply analysis to recognize relationships that these learnerscan develop their cognitive models and high-level abstract knowledge. In theseinteractive communication environments, these learners can become rule-makers todesign their own RLOs with their complex higher-order think skills. Besides, not onlycan these learners keep in charge of their critical thinking processes that are conduciveto positive intentions toward learning, but also they can raise their thinking to theanalytical levels.

5.1. Critical thinking skills improvements

In the constructivist PBOL environment, this course combined the text, graphics,audios, streaming videos, questions, and exercises to produce rich, in-depth andinteractive communication experiences for the learners. As indicated by Wiley (2000),besides, the main design principles of the RLOs were to develop and organize thelearning contents and communication issues quickly and powerfully regarding theparticipants’ critical thinking developments. In the Designing and Delivering an

E-Learning Environment course, the learners’ critical thinking skills referred theirmetacognitive strategies that helped them promote their learning and communicationalperformances in a PBOL setting. According to the collected qualitative and quantitiesdata, the learners in this research were concerned about the following the PBOL stepsto design and develop the diverse RLOs:

The researcher asked dynamic questions under diverse considerations. These learnerswere engaged real-world problems to utilize multiple content areas for use of the RLOsby focusing on dilemmas; identifying questions; formulating criteria for judging possibleanswers; and keeping the situations in their minds. The learners tired to explore variousopportunities to learn concepts, apply information, and represent their knowledge in avariety of ways. Therefore, they could produce the RLOs by analyzing arguments andcommunicating with others; work with other learners to gather all necessary data andresources for the RLOs; documenting with the pictures, sound recording, graphics andtexts; identifying stated and unstated reasons; and being aware of similarities anddifferences on reusability. These helped them guide design choices and reducinginterdependence among RLOs. Moreover, these learners tired to articulate thefoundational principles for developing RLOs and to provide a methodology and broadset of guidelines for generating their RLOs in a constructivist PBOL environment.

Claire and Daphne tried to design an interface to present their RLOs-based project.This design provided other learners with the information they need. On the other hand,it was not easy to navigate through this interface. Claire and Daphne presented toomuch information on a screen, as in this case the learners had to spend considerabletime scanning through a littered screen and numerous screens of information. Joe andGrace grouped the text and graphic RLOs, and order into meaningful parts. Theyhoped that the meaningfully structured information would have been easier to perceive


and guide learners’ attention. However, the other learners emphasized that this Web sitepresented the RLOs arbitrarily so that it was boring. Noah, Daphne and Joe pointed outthat color, flashing texts and graphics, and short videos were not the significant of theirattentions for these RLOs. Raymond strongly underlined that he was distracted by theJoe’s and Grace’s RLOs-based projects so that he did not get his attention on what helooked at and listening to for any give stage of this task. Although Claire and Daphneoverlapped information on the screen whereas Joe and Grace discretely assorteddifferent types of information, their projects did not catch the learners’ attention.

Making active investigations empowered their cognitive tools in online milieus. Theselearners represented their ideas, belief, opinions, attitudes and intentions by locatingdiverse knowledge to answer questions without biases; visualizing the collected data in avariety of ways and solutions abstractly; mapping their ideas and concerns to explainand share knowledge; and also looking for alternatives. Besides, they tried to highlighton their own artifact knowledge constructions to represent what they learned. In thisstep, their freedom was very crucial to build artifacts by designing knowledge; workingon tangible cognitive products; and stating alternative conclusions. They learnt how toshare knowledge and communicate with other learners, professors and the courseowner. They posted their products on WebCT by employing to organizing arguments,reconstructing solutions; formulating alternative models; building credible andtransferable resources; and also withholding the unique and critical concerns. On theother hand, none of them could be able to tag the sound, graphics and video RLOswith metadata; so that these RLOs were not stored and referenced in a database.

In this course, the PBOL setting required active communicational engagements that thelearners tried to learn how to deal with complex issues to build the RLOs. However,neither of them could powerfully reflect their different learning types andcommunicational situations to produce RLOs. Besides, the interview data provided within-dept insights about how these learners collaborated with each other in thisconstructivist PBOL milieu. In these interviews, all participants stated strongly that theydid like to learn with the PBOL environment to produce the RLOs, even though theirscores of the two mid-terms and a final exam were very low that the researcher did notanticipate these low achievements from her participants. When they were asked aboutwhy they could be not successful in this course, the eldest one answered that ‘‘. . . Wewere overloaded by various learning activities. All of us had jobs, and we could notfind enough time to think about our projects. . . Therefore, we preferred completing ourassigned PBOL-based learning activities punctually. . .’’.

Furthermore, in the middle of the Semester, one of the participants, Raymond made asevere confession that:

‘‘. . . not only was the course content very heavy, but also the teaching method wasnot suitable for our learning skills. However, we would like to learn with these newlearning models and methods. . . There were much more different learning materialsthan the traditional ones. However, it was impossible for us to focus on the RLOs.We were working, plus . . . eeee . . . had no idea about how we managed out timeeffectively . . .’’.

Moreover, the researcher was observed during the data collection process that thesegraduate learners did not like to learn individually nor in the groups; because theselearners had not taken the serious pragmatic steps of clearly defining their roles and


responsibilities regarding their different learning styles and diverse needs. However, theyrealized very painfully that they were not open minded individuals to be sensitive to thefeelings, level of knowledge, and degree of sophistication of others when they wereworking in the groups. As mentioned by Levy (1997), Halpern (1996) and Paul (1995),on the other hand, the learners were allowed to adapt over time while integratingcritical thinking skills into regular curricular offerings. This was a personal changeprocess. In reality, producing sound and graphics-based RLOs and embedding theminto the course database systems was the most difficult section for all participants dueto their lack of technology skills and knowledge. Moreover, none of them was eager tolearn about how to develop the well-designed sound and graphics files for their projects.Therefore, they could not become more experienced with producing and learning withRLOs and utilize more efficient means of performing tasks.

This PBOL environment tried to provide these learners with this change process byinvolving them actively in their change process. As described by Hord, Huling-Austin,and Hall (1987) there were the three major concern areas in this course that learnersshould have to demonstrate the self (awareness, informational and personal concerns),the task (concerns of management), and the impact (concerns of consequences,collaboration and refocusing). Besides, any development that was going to changelearners’ cognitive abilities tried to deal with these various stages of their concerns. Asnoted by Levy (1997), the applicability and usefulness of the Metathoughts were limitedonly by their creativity, imagination and resourcefulness whereas Metathinking was theact of thinking about thinking, engaging in a critical analysis and evaluation of thethinking process. In this course, the learners own individual contributions were the keyingredients that infused the contents of RLOs with unique features. On the other side,the absence of their metacognitive skills made them failures to profit from theirattempts at Metathoughts. In this case, unfortunately, thinking was not a mentalprocess of producing their ideas, exercising the powers of their intelligences, reasons,logics, judgments, conceptions, memories and inferences.

Like the majority of the participants (72.7%), Rose Mary pointed out that ‘‘. . . I amused to involving in traditional learning methods, and memorize everything . . . after thefinals, just let yourself forget everything what you sucked up . . .’’. Therefore, in thiscourse, developing the learners’ critical thinking skills was placed at the center ofeducational concentration. Also, their higher-order thinking processes that transferringfrom one situation to another had became one of the vital elements of how to build theRLOs powerfully in this class. To sum up, in this study, when producing and learn withthe RLOs, the majority (81.8%) of the participants tried to improve their criticalthinking skills that they tended to explore diverse points of views; learn from theirmistakes; become flexible communicators; seek and provide reasons for what they weredoing; remain relevant to the main point of a discussion; consider the new ideas andopposite information and knowledge; follow their own thinking skills; be sensitive toothers’ ideas and knowledge; seek appropriate solutions; and distinguish betweenconcrete objects and abstract constructs.

As noted by Wiggins and McTighe (1998), the two of these 11 learners believed thatthey had to really understand whereas three of them desired to internalize knowledge.Only one of them, Raymond, mentioned that he would like to grasp the core andessence that he could learn how to produce a various RLOs related to his projects.However, in this study, the majority of the learners (90.9) had difficulty in developing


more positive intentions of their critical thinking developments. The 72.7% of these 11learners did not recognize when they were acting impulsively, thinking rigidly andplanning the production steps of RLOs. Moreover, the majority (81.8%) indicated thatthey did not need high-level communication skills in their group works. On the otherhand, as highlighted Halpern (1996), only critical thinkers were consensus seekers, whoneeded high-level communication skills to achieve agreements in the group structures.

As discussed by Preece et al. (1994), in this study, the knowledge-based level referred tothe conscious and analytic processes that occurred when these learners were confrontedwith novel situations. It was the level that they had to evaluate the unexpectedproblems by using their mental models. On the other hand, the function of their mentalmodels at the knowledge-based level hardly predicted future events, founded causes ofobserved events, and also concluded appropriate actions to cause changes. None ofthese learners, therefore, understand the RLOs-based complex systems at the multitudeof levels, varying in function, structure, generality and form. Moreover, the majority ofthese learners (81.8%) did not describe appropriately how the cognitive principle ofmetaphorical reasoning had been applied to the RLOs production. They did not expressabstract concepts (such as peace, freedom, love, etc.) in more familiar and accessibleforms. From this critical thinking perspective, it might be assumed that these learnerswould have difficulties with interpreting metaphors. Therefore, neither did theyunderstand different concepts, nor developed multiple mental models when working ontheir projects. For example, Rose Mary tried to use tool bar and combine the iconsthat she would like to explain the word happy in a proper manner. On the other hand,this was an extremely poor designed metaphor that caused conceptual dilemmas toother learners.

5.2. Critical thinking patterns

In this study, the dynamic patterns of the learners’ critical thinking via the RLOs in theconstructive PBOL environment are summarized in Table 3. Each stage explains thecritical steps and the set of skills referred to the learners’ cognitive experiences thatstressed the importance of their insights in producing the RLOs.

According to Table 3, the results indicated noticeably that the learners used one ormore communicational tools that they tried to complete their group projects,assignments and practices during the Spring 2004 Semester. The collected data pointedout that these learners demonstrated the use of the preceding problem-solving strategiesand principles. Besides, the learners in this course could express both their negative andpositive reflective comments eagerly. Jennifer intrepidly confessed that ‘‘. . . the overallcommunication atmosphere was highly academic and professional. . . but I had nochance to complete my course works . . . I meant . . . I was a lazy one . . . so learningwith new communication technologies not for me . . .’’. Also, the qualitative data fromthe attitude survey, all learners in this course communicated with each otherasynchronously and sometimes synchronously. On the other hand, when these learnershad to post their own ideas on the discussion topics in the bulletin board area and/orto involve the formal knowledge sharing activities, they always demonstrated extremelylow self-esteems and preferred not to argue about their concerns and project-basedproblems loudly. However, the researcher observed in the face-to-face classes that thesame learners expressed themselves comfortably. Sue reflected that ‘‘. . . I feel more

Table 3The learners’ PBOL-based critical thinking strategies via RLOs

The learners’ critical thinking skills

The categories of the RLOs in thecourse

PBOL-based strategies

Concluding meta-metathoughts

The evaluative biases andstereotypes

The multiple level ofdescriptions

Dedu ve andinduc e reasoning

The interventions

Texts Course contents� Overviews� Summaries

� using of language reveals� seeking information

� challenging other’spoint of views

� ma ing their selections � asking right questions

Procedures� Definitions� Descriptions

� being impartial� realizing their prior

belief and experiences

� describing at differentlevels of analysis

� diff rentiating betweende ctive and inductiverea ning

� keeping their mindopens to diversepoints of view

� Tests� Assessments� Problems� Case studies

� presenting personaljudgments� reflecting objective truth

� addressing thelogical questions

� describing

� rec gnizing the natureof e particular question,tas , concern and situation

� understanding anissues how theyexplain a results

Graphics Simple drawings � concerning personality� counterarguing their

preexisting beliefs

� differentiating variables � lab ling a particular situation � proving whatoriginally caused thesituation

Still pictures � being objective� explaining diverse judgments

� considering the opposite � inf ring specific conclusions� inf ring general conclusions

NA

Animations � reflecting personal values � selecting mostappropriate levels

NA NA

Audios One-way � building their own schemas NA NA NAInteractive � understanding other learners’

experiencesNA NA NA

Streaming Used an existing one NA NA NA NAProduce a new one NA NA NA NA

The RLOs-based cognitiveactivities

The relationships betweenthought and language

Analyzing arguments De sion makings Problem-solving

Metathinking

G.

Ku

rub

aca

k/

Co

mp

uters

inH

um

an

Beh

avio

r2

3(

20

07

)2

66

8–

26

95

2683

ctitiv

k

edusoothk

e

erer

ci


comfortable in the [regular] class. . . because, I could not write fast, and could not thinkfast . . . I could not find the words when I was posting my messages online . . .’’.Therefore, most of them chose (63.6%) to communicate with face-to-face to design andbuild their RLOs.

Moreover, the researcher examined the dynamic critical thinking patterns of thelearners in this constructive PBOL environment when they were working to produce theRLOs in groups. The background of this investigation process was the Paul critiques(1995) that he argued the pedagogical and theoretical bias toward critical thinking, andalso discussed very different assumptions when analyzing the nature of thinking. In thisframework, the researcher disputed the dynamic critical thinking patterns of thelearners as the following: Of 11 learners in this study, all of them could thinkdescriptively rather than normatively with to respect to approach of thinking whereasthe three of them (27.3%) could formulate their conceptual analysis with to respect tomethodology, and the rest of them neither formulated their theoretical process norfocused on the analysis of the RLOs-based cases of well-justified thinking. The sixlearners (54.5%) realized intradisciplinary thinking, the two (18.2%) focused onreflective thinking with to respect to modes of thinking involved. However, none ofthem emphasized the value of their expertise on RLOs. On the other hand, all learnerswould like to make the authority of the expert central, such as the course owner, andnone of them did not realize their expertise areas nor took responsibility of theirlearning process whereas all learners had very limited language skills when speakingabout their RLOs-based projects and writing their research projects, but they realizedthat they were speaking Turkish with very limited words; so that improving theirlanguage skills was the most important issue for them to become critical thinkers.

Monika and Phoebe used icons for a range of functions. They generated menu items asa labeling function and tools for design as a manipulating function. Their task for whichRLOs were being used was an imperative aspect in formatting these RLOsmeaningfulness. Monika strongly noted that ‘‘. . . we tried to define textual informationto build a well-suited domain which represented in a textual form, but we did expressour ideas about how to represent verbal information in iconic forms . . .’’. Expressingverbal information in iconic forms made their task of retrieving information moredifficult; because it was cognitively more challenging for the learners to have to joinvisual forms with verbal ones. Furthermore, Sue and Rose Mary were unsure of theprecise nature of verbal information. The icons were not very useful as cues for them toguide their navigation on the site Monika and Phoebe built.

Of 11 learners in this study, the four of them (36.4%) realized the importance of theview of other learners, when discussing their basic mental skills and processes towardproducing RLOs whereas the rest of them (63.6%) had no ideas about the role ofvalues of thinking in this process. However, none of them viewed debates andargumentations as central to rational thinking when they were producing their RLOs,and also no body played down the significance of dialectic thinking. The two of them(18.2%) underemphasized the affective obstacles to rational thinking, prejudice, bias andself-deception, but the rest of them (81.8%) did not show any interests toward the viewof affect. At the same time, all learners made central role of the course owner asautonomous critical thinker, and also they would like to accept her own biases,prejudices and point of views rather whereas the majority of them (90.9%) needed todeveloped the face-to-face group communications wherein dialogical exchange was the


matter of their RLO-based productions, and also none of them indicated the arts ofadvocating, restructuring and challenging each other’s ideas.

Of 11 learners in this study, none of them enjoyed to become Socratic questioners asintelligent skeptics whereas the six of them (54.5%) understood the significance ofempirical research in settling RLOs-based issues. On the other hand, all of themindicated that teaching required the step-by-step procedures of RLOs, and no bodyvoluntary constructed these processes by themselves. Besides, the four of them (36.4%)emphasized the recalling, encoding, identifying relationships of how to produce theRLOs whereas the rest of them (63.6%) underlined the importance of the courseowner’s assumptions on their works. None of them could not separate the analysis ofthe micro-skills of the RLOs-based productions from normative considerations whereasthe seven of them (63.6%) viewed the macro-skills of the RLOs-based productions fromthe perspectives of categories of problem solving and decision making.

Of 11 learners in this study, all of them highlighted teaching for critical thinking as anintellectual art and all of them paid attention to the significance of teachers developing aphilosophy of education into which autonomy and self-criticism became central values.However, the seven learners (63.6%) decreased the problem of prejudice and bias inproducing RLOs where as the rest of them (36.4%) were not sensitive to the dangers ofbiases and stereotypes to teaching for rational thinking when working on the RLOswhereas the five of them (45.5%) appreciated the significance of rational passions anduniversal intellectual virtues and ethics whereas the rest of them (54.5%) were cautiousabout national intellectual virtues and ethics when designing the production processesof the RLOs. Moreover, the three learners (27.3%) could orient themselves towarddomain-specific thinking with a specialist learning and working within a discipline, andthe seven of them (63.6%) emphasized the link between teaching for critical thinkingand developing moral insight with the rights of the course owners. Finally, the nine ofthem (81.8%) stayed strongly streets upon the relation of learning for critical thinkingto ethics of teaching and the rights of the course owners whereas only one learnersemphasized the link between learning for critical thinking and developing moral insightwith the rights of them when working on their RLOs-based projects where the eight ofthem (72.7%) did not realized the relation of modes of critical thinking to importantethical and philosophical alternatives regarding a way of their lives whereas the rest ofthem (27.3%) believed that the critical quest for reasons was a dominant andintegrating motive for their societies when producing the RLOs.

Sue tried to use graphical RLOs as cognitive aids. She noted that these RLOs resultedin a considerable decrease in the amount of mental effort. However, Claire, Joe andMonika noted that they needed to know how to find these items on the menus Suegenerated. The use of graphical interfaces did not reduce the amount of knowledge thelearners had to remember about the RLOs. Represented knowledge via Sue’s RLOs wasnot highly organized. Therefore, her RLOs did not allow other learners to carry outtheir cognitive activities with minimum effort. They did not use any metal models wheninteracting with these graphical RLOs. On the other hand, Raymond critically statedthat he did not understand different strategies Sue used when designing these cognitiveaids. Therefore, these learners had trouble realizing that one trouble generated another.Monika and Jennifer pointed out that building and learning with RLOs was difficult sothat they experienced frustration, but did not blame themselves. Monika and Jenniferrealized that they could not correctly follow directions provided by Sue. On the other


hand, all learners learnt from their mistakes and transferred knowledge in order tounderstand RLO, which provided these learners with a bridge between their old andnew knowledge. However, learning with these interactive environments took longerthan they expected. Noah tried to use traffic symbols as concrete icons for hisRLOs-based projects. He mentioned that he tried to provide dynamic representationsabout the status of RLOs. In the reality, all other learners declared that there were norelationships between these traffic symbols and the RLOs Noah built. In short, the alllearners realized that they could only produce effective RLOs when they learnt the artand meanings of characteristic principles from universal perspectives and facts.

5.3. Learning impacts and communicational outcomes

The quantitative data obtained from both the online pre-survey and the post-survey,and the traditional class pre-survey and post-survey indicated that there was nosignificant difference between the learners’ critical thinking skills and their intentionstoward building and learning with RLOs in a PBOL environment, and also between thelearners’ critical thinking skills and constructing knowledge networks viaobject-oriented learning the philosophy of RLOs. Determining the relatedness betweenthe critical thinking development and the involvement of the RLOs-based productionswas complicated in this study. This was exactly the sort of the learners’ decisions aboutthe acceptability of facts, contexts and concerns depended on how they evaluated thesource of their knowledge and values.

The qualitative data from interviews provided more insights of how these learners couldimprove their critical thinking skills when building and delivering the RLOs in theduration of the Designing and Delivering an E-Learning Environment course. However,this result differs from the earlier literature that the learners’ previous critical thinkingskills can affect their intentions toward the contexts they are focusing on (Barritt andLewis, 2000; Clark, 1998; Downes, 2002; Ip and Mornson, 2001; McGreal and Roberts,2001; Merrill, 2000; Nesbit, Belfer & Vargo, 2002; Quinn and Hobbs, 2000). The dataobtained from the classroom and online observations as well as interviews highlightedthat there were three main themes that underlined the learning impacts andcommunicational outcomes a PBOL environment on the learners’ critical thinking skillswhen they work with RLOs the nature of critical thinking and obstructions, the important

role of language in critical thinking, and the awareness of all sort of meaningful items in

the communication milieus.In this study, language was the essential medium of the learners’ critical thinking within

which their thoughts took form and gained expressions. All learners realized that they hadto paid close attention to their language to express what and how they thought. Moreover,the majority of these learners (90.9%) pointed out that they had very limited wordtreasures, so that their communications did not guide their interpretations of otherswords nor generated meaningful dialogs among them. Like Raymond and Jennifer, Lisastrongly emphasized that ‘‘. . . I could not express myself . . . my emotions, my thoughts, Ifelt very angry because I cannot use the words . . . I realized after your [the course owner]feedback on my works . . . kept using with same words . . .’’. Moreover, all participantsconfessed that they had to read more books to improve their language abilities. Noahexpressed himself clearly that he had to start to read the elementary school books first.As mentioned by Rudinow and Barry (1994), a main of language is to communicate


information that individuals can articulate their feelings beliefs and motivationspowerfully. Therefore, the learners’ language usage in this study could not help themguide when thinking about discourse analysis due to the lack of their language skills.Moreover, they could not raise very reflective and essential questions about whenproducing their RLOs.

Producing RLOs in a constructivist PBOL environment must be beyond a technologicalsetting for interactive communications (Bonk & Cunningham, 1998; Nesbit et al., 2002).As noted by Bowers (2000) and Cecez-Kecmanovic and Webb (2000), this was crucialfor the learners to work and learn together in this course. The majority of interviewedlearners (81.8%), however, indicated that they preferred to be silent receivers ratherthan being active learners in the traditional and online classes. Therefore, a lack of theiroral and written communication skills generated exactly a lack of confidence andself-esteem when building and delivering RLOs in this course. The ten interviewees(90.9%) pointed out that the learning strategies, methods and activities of the Designing

and Delivering an E-Learning Environment course were extremely different from othercourses that they had taken. Uncertainty created communicational curiosities, anxietiesand conflicts in these learners due to their low comfort levels. Therefore, these learnerswere fewer enthusiastic to participate in the online communicational activities. RoseMary declared that they would like to be salient and passive learners in the course; sothat they were not afraid of spending their efforts and time to think critically onproducing better RLOs in this course.

The qualitative data from the traditional and online class observations, interviews, andthe quantitative data from the survey indicated the learners showed these learning impactsand communicational outcomes when working on their RLOs in this study: Of 11participants, the two of them (18.2%) enjoyed studying with the realistic problems; thesix of them (54.5%) learnt to break larger problems into smaller parts for multi-waysolutions; the three of them (27.3%) respected others’ opinions and ideas; the five ofthem (45.5%) demonstrated the use of the preceding problem-solving skills; the three ofthem (27.3%) used inductive reasoning; the two of them (18.2%) discriminated betweenvalid and faulty inferences, the seven of them (63.6%) enjoyed debating the multipleview points; and also the four of them (36.4%) made validate generalizations.

Rose Mary, Joe and Raymond mentioned that building powerful RLOs was acommunicative process in which they interacted with each other online, and alsounderstanding collaborations was the fundamental for online communications. On theother hand, Sue and Joe strongly highlighted that one of the important problems wasthat they found it difficult to manage the amount of collaboration that was requiredwhen multiple participants were involved. Noah noted that they did not establish socialprotocols among them to enable themselves to cope better with the communication-basedproblems. These communication problems related to the RLOs production process inthe PBOL environment led to less optimal forms of communications and the potentialfor misunderstandings. Especially, Jennifer and Daphne stated that this situation didnot facilitate some of the social pressures of speaking in a group of people online. Forexample, all of these participants found talk in face-to-face classes less threatening.

The problems of online group communications decreased depending on the type ofknowledge networks that was established in the group. As pointed out by Preece et al.(1994), the more open knowledge networks developed their own communicationstructure by opting for the best suited to the problem rather having a complex structure


imposed upon them. For example, their RLOs-based projects included a few video images.Although the learners in the campus were able to clearly see the videos on screen all otherlearners in other locations, they were able to see that person on their screen. Therefore,they gave up integrating video images into their RLOs. They preferred developing stillvisual images, such as photographs, captured pictures, etc.

Online group communications had different properties and process than pairedinteractions. Their group communications were not flexible at changing their division ofeffort to deal with unexpected circumstances. Informal learning and communicationpractices, and shared artifacts, therefore, did not coordinate their project activities. Thisfinding highlighted the problems that needed to be considered in the design process ofRLOs. Describing the informal and formal processes, properties and needs of theselearners were critical to the accomplishment and acceptance of online group workingtogether. Therefore, modeling the knowledge-base from which the RLOs to be masteredwere drawn were the important learning and communicational outcomes in this PBOLenvironment. On the other hand, these learners were not successfully empowering theirjoint activities, such as group–learners, learner–learner, and course owner–learnerinteractive collaborations.

5.4. The overall findings

The researcher gained the information that the overall findings and discussions of thisresearch toward building knowledge networks through PBOL to develop critical thinking

skills via RLOs were related to a lack of the learners’ communication skills.Of 11 learners, the two of them (18.2%) developed appropriate communication skills to

make connections between what they already knew, and new information whereas the oneof this two learners used information in new contexts by look considering diverse points ofview whereas the three of them (27.3%) communicated their viewpoints to evaluate therespective influence of reason and emotion on their approaches whereas the two ofthem (18.2%) recognized their biases and stereotypes easily. Besides, the three learners(27.3%) expressed their ideas in new ways without accepting risks and unknowns; thefour of them (36.4%) tried out new ideas and exchanged points of view with others, butnone of them could listen and be open to differences; and the ten learners (90.9%)reflected, before and during the actions, on the best way to reach their objectives, butnone of them could complete the RLO-based tasks. On the other hand, the three ofthem (27.3%) accepted others as they were, but none of them was responsive to othersand recognize their interests and needs. Finally, none of these participants did not planand carry their RLOs-based tasks with other learners productively; so that they did notuse teamwork successfully whereas the majority of them (81.8%) tried to participateactively in the classroom and online activities with a collaborative attitude; becausethese activities were mandatory.

In this course, different situations produced different interactions among the learners.The learners realize that they had to change their communication attitudes towardonline milieus. Furthermore, they understood that they had to involve in a dynamicchange process in order to increase their productivity. This was an important aspect topowerfully decide how to the RLOs were structured, and also dynamically identify theproducing process by which benefits and problems developed. On the other hand, theselearners underlined that a learner-centered design process in diverse ways empowered


the usability of their RLOs. Finally, all course participants pointed out that there was aparadigm shift to provoke radical changes in online knowledge networks throughPBOL-based environments. As mentioned by Preece et al. (1994), these milieus allowedthem to generate and develop intellectually in innovative and exciting new approaches.

6. Results and conclusions

The main purpose of this study is to build knowledge networks through Project-Based

Online (PBOL) Learning to develop critical thinking skills via Reusable Learning

Objectives (RLOs). In this study, the participants were from the Designing and

Delivering an E-Learning Environment course at the Department of Distance Educationof Social Sciences Institute that they learnt and communicated with a constructivistPBOL milieu. The results and conclusions of the study provide in detail descriptiveanalysis and discussions of three main areas. This section is divided into three sections:(1) The conclusions and recommendations for the learners’ critical thinking skillimprovements, (2) critical thinking patterns, and (3) learning impacts andcommunicational outcomes in a constructivist PBOL milieu when they were producingand learning with RLOs. Furthermore, the overall results and conclusions of this studyare briefly summarized and discussed at the end of this section.

6.1. Critical thinking skill improvements

The participants recognized that the most important educational goal in this course wasto develop their critical thinking skills by producing and learning with the RLOs. Thelearners in this study tried to engage diverse critical thinking activities. They realizedthat they were kept in charge of critical thinking incessantly to accomplish theRLOs-based tasks. On the other hand, these learners appreciated that they had to thinkcritically to become active, productive, hopeful, and psychologically healthier people intheir society. Involving the various critical thinking activities, however, was a verytough effort for these learners that they had to spend extra time and energy to produceand learn with the RLOs by synchronously and/or asynchronously communicating witha PBOL milieu. These process did not help this individuals promote theircommunication progresses and critically reflect upon what they had to share with eachother. As pointed out by Oakes (2002), furthermore, building and learning with RLOsdid not encourage the research participants to effectively transfer their knowledge to thenew contexts. As a result, they did not improve their complex critical thinking skillssuccessfully to construct, produce and demonstrate their knowledge, and alsocommunicate with each other. To sum up, the overall conclusions about the criticalthinking skill improvements in this study are discussed the below:

The learners used one-way communications when interacting with their classmates andthe course owners; preferred to be a passive receiver rather than an active contributor; notfind multi-way answers that were just as good as the ones they had suggested; livedproblems when they could not state the absence of other possibilities; hardly askquestions to develop completely their new ideas; and likely to waste a great deal of timeby producing the RLOs. Although the course owner provided them with effectivefeedback, guidance and reinforcements, the learners needed additional practices to


develop their critical thinking skills efficiently that they did benefit from studyingindividually nor in groups.

RLOs-based designs were complex processes and the learners needed as much supportas possible. These learners were able to appreciate their diverse needs to provide supportfor their design activities. However, they did not provide each other with the clearguidelines for their RLOs-based projects. They only used paper–pencil prototyping andconcept map techniques help each other understand their projects more and identifypossible mistakes. These kinds of supports were not helpful to obtain a clearunderstanding of their projects.

6.2. Critical thinking patterns

In this study, the learners’ critical thinking patterns were related to their concerns abouttheir learning styles and needs. Therefore, they had to change their learning intentionstoward PBOL that this change process required that their concerns be focused oncommunicating with each other. Therefore, RLOs provided them with the flourishingbenefits of PBOL. On the other side, rarely did these participants begin the course withabsolutely no idea of what they had to accomplish and gain tasks, and also why theyhad to improve their critical think skills during the course. These participants had thesort of presumptions and misconceptions about the nature and functions of RLOs,critical learning and PBOL. However, they learnt about that their preconceptions werenot very accurate. In this course, the participants tried to understand a rational basisfor their beliefs and intentions, and a set of standards and procedures for analyzing,testing and evaluating their critical thinking process. However, they did not justify theirbelief and intentions to find out new investigations. To sum up, the overall conclusionsabout the critical thinking patterns in this study are discussed the below:

The learners were not careful about where they directed their attentions; stronglyconnected with their previous belief structures on how and what they thought; mademoral judgments mainly influenced by their cultural backgrounds all the time; rarelypromoted their careful discussions to lead their moral believes; rarely forced the powerof diverse evidences supporting their point of views; likely waste a great deal of timeand labor by producing the RLOs; were a lack of imaginations and inspirations; anddefied strongly to change their critical thinking skills and learning habits.

The structure of communication activities which one carried out solving learningproblems was quite complex in the PBOL milieu. In this case, these learners tried totransform the situations of their learning tasks to discover general relationships.Furthermore, the focus of these activities involved them in clearly identifying theirneeds and motivations, but they did not master a generalized method for solving acertain class of concrete tasks. Therefore, the RLOs-based activities did not generatesituations for critical investigations, for reflecting via diverse models, and analyzingessential features of the content being focused on. The RLOs were not able to providereflexivity guided educational milieus toward communications.

6.3. Learning impacts and communicational outcomes

As highlighted by Wolf (2002), and also Santally, Govinda, and Senteni (2004),producing and learning with the RLOs generated diverse contexts related to the


learners’ critical thinking, and also provided effective interactive communications amongthem in this study. However, this novel tool constructed a powerful learning paradigmthat the research participants could not recognize this model for promoting their logicalskills. The important point was that the PBOL-based learning approach could notcustomize the communication environments for these learners; because they did noteasily give up their familiar and traditional communication ways. The research dataindicated that these participants could change their learning attitudes, strategies andprocedures regarding their familiarity or unfamiliarity with accepting a constructivistPBOL strategies and principles for altering their communication processes anddeveloping critical thinking skills, adapting the RLOs for using new technology-basedenvironments powerfully, and producing the suitable experiences for answering anyquestions related to learning with the RLOs in the PBOL milieus.

Generating creative ideas was one the most difficult factors for their RLOs-basedprojects. Neither online group discussions nor brainstorming were supported toappropriate communication activities among the learners, course owner andresources. As noted by Preece et al. (1994), therefore, they did not communicatetheir ideas to each other so that alternatives were not be considered, possibleconflicts recognized and resolved, elements integrated, and final decisions taken.Furthermore, they did not establish effective formal and informal communications.They did not access to each other and each other’s ideas and RLOs-based designs,because of the lack of effective communications, for instance, a way of trackingideas and discussions. As indicated by Wylie (2001), the PBOL-based milieu was notable to promote ease of online and traditional class communications.

Finally, the learners in this study could not start effectively using the RLOs to improvetheir critical thinking skills. The researcher realized that she had to spend much more timeto accomplish the course goals, objectives and tasks to promote the learners’ intentionstoward PBOL positively. Furthermore, the course owner needed additional scheduledand unscheduled communication opportunities to help these learners share andexchange their own ideas based on their own experiences. On the other hand, thelearners tried to communicate with each other interactively that they had to generate aneffective change process via the RLOs-based process.

6.4. The overall results and conclusions

At the end of the course, none of learners could complete their own RLOs projects andany group assignments successfully. Besides, they could not write their project articles tosubmit at the journal and to present at any Educational Technology conference.Therefore, all these learners could not accomplish any course requirements. On theother hand, the researcher used both summative and formative evaluation methods toassess the learners’ critical thinking developments and their RLOs-based tasks duringthe course. The formative assessment addressed the learners’ self and group tasks thathave been accomplished or not accomplished during the course (Process) whereas thesummative assessment addressed the learners’ self and group tasks that have beenaccomplished or not accomplished at the end of the course (Production):

The Process-based evaluation provided feedback to the course owner for making courseimprovements. In this research, to improve their critical think skills via RLOs, the learnerswere aware of the importance and significance of generating new solutions; learning about


procedural information that they thought divergently so as to discover new concepts andfacts; learning specifics communication strategies and general concepts; involvingconfirmation of their the RLOs-based hypotheses; focusing on deciding what to believeor do; thinking diverse patterned in new methods and models to guide to their criticalthinking results; realizing original consequences to focus on scientific inquiry;transferring the generalization of concepts and skills learned in the RLOs; askingreflective questions to categorize various strategies learned in the RLOs; thinking usingdiverse inferential thinking strategies; reading more books critically to improve theirlanguage skills that they could express their feelings and ideas powerfully; anddeveloping their own time management strategies. The Production-based evaluationpermitted the course owners to reach unbiased, objective answers to questions. At theend of the course, the learners felt that they overused of individualistic learning toimprove their higher-order thinking skills; overwhelmed the number of possibilities andstrictly logical thinking perspectives; not take the ownerships of their learning processes;had no time to accomplish the course achievements; preferred to do all course relatedactivities in the class hours; directed their responses into narrow paths better; rarelyaccomplished all tasks in the course; tended to focus on relevant and diverse details;and were unreasonably challenged to stimulate procedural and divergent thinking bythe course owner.

As discussed by Sicilia and Garcıa-Barriocanal (2003), reusability is an essential andarguably the most important characteristic of the RLOs, and also referrers to potentialand future usage scenarios in this course. Evaluating usability of the RLOs is difficultto measure. Therefore, reusability is measured as the aggregated degree of adequacy foreach of the possible contexts specified that the learners must rethink about how toimprove their critical thinking skills via RLOs by focusing on the more pressingeducational and social problems of our time. Moreover, they have to understand theachievement challenges and assessment concerns of RLOs. Furthermore, through thedata analysis, this research addresses that the diverse questions and concerns towardhigher-order think skills are very crucial for learners to build democratic andmulticultural communities via RLOs. To sum up, not only does this article provide aconstructivist-based PBOL approach to improve learner critical thinking skillssuccessfully to look at different practices and various experiences from the real-lifesituations, but also discuss pedagogical knowledge and design guidelinescomprehensively required for the communication-oriented designs of digital reusablelearning resources.

References

Anderson, T. D., & Garrison, D. R. (1998). Learning in a networked world: New roles and responsibilities. In C.C. Gibson (Ed.), Distance learners in higher education: Institutional responses for quality outcomes (pp. 1–8).Madison, WI: Atwood.

Barrit, C. (2002). Using learning objects in four instructional architectures. Networker Newsletter, 18(7) [Online]Available from http://www.svispi.org/networker/2002/0702a1.htm .

Barritt, C., & Lewis, D. (2000). Reusable learning object strategy: Definition, creation process, and guidelines forbuilding, Version 3.1. Whitepaper published by Cisco Systems. <http://www.reusablelearning.org/Docs/Cisco_rlo_roi_v3-1.pdf> Retrieved 08.02.2004.

Bonk, C. J., & Cunningham, D. J. (1998). Searching for learner-centered, constructivist, and socioculturalcomponents of collaborative educational learning tools. In C. J. Bonk & K. S. King (Eds.), Electronic

http://www.svispi.org/networker/2002/0702a1.htm

http://www.reusablelearning.org/Docs/Cisco_rlo_roi_v3-1.pdf

http://www.reusablelearning.org/Docs/Cisco_rlo_roi_v3-1.pdf


collaborators: Learner-centered technologies for literacy, apprenticeship, and discourse (pp. 25–50). Mahwah,NJ: Lawrence Erlbaum.

Bonk, C. J., Kim, K. J., & Zeng, T. (2006). Future directions of blended learning in higher education andworkplace learning settings. In C. J. Bonk & C. R. Graham (Eds.), Handbook of blended learning: Global

perspectives, local designs (pp. 550–568). San Francisco, CA: Pfeiffer Publishing [Online] Available fromhttp://php.indiana.edu/~cjbonk/c083_bonk_future.pdf.

Bowers, C. A. (2000). Let them eat data: How computers affect education, cultural diversity, and the prospects of

ecological sustainability. Athens, Georgia: University of Georgia Pres.Cecez-Kecmanovic, D., & Webb, C. (2000). A critical inquiry into Web-mediated collaborative learning. In A.

Aggarwal (Ed.), Web-based learning and teaching technologies: Opportunities and challenges (pp. 307–326).Hershey, PA: Idea Group.

Clark, D. J. M. (1998). Developing, integrating, and sharing Web-based resources for materials education.Journal of the Minerals, Metals and Materials Society, 50(5) [Online] Available from<http://www.tms.org/pubs/journals/JOM/9805/Clark>.

Clark, R. C. (1998). Recycling knowledge with learning objects. Training and Development, 52(10), 60–63.Cristea, A. (2003). Adaptive patterns in authoring of educational adaptive hypermedia. Educational Technology &

Society, 6(4), 1–5 [Online] Available fromhttp://ifets.ieee.org/periodical/6_4/1.pdf.Downes, S. (2002). Design and reusability of learning objects in an academic context: A new economy of

education? eLearning: una sfida per l’universita. Moncton, Canada. [Online] Available from http://www.downes.ca/files/milan.doc.

Duval, E. (2001). Standardized metadata for education: a status report. In P. Kommers & G. Richards (Eds.),Proceedings of world conference on educational multimedia, hypermedia and telecommunications 2001

(pp. 458–463). Chesapeake, VA: AACE.Edmonds, R., & Barron, T. (2002). Learning objects in practice. New York, NY: SRI Consulting Business

Intelligence.Faro, A., Giordano, D., & Mineo, S. (2005). Reusable learning objects in a distributed content management

system for a project-based pedagogy. International Journal of Learning Technology, 1(4), 443–457.Fosnot, C. T. (1996). Constructivism: Theory, perspectives, and practice. New York, NY: Teachers College Press.Greene, K. C., & Gilbert, S. W. (1989). Great expectations: Content, communications, productivity, and the role

information technology in higher education. Change, 27(2), 8–18.Halpern, D. F. (1996). Thought & knowledge: An introduction to critical thinking (3rd ed.). Mahwah, NJ:

Lawrence Erlbaum.Hong, H., & Kinshuk (2004). Adaptation to student learning styles in Web based educational systems. In P.

Kommers & G. Richards (Eds.), Proceedings of world conference on educational multimedia, hypermedia and

telecommunications 2004 (pp. 491–496). Chesapeake VA: AACE.Hord, S. W. L., Huling-Austin, L., & Hall, G. E. (1987). Taking charge of change. Alexandria, VA: Association

for Supervision and Curriculum Development.IEEE (2002). Learning object metadata. from <http://ltsc.ieee.org/wg12> Retrieved 23.06.2002.Ip, A., & Mornson, I. (2001). Learning objects in different pedagogical paradigms. In G. Kennedy, M. Keppell, C.

McNaught, & T. Petrovic (Eds.), Meeting at the crossroads, Proceedings of the 18th annual conference of the

Australian Society for Computers in Learning in Tertiary Education (pp. 289–298). Melbourne: BiomedicalMultimedia Unit, the University of Melbourne <http://www.ascilite.org.au/conferences/melbourne01/pdf/papers/ipa.pdf> , Retrieved 7.11.2005.

Jonassen, D. H. (1999). Designing constructivist learning environments. In C. M. Reigeluth (Ed.). Instructional

design theories and models: A new paradigm of instructional theory (Vol. 2, pp. 215–239). Mahwah, NJ:Lawrence Erlbaum.

Jonassen, D. H. (2000). Computers as mindtools for schools: Engaging critical thinking. New Jersey: Prentice-Hall.Kemery, E. R. (2000). Developing online collaboration. In A. Aggarwal (Ed.), Web-based learning and teaching

technologies: Opportunities and challenges (pp. 227–245). Hershey, PA: Idea Group.Kemp, J. E., Morrison, G. R., & Ross, S. M. (1994). Designing effective instruction. New York, NY: Macmillan

College Publishing.Levy, D. A. (1997). Tools of critical thinking: Metathoughts for psychology. Needham Heights, MA: Ally and

Bacon.Magoulas, G., Papanikolaou, K., & Grigoriadou, M. (2003). Adaptive Web-based learning: accommodating

individual differences through system’s adaptation. British Journal of Educational Technology, 34(4) [Online]Available fromhttp://www.dcs.bbk.ac.uk/~gmagoulas/bjet.pdf.

http://php.indiana.edu/~cjbonk/c083_bonk_future.pdf

http://php.indiana.edu/~cjbonk/c083_bonk_future.pdf

http://www.tms.org/pubs/journals/JOM/9805/Clark

http://www.tms.org/pubs/journals/JOM/9805/Clark

http://ifets.ieee.org/periodical/6_4/1.pdf

http://www.downes.ca/files/milan.doc

http://www.downes.ca/files/milan.doc

http://ltsc.ieee.org/wg12

http://www.ascilite.org.au/conferences/melbourne01/pdf/papers/ipa.pdf

http://www.ascilite.org.au/conferences/melbourne01/pdf/papers/ipa.pdf

http://www.dcs.bbk.ac.uk/~gmagoulas/bjet.pdf


McGreal, R., & Roberts, T. (2001). A primer on metadata for learning objects: Fostering an interoperableenvironment. E-Learning, 2(10), 26–29.

Merrill, D. (2000). Knowledge objects and mental models. In D. Wiley (Ed.), The instructional use of learningobjects. [Online] Available from http://cito.byuh.edu/merrill/text/papers/KOMM.PDF.

Moore, P. E., & Tait, A. (Eds.). (2002). Open and distance learning: Trends, policy and strategy considerations.Paris: UNESCO.

Nesbit, J. C., Belfer, K., & Vargo, J. (2002). A convergent participation model for evaluation of LOs. Canadian

Journal of Learning and Technology, 28(3), 105–120.Nichols, M. (2003). A theory for eLearning. Educational Technology & Society, 6(2), 1–10 [Online] Available

fromhttp://ifets.ieee.org/periodical/6-2/1.html.Norton, P., & Wiburg, K. (1998). Teaching with technology. Fort Worth, Texas: Harcourt-Brace.Oakes, K. (2002). An objective view of learning objects. American Society for Training and Development, 56(5),

103–105.Palloff, R. M., & Pratt, K. (1999). Building learning communities in cyberspace: Effective strategies for the online

classroom. San Francisco, CA: Jossey-Bass.Patel, A., & Kinshuk (1997). Intelligent tutoring tools in a computer integrated learning environment for

introductory numeric disciplines. Innovations in Education and Training International Journal, 34(3), 200–207.Patton, M. Q. (1990). Qualitative evaluation and research methods. Newbury Park, London: Sage Publishing..Paul, R. (1995). Critical thinking: How to prepare students for a rapidly changing world. Santa Rosa, CA:

Foundation for Critical Thinking.Picciano, A. G. (2001). Distance learning: Making connections across virtual space and time. Columbus, OH:

Merrill Prentice-Hill.Polsani, P. R. (2003). Use and abuse of reusable learning objects. Journal of Digital Information, 3(4). Article No.

164.Preece, J., Rogers, Y., Sharp, H., Benyon, D., Holland, S., & Carey, T. (1994). Human–Computer Interaction.

Harlow, England: Addison-Wesley.Quinn, C., & Hobbs, S. (2000). Learning objects and instruction components. Educational Technology and

Society, 3(2) [Online] Available fromhttp://ifets.ieee.org/periodical/vol_2_2000/discuss_summary_0200.pdf.Resta, P. (Ed.). (2002). Information and communication technologies in teacher education: A planning guide. Paris:

UNESCO.Robbins, H. A., & Finley, M. (1995). Why teams do not work: What went wrong and how to make it right.

Princeton, NJ: Peterson’s/Pacesetter.Rudinow, J., & Barry, V. E. (1994). Invitation to critical thinking. New York, NY: Harcourt Braee College

Publishers.Rumetshofer, H., & Woß, W. (2003). XML-based adaptation framework for psychological-driven e-learning

systems. Educational Technology & Society, 6(4), 18–29 [Online] Available fromhttp://ifets.ieee.org/periodical/6_3/4.pdf.

Santally, M., Govinda, M., & Senteni, A. (2004). Reusable learning objects aggregation for e-learning coursewaredevelopment at the University of Mauritius. Instructional Technology and Distance Learning, 1(7), 27–34[Online] Available fromhttp://www.itdl.org/Journal/Jul_04/article02.htm.

Sicilia, M. A., & Garcıa-Barriocanal, E. (2003). On the concepts of usability and reusability of learning objects.International Review of Research in Open and Distance Learning, 4(2) [Online] Available fromhttp://www.irrodl.org/index.php/irrodl/article/view/155/236.

Vockell, E., & Deusen, R. M. (1989). The computer and higher-order thinking skills. Watsonville, CA: McGraw-Hill.

Wiggins, G., & McTighe, J. (1998). Understanding by design. Collumbus, OH: Merril Prentice-Hall.Wiley, D. A. (2000). Connecting learning objects to instructional design theory: A definition, a metaphor, and a

taxonomy. In D. A. Wiley (Ed.). The instructional use of learning objects. Online Version. <http://reusability.org/read/chapters/wiley.doc> Retrieved 23.12.2005.

Williams, D. D. (2000). Evaluation of learning objects and instruction using learning objects. In D. A. Wiley(Ed.), The instructional use of learning objects (pp. 1–32). <http://www.reusability.org/read/chapters/williams.doc> Retrieved 08.12.2005.

Wolf, C. (2002). iWeaver: towards an interactive web-based adaptive learning environment to address individuallearning styles. European Journal of Distance Learning (EuroDL) [Online] Available fromhttp://www.eur-odl.org/materials/contrib/2002/2HTML/iWeaver .

http://cito.byuh.edu/merrill/text/papers/KOMM.PDF

http://ifets.ieee.org/periodical/6-2/1.html

http://ifets.ieee.org/periodical/vol_2_2000/discuss_summary_0200.pdf



http://www.itdl.org/Journal/Jul_04/article02.htm

http://www.irrodl.org/index.php/irrodl/article/view/155/236

http://www.irrodl.org/index.php/irrodl/article/view/155/236

http://reusability.org/read/chapters/wiley.doc

http://reusability.org/read/chapters/wiley.doc

http://www.reusability.org/read/chapters/williams.doc

http://www.reusability.org/read/chapters/williams.doc

http://www.eurodl.org/materials/contrib/2002/2HTML/iWeaver



Wylie, D. A. (2001). The instructional use of learning objects. EURODL 2002. [Online]. Available from http://www.eurodl.org/materials/contrib/2002/2HTML/iWeaver.

Yin, R. (1994). Case study research: design and methods. Thousand Oaks, CA: Sage Publishing.

Gulsun Kurubacak is an assistant professor in distance education at the College of Open Education at AnadoluUniversity, Turkey. She undertook graduate studies at Anadolu University, Turkey (MA. EducationalTechnology) and the University of Cincinnati, USA (Ed.D. Curriculum & Instruction), and also has worked apost-doctoral fellow at the College of Education at New Mexico State University, USA (2001–2002). She spentthe 20 years focusing on the democratic and multicultural aspects of distance online communications, finding newanswers, viewpoints and explanations for complex online communicational problems through critical pedagogy,and improving learner critical thinking skills through project-based online learning (PBOL) via onlineasynchronous and synchronous communications.



Documents

Building knowledge networks through project-based online learning: A study of developing critical thinking skills via reusable learning objects