Upload
brantknutzen7021
View
217
Download
0
Embed Size (px)
Citation preview
8/7/2019 Achieving Trans Active CMC Brant Knutzen ICEL 2010 v2
http://slidepdf.com/reader/full/achieving-trans-active-cmc-brant-knutzen-icel-2010-v2 1/22
8/7/2019 Achieving Trans Active CMC Brant Knutzen ICEL 2010 v2
http://slidepdf.com/reader/full/achieving-trans-active-cmc-brant-knutzen-icel-2010-v2 2/22
Abstract
A common failing of e-learning instructional design is when online discussion learning activities
devolve to become just a “public assignment”: each student posts once and no peer interaction is
achieved. This study shows how instructors can successfully design an e-learning environment to
increase student engagement and achieve transactive communication, where a discussion develops
as participants respond to teach other: giving feedback, taking positions, and bringing in evidence to
socially negotiate new knowledge.
Despite the widespread recognition of transactive communication as a goal of Computer-Mediated
Conferencing (CMC), online discussions often merely consist of individual answers to a question
posed, without any interaction (or even recognition) of the work of other students. A graphical
depiction of this learning activity would look like a "lawn", with short individual blades of grass -- no
branching, no vertical building, none of the "bushiness" of a highly transactive discussion. This paper
develops new quantitative and qualitative ways to measure discussion transactivity, and examines the
instructional design that supports and encourages this type of learning activity.
This paper examines a case study of CMC transactivity at a university using 1-to-1 computer access(each student has a computer on their desk) in a blended teaching environment in Hong Kong.
Four variables were examined for their effect on successful student engagement in online discussions:
Teacher facilitation of social grouping of students
Class time to initiate online discussion interaction
Setting open-ended, challenging topic questions that encourage discussion and debate
Assessment system that reinforces production and peer interaction
The students in the case study used the Moodle e-learning environment, and English as the medium
of instruction. The study took place in a tertiary institution with graduate students in the faculty of
Education, using a blend of face-to-face and e-learning instructional methods. When the instructionalenvironment was set up to support all four discussion design conditions, production and interactivity
both increased dramatically and transactive CMC was achieved.
The results of this study indicate that in order to achieve highly productive and transactive online
discussions in a blended 1-to-1 teaching environment, instructors should incorporate a specific set of
design conditions. When these instructional design conditions are successfully incorporated, the
potential benefits of social constructivism as an instructional design paradigm can be realized within a
blended educational environment.
Key terms : transactivity, social negotiation, discourse, educational dialogue
8/7/2019 Achieving Trans Active CMC Brant Knutzen ICEL 2010 v2
http://slidepdf.com/reader/full/achieving-trans-active-cmc-brant-knutzen-icel-2010-v2 3/22
1.0 Introduction
There is widespread recognition among academic researchers of the value of collaborative groups in
the effective construction of knowledge. This efficacy has been found to be linked to the process that
learners utilize in working on the task together (Fischer et al 2002), and the ways that they socially
negotiate arguments and argument sequences (Leitão 2000; Voss & Dyke 2001). Biggs captured the
educational value of discussion when he stated: "Good dialogue elicits those activities that shape,
elaborate, and deepen understanding" (1999 p. 5).
The final ingredient in this recipe for effective knowledge construction is the method by which they
build on the contributions of their fellow learners, which forms the basis for our working definition of
transactivity (Berkowitz & Gibbs 1983).
Studies in this domain define a transactive discussion as a learning process which involves
understanding the ideas of learning partners, comparing their ideas to one's own, then socially
negotiating a common ground upon which a solution is collaboratively built (Teasley 1997). This
development of a shared understanding is dependent upon the reaction to the ideas of others through
elaboration, questioning, and taking opposing positions (Azmitia & Montgomery 1993).
If we accept that transactivity is a key theoretical construct for measuring collaboration, how can we
describe it in easily grasped graphical and quantitative ways? Once we can measure it, how can we
formulate the instructional design conditions which consistently result in more productive learning
activities?
This paper develops a simple method for measuring transactivity, and examines a case study which
demonstrates how to develop highly productive online discourse activities.
1.1 Transactivity
One aspect of transactivity is strictly objective and quantitative, requiring little judgment on the part of
researchers: discourse productivity. In this study we defined the term production to be the average
number of posts per discussion participant. This term is purely quantitative, as it merely requires
counting the number of posts in a discussion, and dividing the total by the number of participants n :
Production = Total number of discussion posts / n
Researchers in the domain of educational psychology and Computer Supported Collaborative
Learning have developed an extensive categorization of transactive contributions in discourse (Joshi
& Rose 2007). Berkowitz & Gibbs (1983) termed each contribution a transact , and some examples
from their coding scheme are listed in Table 1:
Table 1. Types of transacts and their concrete examples from Berkowitz and Gibbs (1983)
Transact Example
Feedback Request Do you understand or agree with my position?
Dyad Paraphrase Here’s a paraphrase of a shared position.
Competitive
Juxtaposition
I will make a concession to your point, but also reaffirm part
of my position.
Extension Here’s a further thought or an elaboration offered in the
spirit of your position.
8/7/2019 Achieving Trans Active CMC Brant Knutzen ICEL 2010 v2
http://slidepdf.com/reader/full/achieving-trans-active-cmc-brant-knutzen-icel-2010-v2 4/22
Because this study does not seek to identify the key segments of discourse which predict a learning
outcome, we chose to lump all contributions to a discussion which fit any transactive category in this
coding scheme into a simple category: feedback . This coarse level of discourse coding granularity
permits all "dialogue acts" (Stolcke et al 2000) which are judged to be a response to the ideas of a
collaborative learning partner to be simply categorized into a single type. Any responsive transact
was therefore coded as feedback in this study, and the term interactivity was defined as the meannumber of feedback posts per participant:
Interactivity = Total number of feedback posts / n
While the initial basis of a transactive discussion is a quantity of posts in the context, it is only in the
considered response to learning partners that collaboration can take place. While you can have
production without interactivity, you cannot have interactivity without production. Therefore we chose
to define transactivity as the product of quantity and responsiveness, or Production times Interactivity:
Transactivity = Production × Interactivity
An additional way to quantitatively describe a discussion is to measure the level of group activity. The
same way an oral discussion with only one person in it cannot have interactivity, the interactivity level
of an online discussion can be measured by the average number of posts per discussion topic. This
measures the level of group activity:
Group Activity = Total number of discussion posts / number of topics
1.2 Prior Research
In prior research the principal researcher conducted a case study on the CMC production of
adolescent students in a 1-to-1 computer access environment using the Moodle LMS at an English-
medium international secondary school in Hong Kong (Knutzen 2010). In that study evidence was
found that four instructional design conditions had to be met in order to create a learning environment
where transactive online discourse would occur:
Teacher facilitation of social grouping of students
Class time to initiate online discussion interaction
Setting open-ended, challenging topic questions that encourage discussion and debate
Assessment system that reinforces production and peer interaction
When these design conditions were met, student engagement increased dramatically and the
discourse production increased by an order of magnitude, from an average of 1 per student to over 10.
1.3 The current study
The purpose of the current study is to extend this prior research on the instructional design of online
discussions in a blended learning environment. Based on our experience and previous research in
the field, we predicted that when the four instructional design variables met the conditions developed
in the prior study, student engagement and discussion production in the current study would increase
dramatically. Furthermore, we predicted that further data collection and analysis would reveal a
radical improvement in other indicators of a transactive discussion, such as interactivity and group
activity.
Our predictions can be summarized in the two hypotheses:
H1. Redesign of instructional design variables to meet the four conditions specified would
have a significant positive effect on production in the resulting online discussions.
8/7/2019 Achieving Trans Active CMC Brant Knutzen ICEL 2010 v2
http://slidepdf.com/reader/full/achieving-trans-active-cmc-brant-knutzen-icel-2010-v2 5/22
H2. Redesign of instructional design variables to meet the four conditions specified would
have a significant positive effect on transactivity in the resulting online discussions.
To facilitate this examination of the impact of the instructional redesign, the researchers built on
previous research in this field to develop simple quantitative definitions of production, interactivity, and
transactivity. In order to create a graph which visually illustrates the comparison between the student
work products in the two classes, the researchers developed a new information visualization
technique for displaying production and interactivity in an online discussion.
The development of this new graphical technique to represent contributions to a discussion forum
combines the bush-like illustration of social networking and concept sharing in the FlowerGarden data
visualizations of Lantin & Judelmann (2006), with the vertical user paths of the PeopleFlowers data
portraits of Xiong & Donath (1999). By using elements of each the researchers hoped to better
represent the interactivity component which is key to the present analysis of transactivity in a
discussion.
The researchers developed an information visualization (see example in Figure 1) where each
participant in a discussion topic is represented along a horizontal axis, and the height of the graphdenotes the time since the discussion began. Each post along that user timeline is indicated by an
icon which represents the participant, and the interaction is clearly delineated by an arrow which
points from the feedback post back to the post to which it is responding. This new type of data portrait
clearly illustrates the "bushiness" of a highly transactive discussion, and is thus named the
BushGraph.
Figure 1. An example BushGraph illustrating the dynamics of a transactive online discussion.
8/7/2019 Achieving Trans Active CMC Brant Knutzen ICEL 2010 v2
http://slidepdf.com/reader/full/achieving-trans-active-cmc-brant-knutzen-icel-2010-v2 6/22
2.0 Method
2.1 Sample
This case study documents the instructional redesign of an existing blended post-graduate course
titled "eLearning Strategies and Management" at a university in Hong Kong. The "before re-design"
condition was taught by the second author, it had 33 participants and is labeled Class #1, while the"after re-design" condition was taught by the first author, it had 20 participants is labeled Class #2.
2.2 Procedure
The two authors collaborated on the instructional design of the discussion component of Class #2,
and all other design variables were controlled as much as possible. Both classes were taught face-to-
face in the same room using the same blended learning environment: laptops arranged 1-to-1 with
seats around large tables which sat up to 12 students. Both classes used the Moodle LMS to support
the e-learning activities. The course content in both classes remained virtually the same: the lecture
materials, reading assignments, and discussion topic questions were carried over to the new course
design. The only intentional design difference was the modification of the online discussion activities
so that the design variables met the four conditions specified:
Teacher facilitation of social grouping of students
Class time to initiate online discussion interaction
Setting open-ended, challenging topic questions that encourage discussion and debate
Assessment system that reinforces production and peer interaction
2.2.1 Design Variable #1 - Teacher facilitation of social grouping of students
In the initial class session of Class #2, the first author encouraged the students to form small groups
of three to five students for the first oral discussion, and to move to sit close together to facilitate
conversation. This same self-selected grouping was then applied to the online discussions: one
student from each group would create a discussion topic (usually by copying the topic questions), and
then each student in the group would reply to that post in order to form a small group discussion
within the forum.
2.2.2 Design Variable 2 - Class time to initiate online discussion interaction
For each in-class discussion based on the previous lecture segment and the current topic questions,
class time was given for the groups to develop oral arguments, on average about 5 minutes. Next,
the online discussion with matching topic questions was opened on the LMS, and more class time
given (5 - 10 minutes) for the students to take that discussion online, and develop at least their initial
position statements on the topic at hand. The interaction is termed "semi-synchronous" because the
initial posts are put up in real-time, and often the students look around at their learning partner'sactivities and get a few response posts up before time is called for the class to move on to the next
lecture topic.
2.2.3 Design Variable 3 - Setting open-ended, challenging questions that encourage discussion and
debate
Most of material in the course could be easily adapted to create open-ended discussion topic
questions and sub-topics. These types of questions are excellent for basing a Question & Answer (Q
& A) discussion on, as the students can adopt various perspectives and argumentative positions.
Although the lecture materials, reading assignments, and discussion topic questions were all virtually
identical between Class #1 and Class #2, there was more effort made in Class #2 to recap the issue
8/7/2019 Achieving Trans Active CMC Brant Knutzen ICEL 2010 v2
http://slidepdf.com/reader/full/achieving-trans-active-cmc-brant-knutzen-icel-2010-v2 7/22
at hand: restate the topic questions, link to issues from the lecture material, and include thought-
provoking graphics which could be incorporated into the discussion.
2.2.4 Design Variable 4 - Assessment system that reinforces production and peer interaction
The students in Class #2 were trained in the same peer-marking participation-based marking system
developed in the 2007 study, where the initial Summary posts are rated by groupmates as a 6 (out of
10 possible), and every Q & A post thereafter is rated a 10. This marking system uses no subjective
evaluation of content, as the overall mark is completely based on the quantity of contributions to the
discussion, and the arithmetic mean calculated by Moodle and stored as the final grade for the
learning activity.
On every discussion forum this marking system was restated, specifying exactly what was expected
for each type of post, and how it was to be rated. In addition, the instructions specified that the
subject line of the post should indicate the type of post and the expected rating. This helped the
learning partners identify the first post from each participant and mark it with the specified starting
grade of 6.
For this study the discussion instructions requested that the students produce a fourth type of post:
Reflection. This type of contribution was designed to be the culmination of the socially negotiated
knowledge produced by the learning activity, answering the questions "What do you think about this
new material? Has the exposure to other people's opinions changed your viewpoint?". If the summary
post was the trunk of the tree of knowledge, and the Q & A posts were the branches, the Reflection
posts are meant to be the fruit, where the students think critically about what they have learned.
2.3 Measures
All of the student posts in the five examined discussions were coded as either a stand-alone
contribution, or as a feedback response to a previous post. The student work produced in the onlinediscussion learning activities were quantitatively analyzed to produce a data-based comparison of the
student work product between Class #1 and Class #2, where:
Production = Total number of discussion contributions / n
Interactivity = Total number of feedback posts / n
Group Activity = Total number of posts per discussion topic / n
Transactivity = Production × Interactivity
3.0 Results
3.1.1 Results - Discussion #1 (Class #1)
Table 2. Example of discussion #1 instructions from Class #1
Discussion instructions:
Post your comments here.
8/7/2019 Achieving Trans Active CMC Brant Knutzen ICEL 2010 v2
http://slidepdf.com/reader/full/achieving-trans-active-cmc-brant-knutzen-icel-2010-v2 8/22
The discussion instructions provided for Class #1 are extremely minimal. They do not restate thetopic questions from the lecture, link back to pertinent issues, provide graphics to illustrate the topic,or list sub-topic questions.
Table 3. Record of group activity produced by the students on discussion #1 (Class #1)
(Note: student names blocked to protect privacy)
The record of group activity for Class #1 on Discussion #1 primarily shows that there was no group
formation: each student posted in their own discussion topic, and only three posts were technically
replies. Because the learning activity instructions did not direct the formation of small groups, most
students (particularly in Asia) are very hesitant to post replies to another student, as this may be
interpreted as a confrontational challenge. Instead, each student will start their own discussion topic,
and often not read a single post from another student.
8/7/2019 Achieving Trans Active CMC Brant Knutzen ICEL 2010 v2
http://slidepdf.com/reader/full/achieving-trans-active-cmc-brant-knutzen-icel-2010-v2 9/22
The BushGraph in Figure 2 representing Discussion #1 activity by Class #1 is very flat, more of a
"lawn of grass" with individual posts rather than the "bushiness" characteristic of a transactive
discussion. Less than half of the students (0.42) participated in the discussion, and there was not a
single reply responding to another post! Even the three posts which are posted as replies do not
actually respond to the previous post in that discussion topic, but are stand-alone posts makingindependent statements, and were not coded as feedback.
Figure 2. The BushGraph representing discussion #1 (Class #1). Note: the three top-level posts shown are
not responses to the previous posts in the same topic discussion, so they were not coded as feedback.
8/7/2019 Achieving Trans Active CMC Brant Knutzen ICEL 2010 v2
http://slidepdf.com/reader/full/achieving-trans-active-cmc-brant-knutzen-icel-2010-v2 10/22
3.1.2 Results - Discussion #1 (Class #2)
Table 4 shows an example of the comprehensive instructions which Class #2 were given for all online
discussions. They specifically state what the students are expected to do, how much time they will be
given for oral discussion, and the topic questions from the lecture. In addition, the peer-marked
participation-based assessment system is methodically laid out in a series of steps, so the students
know what to do and what sort of grade they will get if they participate in the discussion. Also note the
deadline provided (two weeks from the start date), which is essential for successful moderation and
management of the learning activity.
Table 4. Example of the same discussion #1, but using the progressive peer-marking participation-based
system for Class #2 specified in the instructions. Note: the terms LMS (Learning Management System) and VLE
(Virtual Learning Environment) are interchangeable in this context.
Discussion Instructions:
VLE Comparison Discussion
As the students present their information about the selected VLEs, take notes.
Afterward, the class will have 5 - 10 minutes to discuss and compare them.
Address the following questions in your discussion:
1. What are the common factors that seem to be present in all VLEs?
.
2. Is one system obviously better than another system?
.
3. Who are the various VLEs designed for (administrators, teachers, or students?)
.
4. Are there any specific issues special to Hong Kong students?
--------------------------------------------------------------------------------
(This discussion is peer assessed using a 6 / 10 / 10 / 10 system)
Each table group should Create a Discussion Topic and post:
1. A summary of the material from the lecture, the resources, and their research online. (Subject line:
First post = 6) 2. Read other people's summaries and reflections. What questions do you have for them? (Subject
line: Q & A = 10) 3. Read the questions that have been posted to your group. Do some research, and construct your
answer to the question. (Subject line: Q & A = 10) 4. Reflections on summary posts. What do you think about this new material? Has the exposure to
other people's opinions changed your viewpoint? (Subject line: Reflection = 10)
Each student should put up a minimum of 4 posts. The rating grades will be averaged into a single
Participation grade for the forum.
The more you post, the better your grade, but be SURE to keep the quality upwith thoughtful posts.
Include pictures if you feel they illustrate the concept.
Note: participation on this discussion must be posted by Oct 12, 6:30pm (posts after that date cannot be
rated)
8/7/2019 Achieving Trans Active CMC Brant Knutzen ICEL 2010 v2
http://slidepdf.com/reader/full/achieving-trans-active-cmc-brant-knutzen-icel-2010-v2 11/22
Table 5 shows the record of group activity for Discussion #1 produced by Class #2, and the effect of
the procedure creating small self-selected groups based on physical seating proximity. Each small
group of 3 to 5 students forms a nexus around which a discussion focus develops. Groupmates are
recognized as learning partners, and the social barriers of responding to each other are reduced.
Table 5. Record of group activity produced by the students on discussion #1 (Class #2).
(Note: student names and pictures blocked to protect privacy)
The BushGraph in Figure 3 qualitatively illustrates the "bushiness" of a discussion which has achieved
transactivity. Reply posts which give feedback to previous posts are indicated with an arrow pointingback to the post they are responding to.
Figure 3. The BushGraph representing discussion #1 (Class #2)
8/7/2019 Achieving Trans Active CMC Brant Knutzen ICEL 2010 v2
http://slidepdf.com/reader/full/achieving-trans-active-cmc-brant-knutzen-icel-2010-v2 12/22
Table 6 shows the quantitative descriptive statistics comparing discussion #1 between the two
classes. The Class #2 discussion is far more productive and transactive, reflecting the higher student
motivation and engagement with this learning activity.
Table 6. Quantitative descriptive statistics comparing discussion #1 between classes (based on Figures 2 and 3)
3.2 Results - Discussion #2 (comparing Class #1 to Class #2)
Figures 3 and 4 show how the BushGraph provides a quick qualitative overview of the discussionactivity, and a sense of the transactivity achieved. Again, Class #2 achieves a much higher vertical
dimension, reflecting the layers of responsive feedback developed as the learning partners socially
negotiate an understanding of the topic. Two of the members of the ManU group found the
interactivity available in the "4 in Group" group to be more rewarding than their own group, and
actually posted more in that discussion topic than their own!
Figure 4. The BushGraph representing discussion #2 (Class #1)
Class #1 #2
Production 0.42 3.40
Interactivity 0.00 1.75
Group Activity 1.27 11.3
Transactivity 0.00 5.95
8/7/2019 Achieving Trans Active CMC Brant Knutzen ICEL 2010 v2
http://slidepdf.com/reader/full/achieving-trans-active-cmc-brant-knutzen-icel-2010-v2 13/22
Figure 5. The BushGraph representing discussion #2 (Class #2)
Table 7 quantifies the huge difference in student engagement with the discussion learning activity
between the two classes. The production for Class #2 is nearly 14 times higher than that of Class #1.Some of the difference in motivation levels is due to the fact that only Discussion #3 was assessed for
Class #1, while for Class #2 every discussion activity contributed to the overall grade.
Table 7. Quantitative descriptive statistics comparing discussion #2 between classes (based on Figures 4 and 5) Class #1 #2
Production 0.36 5.00
Interactivity 0.03 2.80
Group Activity 1.09 20.00
Transactivity 0.01 14.00
3.3 Results - Discussion #3 (comparing Class #1 to Class #2)
Figure 6 displays the BushGraph generated by the most productive discussion for Class #1:
Discussion #3, where they achieved an average of 2.03 posts per student. This jump in production is
undoubtedly due to the fact that this discussion was the only discussion learning activity worth points:
about 3% of the total course grade. This discussion posed some of the key issues of the entire e-
learning course, and there were no specific answers available in the extant literature, so the students
were required to post their own opinions. Where there is no established single correct answer, the
topic is open to debate. Even here the lack of student groupings limited the interactivity to 0.03, so
the overall transactivity score was only 0.01.
8/7/2019 Achieving Trans Active CMC Brant Knutzen ICEL 2010 v2
http://slidepdf.com/reader/full/achieving-trans-active-cmc-brant-knutzen-icel-2010-v2 14/22
Figure 6. The BushGraph representing discussion #3 (Class #1)
(Note: long horizontal graph broken and displayed in two parts)
Figure 7 shows the BushGraph for the learning activity of Class #2 generated by Discussion #3. This
was also the most productive discussion for Class #2, with an average of 5.56 posts per student. The
students responded to the challenging and open-ended topic, and at this point in the course (Session
6 of 10) they had grown comfortable with their learning parters. The 4G group in particular was highly
productive and interactive in their discussion topic, generating 45 replies. This group led the early
exploration and development of possible answers on this discussion, closely watched by the other
groups. The principal researcher also played an active moderator role, putting up 7 posts in their
discussion topic to praise work, answer questions, and highlight questionable logic.
8/7/2019 Achieving Trans Active CMC Brant Knutzen ICEL 2010 v2
http://slidepdf.com/reader/full/achieving-trans-active-cmc-brant-knutzen-icel-2010-v2 15/22
Figure 7. The BushGraph representing discussion #3 (Class #2)
Table 8 highlights the value of interactivity and group activity in producing a transactive discussion.
Even though Class #1 achieved 37% of the production of Class #2, their interactivity on this
discussion was only 10%, and only 6% of the group activity, compared to that of Class #2. The
resulting transactivity score for Class #1 was less than 4% that of Class #2.
Table 8. Quantitative descriptive statistics comparing discussion #3 between classes (based on Figures 6 and 7)
Class #1 #2
Production 2.03 5.56
Interactivity 0.42 4.15
Group Activity 1.26 22.60
Transactivity 0.85 23.07
8/7/2019 Achieving Trans Active CMC Brant Knutzen ICEL 2010 v2
http://slidepdf.com/reader/full/achieving-trans-active-cmc-brant-knutzen-icel-2010-v2 16/22
3.4 Results - Discussion #4 (comparing Class #1 to Class #2)
Figure 8 shows that Class #1 is starting to overcome the lack of teacher-supported groupings, and
develop their own learning partner groupings. This discussion generated the highest level of
interactivity for Class #1 (0.55 per student), perhaps due to the localization of the topic (about
technology use in Hong Kong) making the topic more approachable and interesting.
Figure 8. The BushGraph representing discussion #4 (Class #1).
Figure 9. The BushGraph representing discussion #4 (Class #2).
8/7/2019 Achieving Trans Active CMC Brant Knutzen ICEL 2010 v2
http://slidepdf.com/reader/full/achieving-trans-active-cmc-brant-knutzen-icel-2010-v2 17/22
Figure 9 shows that Class #2 also found this topic interesting and approachable, with a production
level of 6.16 per student. Group activity was very high as students felt comfortable and made multiple
posts to bring in outside resources to the discussion. Table 9 sums up the quantitative gap between
the Class #1 activity and Class #2 on Discussion #4, where the overall Transactivity score of Class #2
was 33 times that of Class #1!
Table 9. Quantitative descriptive statistics comparing discussion #4 between classes (based on Figures 8 and 9)
Class #1 #2
Production 1.18 6.16
Interactivity 0.55 3.45
Group Activity 1.95 24.60
Transactivity 0.65 21.22
3.5 Results - Discussion #5 (comparing Class #1 to Class #2)
Figure 10. The BushGraph representing discussion #5 (Class #1).
Figure 11. The BushGraph representing discussion #5 (Class #2).
8/7/2019 Achieving Trans Active CMC Brant Knutzen ICEL 2010 v2
http://slidepdf.com/reader/full/achieving-trans-active-cmc-brant-knutzen-icel-2010-v2 18/22
Figure 11 shows how the BushGraph yields valuable insight into the tremendous differences between
the group dynamics in the discussion topic groups: "4G" has a leader who tends to dominate, "5-3-2"
actually has very little interaction, "5 Girls" is primarily a conversation between two students, and the
"4 in Group" discussion is a wild melee with posters responding to a wide range of other participants.
Table 10. Quantitative descriptive statistics comparing discussion #2 between classes (based on Figures 10 and 11)
Class #1 #2
Production 0.55 6.25
Interactivity 0.00 3.55
Group Activity 1.00 25.00
Transactivity 0.00 22.19
4.0 Analysis
The visual representation of the BushGraphs clearly shows the qualitative difference between the
discussions of Class #1 and Class #2, on multiple dimensions. The vertical dimension is much higher
for all Class #2 graphs compared to Class #1, reflecting the higher productivity and group activity.
The proliferation of arrows in the Class #2 graphs reflects the much larger quantity of feedback posts
between users, which is the most direct measure of interactivity. The grouping of students results in a
series of discussion foci with three to five participants, which serves as the nexus around which forms
the "bush" of replies characteristic of a transactive discussion.
Table 11 summarizes the quantitative descriptive statistics for the five examined discussions. These
statistics state the case even more unequivocally than the BushGraphs: on all measures the
discussions of Class #2 score dramatically higher than Class #1. The transactivity scores are
exponentially higher for Class #2, reflecting the higher student motivation to produce work being
multiplied by the degree of interactivity in the discussions.
8/7/2019 Achieving Trans Active CMC Brant Knutzen ICEL 2010 v2
http://slidepdf.com/reader/full/achieving-trans-active-cmc-brant-knutzen-icel-2010-v2 19/22
Table 11. Summary of the quantitative data comparison between Class #1 and Class #2.
Class #1
Transactivity Score
Discussion # Production Interactivity Group Activity (Production × Interactivity)
1 0.42 0 1.27 0.002 0.36 0.03 1.09 0.01
3 2.03 0.42 1.26 0.85
4 1.18 0.55 1.95 0.65
5 0.55 0 1 0.00
Marg. Means 0.91 0.20 1.31 0.30
Class #2
Transactivity Score
Discussion # Production Interactivity Group Activity (Production × Interactivity)1 3.4 1.75 11.3 5.95
2 5 2.8 20 14.00
3 5.56 4.15 22.6 23.07
4 6.15 3.45 24.6 21.22
5 6.25 3.55 25 22.19
Marg. Means 5.27 3.14 20.70 17.29
Another issue that must be addressed with the peer-marking, participation-based rating of discussion
posts is the reliability and validity of this assessment method. How consistent are the gradesproduced by this method, and how well do they correlate with grades on teacher-assessed learning
activities?
Table 12 shows that the correlations, with few exceptions, are quite high between the gradesproduced by the five examined discussion learning activities. This indicates that the peer-markedparticipation-based discussion grading method has an acceptable level of test-retest reliability.
Table 12. Correlations between grades produced by discussions in Class #2.
Discussion # 1 2 3 4 5
1 0.387 0.637** 0.666** 0.667**
2 0.387 0.656** 0.308 0.542*
3 0.637** 0.656** 0.527* 0.836**
4 0.666** 0.308 . 0527* 0.578**
5 0.667** 0.542* 0.836** 0.578**
Marg. Means 0.589 0.476 0.664 0.520 0.656
* significant at the p < .05 level (2-tailed)** significant at the p < .01 level (2-tailed)
8/7/2019 Achieving Trans Active CMC Brant Knutzen ICEL 2010 v2
http://slidepdf.com/reader/full/achieving-trans-active-cmc-brant-knutzen-icel-2010-v2 20/22
We surmise that the variability in these correlations is due to the differences in the content being
examined in the discussion, and the topic questions employed. Different types of discussions appeal
to different types of students, which alters their learning activity behavior, and the resulting patterns of
grades produced. Another factor which may play a role in reliability is the varying student attendance
for the real-time in-class interaction.
The reflective journal assignment is a collection of all the reflection posts from the discussion forums,
which the student then re-examines and reflects on in the light of later learning and meta-cognition.
This essay assignment is teacher-assessed with a weight of 5% of the final course grade. The final
project is the design and implementation of an e-learning course by each student based on their
uniquely selected learning context. It is a coherent collection of online learning activities which are
designed to meet their stated course learning outcomes, and is teacher-assessed with a weight of 50
percent of the final course grade.
Table 13. Correlations between discussion grades and the teacher-assessed learning activities and course total
grades in Class #2.
Discussion # Reflective Journal Final Project Course Total
1 0.631* 0.608* 0.735*
2 0.791* 0.757* 0.856*
3 0.829* 0.919* 0.845*
4 0.615* 0.675* 0.622*
5 0.793* 0.885* 0.838*
Marginal Means 0.732 0.769 0.779
* significant at the p < .01 level (2-tailed)
As table 13 shows, the correlations between the peer-marked participation-based grades on the five
examined discussions and these teacher-assessed learning activities are very strong (range from
0.608 to 0.919), with every single one significant at the p < .01 level. This would indicate that the
grades produced by the peer-marked participation-based discussion forums are definitely measuring
the same attributes of student behavior as the teacher-assessed learning activities. The final project
is produced by a series of seven formative stages with feedback, and it is the culmination of all the
learning activities in the course. Because this assignment is thoroughly examined and scored by the
instructor on multiple scales based on an exacting rubric, the high correlation with the discussion
grades (marginal mean of .769) indicates a very high validity for interpreting this participation-based
grading method as an accurate measure of student performance.
Table 13 also shows a high degree of correlation between the individual discussion grades and the
course total grades (marginal mean of .779), each correlation significant at the p < .01 level. Because
the Class #2 curriculum had over 50 assessed learning activities, these correlations indicate that the
individual discussions have a very high validity for interpreting this participation-based grading method
as an accurate indicator of overall student performance in the course.
As a final analysis of the validity of the participation-based grading method, we investigated the
relationship between the teacher-assessed final project learning activity and the transactivity
characteristics of the discussions. Given that there is a strong correlation between the grades of the
five examined discussions and the project grades, the correlation between these coefficients and thedescriptive statistics of each discussion could prove illuminating.
8/7/2019 Achieving Trans Active CMC Brant Knutzen ICEL 2010 v2
http://slidepdf.com/reader/full/achieving-trans-active-cmc-brant-knutzen-icel-2010-v2 21/22
Table 14 shows the correlations between the grades correlation coefficients and the discussion
descriptive statistics. They range from .61 to .82, which indicate a very high level of correlation. We
interpret these correlations to indicate that as a discussion reaches higher levels of production,
interactivity, group activity, and the resulting transactivity, there tends to be a higher correlation
between the discussion grades and the project grades.
The most important factor would appear to be interactivity: as students respond more to each other in
a discussion, the resulting discussion grades correlate more strongly with the teacher-assessed
project grade.
Table 14. Correlations between discussion descriptive statistics and the relationship between project grades
and discussion grades
Discussion Descriptive Statistic Correlation between elements
Production 0.61
Interactivity 0.82
Group Activity 0.65
Transactivity 0.76
Given the assumption that a teacher-assessed activity (particularly a final project worth 50% of the
final grade) is highly accurate in measuring student performance, this 0.82 correlation would therefore
indicate that as interaction in a discussion goes up, the learning activity becomes more accurate in
predicting student performance.
5.0 Discussion
The primary aim of this study was to confirm and extend prior research on the effect of instructional
design on discussion transactivity. We also sought to extend Xiong's PeopleGarden (1999) data
portrait to better graphically illustrate discussion interactivity.
We did find evidence supporting our first hypothesis that the redesign of the instructional design
variables to meet the conditions developed in study #1 would have a significant positive effect on the
production in the resulting online discussions. In addition, we found evidence to support our second
hypothesis: the redesign had a significant positive effect on several measures of transactivity,
including interactivity, group activity, and transactivity as a calculated product of discussion productionand interactivity. The peer-marked participation-based scoring method was found to have high
reliability and validity, with strong correlations to teacher-assessed learning activities and the course
total grade.
There were some limitations to the study. The sample size is small and almost exclusively Asian
students. Replication and extension of this study with a larger sample size in other cultural settings
would provide additional perspectives with greater predictive validity. Replication would also address
a possible additional factor in the observed increase in discussion production and transactivity:
differences in instructor teaching style and emphasis on the importance of online discussions in a
blended learning environment.
Further directions for research in this area could seek to determine a causal direction: is the highly
interactive discussion environment conducive to the creation and demonstration of higher levels of
8/7/2019 Achieving Trans Active CMC Brant Knutzen ICEL 2010 v2
http://slidepdf.com/reader/full/achieving-trans-active-cmc-brant-knutzen-icel-2010-v2 22/22
understanding, or are students with higher levels of understanding more capable and willing to create
interactive discourse?
6.0 Conclusion
The results of this study indicate that in order to achieve highly productive and transactive online
discussions in a blended 1-to-1 teaching environment, instructors should incorporate a specific set ofdesign conditions:
Teacher facilitation of social grouping of students
Class time to initiate online discussion interaction
Setting open-ended, challenging topic questions that encourage discussion and debate
Assessment system that reinforces production and peer interaction
When these instructional design conditions are successfully incorporated, the potential benefits of
social constructivism as an instructional design paradigm can be realized within a blended educational
environment.
7.0 References
Azmitia, M. and Montgomery, R. (1993) "Friendship, transactive dialogues, and the development of
scientific reasoning", Social Development , Vol. 2, No. 3, pp 202-221.
Berkowitz, M. and Gibbs, J. (1983) "Measuring the developmental features of moral discussion",
Merrill-Palmer Quarterly, Vol. 29, pp 399-410.
Biggs, J. (1999) "What the Student Does: teaching for enhanced learning", Higher Education
Research & Development , Vol. 18, No. 1, pp 57.
Fischer, F., Bruhn, J., Gräsel, C., and Mandl, H. (2002) "Fostering collaborative knowledge
construction with visualization tools", Learning and Instruction, Vol. 12, pp 213-232.
Josh, M. and Rose, C. (2007) "Using Transactivity in Conversation for Summarization of Educational
Dialogue", Proceedings of the SLaTE Workshop on Speech and Language Technology in Education.
Knutzen, K.B. (2010) "Achieving Productive Computer-mediated Conferencing", Presented at the 3C
Teaching and Learning Innovation Symposium, Polytechnic University, Hong Kong
Lantin, M. and Judelman, G. (2006) "FlowerGarden: An Interactive Visualization of Concept-Sharing",
Proceedings of the CHI 2006 Conference on Human Factors in Computing Systems.
Leitão, S. (2000) "The potential of argument in knowledge building" Human Development, Vol. 43, pp
332-360.
Stolcke, A., Ries, K., Coccaro, N., Shriberg, J., Bates, R., Jurafsky, D., Taylor, P., Martin, R., Van
Ess-Dykema, C. and Meteer, M. (2000) "Dialogue act modeling for automatic tagging and recognition
of conversational speech", Computational Linguistics, Vol. 26, No. 3, pp 339 –373.
Teasley, S. D. (1997) "Talking about reasoning: How important is the peer in peer collaboration?", In:
L.B. Resnick, R. Säljö, C. Pontecorvo and B. Burge, Editors, Discourse, tools and reasoning: Essays on
situated cognition, Springer, Berlin (1997), pp 361 –384.
Voss, J.F. and Van Dyke, J.A. (2001), "Argumentation in Psychology", Discourse Processes, Vol. 32,
No. 2/3, pp 89-111.
Xiong, R. and Donath, J. (1999), "PeopleGarden: creating data portraits for users", Proceedings of the
12th annual ACM symposium on User interface software and technology.