Exploring the Lack of Dialogue in Computer-Supported Collaborative Learning

Mark Guzdial and Karen Carroll

With David Jaggie

Collaborative Software Lab

College of Computing/GVU

Georgia Institute of Technology

http://coweb.cc.gatech.edu/csl

Guzdial & Carroll 2

Story

We have theories and mechanisms for how people learn collaboratively: Dialogue

But they don’t fit what we measure in actual CSCL practice Far too little dialogue

Yet, we do measure individual learning Where is it coming from?

Some potential answers: Vicarious learning, mix of media and experiences

Guzdial & Carroll 3

Where Collaborative Learning Comes From

Jeremy Roschelle’s Convergent Conceptual Change

Heisawn Jeong’s Shared Understanding Learning results from and through

dialogue

Guzdial & Carroll 4

Convergent Conceptual Change

Jeremy observed cycles of: Posing hypotheses,

Testing the hypotheses, and then

Discussing the results of the test and the formation of the next set of hypotheses

Guzdial & Carroll 5

Example

D: But what I don’t understand is how the lengthening, the positioning of arrow

C: Ooh you know what I think it is? It’s like the line. Fat arrow is the line of where it pulls that down. Like see how that makes this dotted line. That was the black arrow. It pulls it.

D: You’re saying this [dotted line] is the black arrow C: Yeah D: And I pull it the other arrow [points to vel with mouse

cursor] like C: like on its hinge. It pulls the other arrow on the hinge

down to the tip of the black arrow

Guzdial & Carroll 6

How much dialogue is there?

49 dialogue turns (from paper) in 30-45 minutes (Roschelle, personal communication)

Heisawn Jeong’s results are not dissimilar Also emphasize significant individual

contributions and turn-taking

Guzdial & Carroll 7

What we see in CSCL:The “Dirty Little Secret”

In voluntary usage, less than 1.0 note per student per week, with rarely more than a single turn per thread CaMILE and Newsgroups: Average 0.4 notes per student per

week (Guzdial, 1998) Average length of a thread: 2.2 notes Averaging over 400 newsgroups, 2.1 notes per thread (Terveen

and Hill, 1998)

CSILE: 2.69 notes per thread (Hewitt & Teplovs, 1999) CoNote: 0.80 notes per student per week (Davis &

Huttenlocher, 1995 AnswerGarden: 0.10 notes per student per week (Ackerman,

1994)

Guzdial & Carroll 8

But Learning IS Happening!

Three studies with similar interaction patterns, but with externally measurable individual learning results. ReCoNote (Miyake & Masukawa, 2000): 0.58 notes

per student per week SpeakEasy (Hoadley & Linn, 2000): 1.3 notes per

student per week CoWeb (Rick, Guzdial, Carroll, Holloway-Attaway, &

Walker, 2002): 2.22 notes per student per week

Guzdial & Carroll 9

How is it happening?

What are the processes of meaning construction (Koschmann, 2002), if not the kinds of dialogue that Roschelle and Jeong describe?

Guzdial & Carroll 10

Some Hypothesized Mechanisms

(From Expected to Emergent)

Vicarious learning (Lee et al., 1999; McKendree et al., 1998): “I feel that others are speaking for me, and I learn from watching their dialogue.”

Dialogue served as catalyst for external reflection.

Dialogue incited or interacted with classroom (or other external) discussion.

Dialogue is reflecting external discussion.

Guzdial & Carroll 11

Study Design

Using log files, identified users who Had posted to a CSCL system (CoWeb), so they

knew the mechanics, Visited a page, But chose not to post.

We replicated and printed the discussion as it existed at the time of not posting.

We presented it to the student and ask them why didn’t they post.

Guzdial & Carroll 12

Interview Guide

It looks as if you read comments on this page, but didn't post. Can you remember why?

Did you ever go with something to say and find someone else had already posted it? Did you ever come up with something to say, go away to think about it, and return to find someone else had said it?

When you read comments which sparked something you wanted to say, did you generally post it right away or go away and think about it for a while first?

Did you ever think of something to say and then bring it up later in class instead of posting?

When you went to the CoWeb, did you usually go with something to say, or were you just browsing through other people's comments?

Guzdial & Carroll 13

Two Sets of Interviews

With English students, where we had measured learning (Carroll) These are the quotes that are in the paper

With CS students this last semester (Jaggie) No evidence of learning per se, but looking

for similar mechanisms

Guzdial & Carroll 14

Evidence of Vicarious LearningK: Apparently, you looked at this page. My impression from looking

at the CoWeb is that you posted pretty frequently. But this is one where you didn’t post—where you looked at it and you didn’t write anything. Do you remember this?

Student P: Yeah, yeah, I remember this.K: Do you have any idea why?Student P: Oh, yeah, I saw this part, and I was going to answer it,

but, I don’t know, I,… before I could answer, somebody already came up with the answer, StudentC or StudentJ came up with the answer.

K: Did you ever find that you had a question you wanted to ask, and then somebody else had asked it already?

Student P: A couple of times, yeah. Yeah. I guess, even if I did have a question, I mean, the questions were there, but it just didn’t come out, somebody had already worded it, so I’m like oh yeah, I wanted to-I was wondering about this, too.

Guzdial & Carroll 15

More on Vicarious Learning

K: Did you ever go with an idea of something you wanted to say and find that someone else had already posted it?

Student Z: mm-hmmm. Several times. … I probably would have had some interesting discussions myself if I had, but like, I like when Student P, how he, I like reading what he writes, too, ‘cause it’s very interesting to me, ‘cause he thinks sorta like I do; like he likes to have facts to base things up; I understand how he thinks, and like a lot of people in the class, a few of them have the same views and a few of them don’t and I like, it just interests me, you know, kind of, I like how people think.

Guzdial & Carroll 16

More than Vicarious Learning:Vicariously Resolving Conflict

David: How would you use the CoWeb to study for the midterm?

Student 3: I looked at the question, and then I try to figure out the questions, and I looked at the responses.

David: Did the responses generally match what you came up with?

Student 3: Sometimes yes, sometimes no, depending on whether I was right or wrong, or whether they were right or wrong. Or both.

David: What would you do it there's a discrepancy?

Student 3: Hmm...if there's a discrepancy between 2 answers? Just wait and see what happens.

David: Just hang back and see who wins?

Student 3: Yeah.

Guzdial & Carroll 17

Learning Benefit without Posting:Dialogue as Catalyst for Reflection

K: Somebody said what you were going to say?Student P: Right, like, I found stuff on this, I mean I came

here, I saw this and I decided to do some research, so I went to some other web page and I looked it up and I found out stuff, I was actually ready to post, I come back, and somebody had already done it…Like I used this as kind of, you know, something that would help me think about a particular topic and a particular direction, like when you just read a book, it’s really difficult to pick a subject and go in a particular direction, like if somebody thinks about something and gives a question, it’s easier to channel your efforts towards a particular thing.

Guzdial & Carroll 18

Research as DialogueIncited by CSCL

David: Ok. Is there ever a time you saw something on the CoWeb, and had a question about it, and then went off and did your own thing—looked it up, did some...

Student 4: A lot of times it's faster, if it's a really dumb question--really pretty simple, that I should have known--or that a lot of people will know, then instead of posting it...there's always a bunch of groups working on the same project, the same program, at any given time, so I'll just find someone else I know and ask them if they know it. And then, if I ask a couple people and they don't know it, then I realize it may not be that easy of a question, so then I'll go ahead and post it.

Guzdial & Carroll 19

Interaction between Class and External Discussion

K: If you could just look at this and see if you can remember when you looked at it and why you didn’t post anything.

Student Z: Trying to remember…the majority of the time, when I looked at a page I was also trying to get other ideas from what the class was doing. See how the class was responding to things. ‘cause usually from that, the class would pick up on attitudes of the teacher, and stuff like that; that way I can gear myself towards the teacher, ‘cause every teacher you write to is different, had completely different writing style than I did in high school, and sometimes when I can go through, I can pick up other ideas and other views of things that are brought up in class, ‘cause I think this was on like the open forum, and from that I can kinda understand what other people were thinking, and possibly get an idea of where the teacher wants us to go with things.

Guzdial & Carroll 20

Posting in Response to a Group (External) Discussion

David: When we were setting up this interview, you said you didn't always post under your name. You'd post under your group's name.

Student 4: Well, a lot of times, we're working on the group, working on the program...the way we split up the tasks, it's never like "One person do this, one person do this," it's usually "two people work on this part, two people work on this big part." If they want to split it up between that, then whatever. The work is usually two people...at least two people at the same time. So when we post…I don't know if it just felt uncomfortable, or like I'm kind of taking the credit for the question...usually we just post under the group name so that everybody gets equal credit.

Guzdial & Carroll 21

Summary:Early Hypothesis Generation

The most significant point of this paper: We don’t know the mechanism of learning that’s going on in actual CSCL practice.

Our studies’ seek to generate and gather evidence for hypotheses for these mechanisms. There is probably more than one.

We do not have evidence that any of these mechanisms do lead to learning. We only offer evidence that these

mechanisms/strategies are occurring.