Computer Supported Collaborative Learning in a Clerkship: 

An Exploratory Study on the Relation of Discussion Activity and Revision of Critical Appraisal Papers

Willem J.M. Koops1,2, Cees P.M. van der Vleuten2, Bas A. de Leng2 , Luc H.E.H. Snoeckx3

1 Department of Medical Education, Máxima Medical Center, Veldhoven, the Netherlands

2 Department of Educational Development and Research,3 Department of physiology,

Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands

Department 2

Critical Appraisal

"The process of assessing and interpreting evidence (usually by published research)

by systematically considering its validity, results and relevance to the individual's work"

[Manley et.al. 2009; Rhodes et.al. 2007; Feltovich et.al. 1989]

Department 3

Critical Appraisal of a Topic (CAT)

Students’ Task: 1) investigation for research articles regarding the clinical problem

2) critical appraisal of the selected research articles

3) presentation of a relevant conclusion regarding the clinical problem

[Hyde et.al. 2000; Parkes et.al. 2009]

Department 4

1. more task-focussed activity in discussion

2. more discussion activity on the content of critical appraisal topics

Objective of Present Study

Department 5

Study Design

16 discussion groups

structured asynchronous CSCL discussion

unrevised CAT papers

47 CAT papers

revised CAT papers

Department 6

Structured Discussion

Critical Appraisal Topics Topic Elements

(1) Literature searchPreparation for executing the literature search

Strategy of the literature search

Results of the literature search

(2) Appraisal of the selected article(s)  

Study design

Study method

Study outcome

(3) ConclusionEvidence table of appraised article(s)

Relevant conclusion regarding clinical problem

Department 7

Content-Analysis of Students´ Activity

(Baker et.al. 2007)

Department 8

  analysis units ofrevised paper discussions

(n=24)

analysis units of unrevised paper

discussions (n=23)

frequency frequencyCollaborative problem solving activities (total of category 1.-7.)

1028 554

     Outside activity Category 1.

29 26

     Non-task-focussed activity(total of category 2. & 3.)

239 143

Category 2. Social Relation 189 96Category 3. Interaction Management 50 47     Task-focussed activity(total of category 4.-7.)

760 385

Category 4. Task Management 64 28Category 5. Opinions 481 269Category 6. Argumentation 186 80Category 7. Broaden and Deepen 29 8

Discussion Activity

Department 9

  analysis units ofrevised paper

discussions (n=24)

analysis units of unrevised paper

discussions (n=23)

median (min.-max.) median (min.-max.)Collaborative problem solving activities(total of category 1.-7.)

36 (15-94) 20 (1-53)

     Outside activity Category 1.

1 (0-10) 1 (0-4)

     Non-task-focussed activity(total of category 2. & 3.)

8 (0-31) 6 (0-15)

Category 2. Social Relation 6 (0-26) 3 (0-11)Category 3. Interaction Management 2 (0-19) 1 (0-7)     Task-focussed activity(total of category 4.-7.)

29 (11-65) 15 (0-45)

Category 4. Task Management 2 (0-19) 1 (0-7)Category 5. Opinions 18 (10-40) 10 (0-30)Category 6. Argumentation 8 (0-19) 4 (0-9)Category 7. Broaden and Deepen 0 (0-5) 0 (0-5)

Discussion Activity

Department 10

  analysis units ofrevised paper

discussions (n=24)

analysis units of unrevised paper

discussions (n=23)

Mann-Whitney U test

median (min.-max.) median (min.-max.) p-valueCollaborative problem solving activities(total of category 1.-7.)

36 (15-94) 20 (1-53) P<.001

       Outside activity Category 1.

1 (0-10) 1 (0-4) P<.716

       Non-task-focussed activity(total of category 2. & 3.)

8 (0-31) 6 (0-15) P<.179

Category 2. Social Relation 6 (0-26) 3 (0-11) P<.071Category 3. Interaction Management 2 (0-19) 1 (0-7) P<.610       Task-focussed activity(total of category 4.-7.)

29 (11-65) 15 (0-45) P<.000

Category 4. Task Management 2 (0-19) 1 (0-7) P<.107Category 5. Opinions 18 (10-40) 10 (0-30) P<.000Category 6. Argumentation 8 (0-19) 4 (0-9) P<.005Category 7. Broaden and Deepen 0 (0-5) 0 (0-5) P<.016

Discussion Activity

Department 11

Discussed Critical Appraisal Topics

  

analysis units of revised paperdiscussions

(n=24)

analysis units ofunrevised paper

discussions (n=23)

Critical appraisal topics of discussion median (min-max) median (min-max)

(1) Literature search regarding the clinical problem 10 (3-30) 8 (0-22)

(2) Appraisal of the selected article(s) 

10 (1-33) 5 (0-15)

(3) Relevant conclusion regarding the clinical problem 7 (0-13) 2 (0-9)

Department 12

Discussed Critical Appraisal Topics

  

analysis units of revised paperdiscussions

(n=24)

analysis units ofunrevised paper

discussions (n=23)

Mann-WhitneyU test

Critical appraisal topics of discussion median (min-max) median (min-max) P-value

(1) Literature search regarding the clinical problem 10 (3-30) 8 (0-22) P< .070 

(2) Appraisal of the selected article(s) 

10 (1-33) 5 (0-15) P<.007 

(3) Relevant conclusion regarding the clinical problem 7 (0-13) 2 (0-9) P<.000

Department 13

Discussed Elements of Critical Appraisal Topics

Critical Appraisal

TopicsTopic Elements

CAT task elements identified in: Chi-squarerevised paper

discussionsN=24

unrevised paperdiscussions

N=23 P-value

(1) Preparation for executing literature search 15 5 P<.005

Strategy of the literature search 24 12 P<.000

Results of the literature search 11 7 P<.278

(2) Study design 15 0 P<.000

Study population 24 11 P<.000

Study outcome 4 0 P<.041

(3) Evidence table of appraised research 6 2 P<.137

Relevant clinical conclusion regarding clinical problem 18 6 P<.001

Department 14

Discussion and Conclusion

extensive social and task-focussed discussion

significant levels of knowledge construction

Consistent with former CSCL research in classroom

Department 15

Discussion and Conclusion

Extensive discussed critical appraisal topics

high discussed elements of critical appraisal topics

former studies on individual CAT task: less elementspresent study: extended with discussion

Department 16

Limitations

• 1 control of variables

• 2 content-analysis

• 3 small sample

Department 17

Further Research

• Affect of interaction on knowledge construction

• Design of task and environment as a motivational framework to

support students discussion activity

Department 18

Conclusions

• A Computer Supported Collaborative Learning environment can support medical students in critically appraising clinical problems encountered during learning in the workplace.

• An increase in activity during the discussions seems to be related to more task-focussed activities and more discussion of critical appraisal topics and its elements.

Department 19

Department 20