Peer Feedback in Academic Writing using Google Docs2012-08-30

• Björn Hedin• KTH Royal Institute of Technology• Media Technology• Stockholm, Sweden• bjornh@kth.se

Annotation and Social Annotation

• Annotations: - “marks on (or attached to) readingmatter” including

underlining, highlighting and comments.• Different from Notes

- “marks that are not co-located with the text to which they refer”.

• Plenty of research indicate that notes and annotations are very important for learning.

• Social annotation tools add a collaboration aspect to annotations, which allows users to - share,- interact and - collaborate on annotations attached to documents.

Social Annotation Tools

• Demo- Diigo

- Google Docs/Drive

- Crocodoc

What We Have Done

• Started with peer feedback in academic writing in 2004- Either comments in separate threads or problematic

commenting in word documents• B.Sc. Thesis in 2010 investigated using SA tools instead,

promising results• 2011 – Tests in 2 groups, good results• 2012 – Google Docs much improved, full scale test• Investigated

- What are the students’ (and teachers’) attitudes to using Google Docs for Peer Feedback?

- What are the student’s attitudes to the use of screen vs. paper when reading documents?

- Which effects can recent and upcoming technical developments have on the above two questions?

Material

• B.Sc. Degree project course • Group supervision

- Meet 6 times and give written and oral feedback to each other on progress of report.

- Written in pairs. 74 students produced 37 final reports (exjobb).

- About 8-10 students per groups meaning 4-5 reports.- Hand in report about a week before seminar and use SA

tool (google docs) for Peer Feedback Marking.- Discuss on seminars, often using projector.

• Evaluated using questionnarie (60% response rate), focus groups and interviews with teachers.

• Demo

Results: Using Google Docs for Writing

• 11% of students prefer word processor over Google Docs when writing individual reports

• 89% prefer using Google Docs when writing group- or pair reports

• Main problem seen with Google Docs was possibilities to format exactly as you want. Also better dictionaries.

• Most students used google docs for writing and in the final version imported into MS Word for final layout.

Reading on Screen vs Paper

“If given a pdf-file to read before a seminar, how often do you read it on screen compared to print-

ing it on paper.”

0%

5%

10%

15%

20%

25%

30%

35%

40%

100%80%60%240%20%0%

Why Screen and Why Paper?

• Why screen?- Simpler to read it on a screen than to print it- Environmental aspects- Economical aspects- The ability to search in the texts- Easier to organize files than paper

• Why Paper?- Most mentioned a better reading experience, especially

for longer texts.- 8 of 26 mentioned better possibilities to annotate the text

But…

"How easy is it to comment using Google Docs?"

0%

10%

20%

30%

40%

50%

60%

Very EasyQuite EasyModerately difficult/easyQuite DifficultVery Difficult

Usefulness of feedbackUsefulness of getting written peer feedback, written supervisor feedback and oral feedback (scale 1-5, 5 is best)

Series10

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Written Peer Feedback (σ = 0.86)Written Supervisor Feedback (σ = 0.87)Oral Feedback (σ = 1.10

Usefulness of feedback (2)

• 4 student thought the written group feedback was more useful than the written supervisor feedback,

• 21 students thought the written supervisor feedback was better than the written group feedback,

• 13 students thought they were equally useful.

Comments in Document or in Separate Thread?

"“Is it best to comment on reports directly the document (as in Google Docs) or in a separate threaded fo-

rum?"

0%

20%

40%

60%

80%

Clearly Google DocsGoogle DocsUndecidedSeparate ThreadCleary Separate Thread

Problems?

• Not really many problems found, but maybe…- Pdf files doesn’t work well with Google Docs- Perhaps more focus on details

Benefits

• Peer Feedback Marking in general a very good peer learning activity (see research by Nancy Falchikov)- Student get insights in other theses and problems- Useful to give and receive feedback

• Comments in context• Increased interaction among the students in giving

feedback.• Simple matters (spelling and so on) easily addressed • Seminar time can be spent with well-prepared

students• Students really read and comment texts

- Difficult to get away with not reading the texts• So simple that even teachers want to use it

Future and Conclusions

• Increased use of tablets like iPad- Even easier to read and annotate than on computers

• Constantly better cloud-based word processing with annotation features

• SA tools can be used for much more!• Students and teachers were all very enthusiastic, so

it’s worth giving a try!