Embed Size (px)
Citation preview
Institute for Learning Innovationand Development
Fair shares for all?The role of peer feedbackin group assignments
Adam [email protected]
2
TEAMMATES
https://teammatesv4.appspot.com Instructor account requires Google
account
3
The aim is to encourage and develop good team skills
4
Formative feedbackon teamwork criteria
5
Peer feedback on contribution to group project
6
Rewarding contributionsbut the average remains the
same
63%
51% 68% 63% 70%
+7+0+5-12
7
MANG1017 Business Skills First year, second semester option 200 students Assessment: group project report 12 supervision classes (16-20 students) 50 teams of 4 students?
8
First formative feedback After two weeks – 154/202 (76%)
Participation in class discussions
Communication outside class
Organising and planning work
Being helpful to other team members
0 10 20 30 40 50 60 70 80 90 100
65.5
52.7
62.1
59.9
31.7
40.2
34.9
39.0
2.7
7.0
3.1
1.2
Needs improvement Good Excellent
9
Second formative feedback After six weeks – 145/202 (72%)
Strongly Agree
Agree
Disagree
Strongly Disagree
0 10 20 30 40 50 60 70 80 90 100
51
42
5
3
51
39
7
2
54
38
5
3
This student makes a significant contribution to the class discussions
This student is making a significant contribution to the leadership or organisation of the project
This student is making a significant contribution to the project's deliverables.
10
Summative feedback Immediately after submission of report Anonymous 20% penalty for non-completion (9
students)
Please divide 100 points between all members of your team in proportion to their overall contribution to the work.
Group project report80%
Individual & collective research 20%Clarity of analytical thought 20%Specific recommendations 20%
Referencing and presentation 20%
Attendance and contribution to in-class discussion:tutor will award up to 15%
Using TEAMMATES to give feedback 3 times = 5%
40%
The individual grade for each report will be scaled by the feedback from the other members of each group.
Grade calculations
60%
20% penalty for anyone who
didn’t submit the summative peer
feedback
12
How to distribute 100 points Suppose your group has four members, including yourself.
If you thought that everyone did an equal share of the work then you would award 25 points each.
Suppose you thought that yourself and one other person put in more effort and gave yourself an extra 10 for a total of 35, then the points might be 35, 35, 15 and 15 – but is that fair? Did you really contribute more than twice as much as someone who only got 15?
How about 30, 30, 20, 20? Does that seem fairer? Did the other strong team member actually do a bit more than you?
So 33, 27, 20, 20? Did the two weaker team members do the same?
How about 33, 27, 22, 18? If you have awarded someone more than twice or less than half
their equal share of the points, please give your reasons
17
Impact of peer feedback on score
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 1100
5
10
15
20
25
30
35
40
45
Scores by mark band
raw project scores individual modified scores
Average 57.6%Median 60.0%SD 16.7
18
Impact of peer feedback on score
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 1100
5
10
15
20
25
30
35
40
45
Scores by mark band
raw project scores individual modified scores
Average 57.6%Median 60.0%SD 16.7
Average 57.8%Median 61.6%SD 21.2
19
Impact of peer feedback on grades
Zero Fail 3rd 2/2 2/1 First0
10
20
30
40
50
60
70
Scores by grade
raw project scores individual modified scores
20
Were students fair?Students awarding themselves:
Average
more than 10 points more 10 13.15-10 points more 29 7.4up to 5 points more 103 2.6the same 17 0.0up to 5 points less 38 -0.85-10 points less 0more than 10 points less 0Within 3 points of average 103 55%
2 of the 50 groups awarded equal shares
21
Dealing with unfairness One student awarded herself 40 and her
three teammates 20 each Her teammates all awarded her 20 or less We adjusted her points to equal shares:
25 points each
22
Groups of one Group of two – e.g. assignment score 57%– Student A = 120, student B = 80– Individual grades are 61% and 52%
Group of one – e.g. assignment score 57%– No reward for challenge of solo project
We decided to use a factor of 133– So this student’s grade is 65%
23
Next steps Student evaluation survey:– Did they think their grades were fair?– Did they act on the formative feedback?– Their thoughts on assigning 100 points
Further data analysis and correlation Presentation at ALT-C in September Fine-tune process for next year!
24
TEAMMATES doing its job!
Student to tutor:Hi, I've given everyone an equal 20 in my group but I fear other will favour themselves and give themselves an extra 5-10, and I will get less marks. This system favours people being slightly biased to themselves, enough so that they don't get picked up on. I fear people won’t be as fair as I have been.
25
TEAMMATES doing its job!
Tutor to student:With feedback comes a level of anxiety for everyone. The system we are using spots patterns in groups so strong gaming is identified.We of course also look at the feedback from class tutors as well as feedback from students so there are checks and balances in the system.
26
TEAMMATES doing its job!
His feedback scores:[student 1] = 20 (12.0) – weak student[student 2] = 4 (29.2) – team with 4[self score] = 20 (11.6)[student 4] = 4 (29.0) – team with 2[student 5] = 10 (18.2)
27
TEAMMATES doing its job!
Student 2 feedback comment:He showed up to one meeting outside class, and in class was always late and distracting from the group work. His part of the project was entirely rewritten by myself and [student] as it was incoherent and had not answered the question.
28
TEAMMATES doing its job!
Student 4 feedback comment:He showed up to one meeting and his submission had to be totally rewritten as it was incoherent and had not addressed the question at hand, he did not attend the group session to help right this, he showed no effort what so ever.
