Upload
scot-chambers
View
214
Download
0
Embed Size (px)
Citation preview
UCL Arena Exchange Seminar
Improving assessment and feedback scores
Improving assessment and feedback scores
Andrew Gardner,Senior Lecturer in the Archaeology of the Roman Empire
Departmental Tutor, UCL Institute of Archaeology
Rachel Rees, Programme Tutor for BSc Speech Sciences 2009-13 (followed by Christina Smith)
Programme Tutor for MSc Speech and Language Sciences 2013-present
Division of Psychology and Language Sciences
Assessment and feedback scores, UCL and the sector 2015
UCLSector
avg RG avgAssessment and feedback
61 73 69
5. The criteria used in marking have been clear in advance. 63 77 72
6. Assessment arrangements and marking have been fair. 74 78 77
7. Feedback on my work has been prompt.58 70 67
8. I have received detailed comments on my work.55 73 65
9. Feedback on my work has helped me clarify things I did not understand. 56 68 63
Archaeology Sector avg
5. The criteria used in marking have been clear in advance.
79 77
6. Assessment arrangements and marking have been fair.
92 78
7. Feedback on my work has been prompt. 65 70
8. I have received detailed comments on my work.
73 73
9. Feedback on my work has helped me clarify things I did not understand.
73 68
Archaeology and the sector
Speech Sciences
Sector avg
5. The criteria used in marking have been clear in advance.
92 77
6. Assessment arrangements and marking have been fair.
81 78
7. Feedback on my work has been prompt. 75 70
8. I have received detailed comments on my work.
92 73
9. Feedback on my work has helped me clarify things I did not understand.
64 68
Speech Sciences and the sector
Andrew Gardner
Departmental TutorSenior Lecturer in the Archaeology of the Roman EmpireUCL Institute of Archaeology
Assessment & Feedback in context
Assessment & Feedback in context
BSc Arch BA Arch BA Arch & Anth
5. The criteria used in marking have been clear in advance.
80 84 70
6. Assessment arrangements and marking have been fair.
90 95 80
7. Feedback on my work has been prompt.
80 68 60
8. I have received detailed comments on my work.
90 84 70
9. Feedback on my work has helped me clarify things I did not understand.
90 74 80
ARCLG_SHS response rate = 77% BSc Arch response rate = 83% BA Arch response rate = 83% BA Arch & Anth response rate = 71%
Assessment & Feedback mechanisms
Assessment & Feedback mechanisms
Assessment & Feedback mechanisms
Student Progress
“I would also like to comment on the outstanding system of monitoring student progress […]. This system of student progress monitoring is relevant for all of the taught programmes in the Institute of Archaeology. I was also provided with the template used to monitor student progress, the UNDERGRADUATE STUDENT PROGRESS FORM. This system is a model of excellent practice that should be flagged to UCL. The activities and methods of monitoring have been meticulously planned and provide fantastic opportunities for students to engage in critical self-reflection, discuss performance with a member of staff and for year tutors to carefully monitor progression of students through the degree programme. I have no doubt that it is these kinds of activities that not only nurture student progression but are reflected in the attainment levels on modules and in the NSS results.”
External Examiner 2015
These mechanisms support feedback which is:
• Timely
• Detailed
• Constructive
• Contextualised
Rachel Rees Programme Tutor for BSc Speech Sciences 2009-13 (followed by Christina Smith)Programme Tutor for MSc Speech and Language Sciences 2013-presentDivision of Psychology and Language Sciences
BSc Speech Sciences
Number Statement % Agreeing
5 The criteria used in marking have been clear in advance.
92
8 I have received detailed comments on my work. 92
9 Feedback on my work has helped me clarify things I did not understand.
64
BSc Speech Sciences: OPINIO surveys for individual modules for SAME cohort Yrs 2-4
“I received constructive feedback on my coursework.”
Rated on a 5 point scale:
Strongly Agree
Agree
Neither Agree nor Disagree
Disagree
Strongly Disagree
How did responses vary ACROSS modules in terms of the %s students that agreed?
“I received constructive feedback on my coursework.”Year of Module % Strongly Agree
or Agree
4 94
4 93
3 92
2 81
3 62
2 35
2 32
3 29
2 24
3 14
What is surprising about these figures?
What could explain it?
What have students said?
• Feedback is “too vague”, “too broad”.
• “It’s a huge boost to just to be told ‘the way you did X was really good’. ”
What have we learnt?
• Be specific
• Be positive and be specific about those remarks too
5
What have students said?
“okay, you didn’t do blah, you didn’t consider whatever,” and we get told, ‘this is what you didn’t do,’ but not what to do to fix the problem. ‘So, this is the problem with your coursework,’ but there’s no kind of action plan or suggestions.”“ As opposed to saying, ‘oh, you didn’t do enough with this,’ more kind of,…. ‘in order to do it next time, you could think about X, Y and Z.”“I want to know how to get a better grade!”
“I also think that one of the major skills that was taught us in the course is self-reflection, so, to give ourselves feedback.”
What have we learnt?
Provide specific suggestions for how to improve
Provide feedback that can be turned into an internal checklist
Hattie and Timperley (2007):
Effective learners have self-regulation strategies. These are cognitive routines that they can draw on while preparing work.
If feedback (from others) could more easily be converted into self-regulation strategies there should be more carryover to future coursework.
Examples of feedback that ticks the boxes
Rather than:
“Good answer to first viva question”
“Q1: You considered how factors for prioritisation (e.g. severity, type of communication difficulty, practical resources) can interact and gave a clear practical example of this (relating to the need for intensive therapy for severe speech disorders).
• Positive remark
• Specific
Examples of feedback that ticks the boxes
Rather than:
“You are clearly confused about the X speech processing model”
“If you revise the difference between stored motor programs and motor execution, you could improve the third hypotheses, which is currently inaccurate”
• Specific• More positive
tone• Suggestion for
action to take to improve
Examples of feedback that ticks the boxes
Rather than:
“The explanation of your intervention programme was patchy”
“You teased out most of the clinical decisions you made but remember to provide clear rationale for each one”
• Specific• Suggestion
that could be converted to internal checklist.
Link between internal checklist and marking criteria?
A+• Each clinical decision is explicitly described and
backed up with rationale
Marking Criteria
• Specific• Tried and tested (if possible, marker completes
the coursework and notes what makes a good piece of work)
• Can be also use as an “internal checklist”
Any questions?
Useful resources
• Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81–112.