UCL Arena Exchange Seminar Improving assessment and feedback scores

UCL Arena Exchange Seminar

Improving assessment and feedback scores

Improving assessment and feedback scores

Andrew Gardner,Senior Lecturer in the Archaeology of the Roman Empire

Departmental Tutor, UCL Institute of Archaeology

Rachel Rees, Programme Tutor for BSc Speech Sciences 2009-13 (followed by Christina Smith)

Programme Tutor for MSc Speech and Language Sciences 2013-present

Division of Psychology and Language Sciences

Assessment and feedback scores, UCL and the sector 2015

UCLSector

avg RG avgAssessment and feedback

61 73 69

5. The criteria used in marking have been clear in advance. 63 77 72

6. Assessment arrangements and marking have been fair. 74 78 77

7. Feedback on my work has been prompt.58 70 67

8. I have received detailed comments on my work.55 73 65

9. Feedback on my work has helped me clarify things I did not understand. 56 68 63

Archaeology Sector avg

5. The criteria used in marking have been clear in advance.

79 77

6. Assessment arrangements and marking have been fair.

92 78

7. Feedback on my work has been prompt. 65 70

8. I have received detailed comments on my work.

73 73

9. Feedback on my work has helped me clarify things I did not understand.

73 68

Archaeology and the sector

Speech Sciences

Sector avg

5. The criteria used in marking have been clear in advance.

92 77

6. Assessment arrangements and marking have been fair.

81 78

7. Feedback on my work has been prompt. 75 70

8. I have received detailed comments on my work.

92 73

9. Feedback on my work has helped me clarify things I did not understand.

64 68

Speech Sciences and the sector

Andrew Gardner

Departmental TutorSenior Lecturer in the Archaeology of the Roman EmpireUCL Institute of Archaeology

Assessment & Feedback in context

Assessment & Feedback in context

BSc Arch BA Arch BA Arch & Anth

5. The criteria used in marking have been clear in advance.

80 84 70

6. Assessment arrangements and marking have been fair.

90 95 80

7. Feedback on my work has been prompt.

80 68 60

8. I have received detailed comments on my work.

90 84 70

9. Feedback on my work has helped me clarify things I did not understand.

90 74 80

ARCLG_SHS response rate = 77% BSc Arch response rate = 83% BA Arch response rate = 83% BA Arch & Anth response rate = 71%

Assessment & Feedback mechanisms

Assessment & Feedback mechanisms

Assessment & Feedback mechanisms

Student Progress

“I would also like to comment on the outstanding system of monitoring student progress […]. This system of student progress monitoring is relevant for all of the taught programmes in the Institute of Archaeology. I was also provided with the template used to monitor student progress, the UNDERGRADUATE STUDENT PROGRESS FORM. This system is a model of excellent practice that should be flagged to UCL. The activities and methods of monitoring have been meticulously planned and provide fantastic opportunities for students to engage in critical self-reflection, discuss performance with a member of staff and for year tutors to carefully monitor progression of students through the degree programme. I have no doubt that it is these kinds of activities that not only nurture student progression but are reflected in the attainment levels on modules and in the NSS results.”

External Examiner 2015

These mechanisms support feedback which is:

• Timely

• Detailed

• Constructive

• Contextualised

Rachel Rees Programme Tutor for BSc Speech Sciences 2009-13 (followed by Christina Smith)Programme Tutor for MSc Speech and Language Sciences 2013-presentDivision of Psychology and Language Sciences

BSc Speech Sciences

Number Statement % Agreeing

5 The criteria used in marking have been clear in advance.

92

8 I have received detailed comments on my work. 92

9 Feedback on my work has helped me clarify things I did not understand.

64

BSc Speech Sciences: OPINIO surveys for individual modules for SAME cohort Yrs 2-4

“I received constructive feedback on my coursework.”

Rated on a 5 point scale:

Strongly Agree

Agree

Neither Agree nor Disagree

Disagree

Strongly Disagree

How did responses vary ACROSS modules in terms of the %s students that agreed?

“I received constructive feedback on my coursework.”Year of Module % Strongly Agree

or Agree

4 94

4 93

3 92

2 81

3 62

2 35

2 32

3 29

2 24

3 14

What is surprising about these figures?

What could explain it?

What have students said?

• Feedback is “too vague”, “too broad”.

• “It’s a huge boost to just to be told ‘the way you did X was really good’. ”

What have we learnt?

• Be specific

• Be positive and be specific about those remarks too

5

What have students said?

“okay, you didn’t do blah, you didn’t consider whatever,” and we get told, ‘this is what you didn’t do,’ but not what to do to fix the problem. ‘So, this is the problem with your coursework,’ but there’s no kind of action plan or suggestions.”“ As opposed to saying, ‘oh, you didn’t do enough with this,’ more kind of,…. ‘in order to do it next time, you could think about X, Y and Z.”“I want to know how to get a better grade!”

“I also think that one of the major skills that was taught us in the course is self-reflection, so, to give ourselves feedback.”

What have we learnt?

Provide specific suggestions for how to improve

Provide feedback that can be turned into an internal checklist

Hattie and Timperley (2007):

Effective learners have self-regulation strategies. These are cognitive routines that they can draw on while preparing work.

If feedback (from others) could more easily be converted into self-regulation strategies there should be more carryover to future coursework.

Examples of feedback that ticks the boxes

Rather than:

“Good answer to first viva question”

“Q1: You considered how factors for prioritisation (e.g. severity, type of communication difficulty, practical resources) can interact and gave a clear practical example of this (relating to the need for intensive therapy for severe speech disorders).

• Positive remark

• Specific

Examples of feedback that ticks the boxes

Rather than:

“You are clearly confused about the X speech processing model”

“If you revise the difference between stored motor programs and motor execution, you could improve the third hypotheses, which is currently inaccurate”

• Specific• More positive

tone• Suggestion for

action to take to improve

Examples of feedback that ticks the boxes

Rather than:

“The explanation of your intervention programme was patchy”

“You teased out most of the clinical decisions you made but remember to provide clear rationale for each one”

• Specific• Suggestion

that could be converted to internal checklist.

Link between internal checklist and marking criteria?

A+• Each clinical decision is explicitly described and

backed up with rationale

Marking Criteria

• Specific• Tried and tested (if possible, marker completes

the coursework and notes what makes a good piece of work)

• Can be also use as an “internal checklist”

Any questions?

Useful resources

• Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81–112.