What makes good feedback good?
Prof. Margaret Price
Director, ASKe Pedagogy Research Centre Faculty of Business
http://www.brookes.ac.uk/aske
Dynamic Assessment Context
Assessment
Massification
Assessment diversity
Holistic approach to assessment
Assessment
drives learningTransparency and
accountability
Modularity
Purpose of Assessment – various stakeholders
Fragmentation
of communities
Target culture
Authentic assessment
Uniform Quality Assurance and accreditation systems
(Bologna 1999)
Employability outcomes
Cultural diversity and internationalisation of curriculum
Academic Literacies
National Student Survey
League
tables
Student
engagement
Change in HE and effects on assessment
Pedagogic developments (e.g. acknowledgement of assessment as key driver of learning, dialogic feedback; discourse of assessment – fairness, cheating, grade inflation; complexity of assessment)
Measurement in HE (e.g.league tables, quality processes and measures, fragmentation).
Student voice (fees, influence of prior education, student engagement)
Market (employability, graduate jobs, authenticity in assessment)
There has been quite a bit of research on feedback…
Most focuses on the student experience of feedback • Don’t read it (Hounsell,1987; Gibbs and Simpson, 2002)• Too vague (Higgins, 2000; Walker, 2009)• Not understood (Lea and Street, 1998; Weaver 2006;
Sadler 2010)• Subject to interpretation (Ridsdale, 2003; Orsmond and
Merry, 2011)• Unidirectional (Nicol 2010)• Seen as a self contained event (Carless 2011)• Damages self-efficacy (Wotjas, 1998) • Seen as not ‘relational’ to the student (Price et al. 2010)
The aim of the project:
to understand better the domains that influence whether feedback is viewed as ‘good’ or ‘bad’ by students (and in NSS scores).
What makes good feedback good?
An ASKe Pedagogy Research Centre project in collaboration with Cardiff, funded by the HEA
Domains pertaining to the feedback itself (the traditional focus of advice to improve feedback)
Technical factors – legibility and detail – if the feedback is not legible or able to be interpreted (incomprehensible ticks/remarks) then students were dissatisfied with the feedback
Particularity of feedback – evidence of engagement with particular piece of work, personalisation valued over ‘standardised criterion-based feedback
Recognition of student effort – evidence of time spent by markers as well as supportive detail.
Domains relating to the context of feedback
Assessment design - clarity of purpose, relevance, realistic (e.g word count, time)
Feedback pre-conditions – clear criteria, task, instructions… dialogue.
Marker predictability – particularly if the marker is not the person who had briefed them or provided formative feedback.
Sites of Silence Timing Design of assessment patterns
Domains pertaining to the development & expectations of the student
Student mark expectations – some influence but not overwhelming. Effort often equated with marks and influences the type of feedback seen as useful.
Student epistemology, resilience and beliefs –dualistic students and model answers and specificity; –intrinsic (learning) and extrinsic (mark) motivations.–poor self evalaution and reliance on feedback–criticism, critique, transaction or conversation
Domains pertaining to the development & expectations of the student
Student mark expectations – some influence but not overwhelming. Effort often equated with marks and influences the type of feedback seen as useful.
Student epistemology, resilience and beliefs –dualistic students and model answers and specificity; –intrinsic (learning) and extrinsic (mark) motivations.–poor self evalaution and reliance on feedback–criticism, critique, transaction or conversation
Overview
Domains overlap, compensate each other, have strong interrelationships, are not mutually exclusive.
Domains of influence are not causal, prioritised, they are context dependent and influences outside the ‘feedback’ artefact are important.
Student perceptions of feedback are shaped by:• Some aspects of the feedback itself – a necessary but insufficient
condition for feedback being seen as good
• Pre-feedback conditions
• Qualities and perspectives of the student
What can we do?Feedback itself
Technical factors Particularity of feedback Recognition of student effort
Time – there are no economies of scale in assessment
Staff training, development and support
Limit anonymous feedback
What can we do? The context of feedback
Assessment design–https://www.plymouth.ac.uk/uploads/production/document/path/2/2559/assignment_brief_design_guidelines.pdf
–Team approach to design
Feedback pre-conditions–Situate feedback in on-going dialogue.
(consider class sizes, availability of staff, allocation of class time to briefing and discussion)
Marker predictability–Avoid contracting out marking–Clarity about purpose and expectations of assessment to avoid mismatch between formative and final feedback.
What can we do? The development & expectations of the student
Student mark expectations–Develop students’ self evaluative abilities
Student epistemology, resilience and beliefs –Develop students’ assessment literacy
Assessment Literacy
appreciation of assessment’s relationship to learning;
conceptual understanding of assessment
understanding of the nature, meaning and level of assessment criteria and standards;
skills in self- and peer assessment;
familiarity with technical approaches to assessment
possession of the intellectual ability to select and apply appropriate approaches and techniques to assessed tasks
(Price et al, 2012)
Intentional development of assessment literacy
What can we do?
1.Planning at programme level
2.Pre-assessment
3.Assessment activity
4.Post assessment (Feedback)
5.Beyond the programme
16
Take a programme view
• Staff team need a programme view Where there is a greater sense of the holistic programme, students are more likely to achieve the learning outcomes than students on programmes with a more fragmented sense of the programme.
(Havnes 2007)
•Develop assessment strategy including intentional development of assessment literacy at key points in programme
Active engagement with feedback
Explicit Criteria
Completion and submission of work
Students Active engagement with criteria
Rust C.,O’Donovan B. & Price M. (2005)
Developing assessment literacy
Developing assessment literacy: pre assessment
• Students need to
• know how to negotiate the assignment task
• understand expectations
• learn self efficacy to enable independence
Active engagement with feedback
Explicit Criteria
Completion and submission of work
Students Active engagement with criteria
Rust C.,O’Donovan B. & Price M. (2005)
Developing assessment literacy - Pre assessment
Assessment standards are difficult
Assessment judgements rely on local, contextualised interpretations of quality underpinned by tacit understanding of ‘quality’ shared by members of an assessment community
(Knight, 2006)
A key issue in assessment is that students often do not understand what is a better piece of work and do not understand what is being asked of them particularly in terms of standards and criteria.
(O’Donovan et al., 2001)
Active engagement with feedback
Explicit Criteria
Completion and submission of work
Students Active engagement with criteria
Rust C.,O’Donovan B. & Price M. (2005)
Developing assessment literacy - Assessment activity
Assessment activity
Students must learn to and identify their performance gap for themselves
‘in the act of production itself’ (Sadler, 1989, p121).
Self assessment
Drafting and redrafting
Peer review
Peer assessment
Peer assisted learning
Active engagement with feedback
Explicit Criteria
Completion and submission of work
Students Active engagement with criteria
Rust C.,O’Donovan B. & Price M. (2005)
Developing assessment literacy: post assessment
Beyond the programme: Assessment literacy and community
Students•Self evaluative ability, independent learning and employability (Boud, 2009)
•Confidence and capability to participate
•Useful evaluation of experience (Price et al 2010)
Staff•Assessment confidence (Handley et al,2013)
•Acknowledging professional judgement •Discourse of assessment and feedback dialogue (Price, 2005)
Summary
Look beyond the feedback artefact itself. An overemphasis on technical factors at the expense of contextual elements such as good teacher student relationships can be detrimental.
Develop student assessment literacyIndependent learners will cope better with the imprecise nature of assessment and feedback
Remember: you don’t need to get it all right all the time