Embed Size (px)
Citation preview
Making LearningHappen
- and getting assessment andfeedback working better!
Dundalk Institute of Technology
www.phil-race.co.uk
Dundalk Institute of Technology21st October 2010
Phil RaceBSc PhD PGCE FCIPD SFHEA NTF
Emeritus Professor
Leeds Metropolitan University
As I explained...
I did some things both with the smallgroup of senior staff in the morning, andwith the larger group in the afternoon.
However, I did some different thingswith each group too, and the
www.phil-race.co.uk
with each group too, and thepresentation here combines most ofthese.
Therefore, whichever group you were in,there will be things to learn from theslides you haven’t seen.
Name labels…Name labels…
Please write your first name, big and bold,on a coded sticky label, and stick it toyour clothing (not onto a fabric whichwould be damaged).
A3A3
Every student comeswith a free
person
www.phil-race.co.uk
I’m aperson, nota number
Getting to know studentsGetting to know students
Asking students to write their names on labelshelps them to get to know each other, and helpsus to know what they want to be called.
When they feel they’re known to us asindividuals (in a good way), they’re less likely todrop out – especially in big groups in HE.drop out – especially in big groups in HE.
Labels are cheap and environmentally-friendly.
The codes can be used to get students intodifferent group configurations, or to quiz thewhole class.
This slide has three mainThis slide has three mainpurposes….purposes….
To focus the data projector, whenTo focus the data projector, whennecessary, and…necessary, and…
Annoy all the folk near the back,Annoy all the folk near the back,
www.phil-race.co.uk
Annoy all the folk near the back,Annoy all the folk near the back,and...and...
...to persuade colleagues never to...to persuade colleagues never toput this sort of detail onto aput this sort of detail onto aslide!slide!
About Phil…About Phil… Born a Geordie
First a musician
Then a writer
Then a scientist
Then a researcher
Then a lecturer and warden
Got interested in how students
And now ‘retired’! (1995, 2009)
Working with students onlearning techniques
And lecturers on teaching andassessment strategies
And trainers on training design
Currently… Got interested in how studentslearn
And the effects assessmentand feedback have on them
And how we teach them
Gradually became aneducational developer
Currently…
Emeritus Prof: Leeds Met
Travelling around as usual!
Based at Newcastle and Leeds
And on trains
And still a Geordie.
And an expert…on train routes and timetables!
What’s an ‘Emeritus Professor?’
Someone really old?
Someone who has not been sacked for grossmoral turpitude?
Someone who’s still on the books, but not onthe payroll?
www.phil-race.co.uk
the payroll?
Or perhaps as in ‘edentate’, the ‘e’ signifies‘without’.
Thanks to students
I could not do what I do nowadays if Idid not continue to spend a fair bit of mytime working with students – they’vetaught me most of what I know.
www.phil-race.co.uk
You will be able to downloadmy main slides
You don’t need to take notes.
But of course do feel free to make notes ofanything you think of – your own thoughts,questions, issues and so on.
I’ll put the main slides I use today up on
www.phil-race.co.uk
I’ll put the main slides I use today up onmy website before the day is out.
So sometimes I’ll go fast.
Intended learning outcomesBy the end of today, you should be better-able to
1. Think afresh about how students really learn;
2. Identify, and address in your teaching, seven factorswhich underpin successful learning, teaching,feedback and assessment;
3. Learn from ‘what the gurus tell us’ about assessment3. Learn from ‘what the gurus tell us’ about assessmentand feedback;
4. Make assessment more valid, reliable, transparent,authentic, inclusive – and manageable;
5. Give more students better feedback in less time!
Post-it task
On a post-it, in your best handwriting, jot downyour own completion of...
Teaching and assessment would be much betterfor me if only I ....
Please swap all post-its until you don’t knowwhose you have.
www.phil-race.co.uk
whose you have.
If asked, read out with passion and dramawhat’s on the post-its you now have.
Post-it task
On a post-it, in your best handwriting,please write your own short completionof:
“assessing students’ work and givingthem feedback would be much better
www.phil-race.co.uk
them feedback would be much betterfor me if only I …”
Please swap post-its until you don’tknow whose you have.
If asked, read out with passion anddrama what’s on the post-it you nowhave.
Feedback to students would workmuch better for me if only I…
Thought they’d read and digest it.
Could give them back their essays to keep.
Thought it would make a difference.
Was able to do it more quickly.
Could be in the right frame of mind when meeting them face toface.
Could be sure they would understand what I’m trying to tellthem.
www.phil-race.co.uk
them.
Could get them to turn up to receive feedback.
Had some expectations that they would use it to improve theirnext mark.
Cared less about the judgements I make, and how to articulatethese in writing.
Could get more of them to attend workshops, and engage in thelearning activities.
Knew the students better.
Could discuss this with my students.
Feedback to students would workmuch better for me if only I…
Knew them personally, and their profiles.
Knew how to phrase it in terms students understand.
Saw evidence of improvement as a result of that feedback.
Knew what to say to them.
Made more time for giving effective feedback.
Could be clearer about why they got the grade they did.
Knew they would read the comments and not just look at thegrade.
www.phil-race.co.uk
grade.
Could identify better what the student needs to do to improve thenext piece of work – feedforward.
Could do this as a dialogue.
Didn’t take so long over it.
Had the chance to discuss it and explain it.
Had a clue what I’m talking about!
Using postUsing post--its to find outits to find outstudents’ real intended learningstudents’ real intended learning
outcomesoutcomes
21 February 201121 February 2011 1616
outcomesoutcomes
PostPost--its…its… A small, equal opportunities,
non-threatening space.
Just about everyone is willing to jot somethingdown on a post-it in answer to a question,whereas they may not offer a spoken answer toa question, or write responses on a blank sheeta question, or write responses on a blank sheetof paper.
Post-its allow everyone the same opportunity torespond, including the quiet or shy students.
Post-its can be swapped, and students can readout someone else’s ideas, in the relative comfortof anonymity.
Finding out where a group isFinding out where a group isstarting from…starting from…
Post-its are particularly useful for open-endedquestions, such as ‘economics would be muchbetter for me if only I …’
Responses can be posted on a flipchart or wall,and used as an exhibit.
They can be photocopied and returned to They can be photocopied and returned tostudents.
Post-its can be a fast way of finding out whatthe real intended learning outcomes are for agroup.
They can also provide a measure of the learningincomesincomes of the group.
FaceFace--toto--face communicationface communication
Monday, 21 February 2011Monday, 21 February 2011 1919
Making all the channels workMaking all the channels work
Face-to-face one-to-one feedbackactivity
Please work in pairs, moving around the room,talking to different people using the scriptwhich follows…
The script:
A ‘Hello’.
B ‘Hello’.
Monday, 21 February 2011 Teaching smarter (Phil Race) 20
B ‘Hello’.
A ‘You are late’.
B ‘I know’.
Try to do it completely differently each time.
The power of face-to-facecommunication
When explaining assessment criteria to students, andwhen linking these to evidence of achievement of theintended learning outcomes, we need to make the mostof face-to-face whole group contexts and,,,
Tone of voice
Body language
Facial expression
Monday, 21 February 2011 Teaching smarter (Phil Race) 21
Facial expression
Eye contact
The chance to repeat things
The chance to respond to puzzled looks
Some things can’t work nearly so well just on paper oron screens.
Useful sources Knight, P and Yorke, M (2003) Assessment, learning and
employability Maidenhead, UK SRHE/Open UniversityPress.
Bowl, M (2003) Non-traditional entrants to highereducation ‘they talk about people like me’ Stoke on Trent,UK, Trentham Books.
Gibbs, G (2010) Using assessment to support studentlearning Leeds: Leeds Met Press.learning Leeds: Leeds Met Press.
Boud, D and Associates (2010) Assessment 2020: sevenpropositions for assessment reform in higher educationSydney: Australian Learning and Teaching Council.
Joughin, G (2010) A short guide to oral assessmentLeeds: Leeds Met Press.
See alsohttp://www.bioscience.heacademy.ac.uk/ftp/newsletters/bulletin22.pdf
More...
Nicol, D J and Macfarlane-Dick Formativeassessment and self-regulated learning: Amodel and seven principles of good feedbackpractice. Studies in Higher Education (2006),Vol 31(2), 199-218
Gordon Stobart (2008) Testing Times – uses
www.phil-race.co.uk
Gordon Stobart (2008) Testing Times – usesand abuses of assessment London:Routledge.
My reference materials Race P (2010) Making learning happen: 2nd edition London:
Sage Publications.
Race, P and Pickford, R (2007) Making teaching WorkLondon: Sage Publications.
Race, P (2007) How to get a good degree: 2nd editionMaidenhead: Open University Press.
Race, P (2006) The Lecturer’s Toolkit: 3rd edition London: Race, P (2006) The Lecturer’s Toolkit: 3rd edition London:Routledge.
(there’s a ‘compendium of writings on feedback’ and anotheron ‘assessment’ as downloads on my website).
Seven factors underpinningsuccessful learning
I’d like us to take a fresh look at howlearning really happens, so thatassessment can be better tuned intoseven factors underpinning successfullearning.
www.phil-race.co.uk
learning.
Ripples model – big slide sequence– downloadable separatelyfrom my website
The ten most important words
You need to remember these – I don’tput these slides on my website!!
www.phil-race.co.uk
Context
Assessment and feedback take up a greatdeal of our time and energy – and also a greatdeal of students' time and energy.
Yet it can be argued that assessment andfeedback are broken in higher education
www.phil-race.co.uk
feedback are broken in higher educationnowadays. They take too much of our timeand we don’t always measure the right things.
What are students like?
More value-conscious, more demanding, lesswilling to put up with stuff?
More likely to sue if they are unhappy?
More likely to be involved in quality assurance?
Less likely to commit 100% of their time to
www.phil-race.co.uk
Less likely to commit 100% of their time tostudy hence increased risk of attrition?
More diverse in their experiences of supportprior to enrolment (and expectations onarrival)?
More dependent on electronically-providedtexts and less willing to buy books?
The digital age - my personalview
1. ‘Digital’ should be regarded as a tool, not asan end in itself. As long as ‘digital’ is ‘special’,we haven’t yet grown up.
2. We need to base our thinking about how bestto use digital assessment and feedback on the
www.phil-race.co.uk
to use digital assessment and feedback on thecollective wisdom we already have – it’s notsomething to re-invent.
3. By far the most significant ‘learningenvironment’ is inside the head of any learner,not outside of it.
We desperately need to makeassessment work better
In the UK, we know from the NationalStudent Survey that since 2005 theevidence suggests that studentsnationally find assessment among theleast satisfactory elements of theirexperience of higher education. We also
www.phil-race.co.uk
least satisfactory elements of theirexperience of higher education. We alsoknow that assessment take up evermore of our time and energy.
Making exams fun
1. Read out the instructions to candidates– rather differently!
2. Include a silly question (just for onemark).
3. Give the senior invigilator a brick.
www.phil-race.co.uk
3. Give the senior invigilator a brick.
4. Ask for a volunteer candidate toaccompany the invigilator to the toilet?
Assessment is broken in highereducation
Student numbers have grown: we can’t usethe same processes and instruments for asystem where getting on for 50% of the 18-30 year old population study in post-compulsory education, compared to 5% acouple of decades ago.
www.phil-race.co.uk
couple of decades ago.
The world has opened up, so that ourassessment processes and practices needto be more compatible with those in quitedifferent cultures and traditions.
And more...
It is widely accepted now that assessment isthe major driver for student learning, and ifassessment is not working as a good driverfor learning, the effectiveness of our entirehigher education provision is jeopardised.
We need to continue to diversify the
www.phil-race.co.uk
We need to continue to diversify theassessment processes and instruments weuse, so that no students are repeatedlydisadvantaged by the predominance ofparticular assessment formats.
In short,
We assess far too much,
Using the same old ways far too often,
Assessment takes far too much of ourtime – and far too much time of ourstudents.
www.phil-race.co.uk
students.
And we drive down the quality oflearning by our assessment.
And what’s wrong with feedback?
Students get it too late.
And too often, it’s just words on paper.
It doesn’t help them enough.
They often don’t take enough notice of
www.phil-race.co.uk
They often don’t take enough notice ofit.
But what’s really wrong withfeedback?
It’s one-way – monologic.
What students want is dialogue.
They want to talk to us about their work.
But they’re scared to talk to us, for
www.phil-race.co.uk
But they’re scared to talk to us, forvarious reasons:
In case it leads to lower marks;
In case they’re ‘found out’;
In case they feel stupid.
What the gurus tell uson assessment,
feedback and learning
Towards making learning happen
www.phil-race.co.uk
Towards making learning happenbetter, using smarter assessment
and feedback
Albert Einstein
“It is simply madness to keep doing thesame thing, and expect different results”
www.phil-race.co.uk
Sally Brown, 2009
Concentrating on giving students detailedand developmental formative feedback is thesingle most useful thing we can do for ourstudents, particularly those who have had astruggle to achieve entry to higher education.
www.phil-race.co.uk
Assessment and feedback are two of the besttools available to us, to support studentachievement, progression and retention.
“With the changing economy, no onehas lifetime employment, but communitycolleges provide lifetime employability”.
President Barack Obama
www.phil-race.co.uk
Gibbs, 2010: tactics to improvestudent learning
1. Capture student time and effort, distributingthat effort appropriately across topics andweeks.
2. Generate high-quality learning effort, orientedtowards clear and high standards.
3. Provide sufficient feedback, often enough,and in enough detail.
4. Focus feedback on students’ performance, onactions under their control, rather than onstudents themselves or their characteristics.
Gibbs, 2010: tactics to improvestudent learning
5. Make feedback timely, while it still matters tostudents, in time for them to use it towardsfurther learning, or to receive furtherassistance.
6. Link feedback to what students believethey are supposed to be doing.
www.phil-race.co.uk
they are supposed to be doing.
7. Ensure that feedback is not only received,but is attended to, so that students act onit to change their future learning andperformance.
Life is too short to…
Spend time and energy writing feedbackwhich won’t actually be used by students(sometimes not even collected by them);
Write feedback just for external examiners tosee.
www.phil-race.co.uk
see.
Approach giving feedback only in the ‘read-write’ dimension, when many students gainmore from it through auditory, or visual, orkinaesthetic channels.
(see Neil Fleming’s excellent (and free!)‘VARK’ work on www.vark-learn.com).
The read-write problem
Higher education has become a read-writeindustry!
Yet only since 1791 have we been usingwritten exams and written feedback.
Feedback on paper is probably the most time-
www.phil-race.co.uk
Feedback on paper is probably the most time-wasting, least effective and most dangerousway to give students feedback.
There is still a lot going for oral assessmentand feedback – see Joughin 2010.
Boud et al 2010: ‘Assessment 2020’Assessment has most effect when...:
1. It is used to engage students in learning that isproductive.
2. Feedback is used to actively improve student learning.
3. Students and teachers become responsible partners inlearning and assessment.
4. Students are inducted into the assessment practices
www.phil-race.co.uk
4. Students are inducted into the assessment practicesand cultures of higher education.
5. Assessment for learning is placed at the centre ofsubject and program design.
6. Assessment for learning is a focus for staff andinstitutional development.
7. Assessment provides inclusive and trustworthyrepresentation of student achievement.
Boud et al, 2010David Boud (University of Technology, Sydney), Royce Sadler (Griffith University), GordonJoughin (University of Wollongong), Richard James (University of Melbourne), MarkFreeman (University of Sydney), Sally Kift (Queensland University of Technology), FilipDochy (University of Leuven), Dai Hounsell (University of Edinburgh), Margaret Price (OxfordBrookes University), Tom Angelo (La Trobe University), Angela Brew (Macquarie University),Ian Cameron (University of Queensland), Denise Chalmers (University of Western Australia),Paul Hager (University of Technology, Sydney), Kerri-Lee Harris (University of Melbourne),Claire Hughes (University of Queensland), Peter Hutchings (Australian Learning andTeaching Council), Kerri-Lee Krause (Griffith University), Duncan Nulty (Griffith University),Ron Oliver (Edith Cowan University), Jon Yorke (Curtin University), Iouri Belski (RMITUniversity), Ben Bradley (Charles Sturt University), Simone Buzwell (Swinburne University ofTechnology), Stuart Campbell (University of Western Sydney), Philip Candy (University ofTechnology), Stuart Campbell (University of Western Sydney), Philip Candy (University ofSouthern Queensland), Peter Cherry (Central Queensland University), Rick Cummings(Murdoch University), Anne Cummins (Australian Catholic University), Elizabeth Deane(Australian National University), Marcia Devlin (Deakin University), Christine Ewan(Australian Learning and Teaching Council), Paul Gadek (James Cook University), SusanHamilton (University of Queensland), Margaret Hicks (University of South Australia), MarnieHughes-Warrington (Monash University), Gail Huon (University of Newcastle), Margot Kearns(University of Notre Dame, Sydney), Don Maconachie (University of the Sunshine Coast), ViMcLean (Queensland University of Technology,) Raoul Mortley (Bond University), KylieO’Brien (Victoria University), Gary O’Donovan (University of Tasmania), Beverley Oliver(Curtin University), Simon Pyke (University of Adelaide), Heather Smigiel (FlindersUniversity), Janet Taylor (Southern Cross University), Keith Trigwell (University of Sydney),Neil Trivett (University of Ballarat), Graham Webb (University of New England).
David Nicol et al...Good feedback practice:1. Helps clarify what good performance is (goals, criteria,
expected standards);2. Facilitates the development of self-assessment
(reflection) in learning;3. Delivers high quality information to students about
their learning;
Nicol, D J and Macfarlane-Dick: Formative assessment and self-regulated learning: A model and seven principles of good feedbackpractice. Studies in Higher Education (2006), Vol 31(2), 199-218
www.phil-race.co.uk
their learning;4. Encourages teacher and peer dialogue around
learning;5. Encourages positive motivational beliefs and self-
esteem;6. Provides opportunities to close the gap between
current and desired performance;7. Provides information to teachers that can be used to
help shape the teaching.
The guru Royce Sadler
“The indispensable conditions for improvement arethat the student comes to hold a concept of qualityroughly similar to that held by the teacher, is able tomonitor continuously the quality of what is beingproduced during the act of production itself, and hasa repertoire of alternative moves or strategies from
www.phil-race.co.uk
a repertoire of alternative moves or strategies fromwhich to draw at any given point. In other words,students have to be able to judge the quality of whatthey are producing and be able to regulate what theyare doing during the doing of it”. (Sadler 1989), myitalics)
Royce Sadler......is the most cited author on formative feedback,
writing about it since the mid-1980s. He isdoing his absolute best work now.
Sadler, D R (2009) Indeterminacy in the use of preset criteria forassessment and grading Assessment and Evaluation in HigherEducation 34:2 159-179
Sadler, D R (2009) Grade integrity and the representation of academic
www.phil-race.co.uk
Sadler, D R (2009) Grade integrity and the representation of academicachievement Studies in Higher Education, 34:7, 807-826
Sadler, D R (2007) Perils in the meticulous specification of goals andassessment criteria Studies in Higher Education I34:7 807-26.
Sadler, D R (2005) Interpretations of criteria-based assessment andgrading in higher education Assessment and Evaluation in HigherEducation 30 175-194.
Sadler, D R (2002) Ah! ... So that’s ‘Quality’ In Schartz, P and Webb, G(eds) Assessment Case Studies: experience and practice from highereducation London, Kogan Page.
The Weston Manor Group2007
ASKe: AssessmentManifesto for Change
Professor Trudy Banta, Dr Simon Barrie, Professor Sally Brown,Cordelia Bryan, Dr Colin Bryson, Jude Carroll, Professor Sue
www.phil-race.co.uk
Cordelia Bryan, Dr Colin Bryson, Jude Carroll, Professor SueClegg, Professor Linda Drew, Professor Graham Gibbs, Dr KarenHandley, Professor Anton Havnes, Dr Mary Lea, Dr JanetMacdonald, Professor Ranald Macdonald, Dr Debra Macfarlane,Dr Susan Martin, Professor Marcia Mentkowski, Dr StephenMerry, Professor David Nicol, Professor Andy Northedge,Professor Lin Norton, Berry O’Donovan, Dr Thomas Olsson, DrSusan Orr, Dr Paul Orsmond, Professor Margaret Price,Professor Phil Race, Clive Robertson, Dr Mark Russell, Dr ChrisRust, Professor Gilly Salmon, Professor Kay Sambell, ProfessorBrenda Smith, Professor Stephen Swithenby, Professor MantzYorke.
The Weston Manor Group:Assessment Standards: a Manifesto for
change1. The debate on standards needs to focus on how high
standards of learning can be achieved throughassessment. This requires a greater emphasis onassessment for learning rather than assessment oflearning.
2. When it comes to the assessment of learning, weneed to move beyond systems focused on marks and
www.phil-race.co.uk
need to move beyond systems focused on marks andgrades towards the valid assessment of theachievement of intended programme outcomes.
3. Limits to the extent that standards can be articulatedexplicitly must be recognised since ever moredetailed specificity and striving for reliability, all toofrequently, diminish the learning experience andthreaten its validity. There are important benefits ofhigher education which are not amenable either to theprecise specification of standards or to objective
The Weston Manor Group:Assessment Standards: a Manifesto for
change
www.phil-race.co.uk
precise specification of standards or to objectiveassessment.
4. Assessment standards are socially constructed sothere must be a greater emphasis on assessment andfeedback processes that actively engage both staffand students in dialogue about standards. It is whenlearners share an understanding of academic andprofessional standards in an atmosphere of mutualtrust that learning works best.
Monday, 21February 2011
5. Active engagement with assessment standards needsto be an integral and seamless part of course designand the learning process in order to allow students todevelop their own, internalised conceptions ofstandards, and to monitor and supervise their ownlearning.
6. Assessment is largely dependent upon professional
The Weston Manor Group:Assessment Standards: a Manifesto for
change
www.phil-race.co.uk
6. Assessment is largely dependent upon professionaljudgement, and confidence in such judgementrequires the establishment of appropriate forums forthe development and sharing of standards within andbetween disciplinary and professional communities.
Monday, 21February 2011
Assessment ReformGroup 2002
Testing, motivation and learning
www.phil-race.co.uk
Testing, motivation and learningProfessor Paul Black King’s College, London
Professor Patricia Broadfoot University of Bristol
Professor Richard Daugherty University of Wales, Aberystwyth
Professor John Gardner Queen’s University, Belfast
Professor Wynne Harlen University of Bristol
Dr Mary James University of Cambridge
Dr Gordon Stobart Institute of Education, London
Professor Dylan Wiliam King’s College, London
Assessment Reform Group 2002Do more of this ...
Provide choice and help pupils to take responsibility for theirlearning.
Discuss with pupils the purpose of their learning and providefeedback that will help the learning process.
Encourage pupils to judge their work by how much they have learnedand by the progress they have made.
Help pupils to understand the criteria by which their learning isassessed and to assess their own work.assessed and to assess their own work.
Develop pupils’ understanding of the goals of their work in terms ofwhat they are learning; provide feedback to pupils in relation to thesegoals.
Help pupils to understand where they are in relation to learning goalsand how to make further progress.
Give feedback that enables pupils to know the next steps and how tosucceed in taking them.
Encourage pupils to value effort and a wide range of attainments.
Encourage collaboration among pupils and a positive view of eachothers’ attainments.
Define the curriculum in terms of what is in the tests to thedetriment of what is not tested.
Give frequent drill and practice for test taking.
Teach how to answer specific test questions.
Allow pupils to judge their work in terms of scores or grades.
Allow test anxiety to impair some pupils’ performance (particularlygirls and lower performing pupils).
Assessment Reform Group 2002Do less of this ...
www.phil-race.co.uk
girls and lower performing pupils).
Use tests and assessment to tell students where they are in relationto others.
Give feedback relating to pupils’ capabilities, implying a fixed viewof each pupil’s potential.
Compare pupils’ grades and allow pupils to compare grades, givingstatus on the basis of test achievement only.
Emphasise competition for marks or grades among pupils.
How not to givefeedback
Some actual examples!!Some actual examples!!
Mark = 30%
This essay is abysmal. There is no evidencewhatsoever that you (1) read and understoodthe assignment topic with due care, (2)readthe accompanying guidelines, or (3) readanything at all on the debate between legalanything at all on the debate between legalmoralism and legal liberalism or any of thesupplied notes either. Hence the mark forthis long, rambling piece of ‘work’.Considerable work is required for a passstandard or higher to be obtained.
More…
I really can not pass this at all;
It completely fails to address the questiontopic;
It presents totally inadequate information;It presents totally inadequate information;
It is basically an incoherent ramble.
A waste of your time writing it, and minereading it. You could do so much better withsome real study of the material.
Feedback, and…
Feed-forward
www.phil-race.co.uk
Feed-forward
Ripples on a pond….
Wanting/
Assessingmaking informed
judgements
Coaching,explaining, teaching
Motivates students – helps them to want to learn;
Helps students to identify what they need to donext;
Helps students to take action to improve their learning;
Helps students to make sense of what they arelearning;
Feedback (including feed forward) works wellwhen it:
www.phil-race.co.uk
Doing
Wanting/Needing
Doing
Feedback
Making sense
learning;
Helps them to further make sense of theirlearning by engaging in real dialogue with tutorsand peers;
Helps them to make informed judgements ontheir own work, and each others’ work:
Helps them to reflect on their past work in waysthey can use to improve their future work.
Feedback works badly when it…
Is just a monologue from tutors to students;
Saps students’ confidence;
Directs students’ activities in inappropriate directions;
Fails to articulate with learning outcomes;
Fails to relate clearly to evidence of achievement ofthe assessment criteria;
www.phil-race.co.uk
the assessment criteria;
Relates only to what is easy to assess rather thanwhat is at the heart of learning;
Focuses on failings rather than achievements.
Why does formative feedback causeus problems?
If the issues surrounding giving feedbackeffectively and efficiently werestraightforward, they would have been solvedlong ago!
Nevertheless, a growing body of researchindicates that giving effective feedback is the
www.phil-race.co.uk
indicates that giving effective feedback is themost powerfully positive thing we can do tofoster and maintain student learning.
When we talk to unhappy students, minimal orover-negative feedback is often the biggestbugbear – and the NSS bears out that this iswidespread at present.
Auditing assessment andfeedback
We know from the NSS that we’ve got a problemwith assessment and feedback, and there’s nobetter time to start tackling it than now.
There’s too much summative assessment in oursystems, and not enough opportunity for feed-forward.
www.phil-race.co.uk
forward.
We need to streamline assessment, and make thefeedback associated with assessment work betterfor ourselves and for our students.
Evidencing our good practice
Feedback to students, when written (or word-processed) is one of the most accessibleindicators of the quality of our teaching.
Such evidence is looked at very thoroughly byexternal agencies, professional bodies,funding council, external examiners.
www.phil-race.co.uk
funding council, external examiners.
And they usually talk to students, who knowbest how well (or badly) our feedback actuallyworks.
Seven key questions aboutfeedback…
1. What do we say to students about theirwork?
2. How do we say it?
3. Do they take any notice?
4. How much does it help their learning payoff?
www.phil-race.co.uk
4. How much does it help their learning payoff?
5. How well does it relate to students’ evidenceof achievement of the intended learningoutcomes?
6. How well does it help them to achieve theirnext learning outcomes?
7. How efficient is it for us?
Life is too short to…
Spend time and energy writing feedbackwhich won’t actually be used by students(sometimes not even collected by them);
Write feedback just for external examiners tosee.
www.phil-race.co.uk
Approach giving feedback only in the ‘read-write’ dimension, when many students gainmore from it through auditory, or visual, orkinaesthetic channels.
(see Neil Fleming’s excellent (and free!)‘VARK’ work on www.vark-learn.com).
How to give more and betterformative feedback to more
students in less time!
Feedback on paper…
Handwritten comments on returned assessedwork;
Word-processed comments about assessedwork, e.g. ‘front sheets’ and so on;
Model answers;
www.phil-race.co.uk
Model answers;
Assignment return sheets;
Code letters or numbers written ontostudents’ work;
Word-processed overall reports on the workof a class;…
Face-to-face feedback…
Face-to-face to whole lecture groups;
Face-to-face to individual students;
Face-to-face to groups of students;…
www.phil-race.co.uk
Further feedback processes…
mp3 files giving students audio feedback fromtutors;
Emailed comments on students individualwork, e.g. using ‘track changes’;
Overall comments delivered electronically
www.phil-race.co.uk
Overall comments delivered electronicallythrough a computer conference;
Computer-delivered feedback;
Peer-group feedback associated with peer-assessment;
Feedback you give students who have self-assessed their work.
Individual task on ways of giving studentsfeedback
Think of the main ways you give feedback toyour students. Think also of other ways theyget feedback without you being involved.
Also think of further feedback methods you’rethinking of introducing soon.
www.phil-race.co.uk
thinking of introducing soon.
Privately, make a list of all these ways yourstudents get (or may soon get) feedback ontheir learning – aim for ten or more differentways students get feedback.
Group task
Please go into groups as directed.
As a group, please write each feedbackmethod on a separate post-it, and placethe post-its in position on a flipchart asfollows…
www.phil-race.co.uk
follows…
(but expect to have arguments anddebates around the group – you’reunlikely to agree completely on exactlywhere a post-it should go on the grid)...
Not highly efficientHighly efficientfor us
High learning payofffor students
www.phil-race.co.uk
Low learning payofffor students
Not highly efficientfor us
Highly efficientfor us
High learning payoff
Highly efficient Not highly efficient
4
5
3
25
16
We need to be makingbetter use of the feedbackprocesses in this quadrant
Low learning payoff
Highly efficient Not highly efficient
24
1
2
5 13
4
1
We need to be makingmuch less use of the feedback
processes in this quadrant
Dundalk Institute: high and low-scoring feedback methods....
High-scoring
25 peer assessment
25 peer assessment
25 mcq on moodle wellexplained
Low scoring
1 no feedback
1 consultation dayfeedback
4 written evaluation sheets
5 simply giving scores
www.phil-race.co.uk
explained
20 group peer marking
20 peer observation
20 podcasts of feedback towhole group
16 comments in forum
16 informal peerevaluation
16 face to face to group
5 simply giving scores
8 individual writtenfeedback
York PGT: high and low-scoringfeedback methods....
High-scoring
25 live feedback in class
25 ad hoc verbal, e.g. Inseminar
20 peer group discussion
Low scoring
5 vengeful peer groupmarking
5 face to face one to one
4.5: 3 academics to onestudent
www.phil-race.co.uk
20 peer group discussion
20 peer assessment,assuming fairly
20 ipsative (self)feedback/assessment
18 talking to small groupsabout common problems...
student
4 written feedback,unreadable or too short,
4 written feedback
3 exam marks, nocomment
1.5 written report for allstudents
Kingston highest and lowest scoringfeedback methods
High scoring
25 wikis
(e.g. patient)
25 consumer feedback
25 in class peer feedback(ditto)
Low scoring
4 written feedback onassignments
4 telephone
3 oral feedback to individuals
1 written comments on work
www.phil-race.co.uk
(ditto)
20 students giving verbalfeedback on their ownpresentations
20 tick-box criteria
18 common mistakes
16 short online quiz
16 peer group
15 watching and commenting
14 Excel spreadsheet
1 written comments on work(ditto)
Essex feedback methods:
High scoring:
24 class discussion on essaytitles
23 peer feedback dependent onage, level, subject
20 group tutorials anddiscussions
Low scoring
9 emails
8 detailed comments on essays
6 end of year exam mark
5 no feedback
5 comments on essays
www.phil-race.co.uk
discussions
20 class discussions
20 small groups
20 self assessment
16 critique
16 questions and answers inclass
16 online interactive somethingor other
5 comments on essays
5 office hours
3 handwritten comments onessays
1 one to one tutorials
Brunel feedback methods:
High scoring:
25 set a task, then do it yourselfin front of them
20 Group tutorials
20 Computer-aided assessment
18 U-link discussion groups
Low scoring
1 exams
1 written feedback
3 written feedback on coursework
3 ad-hoc answering of questionsby email etc.
www.phil-race.co.uk
18 U-link discussion groups
15 peer group assessment
15 outside feedback e.g. friends,relatives…
15 group feedback: oral
by email etc.
York St John feedback methods:
High scoring:
25 web ct tests
25 verbal/non-verbal feedbackwithin teaching
24 webct discussion boards
23 looking at past work from self
Low scoring
3 marginal notes feedback onessays
5 feedback on front-sheets
6 just a grade
10 one-to-one tutorials
www.phil-race.co.uk
23 looking at past work from self
23 looking at past work fromstudents
20 post-its on feedback (notgoing through all channels)
20 peer-feedback during actionlearning
20 show-and-tell with familiesand friends
15 group projects with peers
15 email response to drafts
10 one-to-one tutorials
Curtin Universityfeedback methods:
High scoring:
25 verbal instant feedback onactivities e.g. in prac class
25 summary feedback to wholeclass
20 students mark each otherswork providing feedback on
Low scoring
9 quiz e.g. paper-based mcq
4 written feedback on individuals’work
3 as above
4 providing good and bad
www.phil-race.co.uk
work providing feedback onmarking scheme
20 next class put up examples onscreen
20 poster presentation intutorials…
16 small group feedback by tutorin seminar
4 providing good and badexamples of work before it isdue
1 exams without feedback
Kingston Universityfeedback methods:
High scoring:
25 repeat exercises
25 service users in simulations
25 group discussion
23 group feedback from us tothem
Low scoring
5 just a grade
2.5 grade
2 written comments
1 written comments
6 exam marks
www.phil-race.co.uk
them
22 generic comments
20 one-to-one feedback in labs
20 products give instant feedback
20 poster presentations
16 face-to-face between sessions
16 diagnostic tests
6 exam marks
Syddansk Universitet feedback methods:
High scoring:
25 feedback during activelearning during teaching
20 oral presentation with instantfeedback from teacher andclass
20 peer review students
Low scoring
4/3/2 normal written exam withmark only
www.phil-race.co.uk
20 peer review studentscritiquing paper
15 students in tutor-less groupsreviewing their learning andmaking summaries
12 discussion session andsolutions on Blackboard
University of Central Lancashire feedbackmethods:
High scoring:
25 student evaluation where theyanalyse and review their ownwork
20 gold star
One-to one meetings beforehanding in assignment
Low scoring
2 written comments usingcodes – front sheet
1 cover sheets
4 email comments on individualdrafts
www.phil-race.co.uk
handing in assignment
20 one-to-one meetings atstudents’ request
20 individual presentations withimmediate feedback
18 peers discussing their ownwork
12 web-based feedback
3 individual written comments ontext
De Montfort feedback methods:
High scoring:
25 visual and verbal feedback onbest practice from tutors one-to-many
25 students talking to each other
25 work placement
Low scoring
1 PDP 1-1tutorial
1 module evaluation forms
4 individual critiques
4 general feedback on blackboard
2 written feedback sheets in a
www.phil-race.co.uk
20 several-to-one viva
20 common pitfalls and what agood one looks like – one tomany
20 external competitions
16 peer-assessment feedback
2 written feedback sheets in abox
5 generic written feedback sentout
University of Aalborg methods:
High scoring:
20 students discussing groupwork
20 ditto
20 peer assessment
20 students reviewing each other
Low scoring
2 individual oral examination
3 comments on assignmentsheets
3 written feedback on exercises
3 giving marks to exercises
www.phil-race.co.uk
20 students reviewing each other
16 peer assessment
3 giving marks to exercises
5 face-to-face feedback, andwritten comments
Leeds Met feedback methods:
High scoring:
25 verbal feedback to wholegroup
25 general feedback to the wholegroup with individualcomments
25 instant feedback by self
Low scoring
1 written individual (ditto)
5 no feedback
www.phil-race.co.uk
25 instant feedback by selfassessment
16 reflective journal selfevaluation
16 peer feedback
14 whole group verbal feedback
Feedback without marksFeedback without marks
Monday, February 21,Monday, February 21,20112011 8989
Marks often destroy the value ofMarks often destroy the value ofour feedback to students!our feedback to students!
Feedback versus marks or gradesFeedback versus marks or grades
Feedback may be eclipsed by marksor grades.
Students may be blinded by themark or grade, and not even try tomake sense of the feedback theymake sense of the feedback theyreceive.
Just a mark (or grade) is the least effectiveJust a mark (or grade) is the least effectiveform of feedback!form of feedback!
It’s what students look at first.
If the mark is good, they smile andfile – quite often they don’t evenread the feedback.
If it’s low, they frown and bin it –very often without reading thefeedback.
But there are ways round this...But there are ways round this...
Give students back their work withfeedback but with no marks or grades(keeping your record of their marks).
Ask them to work out their marks fromthe feedback you have given them (andthe feedback you have given them (andfrom the feedback you gave to otherstoo).
Suggest that their marks willSuggest that their marks willcount!count!
Tell them that if their self-assessmentscores are within (say) 5% of your ownscores, the higherhigher number will go forwardinto their assessment record.
But explain that you will talk individuallyto those students whose score is
But explain that you will talk individuallyto those students whose score isdifferent by more than 5% from yours.
Get them to selfGet them to self--assess...assess...
Collect their scores or grades – e.g. passa sheet round in a lecture.
Most students (e.g. 9 in 10) will be withinthe 5%.
Arrange to talk to those where the Arrange to talk to those where thedifference is more than 5% or one grade.
Students who underStudents who under--estimateestimatetheir grade…their grade…
These students often need their esteemboosted.
Remind them about the assessmentcriteria, and how these illustrate theintended standards associated with theintended standards associated with thelearning outcomes.
Check that they weren’t just trying notto be seen as over-confident.
Students who overStudents who over--estimate theirestimate theirgrade…grade…
This usually indicates they’ve got a blindspot.
Talk them through their work, and findout exactly where they’ve lost markswhich they thought they had gained.which they thought they had gained.
Check with them that they can now seewhat was being looked for.
Developing this idea further…Developing this idea further…
Think about getting students to indicate theirexpected score or grade at the point of handingtheir work in – for example on a self-assessmentproforma.
Think about getting students to work out howwell they believe they have achieved each of thewell they believe they have achieved each of theintended learning outcomes for the workconcerned…
Or how well they believe they have met each ofthe assessment criteria for the work.
Fishing for feedback?
Feedback is like fish.
If it is not used quickly, it becomesuseless.
(Sally Brown).
www.phil-race.co.uk
Give a man a fish,
Feed him for a day.
Teach a man to fish,
Feed him for a lifetime.
(Chinese proverb).
Make feedback timely, while itstill matters to students, in timefor them to use it towardsfurther learning, or to receivefurther assistance.
(Graham Gibbs)
How to get feedback to a large groupof students within 24 hours
There are serious reservations aboutwritten feedback, but we can make eventhis work much better than it did.
The next two slides are about a way ofgiving students feedback on their work
www.phil-race.co.uk
giving students feedback on their workwithin 24 hours of them doing it.
There are three or more ‘yes, buts’s withthis idea, but please hold these foraround four minutes.
Set the hand-in time to be at awhole-group session
E.g. next Tuesday’s 10-11 lecture,
Deadline = 1003.
Let them pile up all their work at thefront.
www.phil-race.co.uk
At 1003, issue a coloured sheet,containing numbered points (so you cansay in feedback ‘please see point 3, bluesheet’ and so on).
The blue sheet can contain....
Illustration of what is expected asevidence of achievement of each of theintended learning outcomes
Likely mistakes
Features of a good answer
www.phil-race.co.uk
Features of a good answer
Frequently needed explanations
Let the students study this for 3 minutesuntil 1006 – it goes rather quiet!
At 1006...
Go into face-to-face oral mode, until1009....
Spend a few minutes de-briefing thewhole group and talking them throughone point on the handout…
www.phil-race.co.uk
one point on the handout…
…adding tone-of-voice, facialexpression, body language, emphasis,and so on to the feedback.
The rationale...
Since many students will have done thework in the last 24 hours before handingit in, you’re giving them feedback whilethey still remember what they weredoing.
www.phil-race.co.uk
doing.
They know what they didn’t do.
They know what they missed outbecause they couldn’t understand it.
Now take away their work to mark – ina third of the time it used to take you! You waste far less writing the same old
things on one piece of work afteranother, regarding frequently occurringmistakes;
You can make your comments relate
www.phil-race.co.uk
You can make your comments relatemore to each individual piece of work;
This means when students get theirmarked work back with feedback, theyare more likely to use it, as it’s personalto them.
Furthermore...
When they get their marked work back,they’ve already had the chance to makesense of their own piece of work in thelight of the generic feedback you gavethem when they handed it in, and theadditional oral ‘deepening’ of one
www.phil-race.co.uk
them when they handed it in, and theadditional oral ‘deepening’ of oneimportant point.
Back to our intended learningoutcomesDo you now feel better able to...
(two hands = much better, one hand = somewhat better,no hands = no better)
1. Think afresh about how students really learn?
2. Identify, and address in your teaching, seven factorswhich underpin successful learning, teaching,feedback and assessment?
www.phil-race.co.uk
3. Learn from ‘what the gurus tell us’ about assessmentand feedback?
4. Make assessment more valid, reliable, transparent,authentic, inclusive – and manageable?
5. Give more students better feedback in less time?
