Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
4 Oct 2017: 07:00AM UTC/GMT
Student self-assessment: rationale and practice Justtoletyouknow:
Bypar'cipa'nginthewebinaryouacknowledgeandagreethat: Thesessionmayberecorded,includingvoiceandtextchatcommunica'ons(arecordingindicatorisshowninsidethewebinarroomwhenthisisthecase).Wemayreleaserecordingsfreelytothepublicwhichbecomepartofthepublicrecord.Wemayusesessionrecordingsforqualityimprovement,oraspartoffurtherresearchandpublica'ons.
Webinar Series
e-AssessmentSIG
Webinar Hosts Professor Geoff Crisp, PVC Education, University of New South Wales g.crisp[at]unsw.edu.au Dr Mathew Hillier, Office for Learning & Teaching, Monash University mathew.hillier[at]monash.edu
Danny Carroll University of New South Wales
Student self-assessment: rationale and practice Danny Carroll UNSW Business School [email protected]
Theshapeofthistalk
AndradeandDu(2007)
• Reflec'veprac'ces• Thestudy(brieflydescribed)• Self-assessmentaccuracy:quan'ta've• Self-assessment–qualita'vefeedback• Discussion
Developingjudgementprocesses(…OR…Howwronghavewegotthis?)
hRp://sta'c.guim.co.uk/sys-images/Guardian/Pix/cartoons/2012/6/20/1340194927119/students-taking-exams-008.jpg
Self-assessment–whybother?
• Observedinaccuracy&variabilityofstudentself-assessment
• U'lity,validity,reliability
Whycriteriabasedself-assessment?
• Exemplifywhatexpertjudgesvalue• Providecomprehensibleperformancetargetsforperformance&judgement
• Interactwithrubrics,scalesandexemplarstoestablish‘quality’
• Providesaframeworkforfeedback,interpreta'onandcalibra'on
• StartsaprocessofaRen'on&response
Researchques'ons:Generalpresenta'onofself-assessmentaccuracyCanstudentsreadilyimproveaccuracy?Whoimproves,howdotheyimprove?Otherobserva'ons?
Twotasks:similarcriteriadeveloped-studentses'mateperformance
Againsteachcriteria,studentsposi'ontheslidertoes'matetheirjudgmentofthequalityoftheirperformanceonthatcriteria
Myresearch:comparedself-assessment(SA)accuracybetweentwotasks
• SAwasop'onalinallcases–butSApar'cipa'on%washigh
• 2essaytypetaskscomparedineachcourse• Supportofmarkingstaffandstudentswithaccuracytraining–e.g.class-baseddiscussionofcriteriaandtask
Self-assessmentaccuracybands(differencescorefrommarker)
<+10 >10to<+5 >5to<-5 >-5<-10 >-10
Strongover-es7mators Accurate
assessorsStrongunder-es7mators
Over- assessors Under- assessors
Demographics
Course N= %Local %Int %M %F %20-25years
1stYear 131 25 75 59 41 74
2ndYear 178 59 41 38 62 78
Actualself-assessmentac'vityCourse Students %Self-
assessedTask1
%Self-assessedTask2
%DidSurvey
COMM5001(1stYrPG)
131 73 60 73
MGMT2102(2ndYrUG)
178 91 74 85
Summaryfindings…Studentstendedtobeover-assessAccuracygenerallyimprovedInaccurateassessorsgotmoreaccurateAccurateassessors‘wobbled’2ndyearsweremoreaccuratethan1styearsStudentsimprovedbycalibra'ngontheVERYinaccuratecriteriaAccuracy&taskmarkoutcomes…Localsweremoreaccuratethaninterna'onalsMalesreturnedhigherself-marksthanfemales…
Dostudentsover-assess/dotheygetmoreaccurate?
InTask1’s–55–82%ofstudentsover-assessedtheircriteriamarksoutcome
Course-Task
Over Accurate Under
1stYearTask1
64 32 4
Task2 43 54 3
2ndYearTask1
34 51 15
Task2 19 62 17
Taskmarkoutcomesbyini'alassessortype:whobenefits?
Self-assessor Type Task 1 Task 2 Task mark % change
Under-assessors, n= 19 Mean task mark 74.7 77.1 3.2
Accurate assessors n= 67 Mean task mark 72 75.4 4.7
Over-assessors, n= 38 Mean task mark 67.3 73.5 9.2 Strong over-assessors, n=16 Mean task mark 65.1 72 10.6
Studentviews:u'lityofself-assessment D28
Prac7cingself-assessmentregularly,islikelytoleadtoimprovedprofessionaljudgment
E31Ithinkprac7cingself-assessmentaccuracyisuseful
E33Developingaccurateprofessionaljudgmentisanimportantskill
E34Developingaccurateprofessionaljudgmentisasimportantasbeingjudgedbyteachers/experts
E35Iwanttodevelopbe^erself-assessmentjudgments
1stYearPGSurvey 3.94 3.98 4.26 4.06 4.37
3rdyearUGSurvey 3.57 3.51 4.47 4.03 4.26
2ndYearUGSurvey 3.71 3.49 4.17 3.78 4.09
Score is the mean score of all respondents out of a possible 5 (mid-point is 2.5)
Qualita'vefeedback–2ndYearUGManagementCourse:FeedbackCategories
• Generalexperience• Selfregulatorymechanisms• Mo'va'on• Scaffoldingskillsdevelopment–developingprofessionaljudgment
• Meta-cogni'veskillsdevelopment
Selfregula'on
‘Itmademerealisewhichpartsiexcelledatandwhichpartsneededmorework’.‘…ItalsoletsmetoseewhetherIhaveansweredalltheques'onsornot’.
Recognisingstandards/developingjudgment
‘Thoughtitwasagreatwayforustoreflectonourstrengthsandweaknesses’‘…asanexchangestudentinmyfirstsemester…I'mreallyunfamiliarwithwhatcons'tutesthedifferentgrades…’‘quiteusefulindevelopingjudgementskills’
Metacogni'on
‘Selfassessmentgeneratesthebuildingblockstoimprovement.Iden'fyingwhatstepsandtheelementsofyourworkandwhatneedsimprovementisundoubtedlyimportant’
BibliographyBoud,David(2000)"Sustainableassessment:rethinkingassessmentforthelearningsociety."Studiesincon,nuingeduca,on22(2):151-167.Bandura,A.(2001)“Socialcogni'vetheory:Anagen'cperspec've”.AnnualReviewofPsychology,52(1),1-26.Cornelius-White,Jeffrey(2007)"Learner-centeredteacher-studentrela'onshipsareeffec've:Ameta-analysis."ReviewofEduca'onalResearch77(1):113-143.Carroll,D.,(2013).Businessstudent’saxtudestocriteriabasedself-assessmentandself-efficacy.Proceedingsofthe30thasciliteconference,Sydney,Australia.hRp://www.ascilite.org/conferences/sydney13/program/papers/Carroll.pdfCarroll,D.(2014)Benefitsforstudentsfromachievingaccuracyincriteria-basedself-assessment,IAEAConferenceProceedings.hRp://www.iaea.info/documents/paper_226e233d61.pdfDunning,D.,Heath,C.,&Suls,J.M.(2004).Flawedself-assessment.PsychologicalScienceinthePublicInterest,5(3),69-106.Haxe,John.(2008)Visiblelearning:Asynthesisofover800meta-analysesrela'ngtoachievement.Routledge.Lew,MagdeleineDN,W.A.M.Alwis,andHenkG.Schmidt.(2010)"Accuracyofstudents'self-assessmentandtheirbeliefsaboutitsu'lity."Assessment&Evalua,oninHigherEduca,on35(2):135-156.
Webinar Series
Webinar Session feedback With thanks from your hosts
Professor Geoff Crisp, PVC Education, University of New South Wales g.crisp[at]unsw.edu.au
Dr Mathew Hillier, Office of Learning & Teaching Monash University mathew.hillier[at]monash.edu Recording available http://transformingassessment.com
e-AssessmentSIG