Myths to Truths: Acting on evidence about student learning from assessment

  • Published on
    20-Jun-2015

  • View
    183

  • Download
    0

Embed Size (px)

DESCRIPTION

Keynote address to Durban University of Technology's Annual Learning, Teaching and Assessment Symposium 8,9,10th October 2014.

Transcript

<ul><li> 1. Myths to Truths: Acting on evidenceabout student learning fromassessmentDr Tansy Jessop, Head of Learning &amp; Teaching,University of WinchesterAssessment for Learning SymposiumDurban University of Technology9 October 2014</li></ul> <p> 2. Myths about assessment andfeedbackSisyphus rolls a boulderup a hillan eternity of endlesslabour, useless effort andfrustrationHomer, 8th Century BC 3. 21st century equivalentYou end up assessing forassessments sake rather thanthinking about what the assessmentis for.Programme Leader, Winchester(2008) 4. Three foundations1) Assessment drives what students payattention to, and defines the actualcurriculum (Ramsden 1992).2) Feedback is significant (Hattie, 2009; Blackand Wiliam, 1998)3) Programme is central to influencing change. 5. Of, for or as? 6. Troubling prepositions Assessment of Learning (Gipps 1994) Assessment for Learning (Gipps 1994) Assessment as learning (Torrance 2007) 7. Assessment for Learning Formative assessment The relationship between formative and summative Feedback feeding forward Helping students internalise goals and standards More for less? Efficiencies? Curriculum design 8. Three TESTA premises1) Assessment drives what students payattention to, and defines the actualcurriculum (Ramsden 1992).2) Feedback is significant (Hattie, 2009; Blackand Wiliam, 1998)3) Programme is central to influencing change. 9. Todays talk TESTA: an evidence-based approach Four areas of AFL Evidence for common myths Making AFL work ideas, practices,examples 10. TESTA: Transforming the Experienceof Students through Assessment Higher Education Academy funded research project(2009-12) Evidence-based research and change process Programme the central unit of change Based on assessment for learning principles 11. TESTA Cathedrals Group Universities 12. EdinburghEdinburgh NapierGreenwichCanterbury ChristchurchGlasgowLady Irwin College University of DelhiSheffield HallamUniversity of West Scotland 13. TESTAis a way of thinkingabout assessment andfeedbackGraham Gibbs 14. Based on assessment for learningprinciples Students need to distribute effort and spend timeon task Tasks with challenging and high expectations Internalising understand goals and standards Prompt feedback Detailed, high quality, developmental feedback Dialogic cycles of feedback Deep learning beyond factual recall 15. TESTA Research Methods(Drawing on Gibbs and Dunbar-Goddet, 2008,2009)2000 ASSESSMENTEXPERIENCEQUESTIONNAIRES45 PROGRAMME90 FOCUS GROUPSAUDITSProgrammeTeamMeeting 16. Myth 1: Modules and semesters helpstudents to learn betterThe weak spot? 17. Is the module the right metaphorfor learning?modulus (Latin): small measureinterchangeable unitsstandardised unitssections for easy constructionsa self-contained unit 18. How well does IKEA 101 packagingwork for Engineering 101?Furniture Bite-sized Self-contained Interchangeable Quick and instantaneous Standardised Comes with writteninstructions ConsumptionStudent Learning Long and complicated Interconnected Distinctive Slow, needs deliberation Varied, differentiated Tacit, unfathomable,abstract Production 19. What students say about thesystem Its difficult because your assignments are so detached from the nextone you do for that subject. They dont relate to each other. Because its at the end of the module, it doesnt feed into our futurework. We dont get much time to think. We finish one assignment and theother one is knocking at the door. In the annual system there is more time to learn and do assignments.In the semester system everything is so rushed up. In the annualsystem the teachers say that they had more time to explain in detail. 20. about shared practicesYoull get really detailed, really commenting feedback from onetutor and the next tutor will just say Well done.Some of the lecturers are really good at feedback and others dontwrite feedback, and they seem to mark differently. One person willtell you to reference one way and the other one tells yousomething completely different. 21. about shared standardsEvery lecturer is marking it differently, which confuses people.Weve got two tutors- one marks completely differently to theother and its pot luck which one you get.They have different criteria, they build up their own criteria.Q: If you could change one thing to improve what would it be?A: More consistent marking, more consistency across everythingand that they would talk to each other. 22. How can assessment and feedbackhelp to join the dots? 23. TESTA changes based on evidence1) Integrated assessment across modules2) Multi-stage assessments (formative feedback feedingforward)3) Changes in quality assurance and degree validationprocesses from lego assembly of degrees module bymodule via email to discussion and team baseddevelopment4) Strengthening team approaches to marking 24. Myth 2: Assessment of learning ismore important because it countsHercules attacked the manyheads of the hydra, but assoon as he smashed onehead, two more wouldburst forth in its place!Peisander 600BC 25. How much summative assessment istaking place? Range of UK summative assessment 12-68 over threeyears Indian and NZ universities 100s of small assessments busywork, grading as pedagogies of control Average in UK about two per module, about 40 in threeyears 26. More summative = more learning? 27. A students lecture to professorsThe best approach from the students perspective is to focus onconcepts. Im sorry to break it to you, but your students are notgoing to remember 90 per cent possibly 99 per cent of whatyou teach them unless its conceptual. when broad, over-archingconnections are made, education occurs. Most detailsare only a necessary means to that end.http://www.timeshighereducation.co.uk/features/a-students-lecture-to-professors/2013238.fullarticle#.U3orx_f9xWc.twitter 28. What students say A lot of people dont do wider reading. You just focus on your essayquestion. I always find myself going to the library and going These are the booksrelated to this essay and thats it. Although you learn a lot more than you would if you were revising for anexam, because you have to do wider research and stuff, you still dont doresearch really unless its directly related to essays. Unless I find it interesting I will rarely do anything else on it because Ihavent got the time. Even though I havent anything to do, I dont havethe time, I have jobs to do and I have to go to work and stuff. 29. Effort Map: High Veld Vs AlpsWeek 6 Week 12Max effortModest EffortLow Effort29 30. Solutions to assessment displacinglearning? 31. TESTA changes based on evidence Reducing summative assessment Increasing and making formative assessment work Linking formative tasks to summative Setting tasks involving research, case studies andauthentic assessment Focusing on assessment which builds up and is alignedwith learning on the module constructive alignment 32. Myth 3: Formative assessment isdifficult to do, and not worth doing 33. Defining formative assessment Definitional fuzziness Mantz Yorke (2003) Basic idea is simple to contribute to student learningthrough the provision of information aboutperformance (Yorke, 2003). A fine tuning mechanism for how and what we learn(Boud 2000) 34. TESTAs definition of formative Ungraded, required and eliciting feedback 35. What students say about formativetasks It was really useful. We were assessed on it but we werent officially given agrade, but they did give us feedback on how we did. It didnt actually count so that helped quite a lot because it was just apractice and didnt really matter what we did and we could learn frommistakes so that was quite useful. I find more helpful the feedback you get in informal ways week by week, butthere are some people who just hammer on about what will get them abetter mark. Hes such a better essay writer because hes constantly writing. And wedont, especially in the first year when we really dont have anything to do.The amount of times formative assignments could have taken place 36. What prevents students from doingformative tasks If there werent loads of other assessments, Id do it. If there are no actual consequences of not doing it, most students aregoing to sit in the bar. Its good to know youre being graded because you take it moreseriously. I would probably work for tasks, but for a lot of people, if its notgoing to count towards your degree, why bother? The lecturers do formative assessment but we dont get any feedbackon it. 37. Why summative cant do thesame job as formative Grades the administrative device that actively divertedstudents from really learning anything (Becker, 1968). Feedback on summative tasks is more readily dismissedwhen there is a grade (Black and William 1998, Orrell 2006,Taras 2002; 2008). Timing of summative tasks, often too late to act onfeedback. 38. Good cop, bad cop? 39. Good cop, bad cop? False dichotomy is unhelpful Rebuilding formative-summative relationship Linked, integrated, multi-stage assessment 40. How can we improve the currencyand practice of formative tasks? 41. TESTA changes based on evidence Increase formative assessment Require formative tasks, using QA and validation processes Public tasks to motivate students to undertake formativetasks (presentations, posters, blogs) Authentic and challenging tasks linked to research, casestudies and large projects Multi-stage tasks formative to summative Set expectations about formative in first year Be consistent as a programme 42. Public Domain 43. Ideas for embedding formative tasks1. The Case of American Studies (Multi-stage linkedformative-summative tasks)2. The Case of BA Primary (Blogs)3. The Case of MA Education (Triads)4. The Case of Sports Psychology (Expectation setting)5. Formative MUST be programmatic it wont work ifone keen maverick lecturer does it. 44. Myth 4: Feedback is writtenmonologue from lecturer to student Getting feedback from other students in my classhelps. I can relate to what they are saying and take iton board. Id just shut down if I was getting constantfeedback from my lecturer. I read it and think Well, thats fine but Ive alreadyhanded it in now and got the mark. Its too late. 45. What students say I read through it when I get it and thats about it really. Theyall go in a little folder and I dont look at them again most ofthe time. Its mostly the mark really that you look for. Im personally really bad at reading feedback. Im the kind ofperson, and I hate to admit it, but Ill look at the mark andthen be like well stuff it, I cant do anything about it. 46. Two educational paradigms 47. Transmission Model 48. Social Constructivist model 49. How can we help feedback to feedforward? 50. TESTA changes based on evidence Cycles of feedback through self and peer review of work Developing dialogue through cover sheets Students initiating feedback through questions Using technology to personalise feedback Getting students to give feedback to teachers formative evaluation 51. Impacts at Winchester Improvements in NSS scores on A&amp;F from bottomquartile in 2009 to top quartile in 2013 Three programmes with 100% satisfaction ratings postTESTA All TESTA programmes have some movement upwardson NSS A&amp;F scores Programme teams are talking about A&amp;F and pedagogy Periodic review processes are changing for the better. 52. www.testa.ac.uk 53. ReferencesBecker, H. (1968) Making the grade: the academic side of college life.Boud, D. (2000) Sustainable Assessment: Rethinking assessment for the learning society, Studies in ContinuingEducation, 22: 2, 151 167.Gibbs, G. &amp; Simpson, C. (2004) Conditions under which assessment supports students' learning. Learning and Teaching inHigher Education. 1(1): 3-31.Gibbs, G. &amp; Dunbar-Goddet, H. (2009). Characterising programme-level assessment environments that support learning.Assessment &amp; Evaluation in Higher Education. 34,4: 481-489.Harland, T. et al. (2014) An Assessment Arms Race and its fallout: high-stakes grading and the case for slowscholarship. Assessment and Evaluation inn Higher Education.http://www.tandfonline.com/doi/abs/10.1080/02602938.2014.931927Hattie, J. (2007) The Power of Feedback. Review of Educational Research. 77(1) 81-112.Jessop, T. and Maleckar, B. (2014). The Influence of disciplinary assessment patterns on student learning: a comparativestudy. Studies in Higher Education. Published Online 27 August 2014http://www.tandfonline.com/doi/abs/10.1080/03075079.2014.943170Jessop, T. , El Hakim, Y. and Gibbs, G. (2014) The whole is greater than the sum of its parts: a large-scale study of studentslearning in response to different assessment patterns. Assessment and Evaluation in Higher Education. 39(1) 73-88.Jessop, T, McNab, N &amp; Gubby, L. (2012) Mind the gap: An analysis of how quality assurance processes influence programmeassessment patterns. Active Learning in Higher Education. 13(3). 143-154.Jessop, T. El Hakim, Y. and Gibbs, G. (2011) Research Inspiring Change. Educational Developments. 12(4) 12-15.Nicol, D. (2010) From monologue to dialogue: improving written feedback processes in mass higher education, Assessment&amp; Evaluation in Higher Education, 35: 5, 501 517Sadler, D.R. (1989) Formative assessment and the design of instructional systems, Instructional Science, 18, 119-144. </p>