Embed Size (px)
Citation preview
Abelardo Pardo (@abelardopardo)Faculty of Engineering and IT slideshare.net/abelardo_pardo
The role of data in the provision of feedback at scale
APRU ERT Forum 18 November 2016
Faruk Ateş flickr.com
Abelardo Pardo The role of data in the provision of feedback at scale 2
It’s the learning!
Richard Katz presentation APRU ERT Forum Presentation 16 Nov 2016, Singapore
Abelardo Pardo The role of data in the provision of feedback at scale 3
Abelardo Pardo The role of data in the provision of feedback at scale 4
Tim Pierce flickr.com
Abelardo Pardo The role of data in the provision of feedback at scale 5
Follow a scholarly approachJonathan C
ohen flickr.com
Abelardo Pardo The role of data in the provision of feedback at scale 6
Yvonne Larsson flickr.comPressure Quality/Cost
Student Feedback
Data-driven Support
Examples
Abelardo Pardo The role of data in the provision of feedback at scale 7
Yvonne Larsson flickr.comPressureQuality/Cost
Abelardo Pardo The role of data in the provision of feedback at scale 8
Space & Light flickr.com
Competition
Funding
Enrolment
Expectations
Digital Environments
Abelardo Pardo The role of data in the provision of feedback at scale 9
Tulane Public Relations flickr.com
Instructors know how to support students
but not in the current conditions
Abelardo Pardo The role of data in the provision of feedback at scale 10
Focus on "good instruction"D
aniel Kwok flickr.com
Kirschner, P. A., & Neelen, M. (2016). The cold left-overs of inquiry-based learning. Retrieved from https://3starlearningexperiences.wordpress.com/2016/11/15/the-cold-left-overs-of-inquiry-based-learning/
Abelardo Pardo The role of data in the provision of feedback at scale 11
increased engagement = increase in quality of experience
Matthias Ripp flickr.com
Abelardo Pardo The role of data in the provision of feedback at scale 12
Startup Mena flickr.com
Increase in feedback = increase in engagement
Abelardo Pardo The role of data in the provision of feedback at scale 13
Yvonne Larsson flickr.comPressure Quality/Cost
StudentFeedback
Abelardo Pardo The role of data in the provision of feedback at scale
If you could choose one…• More than 500 meta-analyses
of student achievements
• 100 factors with potential influence
• Feedback in top five
• (74 meta-analyses) Most effective form: video, audio, computer-assisted instructional feedback, and/or related goals
14Hattie, J. A. (1999). Influences on student learning. Inaugural professorial address, University of Auckland, New Zealand
Abelardo Pardo Using technology to provide feedback on learning strategies
The feedback question gets systematically lower values in student surveys
15
Krause, K.-L., Hartley, R., James, R., & McInnis, C. (2005). The First Year Experience in Australian Universities: Findings from a decade of National Studies. University of Melbourne: Centre for the Study of Higher Education.
Eleaf flickr.com
Abelardo Pardo The role of data in the provision of feedback at scale 16
Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81-112. doi:10.3102/003465430298487
Faruk Ateş flickr.com
"information provided by an agent (teacher, peer, book, parent, self, experience) regarding aspects
of one’s performance or understanding”
Abelardo Pardo The role of data in the provision of feedback at scale 17
Hounsell, D. (2007). Toward more sustainable feedback to students. In D. Boud & N. Falchikov (Eds.), Rethinking Assessment in Higher Education: Learning for the Longer Term. London and New York: Routledge.
Perceived as an administrative chore instead of a pedagogical necessity
Marcin W
ichary flickr.com
Abelardo Pardo The role of data in the provision of feedback at scale 18
Allen Lai flickr.com
Hounsell, D. (2007). Toward more sustainable feedback to students. In D. Boud & N. Falchikov (Eds.), Rethinking Assessment in Higher Education: Learning for the Longer Term. London and New York: Routledge.
Downward spiral: Student disenchantment mounts when feedback is uninformative and still,
less feedback is provided
Solution: provide high-value feedback, rethink the role of student, enhance relation between guidance and feedback
Abelardo Pardo The role of data in the provision of feedback at scale 19
Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81-112. doi:10.3102/003465430298487
Orkom
edix flickr.com
Feedback Levels1. Task Level (understanding,
performance)
2. Process Level (what to do to understand, perform)
3. Self-regulation level (detecting and directing effort)
4. Self level (personal evaluation and affect)
Abelardo Pardo The role of data in the provision of feedback at scale 20
Bjork, R. A., Dunlosky, J., & Kornell, N. (2013). Self-regulated learning: beliefs, techniques, and illusions. Annu Rev Psychol, 64, 417-444. doi:10.1146/annurev-psych-113011-143823
• May not know how to promote comprehension, retention, transfer.
• May not assess properly our own learning
• May be biased when judging our learning
• May rely too much on social beliefs
• Towards “adaptive learners”
Christian W
eidinger flickr.com
Abelardo Pardo The role of data in the provision of feedback at scale 21
Janne Moren flickr.com
Dunlosky, J. (2013). Strengthening the Student Toolbox. Study Strategies to Boost Learning. American Educator, 37(3), 12-21.
Suggest learning strategies to students
Abelardo Pardo The role of data in the provision of feedback at scale 22
Personalised suggestions
Woody1778a flickr.com
Abelardo Pardo The role of data in the provision of feedback at scale 23
Laverrue flickr.comIncrease in personalisation = increase in knowledge
Abelardo Pardo The role of data in the provision of feedback at scale 24
From visualisations to actions
Pardo, A., Mirriahi, N., Martínez-Maldonado, R., Jovanović, J., Dawson, S., & Gasevic, D. (2016). Generating Actionable Predictive Models of Academic Performance. Paper presented at the International Conference on Learning Analytics and Knowledge, Edinburgh.
Kannan B flickr.com
Abelardo Pardo The role of data in the provision of feedback at scale 25
Yvonne Larsson flickr.com
Pressure Quality/Cost
Student Feedback
Data-drivenSupport
Abelardo Pardo The role of data in the provision of feedback at scale 26
PNN
L Pacific Northw
est flickr.com
• Collect data about how students participate in a learning experience
• Translate observations into actions
• Take into account the instructional design
• Expertise of the instructor
• Create and deploy personalised support actions
Abelardo Pardo The role of data in the provision of feedback at scale 27
Intan A3 flickr.com1) Go beyond feedback at the task level
Abelardo Pardo The role of data in the provision of feedback at scale 28
2) Comprehensive*, multimodal, real-time data collection
bodhithaj flickr.com
Abelardo Pardo The role of data in the provision of feedback at scale 29
3) Clean, relevant data sources at the instructor’s fingertipsN
ick Readhead flickr.com
Abelardo Pardo The role of data in the provision of feedback at scale 30
Small Realm
flickr.com
4) Capacity to deploy immediate personalised support actions
Abelardo Pardo The role of data in the provision of feedback at scale 31
IAN RAN
SLEY DESIG
N PLU
S ILLUSTRATIO
N flickr.com
5) Scale to large or highly diverse student cohorts
Abelardo Pardo The role of data in the provision of feedback at scale 32
Yvonne Larsson flickr.com
Pressure Quality/Cost
Student Feedback
Data-driven Support
Examples
Abelardo Pardo The role of data in the provision of feedback at scale 33
Tracking
Abelardo Pardo The role of data in the provision of feedback at scale 34
You should take a more careful look at how symbols are encoded in the video. Would you be able to encode/decode UAL symbols without looking at the video?
Good initial work. However, did you understand the trick to handle encoding with a variable number of bits? Would you be able to provide an example?
Good work. Would you be able to come up with your own machine language and your encoding scheme? Remember that it has to be unambiguous.
Thorough work with the task about machine language encoding. Give it a quick review before the midterm.
Q1 Q2 Q3 Q4
Instructor
Abelardo Pardo The role of data in the provision of feedback at scale 35
AutomaticEmail
Abelardo Pardo The role of data in the provision of feedback at scale 36
1) Email remains the primary and preferred channel of communication with the University
http://libinnovation.blogspot.sg/2016/11/how-do-students-really-prefer-to.html
Abelardo Pardo The role of data in the provision of feedback at scale 37
Effect size (Cohen’s d) = 0.49. Medium positive effect
Effect size (Cohen’s d) = 0.21. Small positive effect
Abelardo Pardo The role of data in the provision of feedback at scale 38
Liu, D. Y.-T., Bartimote-Aufflick, K., Pardo, A., & Bridgeman, A. J. (2016). Data-driven Personalization of Student Learning Support in Higher Education. In A. Peña-Ayala (Ed.), Learning analytics: Fundaments, applications, and trends: A view of the current state of the art. In preparation: Springer.
Instructors define simple rules to create personalised emails from multiple data sources
Student Relationship Engagement System
Not so m
uch flickr.com
Abelardo Pardo The role of data in the provision of feedback at scale 39
ontasklearning.org (in production)
IF [video 3.9.5 not watched] THEN “…."
Abelardo Pardo The role of data in the provision of feedback at scale 40
Alan Levine flickr.com
Workshop to
• Expose academics and educational designers to data sets
• Frame the need for personalised support
• Explore immediate vs long-term actions
• Jumpstart the conversation
• Identify institutional needs
Abelardo Pardo The role of data in the provision of feedback at scale 41
Pardo, A., Jovanović, J., Dawson, S., Gasevic, D., & Mirriahi, N. (2016). Exploring student interactions with preparation activities in a flipped learning experience. Computers & Education, Manuscript under review.
Identify student interactions in flipped learning environment
Yvonne Larsson flickr.com
Abelardo Pardo The role of data in the provision of feedback at scale 42
Jovanović, J., Gasevic, D., Pardo, A., Dawson, S., & Mirriahi, N. (2016). Learning Analytics to Unveil Learning Strategies in a Flipped Classroom. The Internet and Higher Education, Manuscript under review.
1. identify engagement patterns in active learning environments
2. Ground for personalised suggestions 3. Increase student awareness about strategies
Luc Legay flickr.com
Abelardo Pardo The role of data in the provision of feedback at scale 43
Ed Dunens flickr.com
Focus on improvements on the learning experienceData provides insight on engagementCombine data with instructor expertise
Use technology to scale personalisation
Abelardo Pardo (@abelardopardo)Faculty of Engineering and IT slideshare.net/abelardo_pardo
The role of data in the provision of feedback at scale
APRU ERT Forum 18 November 2016
Faruk Ateş flickr.com
