Embed Size (px)
Citation preview
Abelardo Pardo (@abelardopardo)Faculty of Engineering and IT
slideshare.net/abelardo_pardo
The role of data in the provision of feedback at scale
Wally G
obetz flickr.com
ASCILITE Webinar 11 April 2017
Abelardo Pardo The role of data in the provision of feedback at scale 2
Learning Analytics
Situated Actions
Feedback Examples
theilr flickr.com
Abelardo Pardo The role of data in the provision of feedback at scale 3
LearningAnalytics
SituatedActions
theilr flickr.com
Abelardo Pardo The role of data in the provision of feedback at scale 4
Dm
itry Grigoriev flickr.com
Learning Analytics: measure, collect, analyse data about learners to understand and improve their learning and the environment in which it occurs
Abelardo Pardo The role of data in the provision of feedback at scale 5
Derek Bruff flickr.comUnderstanding and
Improving Learning
Algorithms
Dashboards
Clickstreams
Multimodal
SNA
Writing analytics
Learning Designs
Abelardo Pardo The role of data in the provision of feedback at scale 6
James Rhodes flickr.comLet’s not forget:
Learning analytics are about learning
Gašević, D., Dawson, S., & Siemens, G. (2015). Let's not forget: Learning analytics are about learning. TechTrends, 59(1), 64-75.
Abelardo Pardo The role of data in the provision of feedback at scale 7
Derek Bruff flickr.com
Learning is situated in interactions with others and relationships to the environment
Ellis, R. A., & Goodyear, P. (2013). Students' Experiences of E-learning in Higher Education. New York and London: Routledge
Abelardo Pardo The role of data in the provision of feedback at scale 8
Chris M
oore flickr.com
Learning environment has a profound effect
on how data is analysed
Abelardo Pardo The role of data in the provision of feedback at scale 9
senscience flickr.com
Marzouk, Z., Rokovic, M., Liaqat, A., Vytasek, J., Samadi, D., Stewart-Alonso, J., . . . Nesbit, J. C. (2016). What if learning analytics were based on learning science? Australasian Journal of Educational Technology, 32(6).
• Setting goals & monitoring progress • Distributed practice • Retrieval practice • Prior knowledge for reading • Comparative evaluation of writing • Collaborative Learning
Abelardo Pardo The role of data in the provision of feedback at scale 10
Joshua Rappeneker flickr.com
Data Understanding
Actions
Improving
Abelardo Pardo The role of data in the provision of feedback at scale 11
Learning Analytics Model
Abelardo Pardo The role of data in the provision of feedback at scale 12
Learning Analytics
Situated Actions
Feedback
theilr flickr.com
Abelardo Pardo The role of data in the provision of feedback at scale 13Hattie, J. A. (1999). Influences on student learning. Inaugural professorial address, University of Auckland, New Zealand
If you could choose one…• More than 500 meta-analyses
of student achievements
• 100 factors with potential influence
• Feedback in top five
• (74 meta-analyses) Most effective form: video, audio, computer-assisted instructional feedback, and/or related goals
Abelardo Pardo The role of data in the provision of feedback at scale 14
Paula Grubb flickr.com
"Feedback is a process whereby learners obtain information about their work in order to appreciate the similarities and differences between the appropriate standards for any given work, and the qualities of the work itself, in order to generate improved work"
Boud, D., & Molloy, E. (Eds.). (2013). Feedback in Higher and Professional Education: Understanding it and doing it well. London and New York: Routledge.
Abelardo Pardo The role of data in the provision of feedback at scale 15
Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81-112. doi:10.3102/003465430298487
Chris Ballard flickr.com
Feedback Levels
1. Task Level (understanding, performance)
2. Process Level (what to do to understand, perform)
3. Self-regulation level (detecting and directing effort)
4. Self level (personal evaluation and affect)
Abelardo Pardo The role of data in the provision of feedback at scale 16
Nicol, D., & Macfarlane‐Dick, D. (2006). Formative assessment and self‐regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-218. doi:10.1080/03075070600572090
Abelardo Pardo The role of data in the provision of feedback at scale
Good feedback practice1. Clarify what good performance is
2. Facilitate self-assessment
3. Deliver high quality feedback information
4. Encourage teacher and peer dialogue
5. Encourage positive motivation and self-esteem
6. Provide opportunities to close the gap
7. Use information to improve teaching
17
Nicol, D., & Macfarlane‐Dick, D. (2006). Formative assessment and self‐regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-218. doi:10.1080/03075070600572090
Abelardo Pardo The role of data in the provision of feedback at scale 18
Baker, R. S. (2016). Stupid Tutoring Systems, Intelligent Humans. International Journal of Artificial Intelligence in Education. doi:10.1007/s40593-016-0105-0
Rienties, B., Boroowa, A., Cross, S., Kubiak, C., Mayles, K., & Murphy, S. (2016). Analytics4Action Evaluation Framework: A Review of Evidence-Based Learning Analytics Interventions at the Open University UK. Journal of Interactive Media in Education, 2016(1). doi:10.5334/jime.394
Change in focus, from artificial intelligence to intelligence amplification
Tutors are informed about students struggling but it is up to them to decide if/when to act
Abelardo Pardo The role of data in the provision of feedback at scale 19
Learning Analytics
Situated Actions
Feedback Examples
theilr flickr.com
Abelardo Pardo The role of data in the provision of feedback at scale 20
Tracking
Abelardo Pardo Can data help me learn? 21
Jovanovic, J., Gašević, D., Pardo, A., Dawson, S., & Mirriahi, N. (In Press. Accepted 20/Feb/2017). Learning Analytics to Unveil Learning Strategies in a Flipped Classroom. Internet and Higher Education. doi:10.1016/j.iheduc.2017.02.001
Analyse Learning Strategies
Abelardo Pardo The role of data in the provision of feedback at scale 22
Report to students
Kahn, I., & Pardo, A. (2016). Data2U: Scalable Real time Student Feedback in Active Learning Environments. Paper presented at the International Conference on Learning Analytics and Knowledge, Edinburgh, UK. doi:10.1145/2883851.2883911
Abelardo Pardo The role of data in the provision of feedback at scale 23
No statistically significant difference in the rating of feedback (2013 edition, M=3.25, SD=0.97; 2014 edition, M=3.35, SD=1.03); t(389.78) = -0.97, p <0.17
Abelardo Pardo The role of data in the provision of feedback at scale 24
You should take a more careful look at how symbols are encoded in the video. Would you be able to encode/decode UAL symbols without looking at the video?
Good initial work. However, did you understand the trick to handle encoding with a variable number of bits? Would you be able to provide an example?
Good work. Would you be able to come up with your own machine language and your encoding scheme? Remember that it has to be unambiguous.
Thorough work with the task about machine language encoding. Give it a quick review before the midterm.
Q1 Q2 Q3 Q4
Instructor
Abelardo Pardo The role of data in the provision of feedback at scale 25
Algorithm
Abelardo Pardo The role of data in the provision of feedback at scale 26
AutomaticEmail
Abelardo Pardo The role of data in the provision of feedback at scale 27
Helpful feedback
Effect size (Cohen’s d) = 0.49. Medium positive effect
Midterm Scores
Effect size (Cohen’s d) = 0.21. Small positive effect
Pardo, A., Jovanovic, J., Dawson, S., Gašević, D. & Mirriahi, N. (2017). Using Learning Analytics to Scale the Provision of Personalised Feedback. Manuscript in preparation
Abelardo Pardo The role of data in the provision of feedback at scale 28
What if I tell you…
Abelardo Pardo The role of data in the provision of feedback at scale 29
ontasklearning.org
Abelardo Pardo The role of data in the provision of feedback at scale 30
IF [video 3.9.5 not watched] THEN “…."
ontasklearning.org
Abelardo Pardo The role of data in the provision of feedback at scale 31
• Support instructors to create personalised feedback
• Simple rule-base knowledge encoding
• Provide appropriate view of data sources
• Scale to large and highly diverse cohorts
• Will be released as open-source project Q3/4 2017
• First pilots in Q1/2 2017
• Tutorial in LAK 2017
• Contact us if interested
ontasklearning.org
Abelardo Pardo The role of data in the provision of feedback at scale 32
Bill Tyne flickr.com
• Data to understand and improve learning needs to be situated and connected to actions
• Feedback is a powerful focal point to drive data-supported decisions
• Explore the design of theory-driven learning analytics tools that connect data with support actions
Abelardo Pardo (@abelardopardo)Faculty of Engineering and IT
slideshare.net/abelardo_pardo
The role of data in the provision of feedback at scale
Wally G
obetz flickr.com
ASCILITE Webinar 11 April 2017
