Embed Size (px)
DESCRIPTION
Presentation given at the IARU EdTech Horizons Workshop about learning analytics, the main stages in the process, some examples, and finally, how to approach it from the institutional and course level.
Citation preview
Kris
Krüg
Flickr
Analytics for decision making inLearning Environments
IARU EdTech Horizons WorkshopSingapore, 14 November 2014
Dr Abelardo Pardo (@abelardopardo)The University of Sydney
slideshare.net/abelardo_pardo
Abelardo Pardo Analytics for decision making in Learning Environments 2
TristanM
artinflickr.com
LearningAnalytics Collecting
ObservationsAnalysis andVisualization
Interventions InstitutionalAdoption
Abelardo Pardo Analytics for decision making in Learning Environments 3
TristanM
artinflickr.com
LearningAnalytics
NY Times 24/11/12
Abelardo Pardo Analytics for decision making in Learning Environments 4
Abelardo Pardo Analytics for decision making in Learning Environments 5
jbgeronimiflickr.com
Learning is complex and opaque
Uncertainty in Education
“Even in education,however, there aresome propositionsthat give you agreat chance ofcomming out aheadif you bet on themoften enough”
Abelardo Pardo Analytics for decision making in Learning Environments 6
(Felder, 2009)
Learning Analytics
Measurement, collection, analysisand reporting of data about learnersand their contexts, for purposes ofunderstanding and optimizinglearning and the environments in whichit occurs.
Abelardo Pardo Analytics for decision making in Learning Environments 7
(SoLAR, Society for LearningAnalytics Research)
Abelardo Pardo Analytics for decision making in Learning Environments 8
http://ww
w.learninganalytics.net/?p=131
(lastvisitedS
ep2013)
(Siemens & Long, 2011)
www.solaresearch.orgAbelardo Pardo Analytics for decision making in Learning Environments 9
Abelardo Pardo Analytics for decision making in Learning Environments 10
LJubarflickr.com
Fork in the road:uptake?
Abelardo Pardo Analytics for decision making in Learning Environments 11
Derable
flickr.com
High impact area: evolution, revolution, tangibleimprovements.
Abelardo Pardo Analytics for decision making in Learning Environments 12
Yoakimflickr.comFail at large scale and
become marginal
Business Week May 01, 2014
Abelardo Pardo Analytics for decision making in Learning Environments 13
400 data pointsper student
Teacher in-classrecordings
Where should I focus?
Abelardo Pardo Analytics for decision making in Learning Environments 14
visible-learning.org/hattie-ranking-influences-effect-sizes-learning-achievement/hattie-ranking-teaching-effects
(LastAccessed
Nov.
2014)
Abelardo Pardo Analytics for decision making in Learning Environments 15
Mary
Witzig
flickr.com
Personalized Learning in a Digital Environment
Abelardo Pardo Analytics for decision making in Learning Environments 16
Hiking
Artist
Learning Technology fails:hinders, prevents, complicates ...
Abelardo Pardo Analytics for decision making in Learning Environments 17
Anthony
flickr.com
Tech affordancesmust have strong,realisticpedagogicalfoundations
Abelardo Pardo Analytics for decision making in Learning Environments 18
VanessaOflickr.com
“Historically, humanity has made sense of the worldthrough discourse, dialogue, artifacts, myth, story,and metaphor. While those sensemaking approacheswon’t disappear, they will be augmented by data andanalytics.”
George Siemens, www.elearnspace.org/blog/2014/04/11/open-learning-analytics/
Abelardo Pardo Analytics for decision making in Learning Environments 19
JansonH
ews
flickr.com
New
Assessments?
Abelardo Pardo Analytics for decision making in Learning Environments 20
Ashley
Fisherflickr.com
21st century skills
Critical thinkingTeam work skillsCommunication skillsInformation literacyCreativity and Innovation
Discourse Analysis
Abelardo Pardo Analytics for decision making in Learning Environments 21
(De Liddo et al., 2011)
Abelardo Pardo Analytics for decision making in Learning Environments 22
PaulM
ayneflickr.com
Learning Dispositions:Tendency to behave incertain way when learning.
(Buckingham Sum, Deaking Crick, 2012)
Abelardo Pardo Analytics for decision making in Learning Environments 23
WernerK
unzFlickr.com
(Campbell, De Blois, Oblinger 2007, Academic Analytics, EDUCAUSE)
Collect
Report
Predict
Act
Refine
The fivesteps ofanalytics
Abelardo Pardo Analytics for decision making in Learning Environments 24
TristanM
artinflickr.com
LearningAnalytics Collecting
Observations
Example: Moodle Log
• Course name• Date and time• IP Address (geolocation)• User full nane• Action: discussion mark read, forum add
discussion, forum add post, forum deletediscussion, forum update post, notes view,resource view, etc.
• Additional information
Abelardo Pardo Analytics for decision making in Learning Environments 25
(Pardo, 2014)
Abelardo Pardo Analytics for decision making in Learning Environments 26
ChefC
ookeFlickr.comObserve while
working oncourse
activities
Abelardo Pardo Analytics for decision making in Learning Environments 27
Abelardo Pardo Analytics for decision making in Learning Environments 28
Track
Abelardo Pardo Analytics for decision making in Learning Environments 29
Track
Track
A clearly identifiedenvironment within
your computer
Abelardo Pardo Analytics for decision making in Learning Environments 30
Kevin
Davies
flickr.com
(Pardo & Delgado Kloos, 2011)
InstrumentationPersonal Computer
ComputerTool 1
ComputerTool 2
ComputerTool 3
ComputerTool 4
Authenticated
Virtual Computer
ComputerTool
ComputerTool
ComputerTool
ComputerTool
ComputerTool
EventDataset
PersonalData
Encoding
Report
Infer
Act
InstructorStudent
Abelardo Pardo Analytics for decision making in Learning Environments 31
Abelardo Pardo Analytics for decision making in Learning Environments 32
deepwarren
Flickr
Closer to the student
From App to companion
Finished focus group. Potential for studentuptake.
Abelardo Pardo Analytics for decision making in Learning Environments 33
Abelardo Pardo Analytics for decision making in Learning Environments 34
puthoOrP
hotographyflickr.com
LMS interactions, quizzes, response times, sessions,hints requested, questionnaires, interviews, requests,demographics, high school grades, etc.
(Bienkowski, M., Feng, M., & Means, B., 2012)
Abelardo Pardo Analytics for decision making in Learning Environments 35
Naezm
iflickr.com
Learning Record Stores. Tin Can API
Abelardo Pardo Analytics for decision making in Learning Environments 36
Space
&LightFlickr
• Data far away from instructor• Needs curation process
Abelardo Pardo Analytics for decision making in Learning Environments 37
TristanM
artinflickr.com
LearningAnalytics Collecting
ObservationsAnalysis andVisualization
Abelardo Pardo Analytics for decision making in Learning Environments 38
ww
w.snappvis.org
(LastaccessedN
ov.2014)
Bookmarklet in your browser
SAM
Abelardo Pardo Analytics for decision making in Learning Environments 39
(Govaerts et al., 2012)
Piazza
Abelardo Pardo Analytics for decision making in Learning Environments 40
Student Engagement with Videos
Abelardo Pardo Analytics for decision making in Learning Environments 41
Flipped Classroom: Preparation activities
Abelardo Pardo Analytics for decision making in Learning Environments 42
MCQ answers
Abelardo Pardo Analytics for decision making in Learning Environments 43
Mr.
Velocipedeflickr.com
Statisticalanalysis(small)
Predictivealgorithms
ClusteringAlgorithms
RelationshipMining
Data Distillation DiscoverywithModels
(Baker and Yacef, 2009)
Abelardo Pardo Analytics for decision making in Learning Environments 44
(Macfadyen & Dawson, 2010)
Abelardo Pardo Analytics for decision making in Learning Environments 45
Clustering, Classification, Rule inference(Romero, Ventura & García, 2008)
Abelardo Pardo Analytics for decision making in Learning Environments 46
TristanM
artinflickr.com
LearningAnalytics Collecting
ObservationsAnalysis andVisualization
Interventions
Abelardo Pardo Analytics for decision making in Learning Environments 47
DullH
unkflickr.com
Reduce attrition. Phone 10 students this semester.
Which students? When?
Abelardo Pardo Analytics for decision making in Learning Environments 48
neilspencerbruceflickr.com
What aspects to personalize?
Abelardo Pardo Analytics for decision making in Learning Environments 49
colecamp
flickr.com
A/B Testing: Space usage after change incourse/curriculum
Abelardo Pardo Analytics for decision making in Learning Environments 50
Design Principles for Interventions
• Integrated with the activity. Connected withconcepts and assessment.
• Interventions must support learning, notbecome an additional burden.
• Provide measures that are considereddesirable and why.
• Discuss the process with the students.
Abelardo Pardo Analytics for decision making in Learning Environments 51
(Wise, 2014)
Abelardo Pardo Analytics for decision making in Learning Environments 52
TristanM
artinflickr.com
LearningAnalytics Collecting
ObservationsAnalysis andVisualization
Interventions InstitutionalAdoption
Abelardo Pardo Analytics for decision making in Learning Environments 53
Secretlondon123
flickr.com
1 Context2 Questions3 Data + algorithms4 Level of intervention
Abelardo Pardo Analytics for decision making in Learning Environments 54
measure
Joeflickr.com
1 Context: Studentretention
2 Questions: Why dothey leave? Can it beavoided?
3 Data + algorithms: ?4 Level of intervention: ?
Abelardo Pardo Analytics for decision making in Learning Environments 55
Shaylorflickr.com
1 Context: Student success2 Questions: How did they
learn?3 Data + algorithms: ?4 Level of intervention: ?
Abelardo Pardo Analytics for decision making in Learning Environments 56
Salfalko
flickr.com
Data Wrangler: someone comfortable handlingstatistics, manipulating data for visualisation, andcapable of engaging with academics about the studentexperience and course design. It is their job toexperiment with different tools to interpret, visualiseand share information with academics as a basis forgaining actionable insights.
(Powell & MacNeil, 2012)
Kris
Krüg
Flickr
Analytics for decision making inLearning Environments
IARU EdTech Horizons WorkshopSingapore, 14 November 2014
Dr Abelardo Pardo (@abelardopardo)The University of Sydney
slideshare.net/abelardo_pardo
ReferencesBaker, R. S. J. D., and Yacef, K., 2009.The state of educational data mining in 2009: A review and future visions.Journal of Educational Data Mining, 1(1), 3–17.
Bienkowski, M., Feng, M., & Means, B. (2012).Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics. USDepartment of Education.
Liddo, A. De, Shum, S., Quinto, I. (2011)Discourse-centric learning analytics.In Proceedings of the International Conference on Learning Analytics and Knowledge.
Felder, R. M. (2006).The Way to Bet.Chemical Engineering Education, 40(1), 38–39.
Govaerts, S., Verbert, K., Duval, E., & Pardo, A. (2012).The Student Activity Meter for Awareness and Self-reflection.In ACM SIGCHI International Conference on Human Factors in Computing Systems (pp. 869–884).
Macfadyen, L. P., & Dawson, S. (2010).Mining LMS data to develop an “early warning system” for educators: A proof of concept.Computers & Education, 54(2), 588–599
Pardo, A., Delgado Kloos, C., 2011,Stepping out of the box. Towards analytics outside the Learning Management SystemInternational Conference on Learning Analytics and Knowledge, pp, 163-167, ACM New York, USA
Abelardo Pardo Analytics for decision making in Learning Environments 58
References II
Powell, S., & MacNeil, S. (2012).Analytics Series Institutional Readiness for Analytics (Vol. 1, pp. 1–11).JISC Center for Educational Technology & Interoperability Standards, Analytics Series, 1(8).
Shum, S. B., and Crick, R. D. (2012)Learning Dispositions and Transferable Competencies: Pedagogy, Modelling and Learning Analytics.In S. Buckingham Shum, D. Gaševic, and R. Ferguson (Eds.), International Conference on LearningAnalytics and Knowledge (pp. 92–101). ACM Press.
Siemens, G., & Long, P. (2011).Penetrating the Fog: Analytics in Learning and Education.Educause Review, 48(5),
Wise, A., 2014Designing Pedagogical Interventions to Support Student Use of Learning Analytics.In A. Pardo & S. D. Teasley (Eds.), Proceedings of the International Conference on Learning Analytics andKnowledge. ACM Press.
Abelardo Pardo Analytics for decision making in Learning Environments 59
