Analytics for decision making in Learning Environments

Kris

Krüg

Flickr

Analytics for decision making inLearning Environments

IARU EdTech Horizons WorkshopSingapore, 14 November 2014

Dr Abelardo Pardo (@abelardopardo)The University of Sydney

slideshare.net/abelardo_pardo

Abelardo Pardo Analytics for decision making in Learning Environments 2

TristanM

artinflickr.com

LearningAnalytics Collecting

ObservationsAnalysis andVisualization

Interventions InstitutionalAdoption

Abelardo Pardo Analytics for decision making in Learning Environments 3

TristanM

artinflickr.com

LearningAnalytics

NY Times 24/11/12

Abelardo Pardo Analytics for decision making in Learning Environments 4

Abelardo Pardo Analytics for decision making in Learning Environments 5

jbgeronimiflickr.com

Learning is complex and opaque

Uncertainty in Education

“Even in education,however, there aresome propositionsthat give you agreat chance ofcomming out aheadif you bet on themoften enough”

Abelardo Pardo Analytics for decision making in Learning Environments 6

(Felder, 2009)

Learning Analytics

Measurement, collection, analysisand reporting of data about learnersand their contexts, for purposes ofunderstanding and optimizinglearning and the environments in whichit occurs.

Abelardo Pardo Analytics for decision making in Learning Environments 7

(SoLAR, Society for LearningAnalytics Research)

Abelardo Pardo Analytics for decision making in Learning Environments 8

http://ww

w.learninganalytics.net/?p=131

(lastvisitedS

ep2013)

(Siemens & Long, 2011)

www.solaresearch.orgAbelardo Pardo Analytics for decision making in Learning Environments 9

Abelardo Pardo Analytics for decision making in Learning Environments 10

LJubarflickr.com

Fork in the road:uptake?

Abelardo Pardo Analytics for decision making in Learning Environments 11

Derable

flickr.com

High impact area: evolution, revolution, tangibleimprovements.

Abelardo Pardo Analytics for decision making in Learning Environments 12

Yoakimflickr.comFail at large scale and

become marginal

Business Week May 01, 2014

Abelardo Pardo Analytics for decision making in Learning Environments 13

400 data pointsper student

Teacher in-classrecordings

Where should I focus?

Abelardo Pardo Analytics for decision making in Learning Environments 14

visible-learning.org/hattie-ranking-influences-effect-sizes-learning-achievement/hattie-ranking-teaching-effects

(LastAccessed

Nov.

2014)

Abelardo Pardo Analytics for decision making in Learning Environments 15

Mary

Witzig

flickr.com

Personalized Learning in a Digital Environment

Abelardo Pardo Analytics for decision making in Learning Environments 16

Hiking

Artist

Learning Technology fails:hinders, prevents, complicates ...

Abelardo Pardo Analytics for decision making in Learning Environments 17

Anthony

flickr.com

Tech affordancesmust have strong,realisticpedagogicalfoundations

Abelardo Pardo Analytics for decision making in Learning Environments 18

VanessaOflickr.com

“Historically, humanity has made sense of the worldthrough discourse, dialogue, artifacts, myth, story,and metaphor. While those sensemaking approacheswon’t disappear, they will be augmented by data andanalytics.”

George Siemens, www.elearnspace.org/blog/2014/04/11/open-learning-analytics/

Abelardo Pardo Analytics for decision making in Learning Environments 19

JansonH

ews

flickr.com

New

Assessments?

Abelardo Pardo Analytics for decision making in Learning Environments 20

Ashley

Fisherflickr.com

21st century skills

Critical thinkingTeam work skillsCommunication skillsInformation literacyCreativity and Innovation

Discourse Analysis

Abelardo Pardo Analytics for decision making in Learning Environments 21

(De Liddo et al., 2011)

Abelardo Pardo Analytics for decision making in Learning Environments 22

PaulM

ayneflickr.com

Learning Dispositions:Tendency to behave incertain way when learning.

(Buckingham Sum, Deaking Crick, 2012)

Abelardo Pardo Analytics for decision making in Learning Environments 23

WernerK

unzFlickr.com

(Campbell, De Blois, Oblinger 2007, Academic Analytics, EDUCAUSE)

Collect

Report

Predict

Act

Refine

The fivesteps ofanalytics

Abelardo Pardo Analytics for decision making in Learning Environments 24

TristanM

artinflickr.com

LearningAnalytics Collecting

Observations

Example: Moodle Log

• Course name• Date and time• IP Address (geolocation)• User full nane• Action: discussion mark read, forum add

discussion, forum add post, forum deletediscussion, forum update post, notes view,resource view, etc.

• Additional information

Abelardo Pardo Analytics for decision making in Learning Environments 25

(Pardo, 2014)

Abelardo Pardo Analytics for decision making in Learning Environments 26

ChefC

ookeFlickr.comObserve while

working oncourse

activities

Abelardo Pardo Analytics for decision making in Learning Environments 27

Abelardo Pardo Analytics for decision making in Learning Environments 28

Track

Abelardo Pardo Analytics for decision making in Learning Environments 29

Track

Track

A clearly identifiedenvironment within

your computer

Abelardo Pardo Analytics for decision making in Learning Environments 30

Kevin

Davies

flickr.com

(Pardo & Delgado Kloos, 2011)

InstrumentationPersonal Computer

ComputerTool 1

ComputerTool 2

ComputerTool 3

ComputerTool 4

Authenticated

Virtual Computer

ComputerTool

ComputerTool

ComputerTool

ComputerTool

ComputerTool

EventDataset

PersonalData

Encoding

Report

Infer

Act

InstructorStudent

Abelardo Pardo Analytics for decision making in Learning Environments 31

Abelardo Pardo Analytics for decision making in Learning Environments 32

deepwarren

Flickr

Closer to the student

From App to companion

Finished focus group. Potential for studentuptake.

Abelardo Pardo Analytics for decision making in Learning Environments 33

Abelardo Pardo Analytics for decision making in Learning Environments 34

puthoOrP

hotographyflickr.com

LMS interactions, quizzes, response times, sessions,hints requested, questionnaires, interviews, requests,demographics, high school grades, etc.

(Bienkowski, M., Feng, M., & Means, B., 2012)

Abelardo Pardo Analytics for decision making in Learning Environments 35

Naezm

iflickr.com

Learning Record Stores. Tin Can API

Abelardo Pardo Analytics for decision making in Learning Environments 36

Space

&LightFlickr

• Data far away from instructor• Needs curation process

Abelardo Pardo Analytics for decision making in Learning Environments 37

TristanM

artinflickr.com

LearningAnalytics Collecting

ObservationsAnalysis andVisualization

Abelardo Pardo Analytics for decision making in Learning Environments 38

ww

w.snappvis.org

(LastaccessedN

ov.2014)

Bookmarklet in your browser

SAM

Abelardo Pardo Analytics for decision making in Learning Environments 39

(Govaerts et al., 2012)

Piazza

Abelardo Pardo Analytics for decision making in Learning Environments 40

Student Engagement with Videos

Abelardo Pardo Analytics for decision making in Learning Environments 41

Flipped Classroom: Preparation activities

Abelardo Pardo Analytics for decision making in Learning Environments 42

MCQ answers

Abelardo Pardo Analytics for decision making in Learning Environments 43

Mr.

Velocipedeflickr.com

Statisticalanalysis(small)

Predictivealgorithms

ClusteringAlgorithms

RelationshipMining

Data Distillation DiscoverywithModels

(Baker and Yacef, 2009)

Abelardo Pardo Analytics for decision making in Learning Environments 44

(Macfadyen & Dawson, 2010)

Abelardo Pardo Analytics for decision making in Learning Environments 45

Clustering, Classification, Rule inference(Romero, Ventura & García, 2008)

Abelardo Pardo Analytics for decision making in Learning Environments 46

TristanM

artinflickr.com

LearningAnalytics Collecting

ObservationsAnalysis andVisualization

Interventions

Abelardo Pardo Analytics for decision making in Learning Environments 47

DullH

unkflickr.com

Reduce attrition. Phone 10 students this semester.

Which students? When?

Abelardo Pardo Analytics for decision making in Learning Environments 48

neilspencerbruceflickr.com

What aspects to personalize?

Abelardo Pardo Analytics for decision making in Learning Environments 49

colecamp

flickr.com

A/B Testing: Space usage after change incourse/curriculum

Abelardo Pardo Analytics for decision making in Learning Environments 50

Design Principles for Interventions

• Integrated with the activity. Connected withconcepts and assessment.

• Interventions must support learning, notbecome an additional burden.

• Provide measures that are considereddesirable and why.

• Discuss the process with the students.

Abelardo Pardo Analytics for decision making in Learning Environments 51

(Wise, 2014)

Abelardo Pardo Analytics for decision making in Learning Environments 52

TristanM

artinflickr.com

LearningAnalytics Collecting

ObservationsAnalysis andVisualization

Interventions InstitutionalAdoption

Abelardo Pardo Analytics for decision making in Learning Environments 53

Secretlondon123

flickr.com

1 Context2 Questions3 Data + algorithms4 Level of intervention

Abelardo Pardo Analytics for decision making in Learning Environments 54

measure

Joeflickr.com

1 Context: Studentretention

2 Questions: Why dothey leave? Can it beavoided?

3 Data + algorithms: ?4 Level of intervention: ?

Abelardo Pardo Analytics for decision making in Learning Environments 55

Shaylorflickr.com

1 Context: Student success2 Questions: How did they

learn?3 Data + algorithms: ?4 Level of intervention: ?

Abelardo Pardo Analytics for decision making in Learning Environments 56

Salfalko

flickr.com

Data Wrangler: someone comfortable handlingstatistics, manipulating data for visualisation, andcapable of engaging with academics about the studentexperience and course design. It is their job toexperiment with different tools to interpret, visualiseand share information with academics as a basis forgaining actionable insights.

(Powell & MacNeil, 2012)

Kris

Krüg

Flickr

Analytics for decision making inLearning Environments

IARU EdTech Horizons WorkshopSingapore, 14 November 2014

Dr Abelardo Pardo (@abelardopardo)The University of Sydney

slideshare.net/abelardo_pardo

ReferencesBaker, R. S. J. D., and Yacef, K., 2009.The state of educational data mining in 2009: A review and future visions.Journal of Educational Data Mining, 1(1), 3–17.

Bienkowski, M., Feng, M., & Means, B. (2012).Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics. USDepartment of Education.

Liddo, A. De, Shum, S., Quinto, I. (2011)Discourse-centric learning analytics.In Proceedings of the International Conference on Learning Analytics and Knowledge.

Felder, R. M. (2006).The Way to Bet.Chemical Engineering Education, 40(1), 38–39.

Govaerts, S., Verbert, K., Duval, E., & Pardo, A. (2012).The Student Activity Meter for Awareness and Self-reflection.In ACM SIGCHI International Conference on Human Factors in Computing Systems (pp. 869–884).

Macfadyen, L. P., & Dawson, S. (2010).Mining LMS data to develop an “early warning system” for educators: A proof of concept.Computers & Education, 54(2), 588–599

Pardo, A., Delgado Kloos, C., 2011,Stepping out of the box. Towards analytics outside the Learning Management SystemInternational Conference on Learning Analytics and Knowledge, pp, 163-167, ACM New York, USA

Abelardo Pardo Analytics for decision making in Learning Environments 58

References II

Powell, S., & MacNeil, S. (2012).Analytics Series Institutional Readiness for Analytics (Vol. 1, pp. 1–11).JISC Center for Educational Technology & Interoperability Standards, Analytics Series, 1(8).

Shum, S. B., and Crick, R. D. (2012)Learning Dispositions and Transferable Competencies: Pedagogy, Modelling and Learning Analytics.In S. Buckingham Shum, D. Gaševic, and R. Ferguson (Eds.), International Conference on LearningAnalytics and Knowledge (pp. 92–101). ACM Press.

Siemens, G., & Long, P. (2011).Penetrating the Fog: Analytics in Learning and Education.Educause Review, 48(5),

Wise, A., 2014Designing Pedagogical Interventions to Support Student Use of Learning Analytics.In A. Pardo & S. D. Teasley (Eds.), Proceedings of the International Conference on Learning Analytics andKnowledge. ACM Press.

Abelardo Pardo Analytics for decision making in Learning Environments 59