learning Design & learning Analytics

Venia Lengendi

María Jesús Rodríguez-­Trianamaria.rodrigueztriana@epfl.ch

May 30th, 2016 – Tallinn (Estonia)

Aligning

Outline

2

Background

Bachelor:Computer Science

PhD in ICT:Linking LD & LA in blended CSCL

Master:-­ Quality Engineering-­ Computer Science-­ Advanced Studies

Teacher:Computer Science

Researcher:GSIC @ Uva (Spain)

Researcher:React @ EPFL(Switzerland)

2000 2004 2005 2008 2010 Present2014

LD: Learning Design;; LA: Learning Analytics;; CSCL: Computer Supported Collaborative Learning;; IBL: Inquiry Based Learning3

PostDoc:Applying LA to IBL

4

Trends shaping education

-­ Globalisation

-­ The future of the nation state

-­ Are cities the new countries?

-­ Family matters

-­ Technology

5

[OECD, 2016]

European Trends and Grand Challenges in TEL

22 Grand Challenge Problems focusing on:

-­ Connecting Learners

-­ Orchestrating Learning

-­ Contextualising Learning

6

[Fischer et al., 2014] [Eberle et al., 2014]

Strategic goals for 2020:

-­ Change in the learning approach

-­ Competent and motivated teachers

-­ A digital focus in lifelong learning

-­ Align lifelong learning opportunities & labour market needs

-­ Equal opportunities & increased participation in lifelong learning

The Estonian Lifelong Learning Strategy 2020

7

(*) The Estonian Lifelong Learning Strategy 2020 (2014): https://www.hm.ee/sites/default/files/estonian_lifelong_strategy.pdf

8

9[Rodríguez-­Triana et al., 2016] [Nussbaum & Diaz, 2013] [Prieto et al.,2015]

LASI @ Bilbao (Spain) June 28th, 2016

Orchestrating Learning Analytics Workshop (OrLA 2016)

http://lprisan.wix.com/orla2016

10

María JesúsRodríguez-­Triana

EPFL

Luis PabloPrietoEPFL

RobertoMartinez-­Maldonado

UTS

YannisDimitriadisUVa

11

CEITER

Project Vision

Research question

How to provide that:

• Support new learning paradigms

• Integrate technological innovations

• Overcome the adoption challenges

12

• learning environments & tools• LA algorithms & visualizations

Proposal

-­ Pedagogical

-­ Technological

-­ Methodological

13

Proposal

-­ Pedagogical

-­ Technological

-­ Methodological

14

Learning scenario lifecycle

15

Learning Design

Design

ManagementAssessment

Reflection

? ?

?

[Dillenbourg, 2011a] [Dillenbourg, 2011b] [Prieto et al., 2011]

16

Learning Analytics

Design

ManagementAssessment

Reflection

Learning scenario lifecycle

?

[Ferguson, 2012] [Sutherland et al., 2012]

ADA – Aligning learning Design & learning Analytics

17

[Wise, 2014] [Bakharia, et al., 2016][Rodríguez-­Triana et al., 2015b] [Vozniuk et al., 2015]

[Manske et al., 2015]

Benefits• Orchestration support

• Pedagogically grounded tools• LA-­aware learning designs• Supervised designs• Contextualised LA• Formative assessment• Personalization & gamification

Challenges• Obtain learning designs minimizing

the user workload• Inform the teacher about the LA

features of the learning tools• Management of Ethical & Privacy

issues

ECTEL @ Lyon (France) June 28th, 2016

WS on Connecting Learning Analytics and Learning Design – CLAD 2016 http://clad2016.ld-­grid.org/

SI on Connecting Learning Design and Learning Analytics the Interaction Design & Architecture(s) journal: http://www.mifav.uniroma2.it

18

MJ. Rodríguez-­TrianaEPFL

Luis P. PrietoEPFL

Steven WarburtonUS

Paul Salvador InventadoCMU

Peter ScupelliCMU

Davinia Hernández-­LeoUPF

YishayMorPAU

Bart RientiesOU

Proposal

-­ Pedagogical

-­ Technological

-­ Methodological

19

CEITER infrastructure

20

Data warehouse

Data analyses & visualization Orchestration tools

Pedagogically grounded learning tools& other ICT tools

DesignImplementation

[Rodríguez-­Triana et al., 2011] [Rodríguez-­Triana et al., 2016c][Schwendimann et al., 2016] [Vozniuk et al., 2015]

Standards (activity tracking, learning objects)

Ethical & privacy issues

Data filtering& Integration

Design decisions

Pedagogically grounded analyses

JTELSS @ Tallinn (Estonia) June 28th, 2016

WS on Ethics & Privacy in Learning Analytics

http://go.epfl.ch/JTELSS2016_EPLA

21

AndriiVozniukEPFL

María JesúsRodríguez-­Triana

EPFL

DenisGilletEPFL

ORGANISERS

LASI @ Bilbao (Spain) June 28th, 2016

WS on Contextual Learning analytics Enforcing data Ownership (CLEO).

Building Learning Dashboards with Elasticsearch, Logstash & Kibana

http://go.epfl.ch/LASI-­CLEO2016

22

AndriiVozniukEPFL

María JesúsRodríguez-­Triana

EPFL

DenisGilletEPFL

ORGANISERS

CEITER infrastructure

23[Rodríguez-­Triana et al., 2011] [Rodríguez-­Triana et al., 2016c][Schwendimann et al., 2016] [Vozniuk et al., 2015]

Benefits• Richer data -­> More accurate & deep analyses• Interoperability between learning & analyses tools• Ethically & privacy respectful

Challenges• Obtain learning designs• Degradation of privacy settings• Maintain accuracy when students opt out of the data collection• Align learning activities, competencies & assessment• Integrate personal, educational & institutional ecosystems

Proposal

-­ Pedagogical

-­ Technological

-­ Methodological

24

Methodology Benefits

25

• Iterative process• Agile prototyping • Refinement

Developm

ent

• Socio-­Cultural Activity Theory• Design Based Research• Participatory design

• Integration of user needs• Engagement (interests, motivational & gender aspects)• ICT / Data Literacy• Contextual constraintsRe

search

[Rodríguez-­Triana et al., 2015a] [Rodríguez-­Triana et al., 2015b] [Rodríguez-­Triana et al., 2016a]

• Pre-­service & In-­service teachers• School, vocational & university settings

• Enhance ICT & data literacy• Long term adoption• Learning to learnPi

loting &

training

• Mixed methods• Evidence-­based• Authentic studies

• Teaching gains/adoption barriers

• Learning gains/adoption barriersEvaluation

Overcoming CEITER challenges

26[Karasavvidis, 2009]

Learning Evidence

EducationalInnovationAdoption

Currenteducational settings

• Support new learning paradigms • Integrate technological innovations• Overcome the adoption challenges

27

Building our network and planning future research

28

Estonia: Ministry of Education & Research, ArchimedesNear countries: Interreg Central Baltic, EEA & Norway grants, Nordplus

Learning Analytics

Learning Design & Evaluation (IBL)

Infraestructure

(*) Ministry of Education and Research: https://www.hm.ee/en/activities/research-­and-­development/research-­programmes(*) Archimedes foundation: http://archimedes.ee/en/foundation/(*) InterregCentral Baltic: http://centralbaltic.eu/programme-­pages/specific-­objective-­42(*) EEA & Norway grants: http://eeagrants.org(*) Nordic Council of Ministers programme in the area of lifelong learning: http://www.nordplusonline.org/

Building our network and planning future research

29

Learning Analytics

Learning Design (CSCL/IBL)

Infrastructure & HCI (KMS, mobile)

Communities & Networks (ATEE, EUPAS, PAU, GoLab)

Ministries of Education (EUN)

European level: H2020, Erasmus+, Bilateral programs

(*) Horizon 2020: http://ec.europa.eu/programmes/horizon2020/(*) Erasmus+: http://ec.europa.eu/programmes/erasmus-­plus(*) Estonian national scholarship programme for international students, researchers and academic staff: http://adm.archimedes.ee(*) Swiss National Foundation: http://www.snf.ch/en/funding/programmes/sinergia(*) Spanish I+D+I international program: http://www.idi.mineco.gob.es/

Building our network and planning future research

International level: H2020, MSCA, Bilateral programmes (Endeavour Scholarships, ARC)

30

Learning Analytics

Learning Design (CSCL/IBL)

(*) Endeavour Scholarships: http://orau.org/endeavourscholarships/(*) Australian Research Council: http://www.arc.gov.au/

Coming calls

• Spain -­ Programme for Research Aimed at the Challenges of Society (June 16th, 2016)

• Australia -­ Endeavour Scholarships (June 30th, 2016)

• Switzerland -­ Sinergia (December 1st, 2016)

• H2020 (December 8th, 2016)

• ICT-­11-­2017: Collective Awareness Platforms for Sustainability and Social Innovation

• ICT-­15-­2016-­2017 Big Data PPP: Large Scale Pilot actions in sectors best benefitting from data-­driven innovation

31

• Interreg Central Baltic (February 2nd, 2017)

• Skilled and socially inclusive region (vocational education and training)

• Horizontal objectives (Equality, ICT)

Concluding remarks

32

-­ Pedagogical

-­ Technological

-­ Methodological

ADA

How to provide that:

• Support new learning paradigms

• Integrate technological innovations

• Overcome the adoption challenges

learning environments & tools

LA algorithms & visualizationsCEITER

33

?… questions, comments

Thank you

for your attention

María Jesús Rodríguez-­TrianaEmail: maria.rodrigueztriana@epfl.chSkype contact: chus_mjrt

34

References I

35

• [Bakharia et al., 2016] A. Bakharia, L. Corrin, P. de Barba, G. Kennedy, D. Gasevic, R. Mulder, D. Williams, S. Dawson, and L. Lockyer. A Conceptual Framework linking Learning Design with Learning Analytics Learning Analytics. In 6th International Conference on Learning Analytics and Knowledge, 2016.

• [Dillenbourg, 2011a] P. Dillenbourg. Trends in classroomorchestration. Second research & technology scouting report, 2011.• [Dillenbourg, 2011b] P. Dillenbourg. Design for orchestration. April, 2011.• [Eberle et al., 2014] J. Eberle, K. Lund, P. Tchounikine, and F. Fischer, Challenge Problems in Technology-­Enhanced Learning II: MOOCs and

Beyond Perspectives for Research, Practice, and Policy Making Developed at the Alpine Rendez-­Vous in Villard-­ de-­Lans. London: Springer, 2014.

• [Ferguson, 2012] R. Ferguson. Learning analytics: drivers, developments and challenges. International Journal of Technology EnhancedLearning, 4(5/6), 2012

• [Fischer et al., 2014] F. Fischer, F. Wild, R. Sutherland, and L. Zirn, Grand Challenges in Technology Enhanced Learning: Outcomes of the 3rd Alpine Rendez-­Vous. London: Springer, 2014.

• [Karasavvidis, 2009] I. Karasavvidis, Activity theory as a conceptual framework for understanding teacher approaches to information and communication technologies, Computers & Education. 53 (2): 436–444, 2009.

• [Manske et a;;., 2015 ] S. Manske, I.A. Chounta, M. J. Rodríguez-­Triana, D. Gillet, H.U. Hoppe. Exploring Deviation in Inquiry Learning: Degrees of Freedom or Source of Problems. Proceedings of the 23rd International Conference on Computers in Education, ICCE 2015. Hangzhou, China, 2015

• [Nussbaum & Díaz, 2013] M. Nussbaum, A. Diaz. “Classroom logistics: Integrating digital and non-­digital resources”. Computers & Education. 69: 493-­495, 2013

• [OECD, 2016] OECD. Trends Shaping Education 2016, OECD Publishing, Paris. 2016• [Prieto et al., 2011] L. P. Prieto, M. Holenko Dlab, I. Gutiérrez, M. Abdulwahed, and W. Balid. Orchestrating technology enhanced learning: a

literature review and a conceptual framework. International Journal of Technology EnhancedLearning, 3(6):583-­598, 2011.[Prieto et al., 2016 ] L. P. Prieto, K. Sharma and P. Dillenbourg. Studying teacher orchestration load in technology-­enhanced classrooms: A mixed-­method approach and case study. 10th European Conference on Technology-­Enhanced Learning (EC-­TEL 2015), Toledo, Spain, 2015.

• [Rodríguez-­Triana et al., 2016a] M. J. Rodríguez-­Triana, A. C. Holzer, L. P., Prieto, and D. Gillet, Examining the effects of social media in co-­located classrooms: A case study based on SpeakUp. 11th European Conference on Technology Enhanced Learning: Adaptive and adaptable learning. ECTEL2016. Lyon, France, 2016.

• [Rodríguez-­Triana et al., 2015a] M. J. Rodríguez-­Triana, A. Holzer, A. Vozniuk, D. Gillet. “Orchestrating Inquiry-­Based Learning Spaces: an Analysis of Teacher Needs”. Proceedings of the 14th International Conference of Web Learning. ICWL 2015. Guangzhou, China. November 2015.

References II

36

• [Rodríguez-­Triana, et al. 2011] M. J. Rodríguez-­Triana, A. Martínez-­Monés J. I. Asensio-­Pérez. MonitoringCollaboration in Flexible and Personal Learning Environments. Special issue on: EvaluatingEducativeExperiences of Flexible and Personal Learning Environments. ID&A Journal. 11(12):51-­63, November 2011.

• [Rodríguez-­Triana, et al. 2013] M. J. Rodríguez-­Triana, A. Martínez-­Monés, J. I. Asensio-­Pérez, and Y. Dimitriadis. Towards a Script-­AwareMonitoring Process of Computer-­Supported Collaborative LearningScenarios. International Journal on Technology Enhanced Learning. Specialissue on: Learning Analytics. 5 (2):151-­167, 2013.

• [Rodríguez-­Triana et al., 2015b] M. J. Rodríguez-­Triana, A. Martínez-­Monés, J. I. Asensio-­Pérez, and Y. Dimitriadis. Scripting and monitoring meet each other: Aligning learning analytics and learning design to support teachers in orchestrating CSCL situations. British Journal of Educational Technology, vol. 46, no. 2, pp. 330–343, 2015.

• [Rodríguez-­Triana et al., 2015c] Rodríguez-­Triana, M. J., Martínez-­Monés, A., Asensio-­Pérez, J. I., & Dimitriadis, Y. A monitoring-­aware learning design process: Pilot studies in authentic CSCL scenarios. Computing & Informatics, 2015

• [Rodríguez-­Triana et al., 2016b] M. J. Rodríguez-­Triana, A. Martínez-­Monés, S.Villagrá-­Sobrino. Learning Analytics in small-­scale teacher-­led innovations: Ethical and data privacy issues. Journal of Learning Analytics. Special section on: Ethics and Privacy for Learning Analytics. 3 (1): 43-­65, 2016.

• [Rodríguez-­Triana et al., 2016c] M. J. Rodríguez-­Triana, L. P. Prieto, A. Vozniuk, M. Shirvani Boroujeni and B. A. Schwendimannet al. Monitoring, Awareness and Reflection in Blended Technology Enhanced Learning: a Systematic Review, accepted in International Journal of Technology Enhanced Learning.

• [Schwendimann et al., 2016] B. A. Schwendimann, M. J. Rodríguez-­Triana, A. Vozniuk, L. P. Prieto and M. Shirvani Boroujeni et al. Understanding learning at a glance: An overview of learning dashboard studies. International Learning Analytics and Knowledge, Edinburgh, UK, 2016.

• [Vozniuk et al., 2015] A. Vozniuk, M. J. Rodríguez-­Triana, A. C. Holzer, S. Govaerts, D. Sandoz, and D. Gillet, Contextual Learning Analytics Apps to Create Awareness in Blended Inquiry Learning, 14th International Conference on Information Technology Based Higher Education and Training. Lisbon, Portugal, 2015.

• [Wise, 2014] A. F. Wise. Designing pedagogical interventions to support student use of learning analytics. In Proceedings of the 4th International Conference on Learning Analytics And Knowledge -­ LAK ’14, 2014, pp. 203–211.