Embed Size (px)
Citation preview
From Incidental Learning Analytics Research to a
Sustainable Learning Analytics Program
Stefan T. Mol ([email protected])University of Amsterdam
UvA is currently here
UvA
Siemens et al. 2014
The UvAInform Project - History
• Initiated as a proposal from the ICTS Department• Expertise group Education (EGO) Reserved budget of
150K• EGO dislike of (bottom-up) tender procedure with limited
strategic vision• Focus Group Learning Analytics Established Late 2012• UvAInform proposal approved in June 2013• Central infrastructure (LRS)
• (De?)centralized pilots
UvAInform ProcessFocus Group: evaluation criteria
Definition of initial pilots
Discussion with an external expert (Erik Duval)
Challenge 1 – how to agree on the separate pilot(s)
Dilemma – To LRS or not to LRS
Resolution - Definition of LRS as separate project
All faculties present their research ideas
Challenge 2 – How to agree on pilot requirements and budgets
Resolution: 7 small pilots in 3 pilot clusters
Challenge 3 – How to manage ethics & privacy
Resolution – Faculty level ethics approval / Data Governance center
Learning Record Store
• Community sourced, secure, scalable repository/infrastructure • Store and retrieve statement data reliably and ensures a good
scalable storage layer for various types of data and data streams• Scales above 100 billion records.• These data can be made available in a secure and consistent
way for further analysis. • Upon this LRS infrastructure dashboards can be built or
developed for the delivery of (analysed) data to students, educators, and researchers
http://tincanapi.com
Cluster 1: Mirroring of traditional and non-traditional study performance to students
COACH2 - FNWI
Visualize the position of individuals in the context of the group using BB data
Using positioning of individual student to determine support from teaching staff.
Cluster Exam Feedback (qDNA - FMG)
More fine grained mirroring of exam results to provide students and teachers with insight in the development on four competencies (interpretation, analysis, evaluation, inference) and knowledge goals
Cluster 1: Mirroring of traditional and non-traditional study performance to students
Identifying Types of Effective Comparative Feedback and Relevant Mediators (FMG)
Cluster 1: Mirroring of traditional and non-traditional study performance to students
• Goal setting in education (FEB)• Students are instructed/ to formulate goals in a concrete
manner such that they are Specific, Measurable, Attainable, Relevant and Time-bound (SMART).
• Dashboard facilitates individual students to choose from, and set their own goals (and deadlines) against specific course events
• Dashboard shows students how they are scoring/succeeding in attaining the goals compared to their fellows
Cluster 2: Using specific student data to provide feedback for teachers
Learning Analytics Pilot “Turnitin” – Assignments Criteria Validation (FMG)
Visualization of Turnitin scores for students and teachers over (partial) assignments or courses
Do the grading schemes used for the evaluation of written assignments hold any validity, i.e. do they correlate with overall study results?
Teacher insight into Weblectures (FMG)
Visualization of use of video’s/weblectures to find potential problems, difficult material to improve
teaching, provide better remedial
material
Cluster 3: Using other people’s data to provide recommendation system to students
Validating Learning Analytics in Higher Ed. (FGW)
Determine the predictive validity of demographic data, learning styles, motivation, and behavior to optimize prediction of study success
Developing interventions (recommendations based) based on evidence
Positioning a learning analytics project
Centralized vs. decentralized
Research vs. practice
Imposed vs. desired
Technology vs. Pedagogy
Make or buy (resource oriented)
Generic infrastructure vs. pilot specific infrastructure in relation to UvA learning analytics lifecycle
Adoption of best practices or local identification thereof
Lessons Learned UvAInform
Top down approach currently bridge too farYet…• Need for generic infrastructure/standards • Need for access to data• Need for large scale experimentation• International collaboration• Data governance
Towards a Center for Data Governance and Innovation
• Supporting learning analytics (and other big data initiatives)• Evaluating/approving specific learning analytics projects,
ensuring ethical and legal compliance, • Centralization of knowledge • Streamlining policies (e.g. user agreements) so as to
facilitate Learning analytics (and other big data Initiatives)• Facilitating communications amongst key stakeholder• Managing public relations• Later: Establishing decision trees
Working scenario – LA Sandbox
Realize cheap centralized data warehouse for one educational program with full ethics clearance and
student informed consentExperiment with • Open dashboard and different requirements• Data governance• Randomized controlled trials
(dashboards/LMS’s/interventions, etc.)• Continuous delivery (adding data sources)• Intersections with blended learning/digital testing…
Q&A
