Director, Office of Learning and TeachingOctober 29, 2014

Learning Analytics: Exploring Strengths-basedApproaches

Flinders University Workshop| 29 October 2014| Slide 2

• OLT has commissioned projects to identify learning analytics technologies,

processes and polices Australian universities are employing to address

student retention

• 2 funded teams (working collaboratively)

• Target policy, future direction and international comparisons (UniSA, UTS,

Macquarie, UMelb, USC, UNE)

• Identify current technologies and practices (CDU, Griffith, BIITE,

UNewcastle, Murdoch)

OLT Commissioned Projects

Flinders University Workshop| 29 October 2014| Slide 3

Aims

- map current (and proposed) LA activity in Australian institutions

- identify the drivers shaping, and factors affording and constraining, its adoption

- Comparatively situate Australian LA activity within the international context

Outcomes:

- produce a ‘roadmap’ of best practice, principles and resources that can inform and assist institutions in their LA activity

- develop resources that provide an overview of LA including definitions, applications and tools within Australian universities, the overarching barriers and potential opportunities surrounding its deployment, future trends and ethical issues

- A Good Practice Guide that will provide case studies of best practice that will assist institutions in future LA activity.

Project Aims & Outcomes

Flinders University Workshop| 29 October 2014| Slide 4

1. Semi-structured one-on-one interviews with:- internationally-recognised research experts in learning analytics- higher education leaders with responsibility for LA around their institution’s LA strategy,

progress and goals (completed)- DVC’s in Australia around their institution’s LA strategy, progress and goals- Academics using learning analytics

2. Delphi-method exercise - involving panel of international and national experts to explore and provide consensus

on themes arising from interviews and survey

3. Survey data- Snapshot of current institutional level infrastructure, policy and strategy to support and

use LA (survey deployed to DVCAs)- Investigation of academic understanding and use of LA in Australian institutions

Project Methods

Flinders University Workshop| 29 October 2014| Slide 5

URL: http://cduhes.asia.qualtrics.com/SE/?SID=SV_bpU11UZMewG8uc5

Academic Survey:

Flinders University Workshop| 29 October 2014| Slide 6

“Learning Analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs.” (LAK, 2011)

School

Flinders University Workshop | 29 October 2014| Slide 7

‘Learning Analytics is about collecting traces that learners leave behind and using those to improve learning’ (Duval, 2012)

Flinders University Workshop | 29 October 2014| Slide 8

Drivers

• Technological ability to deal with large data sets

• Power of technology to integrate data within and across systems

• Speed with which data can be presented• Use of LMS• New types of data for a digital footprint

Flinders University Workshop | 29 October 2014| Slide 9

Potential benefits• potential for measuring, quantifying,

explicating, describing student experiences and learning patterns

• which can lead to better predicting, forecasting and understanding of learning and teaching

• help universities to improve its services and assist in student retention, improvement of grades and enhance experiences.

Flinders University Workshop| 29 October 2014| Slide 10

• System integration:• Functionality of LMS• Analytics packages for the LMS• Data warehouses

• Other factors include strategy, policy, application uses

Stages of maturity – one factor is system integration

Flinders University Workshop| 29 October 2014| Slide 11

• What could analytics be used for?• What kind of questions would you like to be able to answer

with analytics?

 

Key Questions:

Flinders University Workshop | 29 October 2014| Slide 12

Flinders University Workshop| 29 October 2014| Slide 13

• Kantian – concepts of rights and duty

• Virtue – action is based on virtue of the agent

• Utilitarian – action should benefit the majority

Ethical Frameworks

Flinders University Workshop| 29 October 2014| Slide 14

• Technical considerations / issues• Customisations - Student Information System,

University & Australian context• Data models & student variables to integrate

• (E.G Student demographics, academic details, course & unit details)

• Infrastructure & expertise• Time to establish

Implementation

Flinders University Workshop| 29 October 2014| Slide 15

• Rollout & Deployment• Preparedness of academic community• Use case scenarios and actions• Grade Centre, best practices• Governance, policy, procedure & privacy• University executive reporting & understanding • Potential resourcing implications

Implementation

Flinders University Workshop| 29 October 2014| Slide 16

• What do you think the main reason for using analytics is? Why? 

• Is it ethical to just focus on ‘at risk’ students? What about all of the other students?•  

• Who should have access to reports that can be generated about learning design? • What if we find out that our design needs improvement? • What are the ethics around taking action? • What does this mean for the support that universities should provide to academics to assist them in this?

What if we don’t have the resources to do this? 

• What issues might it raise if the teaching is more public?

Discussion Questions

Flinders University Workshop| 29 October 2014| Slide 17

• What kinds of information should we share with our students? • For example, if we know some students do better with some designs than others, should we tell them?• Student dashboards that show their grade relative to others in the class?

 

• Rolling out analytics• Policy or rollout first? • Should we rollout the student dashboard? What should we include in it? On what basis?

 

• Think about all the different student ‘traces’ in your institution (e.g. library use, swipe cards, grades)

• How could you use this data?• What would the benefits of such data be?• What data would be missing in this approach?

Discussion Questions

Flinders University Workshop | 29 October 2014| Slide 18

But….

• “Everything that can be counted does not necessarily count; everything that counts cannot necessarily be counted.” –Albert Einstein

Flinders University Workshop| 29 October 2014| Slide 19

Duval, E. (2011) Attention Please! Learning Analytics for Visualization and Recommendation. LAK11: 1st International Conference on Learning Analytics and Knowledge, Banff, Canada February 27-March 1, 2011

Learning Analytics and Knowledge (2011) Proceedings of the 1st International Conference on Learning Analytics and Knowledge, Banff, Canada February 27-March 1, 2011

Reamer, F. (1993). The philosophical foundations of social work. New York, Columbia University Press.

References