Data Explorer (Learning Analytics)Site Explorer Guide version 1.0

Contents

1 Overview.............................................................................................................................................................. 2

2 Who is This Guide For?....................................................................................................................................... 2

3 How to Access the System.................................................................................................................................. 2

4 Site Explorer View............................................................................................................................................... 3

5 Student Engagement Indicator (RAG Indicator)..................................................................................................4

5.1 Traffic Light Calculations (TLC)...................................................................................................................4

5.2 Red, Amber, Green (RAG) Indicator...........................................................................................................5

6 Accessing ‘Students’........................................................................................................................................... 6

6.1 My Students................................................................................................................................................ 6

6.2 Student Search........................................................................................................................................... 6

6.3 Student Overview........................................................................................................................................ 7

6.4 Overview Data............................................................................................................................................ 7

6.4.1 General Indicators................................................................................................................................... 8

6.4.2 Data Visualisations.................................................................................................................................. 8

6.4.3 Module Traffic Lights............................................................................................................................. 10

6.5 Module Data.............................................................................................................................................. 10

6.5.1 Data Visualisations................................................................................................................................ 10

6.5.2 Module Specific Traffic Lights...............................................................................................................12

6.6 Notes......................................................................................................................................................... 12

6.7 Bio............................................................................................................................................................. 13

7 Accessing ‘Courses’.......................................................................................................................................... 13

7.1 My Courses............................................................................................................................................... 13

7.2 Course Search.......................................................................................................................................... 14

7.3 Course Overview...................................................................................................................................... 14

8 Accessing ‘Modules’.......................................................................................................................................... 15

8.1 My Modules............................................................................................................................................... 15

8.2 Module Search.......................................................................................................................................... 15

8.3 Module Overview...................................................................................................................................... 16

9 Accessing ‘Department and Course View’.........................................................................................................17

Data Explorer (Learning Analytics) Site Explorer Guide v.1 | 1

1 OverviewLearning Analytics at City provides a dashboard that allows institutions to view student engagement data from multiple systems in one central location. The dashboard, which was created by Jisc, is called Data Explorer.

Currently Data Explorer contains virtual engagement information for undergraduate and postgraduate students studying at City, University of London.

This virtual engagement information is based on when a student has last logged into a module on City’s Virtual Learning Environment (Moodle) and how many times they have clicked on different resources, e.g. lecture slides.

2 Who is This Guide For?This guide is suitable for anyone who has been given Site Explorer access by the system administration team. Individuals with Site Explorer access will have an overview of students within schools or departments, and restricted access to courses and modules.

Examples of individuals with this access are Heads of Academic Services, Course Officers, Wellbeing Officers, Attendance Officers, and any other role the school has requested.

If you would like to request access, please log your request in Service Now and a member of the Educational Technology Team will contact you.

3 How to Access the SystemTo access Data Explorer please use the following webpage: https://datax.jisc.ac.uk/

The login uses City’s institutional authentication page. Select City, University of London, hit login and then enter your username and password.

Data Explorer (Learning Analytics) Site Explorer Guide v.1.0 | 2

4 Site Explorer ViewOnce you have logged into Data Explorer, you will see these icons:

Please note: There are restrictions within the system around access to data. You will only see the data that you have been given access to. This may be at school or department level.

These icons allow access for the following:

ModulesYou can access information on all the modules that you have been granted access to by searching for a specific module. This shows an overview of the students on that module.

CoursesYou can access information on all the courses that you have been granted access to by searching for a specific course. This shows an overview of the students on that course.

Department and Course viewYou can access information on all the departments and courses that you have been granted access to. This shows an overview of students by department or course.

StudentsYou can access information on all the students that you have been granted access to by searching for a specific student. This shows an overview of the student and their engagement with their course and specific modules.

TutorsYou can search for a specific tutor and see an overview of the students they tutor. You will also have the option to log in as that tutor and see their students.

Please note: You can also access the Site Explorer view from the menu found in the top left-hand corner of the dashboard.

Data Explorer (Learning Analytics) Site Explorer Guide v.1.0 | 3

5 Student Engagement Indicator (RAG Indicator)

The RAG (red, amber, green) Indicator is a rule-based tool to identify student behaviours that are often associated with a lack of engagement and risk of failing. The RAG indicator is an overall engagement indicator that is calculated using the Traffic Light Calculations (TLC’s) for each module.

5.1 Traffic Light Calculations (TLC)Student engagement is measured on a per module basis and on three categories:

1. Virtual Engagement – use of online tools such as the virtual learning environment (Moodle)

2. Physical Engagement – attendance of events

3. Assessments – results of assessments and the context of their submission (retakes and late submissions)

The student will receive a Traffic Light Calculation (TLC) for each of these categories.

These TLC’s can be:

Red (empty circle) – Not meeting expectations

Amber (half full circle) – Meeting some expectations

Green (full circle) – Meeting expectations

Grey (full circle with line through it) – Insufficient data to make a decision

The main area of concentration at the moment is the use of virtual engagement data, also known as Moodle data. Currently, we do not have attendance data, but we hope to obtain it in the future. Therefore, all TLC indicators for module attendance will be greyed out. Similarly, grades will be taken from SITS after they have been approved and finalised, therefore data around grades will likely be greyed out until the end of term or academic year.

For an example of how a student’s TLC’s will look, please see below highlighted in red:

Data Explorer (Learning Analytics) Site Explorer Guide v.1.0 | 4

These TLC’s are set via rules by the institution. If a student breaks one of these rules, then they are set to either red (empty circle), or yellow (half-filled circle). When a student breaks the first rule in the category they will be set to yellow (half-filled circle). When a student breaks the second rule in the category (extended risk) then they will be set to red (empty circle). The image below shows the rules (parameters) set around virtual engagement.

In the example above, a student will be set to amber (half-filled circle) when it has been longer than 7 days since their last activity, or if they are using the VLE 30% less than the module average. A student will be set to red (empty circle) when it has been longer than 14 days since their last activity on the VLE, or if they are using the VLE 50% less than the module average. If there is insufficient data within the system, then the student will receive a greyed-out circle with a white line running through it.

5.2 Red, Amber, Green (RAG) IndicatorBased on the 3 separate TLC indicators (VLE Usage, Attendance, Marks) for each module, a student will be given an overall RAG indicator for their course. This RAG indicator follows the same visual system (colour/shape) as the TLC indicators.

You will see the RAG indicator next to a student’s name throughout the system and this guide. Next to the RAG indicator is a percentage. This percentage should be ignored for the time being, as it is not accurate. This is because two thirds of the TLC’s will be greyed out due to insufficient data (attendance & marks).

For a detailed explanation of this please see the Traffic Light Calculator Guide.

Examples of a student’s RAG indicator can be seen below:

Data Explorer (Learning Analytics) Site Explorer Guide v.1.0 | 5

6 Accessing ‘Students’6.1 My Students

If you select ‘My Students’, the system will read “0 Students Found”.

Data explorer uses SITS to assign personal tutors with their tutees and this is shown in ‘My Students’. The site Explorer Role is not linked to any students within SITS and therefore you will not see any students.

You can, however, search for any student within the school or department depending on what access you have been given (below)

6.2 Student SearchSelecting ‘Search’ allows you to search for any student by last name or student identifier.

If more than one student is found, you can sort the list by student name, student ID, and RAG indicator. You can also filter by RAG indicator.

The RAG Indicator shows the students engagement status. Red (empty circle) indicates that a student is not meeting expectations, Amber (half-filled circle) indicates that a student is meeting some expectations, and Green (full circle) indicates that the student is meeting expectations. Grey (full circle with a white line through) indicates that there is insufficient data to calculate engagement for that student. For more detail see Section 5. Student Engagement Indicators (RAG Indicator).

Data Explorer (Learning Analytics) Site Explorer Guide v.1.0 | 6

Clicking on the student name will allow you to see data on an individual student (see Student Overview below).

6.3 Student OverviewYou can access this by clicking a student’s name at any stage (For example, within ‘My Students’, Student ‘Search’, Course view, and Module view).

Once you have clicked on a student’s name you will be shown an overview of that student. This overview will contain information about the student including their ID, course, year of study, tutor, and RAG indicator.

Underneath, you have the following tabs:

Tabs Overview data; showing overall data and indicators

Module data; showing VLE engagement data for each module

Notes; allows recording of notes on student interactions

Bio; provides other information about the student

A detailed explanation of each tab can be found below.

6.4 Overview DataYou can see an overview of a student’s data by clicking the ‘Overview data’ tab on the ‘Student Overview’ page.

6.4.1 General Indicators

General indicators for the student are at the top of the dashboard and have an explanation of what each indicator means.

Data Explorer (Learning Analytics) Site Explorer Guide v.1.0 | 7

Last Active on VLE: The amount of days since this student has last been active on the VLE.

TLC Count: The amount of Red, Amber, Green, and Grey TLC ratings this student currently has in total. A student will have a three TLC counts per module. This will include one for VLE engagement, one for Attendance, and one for Assessment. Therefore, if the student is studying 3 modules, they would have 9 TLC counts in total. The attendance TLC rating will be greyed out for at the moment as there will not be any data. The assessment TLC rating will also likely be greyed out because assessment data will be collected from SITS later in the term/year.

Assignments Complete: The assignments handed in successfully this academic year for all active modules.

Average Marks: The average mark for all completed assignments this academic year.

6.4.2 Data Visualisations

This can be found directly underneath the general indicator boxes. You can choose what visualisation to view by using the Data Type drop down menu. You are also able to choose what date range you want to look at.

An explanation can be found below each visualisation.

VLE Activity: This shows a visualisation of VLE activity for that student. Each tracked VLE event counts as an activity. For example, this could be viewing a content item, logging into the VLE, or submitting an assignment through the VLE.

TLC Count: This visualisation shows the amount of Red, Amber, Green, and Gray TLC ratings this student has had in total. A student will have a three TLC counts per module. This will include one for VLE engagement, one for Attendance, and one for Assessment. Therefore, if the student was studying 3 modules, they would have 9 TLC counts in total. This visualisation shows the daily traffic light breakdown for a student. Where no day is shown, no activity occurred on that day.

Data Explorer (Learning Analytics) Site Explorer Guide v.1.0 | 8

If you select the TLC Count option from the Data Type drop down menu then you will be provided with a breakdown of a student’s TLC Count over time.

The total will increase and decrease depending on the student’s active modules.

You can select the colours you want to hide/see by clicking on the circles underneath the graph. The example to the right is focusing on the red and amber.

The graph to the left is of the same student, but only focusing on the red indicators.

6.4.3 Module Traffic Lights

At the bottom of the dashboard you will see ‘module traffic lights’ which provides a TLC rating for each module. The City system currently has VLE data and therefore you will see a TLC rating for this. Assessment data will be taken

Data Explorer (Learning Analytics) Site Explorer Guide v.1.0 | 9

from SITS after the board of studies has finalised the marks for that student, therefore this is likely to be greyed out until later in the year. Attendance data will not be feeding into the learning analytics system therefore all data points for attendance will be a greyed-out circle with a white line through it as can be seen below.

Clicking on the ‘show results’ button will detail why the issue was triggered

6.5 Module DataYou can also view student data by module. Clicking on the ‘Module data’ tab on the ‘Student Overview’ page allows viewing of the indicators for each module.

You can select the different modules from the ‘Module’ drop-down menu.

6.5.1 Data Visualisations

Using the ‘Data Type’ dropdown menu, you can select different visualisations for that module.

Overall Activity: This shows overall activity for the student compared to the module average. In this view, each tracked event counts as activity. This could be clicking/logging into a VLE, using presentation slides, or attending a lecture.

Data Explorer (Learning Analytics) Site Explorer Guide v.1.0 | 10

Assessment: The Assessment tile shows the number of submitted assignments, if any were submitted late, how many which have been graded and the average grade all submissions

Assessment Breakdown: Assignments associated with this module and marks/grades if submitted. You can also view all or by individual assessment.

By clicking on the grade of assessment, you will see a graph of the assessment mark distribution. The patterned band depicts the student’s marks.

Data Explorer (Learning Analytics) Site Explorer Guide v.1.0 | 11

6.5.2 Module Specific Traffic Lights

At the bottom of the dashboard you will see ‘module traffic lights’ for that module. See 6.4.3 Module Traffic Lights for more detail.

6.6 NotesYou can also view notes that have been left on the student’s profile. Clicking on the ‘Notes’ tab on the ‘Student Overview’ page allows the recording of notes and shows a list of any recorded notes of meetings or other interactions with the student.

By clicking on ‘Record a note’ you can input details of an intervention with a student. If the conversation is confidential you have the option to ‘Protect message.’ If this option is selected, only the tutor who has written the note will be able to open and read once saved.

If a note is protected, you will see this:

Data Explorer (Learning Analytics) Site Explorer Guide v.1.0 | 12

All other people will not be able to view what has been recorded and will see the following:

6.7 BioYou can also view a student’s Bio. By clicking on ‘Bio’ tab on the ‘Student Overview’ page you can view other information about the student. You can also see the student’s progression history.

7 Accessing ‘Courses’7.1 My Courses

If you select ‘My Courses’, the system will read “0 Courses Found”.

Data explorer uses SITS to assign teachers with the students studying on their course. The Site Explorer role is not linked to any courses within SITS and therefore you will not see any courses in ‘My Courses’.

You can, however, search for any course within the school or department depending on what access you have been given (below)

7.2 Course SearchSelecting ‘Search’ allows you to view all the courses in the school or department you have been given access to. You can search by course name, ID, or instance ID.

The default search will only show courses for the current academic year. If you wish to search all academic years you can click the ‘Search all academic years’ box.

You can sort the list by module name, ID, and academic year.

Data Explorer (Learning Analytics) Site Explorer Guide v.1.0 | 13

Under the course name you will see the date in which that course started. To select the course you wish to view, click the name of the course next to the arrow, e.g. BA (Hons) Jiscing >.

Clicking on the BSc Accounting & Finance course starting in 2020-2021 will only show students who started the course that year.

7.3 Course Overview

Selecting a course will give you a list of students studying on that course.

The RAG Indicator shows the students engagement status. Red (empty circle) indicates that a student is not meeting expectations, Amber (half-filled circle) indicates that a student is meeting some expectations, and Green (full circle) indicates that the student is meeting expectations. Grey (full circle with a white line through) indicates that there is insufficient data to calculate engagement for that student. For more detail see Section 5. Student Engagement Indicators (RAG Indicator).

The list can be sorted by name, student ID or RAG indicator and filtered by RAG indicator.

Clicking on a student name will give you an overview of that student, which can be seen in section 6.3 Student Overview.

8 Accessing ‘Modules’8.1 My ModulesIf you select ‘My Modules’, the system will read “0 Modules Found”.

Data Explorer (Learning Analytics) Site Explorer Guide v.1.0 | 14

Data explorer uses SITS to assign teachers with the students studying on their module. The Site Explorer role is not linked to any modules within SITS and therefore you will not see any Modules in ‘My Modules’.

You can, however, search for any module within the school or department depending on what access you have been given (below)

8.2 Module SearchSelecting ‘Search’ allows you to view all the modules in the school or department you have been given access to. You can search by module name, ID, or instance ID.

The default search will only show courses for the current academic year. If you wish to search all academic years you can click the ‘Search all academic years’ box.

You can sort the list by module name, ID, and academic year.

Under the module name you will see the date in which that module started. To select the module you wish to view, click the name of the module next to the arrow, e.g. ‘Final Year Project (structured) – >’.

Clicking on the ‘Final Year Project (structured)’ module starting in 2020-2021 will only show students who started that module that year.

After Selecting a module you will be able to see module data visualisations (below).

8.3 Module OverviewClicking on the module name will provide you with visualisations of module activity.

Date Range: You can select the specific date ranges you want the visualisation to show.

Data Type: An explanation is provided underneath each graph that can be viewed by the data type menu.

Data Explorer (Learning Analytics) Site Explorer Guide v.1.0 | 15

VLE activity – by week: Every activity that a student takes on a VLE counts as activity, such as logging in, viewing a document, or undertaking a quiz. This graph shows the sum of these activities on a weekly basis.

VLE activity – by day: This graph shows the sum of these activities on a daily basis. Where no day is shown, no activity occurred on that day.

VLE activity – student use distribution: Every activity that a student takes on a VLE counts as activity, such as logging in, viewing a document, or undertaking a quiz. This graph shows the sum of these activities for a student, for the selected data range.

VLE Tool Use: This shows the sum of the clicks on the different tool types on your VLE module.

VLE Content Use: This shows the sum of the clicks on the individual content items in your VLE module.

VLE Use vs Average Mark: Scatter plot of VLE activity against all module marks recorded in your Student Records system. Date range applies to VLE activity only.

Assessment: Shows how many assessments have been submitted. Clicking on the number will show the mark distribution and a list of how each student did.

Staff: This shows a list of staff members teaching on that module

Students: This shows and overview of the students studying on that module.

Many of these visualisations are interactive. For example, when selecting VLE Content Use you will see a breakdown of all the content on the module and how many times it has been clicked. If you click the bar in the visualisation next to that content it will show you how many times each student on that module clicked on that specific piece of content.

In the top right-hand corner of every visualisation you will find a burger menu. Here you have the option to view, print and/or download the image.

9 Accessing ‘Department and Course View’

This allows you to view the number of students at risk by department or course.

You will only be able to view the schools and departments you have been given access to.

Data Explorer (Learning Analytics) Site Explorer Guide v.1.0 | 16When presented with this screen you can select which school you would like to access using the drop-down feature and

Selecting a department will give you a list of courses, with a similar breakdown. These courses include all academic years within a course.

You can filter the course by:

StatusThis allows you to filter by active or past courses.

Degree typeThis allows you to filter by undergraduate,

postgraduate, and other.

To select the course you wish to view click the name of the course next to the arrow, e.g. MPhys Physics >.

Selecting a course will give you a list of students on that course. These courses include all academic years within a course.

The list can be sorted by name, student ID or RAG Indicator and can also be filtered by RAG indicator.

Clicking on a student name will give you an overview of that student, which can be seen in the Student Overview section above.

Data Explorer (Learning Analytics) Site Explorer Guide v.1.0 | 17

When presented with this screen you can select which school you would like to access using the drop-down feature and