Analytics Exemplar:

title

Simon Buckingham Shum Knowledge Media InstituteThe Open University, UK

http://simon.buckinghamshum.net

An Analytics Exemplar has the following elements

1. What kind of research questions are we asking?

2. What theories/methodologies are used?

3. What kind of data is required?

4. What analysis tool are we using?

5. What’s the workflow to get the data into the tool?

6. What visualizations and settings are available?

7. What insights do/might we get?

What kind of research questions are we asking?

• Example: Do mentors with different learner-support roles in

the course have different kinds of social networks?

• Or… Do they engage in different kinds of conversations?

If so, then analytics might identify other learners starting to play

that role, or help compare/contrast mentor performance.

What theories/methodologies are used?

Examples:

• tying SNA metrics to a theory of social learning;

• or a systematic method for analysing a network graph;

• or tying particular discourse patterns to a pedagogical

approach;

• or engagement patterns to particular demographics.

What kind of data is required?

Examples:

Quiz data

• Dropout data

• Demographic data

• Forum data

• Video watching data

What analysis tool are we using?

• Example: NodeXL

• http://nodexl.codeplex.com

Workflow for social network analytics in NodeXL: are learners forming effective peer-relationships?

Import data into OpenRefine

Reshape using template

Export data to CSV

Process in NodeXLand generate network

Martin Hawkseyhttp://mashe.hawksey.info/2013/02/lak13-recipes-in-capturing-and-analyzing-data-using-sna-on-canvas-discussions-with-nodexl-for-when-its-not-a-snapp

What visualizations and settings are available?

• Example: NodeXL gallery

• http://www.nodexlgraphgallery.org/Pages/Default.aspx

What insights do/might we get?

Examples:

• Highly rated mentors of type 1 have social graphs like

this…

• Mentors of type 2 however look like this…

• This confirms Bloggs et al’s prediction that…

other example workflows(discourse analytics)

Workflow for quantifying reflections in forum posts: what elements of reflection are evident?

Convert discussion threads in comma-separated file format

Annotate text segments using custom components for UIMA

Convert results in CSV

Ullmann, T. D., Wild, F., & Scott, P. (2012). Comparing Automatically Detected Reflective Texts with Human Judgements. In 2nd Workshop on Awareness and Reflection in Technology-Enhanced Learning. Presented at the 7th European Conference on Technology-Enhanced Learning, Saarbruecken, Germany. Retrieved from http://ceur-ws.org/Vol-931/paper8.pdf

Inspect and analyse data with R

Reason over annotations with Drools

Workflow for academic writing analytics: to what extent does student writing display the hallmarks of scholarly argument?

Extract submitted essay drafts from Course XYZ

Convert to text files for XIP

Analyse using rhetorical parser

Render in custom dashboardAnnotate onto source text

Simsek D, Buckingham Shum S, Sándor Á, De Liddo A and Ferguson R. (2013) XIP Dashboard: Visual Analytics from Automated Rhetorical Parsing of Scientific Metadiscourse. 1st International Workshop on Discourse-Centric Learning Analytics, at 3rd International Conference on Learning Analytics & Knowledge. Leuven, BE (Apr. 8-12, 2013). Open Access Eprint: http://oro.open.ac.uk/37391