Learning Analytics of and in Mediational Processes of Collaborative Learning

Dan SuthersAlyssa Friend WiseBetrand Schneider

David Williamson ShafferH. Ulrich HoppeGeorge Siemens

The collection, analysis and reporting of data traces related to learning in

order to understand, inform and improve the process, outcomes and/or

environments in which it occurs

Learning Analytics

Rapidly growing interest in learning analytics by the CSCL community

CSCL’15: 9 papers + 3 posters + this invited sessions

CSCL’13: No mention in program

A rose by any other name…

Intrigue

Puzzlement

Hesitancy

Energy

Generation of insight through computational analytic methods

Informing of human action and decision making (“closing the loop” in a tighter cycle)

Development of indicators, models and data representations

Some Critical Characteristics of Learning Analytics

Knowledge Bases in Learning , Informatics / Computing & Interaction Design

Information Representation / Visualization

Design

Learning Environment

Design & Infrastructure

One Schematic of Learning Analytics

Adapted from Tyne (2015)

Ethical Responsibilities & Privacy Safeguards

Processing Algorithm Design &

Infrastructure

Data Access, Capture &

Management

Analysis & Creation of

Insight

Processes that Impact Student

Learning & Success

Learning Analytics and CSCL Fundamental shared concern with learning processes and innovating to improve them

Analytics can help uncover mediation of social interactions by physical, digital + conceptual artifacts

- Identify patterns in “big” or “deep” data

As well, learning analytics themselves are material that can further mediate these interactions

- Possibilities for data-informed practices

Focus of the Panel

Explore potential of learning analytics for generating understanding of and participating in mediational processes in CSCL

Session Structure

Four illustrative examples of CSCL learning analytics projects

Comments and questions to the panel by George Siemens, Founding President of SoLAR (Society for Learning Analytics Research)

Extended discussion based on questions from the audience, live +via twitter #cscl2015 #learninganalytics

Mediation of Discussion Forum Activity by “Messages”: A Learning Analytics Approach

Supported by the Social Sciences and Humanities Research Council of Canada

CSCL 2015 ∙ Gothenburg, Sweden

Alyssa Friend Wise Simon Fraser University Vancouver, Canada

With grateful thanks to the entire e-listening research team:

Trisha, Hsiao, Farshid Marbouti, Jennifer Speer, Yuting Zhao, Simone

Hausknecht & Nishan Perera

Origins of Online “Listening”

• From a social constructivist perspective, the goal of online discussions is for learners to build understanding through dialoging with other. At a basic level this involves

Externalizing one’s ideas by contributing msgs to

an online discussion

Taking in the externalizations of others by accessing

existing msgs

The messages are thus conceptual and interactional resources that mediate the

process of discussing online

Distinct Characteristics of Listening Online

• Listeners (rather than speakers) determine timeline by which messages and ideas are accessed

• Large decision space– Frequency and length of log-in sessions– Which posts attended to, in what order, for how long– Revisit posts as many times as needed– Reply when ready, unlimited time to prepare

Not Just the Messages, but their Presentation also Mediates Interaction

11

Microanalytic Case Studies of ListeningDate Time Session Action Duration

(min)Length(words)

Message #

6/3/2011 23:46 1 Read 44.43 413 447

6/3/2011 23:52 1 Read 1.73 60 455

6/4/2011 00:08 1 Scan 0.23 117 459

6/4/2011 00:09 1 Read 12.51 413 460

6/4/2011 23:49 2 Post 3.18 120 477

Dynamic Discussion Map: A record of the discussion to show the historical appearance of the discussion forum at any point in time

Log-file Data of Student Actions

So how do we study this?

Common Online Listening Patterns

Pattern Characteristic Behaviors

Disregardful Minimal attention to others’ posts (few posts viewed; short time viewing). Brief and relatively infrequent sessions of activity in discussions.

Coverage

Views a large proportion of others’ posts, but spends little time attending to them (often only scanning the contents). Short but frequent sessions of activity, focusing primarily on new posts. *May be socially-oriented or content-driven.

FocusedViews a limited number of others’ posts, but spends substantial time attending to them. Few extended sessions of activity in discussions.

ThoroughViews a large proportion of other’s posts; spends substantial time attending to many of them. Long overall time spent listening; considerable revisitiation of posts already read.

Developing Metrics for the Patterns

14

Dimension Metric Definition

Listening Breadth

% of posts viewed Number of unique posts that a student viewed divided by the total number of posts made by others.

% of posts read Number of unique posts that a student read divided by the total number of posts made by others.

Listening Depth

% of real readsNumber of times a student viewed other’s posts that were slower than 6.5 words per second, divided by the total number of views.

Av. length of real reads

Total time a student spent reading posts, divided by the number of reads.

Listening Reflectivity

# of reviews of own posts

Number of times a student revisited posts that he/she had made previously in the discussion

# of reviews of others posts

Number of times a student revisited others’ posts that he/she had viewed previously in the discussion

Identifying Patterns with MetricsBreadth Depth

Connections with Speaking

• Greater revisitation of others’ posts is associated with richer responsiveness

• Greater listening depth (% of real reads) is associated with richer argumentation

• Initially no relationship found between listening breadth and quality of speaking

Designing Learning Analytics to Mediate Learner’s Interactions w/ Messages

Embedded Analytics

Designing Learning Analytics to Mediate Learner’s Interactions w/ Messages

Extracted Analytics

Metric Your Data (Week X)

Class Average (Week X)

% of posts read 72% 87%

% of real reads41% 66%

Av. length of real reads

2.37m 4.12m

#of reviews of own posts

22 13

#of reviews of others’ posts

8 112

Summary

The E-Listening Project as a brief illustration of how:

1. Analytics helped uncover mediation of discussion forum activity by “messages” as conceptual / interactional resources

2. This information could then be used to create material that could further mediate these interactions

Alyssa Friend WiseSimon Fraser University

afw3@sfu.ca@alywise

www.sfu.ca/~afw3/research/e-listening

Continue the conversation and bridge building…