CSCL Research on Online Discussions to Inform DCLA (LAK13 DCLA Workshop)

LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY

LEVERAGING CSCL RESEARCH ANALYZING ONLINE DISCUSSIONS TO

INFORM DCLAALYSSA WISE

SIMON FRASER UNIVERSITYFACULTY OF EDUCATION

LEARNING ENVIRONMENT – ONLINE DISCUSSIONS


ONE CENTRAL CONCERN

CONNECTING THE DEVELOPMENT AND USE OF DISCOURSE-CENTRIC LEARNING

ANALYTICS WITH THEORETICAL UNDERSTANDINGS OF THE QUALITIES OF

PRODUCTIVE ONLINE DISCOURSE


ONE CENTRAL CONCERN

THEORETICAL MODELS OF ONLINE DISCOURSE


TWO KEY POINTS

THERE IS AN ADDITIONAL, NOVEL KNOWLEDGE BASE NEEDED TO LEVERAGE CSCL MODELS OF DISCOURSE TO INFORM

DCLA

CREATING A CLEAR THREAD BETWEEN DISCOURSE GOALS, ACTIONS AND

ANALYTICS IS CRITICAL FOR LEARNERS (AND TEACHERS) TO USE DCLA AS A TOOL FOR SUPPORTING PRODUCTIVE DISCOURSE


TWO KEY POINTS

DCLA > LAK + CSCL

TRANSPARENCY & INTEGRATION


CSCL HAS MANY WELL-DEVELOPED MODELS FOR THEORIZING AND ANALYZING THE QUALITIES OF ONLINE

DISCOURSE

MOST ARE PROCESS MODELS THEORETICALLY (THOUGH ANALYSES HAVE OFTEN DISREGARDED THIS)

MOST ARE GROUP, NOT INDIVIDUAL, MODELS

THERE IS ADDITIONAL, NOVEL KNOWLEDGE NEEDED TO LEVERAGE CSCL MODELS OF DISCOURSE TO INFORM

DCLA

DCLA > LAK + CSCL


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 201

2

3

4

5

THEORETICAL PATTERN

Sharing Information

Negotiating Meaning

Testing & Modifying

Exploring Dissonance

Agreeing & Applying

KNOWLEDGE CONSTRUCTION MODELFROM GUNAWARDENA ET AL., 1997….. TO WISE & CHIU, 2011


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 201

2

3

4

5

ACTUAL PATTERN

(PRETTY GOOD)

Sharing Information

Negotiating Meaning

Testing & Modifying


Agreeing & Applying



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 201

2

3

4

5

ACTUAL PATTERN

(FAILURE TO LAUNCH)

Sharing Information

Negotiating Meaning

Testing & Modifying


Agreeing & Applying



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 201

2

3

4

5

ACTUAL PATTERN

(QUICK CONSENSUS)

Sharing Information

Negotiating Meaning

Testing & Modifying


Agreeing & Applying



AUTOMATING POST CODING, IN REAL TIME

EVALUATING “APPROXIMATIONS” OF GOOD DISCOURSE PATTERNS (DESIRABLE ENDSTATE VS PATHS FOR CHANGE)

EVALUATING DISCOURSE PATTERNS IN PROGRESS (IS “QUALITY” OF A POST PATTERN PARTIALLY DETERMINED BY THOSE YET TO COME)

CREATING ACTIONABLE ANALYTICS (GROUP / INDIVIDUAL LEVELS, INTERPRETABILITY)

ASSESSING INDIVIDUAL’S POSTS IN CONTEXT (SEQUENTIAL ANALYSIS)

ISSUES IN DRAWING ON KC MODEL TO DEVELOP DCLA


TRANSPARENCY & INTEGRATIONLEARNERS HAVE A VARIETY OF PERCEPTIONS ABOUT THE

PURPOSE OF ONLINE DISCOURSE (THAT OFTEN DIFFER FROM THE INTENDED PURPOSE) - HOW THEY PARTICIPATE

IN ONLINE DISCUSSIONS RELATE TO THESE PERCEIVED PURPOSES

(KNOWLTON, 2005; WISE ET AL. 2012; 2013)

LEARNERS (AND TEACHERS) NEED TO SEE THE CONNECTIONS BETWEEN THE GOALS OF DISCOURSE,

RELATED ONLINE ACTIVITY, AND ANALYTIC FEEDBACK IN ORDER TO USE DCLA AS A TOOL FOR SUPPORTING

PRODUCTIVE DIALOGUE



THEORETICAL MODEL

DISCOURSE GUIDELINES

RELATED ANALYTICS

GOAL SETTING

DATA

REFLECTION


SOCIAL CONSTRUCTIVIST PERSPECTIVE - LEARNERS BUILD UNDERSTANDING THROUGH DIALOGING WITH OTHERS

VARYING IMPORTANCE OF SHARING AND SUPPORTING ONE’S OWN IDEAS, BEING EXPOSED TO MULTIPLE VIEWPOINTS,

EXPERIENCING PERSONAL COGNITIVE CONFLICT, NEGOTIATION OF GROUP UNDERSTANDINGS

IN ALL, TWO BASIC (AND RELATED) UNDERLYING PROCESSES:

SPEAKING EXTERNALIZING ONE’S IDEAS BY CONTRIBUTING POSTS TO THE

DISCUSSION

LISTENING TAKING IN THE EXTERNALIZATIONS OF OTHERS BY ACCESSING

EXISTING POSTS

THEORETICAL MODEL


IN ONLINE DISCUSSIONS LEARNERS HAVE A HIGH DEGREE OF CONTROL OVER THE TIMELINE AND PACE OF THEIR

ENGAGEMENT

OPPORTUNITIES FOR THOUGHTFUL LISTENING AND REFLECTIVE SPEAKING

CHALLENGES OF TIME MANAGEMENT, ESPECIALLY IN PROLIFIC DISCUSSIONS

HELPING LEARNERS TO ACTIVELY MONITOR AND REGULATE HOW THEY SPEAK AND LISTEN IN ONLINE DISCUSSIONS IS AN

IMPORTANT TOOL FOR SUPPORTING PRODUCTIVE ENGAGEMENT IN DISCUSSIONS

THEORETICAL MODEL


SPEAKING

MECHANISM FOR SHARING IDEAS WITH OTHERS

VALUE IN SPEAKING THAT IS

- RECURRING, RESPONSIVE AND RATIONALED

- TEMPORALLY DISTRIBUTED

- MODERATE PORTIONED

WHILE “SPEAKING” IS VISIBLE, NOT ALL QUALITIES ARE SALIENT IN THE SYSTEM (ESP. AS RELATED TO TIME)

POST QUALITY INFO VALUABLE, COMPLEX TO ASSES

LISTENING

ATTENDING TO THE IDEAS OF OTHERS IS A CRITICAL, THOUGH “INVISIBLE” PART OF LEARNING THROUGH ONLINE DISCUSSIONS

VALUE IN LISTENING THAT IS

- BROAD (TO CONSIDER MULTIPLE IDEAS)

- INTEGRATED (SO REPLIES ARE INFORMED BY READS)

- REFLECTIVE (TO PROVIDE CONTEXT FOR DISCUSSION FLOW)

EARLY RESEARCH SUGGESTED UNIVERSALLY POOR BEHAVIORS, BUT OUR RECENT WORK SHOWS STUDENTS INTERACT WITH PRIOR MESSAGES IN VERY DISTINCT WAYS

- E.G. COVERAGE, INTERACTIVE, SELF-FOCUSED, TARGETED


RELATED ANALYTICS - EMBEDDED


Metric Definition Criteria

Range Span of days a student logged in to the discussion

Temporal Distribution

# of sessions # of times a student logged in to the discussion

Av session length Total time a student spent in the discussions divided by his / her # of sessions

% of sessions with posts

# of sessions in which a student made a post, divided by his/her total of # sessions

Posts Total # of posts a student contributed to the discussionSpeaking Quantity

Av post length Total # of words posted by a student divided by the # of posts he/she made to the discussion

% of posts read # of unique posts that a student read divided by the total # of posts made by others to the discussion

Listening Breadth

# of reviews (own posts)

# of times a student revisited posts that he/she had made previously in the discussion

Listening Reflectivity

# of reviews (others’ posts)

# of times a student revisited others’ posts that he/she had viewed previously in the discussion

RELATED ANALYTICS - EXTRACTED


CONNECT PURPOSE OF THE LEARNING ACTIVITY WITH THE

INSTRUCTOR’S EXPECTATIONS FOR ENGAGEMENT

Discussion Participation Guidelines

Attending to Others Posts

Broad Listening: Try to read as many posts as possible to consider everyone’s ideas in the discussion. This can help you examine and support your own ideas more deeply. However, when time is limited it is better to view a portion in depth, then everything superficially.

*The visual interface shows posts that you have viewed in blue and new ones in red to help you track this.

DISCOURSE GUIDELINES

CONNECT PURPOSE OF LEARNING ACTIVITY + INSTRUCTOR’S

EXPECTATIONS WITH LEARNING ANALYTICS INDICATORSLearning Analytics Guidelines

Attending to Others’ Posts

% of posts read

The proportion of posts you read (not scanned) at least once.

It is good to read as many posts as possible to consider everyone’s ideas in the discussion However, when time is limited it is better to view a portion in depth, then everything superficially.

ANALYTICS GUIDELINES


GOAL SETTING, DATA & REFLECTIONSUPPORTING INTENTIONAL ATTENTION TO DISCOURSE

GUIDELINES PRESENT (DIVERSE) METRICS AS A STARTING POINT FOR CONSIDERATION, NOT AS ABSOLUTE ARBITERS OF

ENGAGEMENT IN THE ACTIVITY

USE OF CLASS AVERAGE TO PROVIDE CONTEXT FOR #S

STUDENTS SET PERSONAL GOALS FOR PARTICIPATION AND USE THE ANALYTICS TO HELP MONITOR THESE REAL-TIME AND WEEKLY

REFLECTIVE DIALOGUE BETWEEN EACH STUDENT AND INSTRUCTOR ABOUT THEIR PARTICIPATION, GROUNDED IN

ANALYTICS


INITIAL FINDINGSANALYTICS PILOTED WITH SEMESTER-LONG DOCTORAL

COURSE

STRONG STUDENT BUY-IN TO USEFULNESS OF ANALYTICS TO SUPPORT PRODUCTIVE DISCUSSION, BUT DIFFERENT STUDENTS FOUND DIFFERENT ANALYTICS USEFUL FOR

SETTING GOALS AND MONITORING PROGRESS TOWARDS THEM

NO “BIG BROTHER” ISSUES

STUDENT RECOGNITION OF WHEN THEY WERE AND WEREN’T MAKING PROGRESS ON THEIR GOALS


ONE CONCERNTHEORETICAL MODELS OF ONLINE

DISCOURSE

TWO KEY POINTSDCLA > LAK + CSCL


Education

CSCL Research on Online Discussions to Inform DCLA (LAK13 DCLA Workshop)