Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)

7/29/2019 Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)

http://slidepdf.com/reader/full/real-time-visual-analytics-to-evaluate-online-collaboration-166278128 1/61

Real-Time Visual Analytics to Evaluate

Online Collaboration

John McCormick | Sarah Krongard



Agenda

1. Learning analytics defined

2. Discourse and collaboration

3. Social Networks Adapting Pedagogical Practice(SNAPP): A tool for visualizing interaction

4. Future of discourse learning analytics tools

5. Q and A



Horizon Report: Emerging Technologies



Mobile

Learning(1-2 yrs)

TabletPCs(1-2 yrs)

Learning

Analytics(2-3 yrs)

iPad



Definitions

The measurement, collection, analysis and reporting of dataabout learners… for purposes of understanding and optimising

learning and the environments in which it occurs

- 1st international Conference on Learning Analytics and Knowledge (LAK 2011)

----------------…selection, capture and processing of data that will be helpfulfor students and instructors at the course or individual level.

…able to adjust content, levels of support and other personalized

services by capturing, reporting, processing and acting on data

on an ongoing basis in a way that minimizes the time delay

between the capture and use of data.

-Elias, T. (2011). Learning Analytics: Definitions, Processes and Potential



Definitions (cont.)

Learning Analytics:

• is the analysis of many kinds of learner-produced and

learner-related data;

•seeks to monitor learner activity and progress and to

predict learner outcomes; and

• enables interventions and decision making about

learning by instructors and students.

- From Educause/ELI Focus Session on Learning Analytics (2012)



Example 1: Signals: Purdue

Source: http://www.itap.purdue.edu/studio/signals/

http://www.itap.purdue.edu/studio/signals/






Signals: Results

“When instructors follow recommended Course Signals bestpractices, grade performance improvement has been consistently

demonstrated at both the course and departmental level.”

• 10% increase in numbers of As and Bs awarded in semester-

long courses, and a 6.41% decrease in Ds, Fs and withdrawals.

• Greatest improvement seen in students who were initially

receiving Cs and Ds in early assignments, and pull up half a

letter grade or more to a B or C.

• “…the overwhelming response from the students is that Course

Signals is a helpful and important tool that aids in their overall

academic success”

Arnold and Pistilli (2012)



The seven dimensions of ‘Learning Power’

1. Critical curiosity

2. Changing and learning

3. Meaning making

4. Creativity

5. Learning relationships (interdependence)

6. Strategic awareness

7. Resilience

Example 2: Effective Lifelong Learning

Inventory (ELLI)



Source: http://www.slideshare.net/Ruthdeakincrick/learning-dispositions-and-transferable-competences-pedagogy-modelling-and-learning-analytics

http://www.slideshare.net/Ruthdeakincrick/learning-dispositions-and-transferable-competences-pedagogy-modelling-and-learning-analytics
























Source: http://www.heacademy.ac.uk/assets/documents/what-works-student-retention/Northumbria_ELLI_Dec_11.pdf

http://www.heacademy.ac.uk/assets/documents/what-works-student-retention/Northumbria_ELLI_Dec_11.pdf












Spider Diagram



ELLI ResearchFindings from research and development projects indicate that

learners who are aware of their learning power profile using ELLI and

who have structured learning opportunities to improve it, become:

• More reflective and aware of themselves as learners

•

More confident and motivated to learn• More willing and able to take responsibility and ‘own’ their

learning

• More keen to carry on learning

• More likely to attain higher levels and grades• More likely to fulfil their potential

• More sociable and able to learn with and from each other

• More able to manage change and transitions

Source: My Learning Power: Using ELLI in Higher Education



Enquiry Blogger

Source: http://wordpress.org/extend/plugins/eb-enquiryblogbuilder/





Instructor Dashboards



ELLIMent

Source: http://people.kmi.open.ac.uk/ullmann/projects/ELLIMent/index.html



Academic vs. Learning Analytics

Long, P. & Siemens, G. (2011). Penetrating the Fog: Analytics in learning and education.



Initial Thoughts about Learning Analytics?



Discourse & Online Collaboration

• What is the value of collaboration in an online

environment?

•

What are the challenges & opportunities doinstructors face in facilitating online

collaboration?

•

To what extent have the roles of instructorand student changed?



The Importance of Collaboration

• To refine and confirm personal understanding

with a community of learners

• To develop, foster, and practice higher-order

thinking skills

• To share and co-construct knowledge to

support emerging collectives



Online Collaboration

Challenges

• Increased Workload

• Difficulty of Encouraging

and Sustaining High-LevelDiscourse

• Asynchronous Interactions

• Lack of F2F Connections

• Challenge of Self-Motivation

• Technical Difficulties

Opportunities

• Time Allocated for

Thoughtful and Reflective

Dialogue• Permanency of the Written

Word

• Tools Available to Facilitate

Interaction and Engagement• Opportunities for Student

Agency and Efficacy



Making the Shift in Roles from…

… Traditional Instructor to Guide and Facilitator

… Presumed Authority to Collective Credibility

… Student as Passive Recipient to ActiveProducer and Co-Creator



“Learning is in social relationships – situations of cooperation with others…

Both the learner and the community in which

they participate are transformed by thecommunity as a complementary unit thatwork in harmony to promote learning…”

(Dooley, Lindner, Dooley, & Murphy, 2005, p. 49)





SNA Concepts: Centrality

Source: http://www.wolfram.com/mathematica/new-in-8/graph-and-network-analysis/degree-centrality-in-social-networks.html



Node Size: Number of Interactions

Source: http://socialmedialab.ca/wp-content/uploads/2011/02/Fig.1.jpg



Information Brokers



Frequency of Interaction



SNA and Online Discourse

• Very little SNA research on online discussion for learning

General Findings:

•

More evenly distributed activity beneficial• Low hierarchical structures better (groups interacting,

teacher not playing central role)

• More interaction not necessarily better

Toikkanen , T. & Lipponen, L. (2011). The applicability of social network analysis tothe study of networked learning. The Journal of Interactive Learning Environments,19 (4), 365-379.



Social Networks Adapting Pedagogical Practice (SNAPP)



Other SNAPP metrics



What SNAPP can tell you

Dawson, S., Bakharia, A., Lockyer, L., & Heathcote, E. (2011). ‘Seeing’ Networks: Visualising and evaluating student learning networks.Australian Learning and Teaching Council, Canberra, Australia.



Dawson et al findings: Instructor use

“extremely effective in promoting reflection onteaching activities and rapidly assessing overalleffectiveness..”

“However, teachers require additional support and training to better understand therelationship between their implemented online

learning activities and the resultant onlinestudent behavior…”

Dawson, S., Bakharia, A., Lockyer, L., & Heathcote, E. (2011). ‘Seeing’ Networks:

Visualising and evaluating student learning networks. Australian Learning and

Teaching Council, Canberra, Australia.



Interpretation and Learning Design

“Future research and professional development

should seek to merge learning design with

learning analytics and data interpretation. A

sound understanding of learning designprinciples will greatly assist academics in better

interpreting the learning analytics data into

informed pedagogical action”.



Action Research Steps

1. Create visual taxonomy

2. Choose instructors

3. Collect data

4. Send “Visual Key” and “Custom Report”

5. Interview instructors

6. Follow- up with instructors



Visual Key



Visual Features (filtering)



Instructor-Led



Learning Community



Combined Patterns



Learning Community with strong instructor presence



Emerging Learning Community



Emerging Learning Community with strong instructor

presence



Weak Learning Community



Learning Community Development



Two Facilitation Styles



Facilitator Role: Central vs Peripheral

Instructors critical to success of discussion• Blignaut & Trollip, 2003; Ferdig & Roehler, 2004; Greenlaw & DeLoach,

2003; Love, 2002; Roblyer & Wiencke, 2003; Kearsley, 2000; Arnold &

Ducate, 2006; Darabi etal., 2011; Ertmer et al., 2011; Liu & Yang, 2012;

Instructors should take a back seat, let students

construct own knowledge• Burstall, 2000; Li, 2003; Mazzolini & Maddison, 2003; Poole, 2000; Rourke

& Anderson, 2002; Paloff & Pratt, 2001; Wang & Chen, 2010, Baram &Correia, 2009

Learner Population: Undergraduate Business



Learner Population: Undergraduate Business

Typical Pattern: Learning Community with strong instructor

presence

Intervention Suggestions: Reduce number of instructor postings,

add summary or “weave” following discussion, use student-led

facilitation, organize discussions with use of titled posts.

Intervention Plans: Reduce instructor posting in some

discussions and better organize initial posts.



d

Instructor: High Centrality, Low In-Degree



Future learning analytics

Source: http://www.studentsoftheworld.info/sites/people/32917.php



Meerkat-ED



Group interaction over time



Networks of Terms



d d l h d l k d



Individual Discussion Thread: Click on edges



Natural Language Processing

https://www.coursera.org/course/nlp



Dashboards: Desire2Learn





“The ties between people make

the whole greater than the sum

of its parts”

Nicholas Kristof, Sociologist



Questions, Comments, Answers?

Documents

Real-Time Visual Analytics to Evaluate Online Collaboration (166278128)