Identification of Learning Goals in Forum-based Communities

Lehrstuhl Informatik 5(Information Systems)

Prof. Dr. M. Jarke

Julian KrengeZ. PetrushynaMilos KravcikRalf Klamma

ICALT 2011-1

Identification of Learning Goals in Forum-based

Communities

Identification of Learning Goals in Forum-based Communities

Julian Krenge, Zinayida Petrushyna, Milos Kravcik, Ralf KlammaRWTH Aachen University

ICALT 2011, July 6-8Athens, Georgia, USA

11th IEEE International Conference on Advanced Learning Technologies


Prof. Dr. M. Jarke


ICALT 2011-2


Communities

RWTH Aachen University 260 institutes in 9 faculties as Europe’s leading institutions for science and research

Around 31,400 students enrolled in 100+ academic programs

5,000+ international students from 120 countries

Leading German University in External Funding (September 2010)

1,250 spin-offs created around 30,000 jobs in greater Aachen region in 20 years

The Chair of Computer Science I5 http://dbis.rwth-aachen.de

Our research group:Community Information Systems


Prof. Dr. M. Jarke


ICALT 2011-3


Communities

ProjectsROLE: Responsive Open Learning

Environments IST 7th Framework Programme Integrated Project Scientific coordination Self-regulated & social learning http://www.role-project.eu

TEL-Map: Future gazing TEL 7th Framework ICT Coordination and support action Dynamic roadmaps for unknown

learning landscape http://www.telmap.org Portal: http://learningfrontiers.eu

TeLLNet: Teachers‘ Life Long Learning

Social Network Analysis to studyeTwinning network (135 000+ teachers)

Reflections support for web communities Development of meta-competences http://www.tellnet.eun.org

GaLA: Games and Learning Alliance

IST 7th Framework Programme Network of Excellence on Serious

Games http://www.galanoe.eu/

http://www.role-project.eu/�

http://www.telmap.org/�

http://learningfrontiers.eu/�

http://www.tellnet.eun.org/�

http://www.galanoe.eu/�


Prof. Dr. M. Jarke


ICALT 2011-4


Communities

Agenda

Motivation: Support of Self-Regulated Learning

Theoretical Background: Intent & Sentiment Analysis

Context: Community preparing for English Language Tests

Method: Identification of Learning Phases

Evaluation

Conclusion & Outlook


Prof. Dr. M. Jarke


ICALT 2011-5


Communities

Motivation: Self-Regulated Learning

Motivation [Fruhmann, Nussbaumer & Albert, 2010]


Prof. Dr. M. Jarke


ICALT 2011-6


Communities

Theoretical Background

Natural Learning Processing Intent Analysis focuses on classification by a profile of the goals and

intentions present in textual content [Strohmaier et al. 2008] Sentiment Analysis aims to use automated tools to

detect subjective information such as opinions, attitudes, and feelings expressed in text [Lin & He 2009]

Social Network Analysis Identifies patterns of behavior based on occurrence of posts and

threads, not on the contentTheoretical Background


Prof. Dr. M. Jarke


ICALT 2011-7


Communities

Urch Forums (formerly TestMagic)- Community: preparation for English language tests- 120,000+ threads, 800,000+ posts, 100,000+ users over 10 years

Refinement- Focus on most relevant sub-forums- 67,000+ threads, 428,000+ posts, by 21,000+ users- Clique: a temporal complete sub-graph within global community Group of users who appear in several threads together Requirements: 4+ users, occurs 10+ times, 75+% clique

members post in 10+ threads

Context

Concept


Prof. Dr. M. Jarke


ICALT 2011-8


Communities

Cliques

Concept

Thread 1 Thread 2

Thread 3

User of cliqueNon-clique User in threadClique-user missing in thread

Time

12,881 cliques: avg. size 5 users, avg. occurrence 14 times


Prof. Dr. M. Jarke


ICALT 2011-9


Communities

Phases of Learning and Goal Model

Based on [Fruhmann, Nussbaumer & Albert, 2010]Concept


Prof. Dr. M. Jarke


ICALT 2011-10


Communities

Evaluation of Learning Phases

Evaluation

1 week / step

Phase 1 and 2 (low sentiment, questioner, lot of intents)Phase 3 (increasing sentiment, conversationalist)Phase 4 (high sentiment, answering person)

Different users

40% of „footprints“ of cliques align with model for phases


Prof. Dr. M. Jarke


ICALT 2011-11


Communities

Intent Analysis ExpressionsRank Firstword Keyverb Count

1 Want Know 12642 Need Know 10273 Planning Take 8854 Like Know 8545 Going Take 7886 Need Find 7607 How Solve 7088 Need Get 5549 Want Get 552

10 Wanted Know 468

Rank Keyverb Noun Count1 Take Test 8372 Get Score 4783 Spend Time 3664 Solve Problem 3335 Take Exam 2536 Take GRE 2507 Answer Question 231- Buy Homes 2319 Think People 221

10 Take TOEFL 219

Evaluation

Patterns of intent expressions 132,328 expressions of intent detected Most obvious keyverbs: know (solve), take (test)


Prof. Dr. M. Jarke


ICALT 2011-12


Communities

User Feedback on Significance

Evaluation

-7 -2 3 8 13

KnowSolve

AnalyseWorkFind

ShareImproveExplain

GetThink

SupportGivePassTake

YesNo

18 users gave feedback on combinations that reflected their goals Specific verbs have high significance for goal expression


Prof. Dr. M. Jarke


ICALT 2011-13


Communities

Conclusion Issue: Extracting goals in learning communities Chosen criteria help identifying learning phase

- SNA pattern- Intent analysis- Sentiment analysis

Intent analysis patterns are significant for goals of learners as different aspects are expressed

Learners follow the assumed phase model (~40%) Challenge: Support self-reflection & self-regulation Holistic approach enables automatic identification

of learning goalsConclusion & Outlook


Prof. Dr. M. Jarke


ICALT 2011-14


Communities

Announcements Workshop: Self-Regulated Learning in Responsive Open

Learning Environments- Thursday, 11:00-13:30

Poster: How can Psychology inform the Design of Learning Experiences- Thursday, 15:00-18:00

ROLE Widget Enchantment Contest: Deadline – 15 July- Join the competition – create a widget – win a prize!- www.role-project.eu

Conclusion & Outlook

http://www.role-project.eu/�

Business

Identification of Learning Goals in Forum-based Communities