Exploiting Semantic Information for Graph-based Recommendations of Learning Resources

© author(s) of these slides including research results from the KOM research network and TU Darmstadt; otherwise it is specified at the respective slide 21-Sep-12

Prof. Dr.-Ing. Ralf Steinmetz KOM - Multimedia Communications Lab

ECTEL__Sem_Info_rec_learning_resources_v6.0_20120921_MA.pptx

Exploiting Semantic Information for Graph-based Recommendations of Learning Resources

Mojisola Anjorin Thomas Rodenhausen Renato Domínguez García Christoph Rensing

EC-TEL 2012, Saarbrücken

Research Talk

Ranking

Algorithms

Slideshare

Tags ResourcesUsers

Prepare Talk

Read-Up on Basics

Activities

Find Related Work

Friends

Friends

FriendsBlue Group

KOM – Multimedia Communications Lab 2

Resource-Based Learning


Application Scenario: CROKODIL

CROKODIL is a platform offering support for resource-based learning § Semantic Tag Types § Activities § Learner Groups & Friendships § Recommendations

[Anjorin et al, 2011]

http://demo.crokodil.de


§ Motivation: Resource-based Learning § Application Scenario: CROKODIL § CROKODIL’s Extended Folksonomy Model § Ascore and AInheritScore § Evaluation Methodology, Metrics and Results § Conclusion & Future Work

Overview


A folksonomy is a quadruple F:= (U, T, R, Y), where U – Users T – Tags R – Resources Y ⊆ U × T × R - tag assignment

Folksonomy Model

Research Talk

Ranking

Algorithms

Slideshare

Tags ResourcesUsers

[Hotho et al. 2006]


CROKODIL Extends the Folksonomy Model …

Research Talk

Ranking

Algorithms

Slideshare

Tags ResourcesUsers


… with Semantic Tag Types

[Böhnstedt et al. 2009]

Research Talk

Ranking

Algorithms

Slideshare

Tags ResourcesUsers

Genre

Event

Person

Location

Other

Topic


… with Activities

Research Talk

Ranking

Algorithms

Slideshare

Tags ResourcesUsers

Prepare Talk

Read-Up on Basics

Activities

Find Related Work


… with Learner Groups and Friendships

Research Talk

Ranking

Algorithms

Slideshare

Tags ResourcesUsers

Prepare Talk

Read-Up on Basics

Activities

Find Related Work

Friends

Friends

FriendsBlue Group


CROKODIL‘s Extended Folksonomy

FC:= (U, TTyped, R, YT, (A, <), YA, YU, G, friends) where U – users TTyped – typed tags R – learning resources YT ⊆ U × TTyped × R – tag assignment (A, <) – activities with sub-activities YA ⊆ U × A × R – activity assignment YU ⊆ U × A – activity membership

assignment G ⊆ P(U) – groups of learners friends ⊆ U × U – friendship relation

Research Talk

Ranking

Algorithms

Slideshare

Tags ResourcesUsers

Prepare Talk

Read-Up on Basics

Activities

Find Related Work

Friends

Friends

FriendsBlue Group


Resource Recommendations for CROKODIL

http://demo.crokodil.de


Graph-based recommender techniques can be classified as neighbourhood-based collaborative filtering approaches

Graph-based Resource Recommendations

Graph-based Ranking

Algorithm

Resource Score r1 0.9 r2 0.7 r3 0.5 r4 0.2

1 1

2 1

P1

P2

P4

P3

3

4

2

1

2

Folksonomy Graph e.g. FolkRank based on “Random Walk” of PageRank

Recommendation List (ranked resources)

[Desrosiers et al. 2011]



Overview


1.  Add activity nodes Vc = VF ∪ A 2.  Add edges: § activity assignments (u, r, a) § assignments of a user to an

activity (u, a) § activity hierarchies (asub , asuper)

4.  Assign weights to edges: § w(r,a) = w(r,u) = w(u,a)

= max(|Ut,r|) § w(u, a) = max(|Ru,t|) § w(asub,asuper) = max(|Ut,r|, |Ru,t|)

5.  Run graph-based ranking algorithm e.g. FolkRank

AScore

[Abel et al, 2011] Inspired by GFolkRank

Extend the Folksonomy Graph F = (V, E) with Activities

Research Talk

Ranking

Algorithms

Slideshare

Tags ResourcesUsers

Prepare Talk

Read-Up on Basics

Activities

Find Related Work


§ Depending on the tags of a user,

scores are “inherited” over the activity hierarchy

§ Resources and users assigned to activities influence the scores as well

§ Scores are attenuated depending on activity distance § Activity distance between two

activities: the number of hops from one activity to the other

AInheritScore

[Abel et al, 2011] Inspired by GRank

Leveraging Activity Hierarchies to Calculate Scores Research Talk Ranking

Algorithms

Research Talk Prepare Talk

Read-Up on Basics

Find Related Work

...

... ...



Overview


GroupMe! dataset

Evaluation Corpus and Evaluation Metrics

[Abel et al, GroupMe!]

Elements Count Users 649 Tags 2580 Resources 1789 Groups of Resources

1143

Posts 1865 Tag assignments 4366

The mean of the Average Precision over several queries Q

Mean Normalized Precision: The mean of the Precision@k over several queries Q

MAP(Q) =1

|Q|

|Q|�

j=1

1

mj

mj�

k=1

Precision(Rjk)

Mean Average Precision:

MNP(Q,k) =1

|Q|

|Q|�

j=1

Precisionj(k)

Precisionmax,j(k)

[Manning et al 2008]


Tango

Buenos

Aires

DancingFestival

Tango

Buenos

Aires

Dancing Festival

A post is a Pu,r= {(u,r,t)|(u,r,t) ∈ Y} For LeavePostOut, the recommendation task with user as input is harder as with tag as input

Evaluation Methodology: LeavePostOut

[Jäschke et al. 2007]


RTr,t= {(u,r,t)|(u,r,t) ∈ Y} For LeaveRTOut, the recommendation task with tag as input is harder as with user as input

Evaluation Methodology: LeaveRTOut

Tango

Buenos

Aires

DancingFestival

Tango

Buenos

Aires

Dancing Festival


A violin plot is a combination of a box plot and a density trace

Visualization of Results with Violin Plots

[Hintze et al. 1998]


A violin plot is a combination of a box plot and a density trace

Visualization of Results with Violin Plots

Median

3rd Quartile

1st Quartile [Hintze et al. 1998]


Evaluation results with user as input

Evaluation Results for LeavePostOut


















Approaches MAP GFolkRank 0.70 AScore 0.70 AInheritscore 0.47 GRank 0.38 FolkRank 0.19 Popularity 0.00


Evaluation Results for LeaveRTOut



Evaluation Results for LeaveRTOut

Approaches MAP AScore 0.20 GFolkRank 0.20 FolkRank 0.18 GRank 0.14 AInheritscore 0.11 Popularity 0.02


Exploiting hierarchical activity structures as found in CROKODIL can improve the ranking of resources for the purpose of recommending learning resources § AScore § AInheritscore

Future Work § Evaluation using a data set from CROKODIL § User Study § Hybrid approaches

Conclusion and Future Work

www.crokodil.de


Questions & Contact


Statistical Significance Tests – LeavePostOut

More effective than à

Popularity Folk Rank

GFolkRank

AScore GRank AInheritScore

Poularity FolkRank X GFolkRank X X X X X AScore X X X X GRank X X AInheritScore X X X

Significance matrix of pair-wise comparisons of LeavePostOut results Based on Average Precision with a significance level of p = 0.05


Statistical Significance Tests – LeaveRTOut

More effective than à

Popularity Folk Rank

GFolkRank

AScore GRank AInheritScore

Poularity FolkRank X X X GFolkRank X X X X AScore X X X X X GRank X X AInheritScore X

Significance matrix of pair-wise comparisons of LeaveRTOut results Based on Average Precision with a significance level of p = 0.05


Adapted PageRank

!

!

!"

"

"

"

"

# #

$%&'()*+,& Tango0

Buenos

Aires0

Buenos

Aires0

Dancing

Festival0

1

"-.

#-.

#-.

"-.

PageRank‘s intelligent surfer model The ranking of a node is determined by how often the surfer visits the node Adjoining edges are followed with a certain probability – determined by the edge weights The query node acts as the starting point and focus i.e. the surfer returns to this node with a certain probability – determined by the node weights

[Hotho et al. 2006]

Education

Exploiting Semantic Information for Graph-based Recommendations of Learning Resources