Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
1
TU Graz – Knowledge Management Institute
Graz, June 9th, 2010
Towards Understanding the Motivation Behind Tagging
Christian Körner
Knowledge Management Institute Graz University of Technology
2
TU Graz – Knowledge Management Institute
Graz, June 9th, 2010
Outline of Todays Talk
• Introduction • Motivation • Research Questions • Related Work • What happened so far? • Two Different Types of Tagging Motivation • Expected Contribution • Outlook
3
TU Graz – Knowledge Management Institute
Graz, June 9th, 2010
Introduction / 1
• Tagging is an easy and intuitive way to annotate resources
• A lot of current web platforms enable the tagging of resources
• Tags: – are simple strings – add additional metadata to a resource – support re-finding of resources – enable the browsing of a user’s resource collection – mostly do not follow a controlled vocabulary
How and which tags are applied to a resource depends on the user!
4
TU Graz – Knowledge Management Institute
Graz, June 9th, 2010
Introduction / 2
Examples of Social Tagging Systems
4
5
TU Graz – Knowledge Management Institute
Graz, June 9th, 2010
Introduction / 3
Resulting structure of social tagging systems consists of: – Users – Tags – Resources
Folksonomy (all users of a system) Personomy (one user of a system)
6
TU Graz – Knowledge Management Institute
Graz, June 9th, 2010
Motivation
Getting a closer look at the motivation users of tagging systems have
Inferring which users/tags are good for certain tasks: – searching in these systems – ontology learning
Improve tag recommendation engines
Simulation of users and folksonomies
6
7
TU Graz – Knowledge Management Institute
Graz, June 9th, 2010
Research Questions
Is it possible to measure tagging motivation automatically?
How do different motivations influence and transform resulting folksonomies?
Based on these findings: – Can we improve existing mechanisms (such as tag
recommendation)? – Is it possible to simulate whole folksonomies?
7
8
TU Graz – Knowledge Management Institute
Graz, June 9th, 2010
Related Work (excerpt)
[Golder2006] - studies folksonomies as a whole, shows stable patterns. Present a dynamic model of collaborative tagging.
[Nov2009] - different motivations in an online photo sharing system: enjoyment, commitment, self development, reputation
[Heckner2009] - studied resource sharing vs. personal information management in social tagging systems and propose model of information behavior in social tagging systems
But all previous work relies on expert judgement!
9
TU Graz – Knowledge Management Institute
Graz, June 9th, 2010
What happened so far?
Identification of two types of tagging motivation
Developed measures to detect the behavior
Showed how tagging motivation can influence the resulting tags of a folksonomy and ontology learning[Körner2010a]
Evaluated measures to identify the best for the differentiation [Körner2010b]
9
10
TU Graz – Knowledge Management Institute
Graz, June 9th, 2010
Two Different Extreme Types of Tagging Motivation (so far)
Categorizers
Describers
11
TU Graz – Knowledge Management Institute
Graz, June 9th, 2010
Categorizers - Using Tags for Categorization
• Main focus: using tags for mimicking a taxonomy created by their personal preferences • they utilize tags so that their resources can be browsed more easily later • avoid synonyms • use limited tagging vocabulary • use “subjective” tags
11
12
TU Graz – Knowledge Management Institute
Graz, June 9th, 2010
Describers - Using Tags to Describe Resources
• Main focus: describing resources as detailed as possible • support search with their usage of tags • tagging vocabulary can contain synonyms • have an open tagging vocabulary • use “objective” vocabulary
13
TU Graz – Knowledge Management Institute
Graz, June 9th, 2010
(Current) Detection Measures
Agnostic to semantics of used language
Evaluate user behavior of single user (as opposed to the complete folksonomy) – no comparison to complete folksonomy necessary
Inspect the usage of tags NOT their semantics: – How often are tags used? – How good does a user “encode” her resources with tags? – How many tags are used to annotate a single resource – etc.
13
14
TU Graz – Knowledge Management Institute
Graz, June 9th, 2010
Examples of Current Detection Measures
Conditional Tag Entropy
Orphan Ratio
Tags per Resource tpr = |T| / |R|
Tags per Post tpp = |TAS|/|R|
Vocabulary Size vocab = |V|
14
15
TU Graz – Knowledge Management Institute
Graz, June 9th, 2010
What we found out / 1
Tagging motivation varies within and across tagging systems
15
16
TU Graz – Knowledge Management Institute
Graz, June 9th, 2010
What we found out / 2
Tagging motivation can change over time.
Human Subject Study verified that the measures indicate the tagging behavior
16
17
TU Graz – Knowledge Management Institute
Graz, June 9th, 2010
What we found out / 3
Users who are motivated by by description agree on more tags
– Tag agreement among Delicious users for 500 most popular resources
17
18
TU Graz – Knowledge Management Institute
Graz, June 9th, 2010
Results
• Cooperation with KDE Kassel which resulted in a publication at the WWW2010 - more on that later
• Evaluation which measures perform best to differ types - Hypertext 2010
19
TU Graz – Knowledge Management Institute
Graz, June 9th, 2010
Expected Contribution
Getting a closer look at the reasons why users tag
Improve recommendation engines
Enhancing search
Enhancement of automated ontology learning
Possible identification of spammers
19
20
TU Graz – Knowledge Management Institute
Graz, June 9th, 2010
Outlook / Next Steps
Finish the PhD proposal
In-depth look into recommender selection (in cooperation with KDE Kassel)
Investigate additional types of tagging motivation
Using social network analysis for further investigation
Using identified types of tagging motivation to build simulators
21
TU Graz – Knowledge Management Institute
Graz, June 9th, 2010
Conclusion
• Insight into my research on motivation behind tagging
• Quick introduction about tagging • Motivation & Research Questions • Related Work
• Categorizer VS. Describers • Work which was done so far • Expected Contribution & Outlook
22
TU Graz – Knowledge Management Institute
Graz, June 9th, 2010
Thank You For Your Attention
Please feel free to ask questions!
23
TU Graz – Knowledge Management Institute
Graz, June 9th, 2010
References [Ames2007] Ames, M. & Naaman, M. (2007), Why we tag: motivations for annotation in mobile and
online media, in ‘CHI ’07’: Proceedings of the SIGCHI conference on Human factors in computing systems’ ACM, New York, NY, USA, pp.971--980
[Golder2006] Golder S. & Huberman B.; Usage Patterns of Collaborative Tagging Systems; Journal of Information Science; 32(2):198, 2006
[Heckner2009] Heckner, M; Heilemann, M. & Wolff, C. (2009) Personal Information Management vs. Resource Sharing: Towards a Model of Information Behavior in Social Tagging Systems, in ‘Int’l AAAI Conference on Weblogs and Social Media (ICWSM)’.
[Körner2010a] Körner, C.; Benz, D.; Strohmaier, M.; Hotho, A. & Stumme, G. (2010), Stop Thinking, start Tagging - Tag Semantics emerge from Collaborative Verbosity, in 'Proceedings of the 19th International World Wide Web Conference (WWW 2010)', ACM, Raleigh, NC, USA.
[Körner2010b] Körner, C.; Kern, R.; Grahsl, H. P. & Strohmaier, M. (2010), Of Categorizers and Describers: An Evaluation of Quantitative Measures for Tagging Motivation, in '21st ACM SIGWEB Conference on Hypertext and Hypermedia (HT 2010)', ACM, Toronto, Canada.
[Nov2009] Nov, O.; Naaman, M. & Ye, C. (2010), 'Analysis of participation in an online photo-sharing community: A multidimensional perspective.', JASIST 61(3), 555-566.
Christian Körner1, Dominik Benz2, Andreas Hotho3, Markus Strohmaier1, Gerd Stumme2
Stop thinking, start tagging: Tag Semantics arise from Collaborative Verbosity
1Knowledge Management Institute and Know Center,
Graz University of Technology, Austria
2Knowledge and Data Engineering Group (KDE),
University of Kassel, Germany
3Data Mining and Information Retrieval Group
University of Würzburg, Germany
30.04.2010 Körner, Benz et al.: Tag Semantics arise from Collaborative Verbosity @ WWW2010 2 / 20
Where do Semantics come from?
Semantically annotated content is the „fuel“ of the next generation World Wide Web – but where is the petrol station?
Expert-built expensive
Evidence for emergent semantics in Web2.0 data Built by the crowd!
Which factors influence emergence of semantics?
Do certain users contribute more than others?
30.04.2010 Körner, Benz et al.: Tag Semantics arise from Collaborative Verbosity @ WWW2010 3 / 20
The Story
Emergent Tag Semantics
Pragmatics of tagging
Semantic Implications of Tagging Pragmatics
Conclusions
30.04.2010 Körner, Benz et al.: Tag Semantics arise from Collaborative Verbosity @ WWW2010 4 / 20
Emergent Tag Semantics
tagging is a simple and intuitive way to organize all kinds of resources
uncontrolled vocabulary, tags are „just strings“
formal model: folksonomy F = (U, T, R, Y) Users U, Tags T, Resources R
Tag assignments Y ⊆ (U×T×R)
evidence of emergent semantics Tag similarity measures can
identify e.g. synonym tags (web2.0, web_two)
30.04.2010 Körner, Benz et al.: Tag Semantics arise from Collaborative Verbosity @ WWW2010 5 / 20
Tag Similarity Measures: Tag Context Similarity
Tag Context Similarity is a scalable and precise tag similarity measure [Cattuto2008,Markines2009]: Describe each tag as a context vector Each dimension of the vector space correspond to
another tag; entry denotes co-occurrence count Compute similar tags by cosine similarity
5 30 1 10 50
design software blog web programming
… JAVA
Will be used as indicator of emergent semantics!
30.04.2010 Körner, Benz et al.: Tag Semantics arise from Collaborative Verbosity @ WWW2010 6 / 20
= tag
Assessing the Quality of Tag Semantics
JCN(t,tsim) = 3.68 TagCont(t,tsim) = 0.74
Folksonomy Tags = synset
WordNet Hierarchy
Mapping
Average JCN(t,tsim) over all tags t: „Quality of semantics“
30.04.2010 Körner, Benz et al.: Tag Semantics arise from Collaborative Verbosity @ WWW2010 7 / 20
The Story
Pragmatics of tagging
Semantic Implications of Tagging Pragmatics
Conclusions
Tag Similarity measures can capture emergent tag semantics
30.04.2010 Körner, Benz et al.: Tag Semantics arise from Collaborative Verbosity @ WWW2010 8 / 20
Tagging motivation
Evidence of different ways HOW users tag (Tagging Pragmatics) Broad distinction by tagging motivation [Strohmaier2009]:
donuts
duff
marge beer
bart
barty
Duff-beer
bev
alc nalc
beer wine
„Categorizers“…
- use a small controlled tag vocabulary
- goal: „ontology-like“ categorization by tags, for later browsing
- tags a replacement for folders
„Describers“…
- tag „verbously“ with freely chosen words
- vocabulary not necessarily consistent (synomyms, spelling variants, …)
- goal: describe content, ease retrieval
30.04.2010 Körner, Benz et al.: Tag Semantics arise from Collaborative Verbosity @ WWW2010 9 / 20
Tagging Pragmatics: Measures
How to disinguish between two types of taggers? Intuition: Describers use open set of many tags,
Categorizers use small set of controlled tags:
Vocabulary size:
Tag / Resource ratio:
Average # tags per post:
high
low
30.04.2010 Körner, Benz et al.: Tag Semantics arise from Collaborative Verbosity @ WWW2010 10 / 20
Tagging Pragmatics: Measures
Next Intuition: Describers don‘t care about „abandoned“ tags, Categorizers do
Orphan ratio:
R(t): set of resources tagged by user u with tag t
high
low
30.04.2010 Körner, Benz et al.: Tag Semantics arise from Collaborative Verbosity @ WWW2010 11 / 20
Tagging pragmatics: Limitations of measures
Real users: no „perfect“ Categorizers / Describers, but „mixed“ behaviour
Possibly influenced by user interfaces / recommenders
Measures are correlated
But: independent of semantics; measures capture usage patterns
30.04.2010 Körner, Benz et al.: Tag Semantics arise from Collaborative Verbosity @ WWW2010 12 / 20
The Story
Semantic Implications of Tagging Pragmatics
Conclusions
Tag Similarity measures can capture emergent tag semantics
Measures of tagging pragmatics differentiate users by tagging motivation
30.04.2010 Körner, Benz et al.: Tag Semantics arise from Collaborative Verbosity @ WWW2010 13 / 20
Influence of Tagging Pragmatics on Emergent Semantics
Idea: Can we learn the same (or even better) semantics from the folksonomy induced by a subset of describers / categorizers?
Extreme Categorizers
Extreme Describers
Complete folksonomy
Subset of 30% categorizers
= user
30.04.2010 Körner, Benz et al.: Tag Semantics arise from Collaborative Verbosity @ WWW2010 14 / 20
Experimental setup
1. Apply pragmatic measures vocab, trr, tpp, orphan to each user 2. Systematically create „sub-folksonomies“ CFi / DFi by
subsequently adding i % of Categorizers / Describers (i = 1,2,…,25,30,…,100)
3. Compute similar tags based on each subset (TagContext Sim.) 4. Assess (semantic) quality of similar tags by avg. JCN distance
TagCont(t,tsim)= …
JCN(t,tsim)= …
DF20 CF5
30.04.2010 Körner, Benz et al.: Tag Semantics arise from Collaborative Verbosity @ WWW2010 15 / 20
Dataset
From Social Bookmarking Site Delicious in 2006 ORIGINAL Two filtering steps (to make measures more meaningful):
Restrict to top 10.000 tags FULL Keep only users with > 100 resources MIN100RES
dataset |T| |U| |R| |Y|
ORIGINAL 2,454,546 667,128 18,782,132 140,333,714
FULL 10,000 511,348 14,567,465 117,319,016
MIN100RES 9,944 100,363 12,125,176 96,298,409
30.04.2010 Körner, Benz et al.: Tag Semantics arise from Collaborative Verbosity @ WWW2010 16 / 20
Results – adding Describers (DFi)
Almost all sub-folksonomies are better than random-picked ones
40% of describers according to trr outperform complete data!
Optimal performance for 70% describers (trr)
more describers
bett
er s
eman
tics
30.04.2010 Körner, Benz et al.: Tag Semantics arise from Collaborative Verbosity @ WWW2010 17 / 20
Results – adding Categorizers (CFi)
Almost all sub-folksonomies are worse than random-picked ones
Global optimum for 90% categorizers (tpp) removing 10% most extreme describers! (Spammers?)
bett
er s
eman
tics
more categorizers
30.04.2010 Körner, Benz et al.: Tag Semantics arise from Collaborative Verbosity @ WWW2010 18 / 20
The Story
Tag Similarity measures can capture emergent tag semantics
Measures of tagging pragmatics differentiate users by tagging motivation
Sub-folksonomies introduced by measures of pragmatics show different semantic qualities
Conclusions
30.04.2010 Körner, Benz et al.: Tag Semantics arise from Collaborative Verbosity @ WWW2010 19 / 20
Summary & Conclusions
Introduction of measures of users‘ tagging motivation (Categorizers vs. Describers)
Evidence for causal link between tagging pragmatics (HOW people use tags) and tag semantics (WHAT tags mean)
„Mass matters“ for „wisdom of the crowd“, but composition of crowd makes a difference („Verbosity“ of describers in general better, but with a limitation)
Relevant for tag recommendation and ontology learning algorithms
30.04.2010 Körner, Benz et al.: Tag Semantics arise from Collaborative Verbosity @ WWW2010 20 / 20
Guess who‘s a Categorizer from the authors
30.04.2010 Körner, Benz et al.: Tag Semantics arise from Collaborative Verbosity @ WWW2010 21 / 20
Thanks for the attention! Questions? Be verbous
Tag Similarity measures can capture emergent tag semantics
Measures of tagging pragmatics differentiate users by tagging motivation
Sub-folksonomies introduced by measures of pragmatics show different semantic qualities
Evidende of causal link between pragmatics and semantics of tagging!
30.04.2010 Körner, Benz et al.: Tag Semantics arise from Collaborative Verbosity @ WWW2010 22 / 20
References
[Cattuto2008] Ciro Cattuto, Dominik Benz, Andreas Hotho, Gerd Stumme: Semantic Grounding of Tag Relatedness in Social Bookmarking Systems. In: Proc. 7th Intl. Semantic Web Conference (2008), p. 615-631
[Markines2009] Benjamin Markines, Ciro Cattuto, Filippo Menczer, Dominik Benz, Andreas Hotho, Gerd Stumme: Evaluating Similarity Measures for Emergent Semantics of Social Tagging. In: Proc. 18th Intl. World Wide Web Conference (2009), p.641-641
[Strohmaier2009] Markus Strohmaier, Christian Körner, Roman Kern: Why do users tag? Detecting users‘ motivation for tagging in social tagging systems. Technical Report, Knowledge Management Institute – Graz University of Technology (2009)