Embed Size (px)
DESCRIPTION
The availability of social tags has greatly enhanced access to information. Tag clouds have emerged as a new “social” way to find and visualize information, providing both one-click access to information and a snapshot of the “aboutness” of a tagged collection. A range of research projects explored and compared different tag artifacts for information access ranging from regular tag clouds to tag hierarchies. At the same time, there is a lack of user studies that compare the effectiveness of different types of tag-based browsing interfaces from the users point of view. This paper contributes to the research on tag-based information access by presenting a controlled user study that compared three types of tag-based interfaces on two recognized types of search tasks – lookup and exploratory search. Our results demonstrate that tag-based browsing interfaces significantly outperform traditional search interfaces in both performance and user satisfaction. At the same time, the differences between the two types of tag-based browsing interfaces explored in our study are not as clear.
Citation preview
Graz University of Technology
1
Christoph Trattner Hypertext 2012
Evaluating Tag-Based Information Access in Image Collections
Christoph Trattner*,
Yiling Lin, Denis Parra, Zhen Yue, Peter Brusilovsky
*Graz University of Technology, AustriaUniversity of Pittsburgh, USA
Graz University of Technology
2
Christoph Trattner Hypertext 2012
Tagging Systems
“Tagging gained tremendously in popularity over the past few years”
Graz University of Technology
3
Christoph Trattner Hypertext 2012
Tags
Graz University of Technology
4
Christoph Trattner Hypertext 2012
Tags
Graz University of Technology
5
Christoph Trattner Hypertext 2012
Tags
Graz University of Technology
6
Christoph Trattner Hypertext 2012
Problem Statement
Not surprisingly, there was a lot of research in the past few years that for instance investigated the value of tags for efficient search and information retrieval in online information systems
Surprisingly, most of the studies only use information retrieval or network-theoretic measures and ignore the user side
To contribute to this field of research we conducted a controlled user study
1 2 3
Graz University of Technology
7
Christoph Trattner Hypertext 2012
What will be presented?
Are tags useful to be used in search interfaces?
Graz University of Technology
8
Christoph Trattner Hypertext 2012
Dataset
~ 2,000 images ~ 4,200 tags ~ 16,000 tag assignments
Interesting Fact:
Tags were generated by ~100 users from Amazon Mechanical Turk
Graz University of Technology
9
Christoph Trattner Hypertext 2012
InterfacesBaseline
Tag Cloud Search InterfaceFaceted Tag Cloud Search Interface
1
23
Graz University of Technology
10
Christoph Trattner Hypertext 2012
How were the interfaces evaluated?
Graz University of Technology
11
Christoph Trattner Hypertext 2012
Evaluation
Within-subject design, i.e. all of or subjects evaluated all interfaces during the study.
Interfaces were counter balanced
Baseline Tag Cloud Faceted Tag Cloud
1 2 3
Graz University of Technology
12
Christoph Trattner Hypertext 2012
Evaluation 2 types of tasks:
Look-up search task
9 images with different difficulty level
Exploratory search task 3 tasks with different difficulty level
Sample Task:
“Find at least 8 different types of stores/shops in Pittsburgh! Each type of store/shop should have at least two images from different locations, i.e. in total you will have to find at least 16 images.”
2 210 418 626 834 104212501458166618740
40
80
120
160
Ran
k P
os
hard
medium easy
Graz University of Technology
13
Christoph Trattner Hypertext 2012
Evaluation: Look-up Task
Look-up task
Graz University of Technology
14
Christoph Trattner Hypertext 2012
Evaluation: Exploratory Search Task
Exploratroy search task
Graz University of Technology
15
Christoph Trattner Hypertext 2012
Evaluation
All in all, 24 subjects Median age 31 19 reported to be familiar with tagging systems All reported to be to use computers more than 5
hours a day All of them reported to be familiar with search
engines
One session took 90 mins
Graz University of Technology
16
Christoph Trattner Hypertext 2012
What are the results?
Graz University of Technology
17
Christoph Trattner Hypertext 2012
Results: Performance (1/2)
Look-up: no sign. differences between interfaces
Exploratory:- Tag Cloud Interface out-performs baseline- Faceted Tag Cloud Interface almost as slow as baseline
Question 1: What interface performs best?Variables: • Total Actions• Search Time
1 2 3
Graz University of Technology
18
Christoph Trattner Hypertext 2012
Results: Performance (2/2)
On medium difficultly level…
Tag Cloud Interface out-performs baseline interface in search time
Question 2: What is the effect of familiarity and difficulty on the performance of the interfaces?
Variables: • Total Actions• Search Time
1 2 3
Graz University of Technology
19
Christoph Trattner Hypertext 2012
Results: Usage (1/2)Question 3: How are the interfaces used?
=> Log analysis
Results: Search action and click image action used most often Add tag action sign. more used in facet Show more results sign. less used in tag cloud
Graz University of Technology
20
Christoph Trattner Hypertext 2012
Results: Usage
Question 4: Does tag grouping by semantic category affect the usage of these categories?
who where when what other0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
40.00%
45.00%
50.00%
Baseline
Tag Cloud
Facet
Answer: Yes,We found sign. differences between the baseline and the faceted tag cloud interface
Graz University of Technology
21
Christoph Trattner Hypertext 2012
Results: Participants‘ perception of the interfaces
Question 5: What was the perception of the particpants regarding the interfaces?
Post-questionaires after each interface
Scale: 1=very bad….5=very good
Graz University of Technology
22
Christoph Trattner Hypertext 2012
Results: Preference and RatingQuestion 6: What was the preference of the users?
• Post-questionair was handed out to the subjects with overall 7 questions.
Question 7: How are the interfaces rated?
Scale: 1 = very bad….5=very good
Tag cloud interface is
rated best
Graz University of Technology
23
Christoph Trattner Hypertext 2012
Questions?
Why did people like the tag cloud interfaces more than the baseline?
Why was the tag cloud interface better rated than the faceted tag cloud interface?
Why did people recommend the faceted tag cloud interface even if they rated the tag cloud interface higher?
Graz University of Technology
24
Christoph Trattner Hypertext 2012
Results: Comment Analysis (1/3)
Why did people like the tag cloud interfaces?
“The tag cloud provided more information than search only interface”
“I like tag cloud because it gives me new ideas and it is easier to use”
Tag cloud interfaces
provide more
information
Than the search-only
interface
Graz University of Technology
25
Christoph Trattner Hypertext 2012
“It is easy to find the tags that I needed in faceted tag cloud”
Why did people prefer the faceted tag cloud interface?
“I like faceted tag cloud interface, because the interface is clearer and I always know where to find the tag”
Results: Comment Analysis (1/3)
People find relevant
tags more easily with
the faceted tag cloud
interface
Graz University of Technology
26
Christoph Trattner Hypertext 2012
Why did people prefer the tag cloud interface over the faceted tag cloud interface?
“The facet did not seem to identify tags well”
“I think the categorization was not good, it was not relevant to the task”
Results: Comment Analysis (2/3)
?
Bad categorization
Graz University of Technology
27
Christoph Trattner Hypertext 2012
Conclusions of this work
In general, tags are useful in search interfaces
Þ They help the user to find information fasterÞ Less clicks, less search time
Þ They give the users hintsÞ They make the user happier
However, depending on the tag interface design different results can be observed…
Take home message
Not always the most advanced interface design is the best
choice…
Graz University of Technology
28
Christoph Trattner Hypertext 2012
End of Presentation
Thank you!
Christoph Trattner
Graz University of Technology, Austria
![Web Applications: Web 2.0 and Beyondsfigueira/162/thomas_yan_slides.pdf · Social Bookmarking in del.icio.us [GH06] It is expected that the combination of the varying tag collections](https://img.pdfslide.us/doc/110x75/6025c858600a9303624e5664/web-applications-web-20-and-beyond-sfigueira162thomasyan-social-bookmarking.jpg)
