Web2.0 in the light of HCIVisualizing Tags over Time

CS 260

October 25, 2006

Hannes Hesse

Web 2.0 is the network as platform, spanning all connected

devices; Web 2.0 applications are those that make the most of

the intrinsic advantages of that platform: delivering software as

a continually-updated service that gets better the more

people use it, consuming and remixing data from multiple

sources, including individual users, while providing their own

data and services in a form that allows remixing by others,

creating network effects through an "architecture of

participation," and going beyond the page metaphor of

Web 1.0 to deliver rich user experiences.

What?

Tags

vs.

Taxonomies

/articles/cats all articles on cats /articles/africa all articles on Africa /articles/africa/cats all articles on African cats /articles/cats/africa all articles on cats from Africa

Taxonomieshierarchical, exclusive

Tagsnon-hierarchical, inclusive

Each choice reflects a decision about the relative importance of each characteristic. Folder names and levels are in themselves informative, in that, like tags, they describe the information held within them (Jones et al. 2005). Folders like 1. and 2. make central the fact that the folders are about “cats” and “africa” respectively, but elide all information about the other category. 3. and 4. organize the files by both categories, but establish the first as primary or more salient, and the second as secondary or more specific. However, looking in 3. for a file in 4. will be fruitless, and so checking multiple locations becomes necessary.

Despite these limitations, there are several good reasons to impose such a hierarchy. Though there can be too many folders in a hierarchy, especially one created haphazardly, an efficiently organized file hierarchy neatly and unambiguously bounds a folder’s contents. Unlike a keyword-based search, wherein the seeker cannot be sure that a query has returned all relevant items, a folder hierarchy assures the seeker that all the files it contains are in one stable place.

In contrast to a hierarchical file system, a non-exclusive, flat tagging system could, unlike the system described above, identify such an article as being about a great variety of things simultaneously, including africa and cats, as well as animals more generally, and cheetahs, more specifically.

Like a Venn diagram, the set of all the items marked cats and those marked africa would intersect in precisely one way, namely, those documents that are tagged as being about African cats. Even this is not perfect, however. For example, a document tagged only cheetah would not be found in the intersection of africa and cats, though it arguably ought to; like the foldering example above, a seeker may still need to search multiple locations.

“cats”

“africa”

“cats” AND “africa”

Figure 1. A Venn diagram showing the intersection of “cats” and “africa”.

Looking at it another way, tagging is like filtering; out of all the possible documents (or other items) that are tagged, a filter (i.e. a tag) returns only those items tagged with that tag. Depending on the implementation and query, a tagging system can, instead of providing the intersection of tags (thus, filtering), provide the union of tags; that is, all the items tagged with any of the given tags, rather than all of them. From a user perspective, navigating a tag system is similar to conducting keyword-based searches; regardless of the implementation, users are providing salient, descriptive terms in order to retrieve a set of applicable items.

2.1 Semantic and Cognitive Aspects of Classification Both tagging systems and taxonomies are beset by many problems that exist as a result of the necessarily imperfect, yet natural and evolving process of creating semantic relations between words and their referents. Three of these problems are polysemy, synonymy, and basic level variation.

A polysemous word is one that has many (“poly”) related senses (“semy”). For example, a “window” may refer to a hole in the wall, or to the pane of glass that resides within it (Pustejovsky 1995). In practice, polysemy dilutes query results by returning related but potentially inapplicable items. Superficially, polysemy is similar to homonymy, where a word has multiple, unrelated meanings. However, homonymy is less a problem because homonyms can be largely ruled out in a tag-based search through the addition of a related term with which the unwanted homonym would not appear. There are, of course, cases where homonyms are semantically related but not polysemous; for example, searching for employment at Apple may be problematic because of conflicts with the CEO’s surname.

Synonymy, or multiple words having the same or closely related meanings, presents a greater problem for tagging systems because inconsistency among the terms used in tagging can make it very difficult for one to be sure that all the relevant items have been found. It is difficult for a tagger to be consistent in the terms chosen for tags; for example, items about television may be tagged either television or tv. This problem is compounded in a collaborative system, where all taggers either need to widely agree on a convention, or else accept that they must issue multiple or more complex queries to cover many possibilities. Synonymy is a significant problem because it is impossible to know how many items “out there” one would have liked one’s query to have retrieved, but didn’t.

Relatedly, plurals and parts of speech and spelling can stymie a tagging system. For example, if tags cat and cats are distinct, then a query for one will not retrieve both, unless the system has the capability to perform such replacements built into it.

Reflecting the cognitive aspect of hierarchy and categorization, the “basic level” problem is that related terms that describe an item vary along a continuum of specificity ranging from very general to very specific; as discussed above, cat, cheetah and animal are all reasonable ways to describe a particular entity. The problem lies in the fact that different people may consider terms at different levels of specificity to be most useful or appropriate for describing the item in question. The “basic level,” as opposed to superordinate (more general) and subordinate (more specific) levels, is that which is most directly related to humans’ interactions with them (Tanaka & Taylor 1991). For most people, the basic level for felines would be “cat,” rather than “animal” or “siamese” or “persian.” Experiments demonstrate that, when asked to identify dogs and birds, subjects used “dog” and “bird” more than “beagle” or “robin,” and when asked whether an item in a picture is an X, subjects responded more quickly when X was a “basic” level (Tanaka & Taylor 1991). These experiments demonstrate general agreement across subjects.

There is, however, systematic variation across individuals in what constitutes a basic level. Expertise plays a role in defining what level of specificity an individual treats as “basic.” For example, in

!"#!"$"%&"'( )$( &*#"'( "'+"$( ,-%%",&.%+( /$"!$( )%'( !"$-/!,"$0(

1"$-/!,"$(2)*(3"()4$-(3"(,-%%",&"'(&-("),5(-&5"!(6"0+07()$( 4.%8$(

3"&9""%( 9"3( #)+"$:( )%'( /$"!$( 2)*( 3"( )$$-,.)&"'( 3*( )( $-,.)4(

%"&9-!87(-!($"&$(-;();;.4.)&.-%$(6"0+07(/$"!$(&5)&(9-!8(;-!(&5"($)2"(

,-2#)%*:0((

(

!"#$%&'()'*'+,-&.',/'01##"2#'3430&+3)'

<5"( &5!""( .%'.=.'/)4( "4"2"%&$(-;( &5"(2-'"4('"#.,&"'( .%(>.+/!"(?(

5)="(3""%($&/'."'(.%'"#"%'"%&4*(.%(&5"(#)$&7(/$/)44*(.%(&5"(,-%&"@&(

-;(9"3A3)$"'($*$&"2$B(

!!!"#$%&'"#() <5"( !"4)&.-%$5.#( 3"&9""%( !"$-/!,"$( )%'( 4.%8$( .$( )(

9"44A!"$")!,5"'( )!")0( C-$&( #!-2.%"%&4*7( D)+"1)%8( E?FG( 5)$(

2)'"()%)4*$.$(-;(4.%8($&!/,&/!"(-%(&5"(9"3()(5-/$"5-4'(%)2"0))

!!*#"&#() H%)4*$.$( -;( $-,.)4( &."$( )%'( $-,.)4( %"&9-!8$( .$( )%(

"$&)34.$5"'($/3;."4'(-;($-,.-4-+*(EIJG()%'(5)$(!",".="'()&&"%&.-%(

;!-2(#5*$.,.$&$7(,-2#/&"!($,."%&.$&$7(",-%-2.$&$7()%'(%/2"!-/$(

-&5"!()!")$(-;($&/'*0)

!!+,-#()1","%&4*7( &5"( )++!"+)&.-%( )%'( $"2)%&.,( )$#",&$( -;( &)+$(

5)="(3""%('.$,/$$"'()%'('"3)&"'()&(4"%+&5(E?KG0(<5.$('.$,/$$.-%(

5)$(2).%4*( ;-,/$"'(-%( &5"(L/)4.&*(-;( .%;-!2)&.-%(#!-'/,"'(3*(

&)++.%+( $*$&"2$( )%'( &5"( #-$$.34"( &!)'"-;;$( 3"&9""%(

;-48$-%-2."$()%'(,!);&"'(-%&-4-+."$(E?M7(ING0(>/!&5"!2-!"7( &5"(

,5)44"%+"$( -;( $5)!"'( =-,)3/4)!."$( ;-!( '"$,!.#&.-%( 5)="( 3""%(

$&/'."'( .%( &5"( .%;-!2)&.-%( $,."%,"( )%'( 4.3!)!*( $,."%,"(

,-22/%.&."$(;-!(2)%*(*")!$(EFG0)

O"$#.&"( &5"$"( .%'.=.'/)4( ,-%&!.3/&.-%$( 695.,5( 9"( 9.44( !"=.$.&( .%(

2-!"('"&).4(.%(P",&.-%(I:7(&-(;/44*(/%'"!$&)%'(&)++.%+($*$&"2$(9"(

3"4."="( )( 5-4.$&.,( )##!-),5( .$( %","$$)!*0( Q)48"!( EIRG( '"$,!.3"$(

&)++.%+()$(S;"!)4(5*#"!&"@&T7()($&!/,&/!"(-/&(-;(,-%&!-47(95"!"(&5"(

$)2"(&)+(.$()$$.+%"'(&-('.;;"!"%&(!"$-/!,"$(9.&5('.;;"!"%&($"2)%&.,(

$"%$"$7( )%'( &5/$( )$$-,.)&"$( -&5"!9.$"( /%!"4)&"'( !"$-/!,"$0(

U-9"="!7( 3*( ,-%$.'"!.%+( &5"( "%&.!"( 2-'"47( ,-2#/&"!( $*$&"2$(

,-/4'(2)8"(.%;"!"%,"$(&5)&(S'-2"$&.,)&"T(6&-(/$"(Q)48"!V$(&"!2$:(

&5"$"( S;"!)4T( &)+$0( >-!( "@)2#4"7( &)+( $"2)%&.,$( )%'( $*%-%*2$(

,-/4'( #-&"%&.)44*( 3"( .%;"!!"'( 3*( )%)4*W.%+( &5"( $&!/,&/!"( -;( &5"(

$-,.)4( %"&9-!87( )%'( .'"%&.;*.%+( ,"!&).%( #-!&.-%$( -;( &5"( %"&9-!8(

&5)&( /$"( ,"!&).%( &)+$( ;-!( &5"( $)2"( !"$-/!,"7( -!( !"4)&"'( !"$-/!,"$7(

.%&"!,5)%+")34*0(<5"$"(&)+$(2)*(3"($*%-%*2-/$0((

H( /%.;."'( /$"!A&)+A!"$-/!,"( )##!-),5( 2.+5&( 3"( /$";/4( ;-!( 2)%*(

8"*(9"3(&",5%-4-+."$7(.%,4/'.%+B($")!,5()%'(.%;-!2)&.-%(!"&!."=)4X(

.%;-!2)&.-%( -!+)%.W)&.-%7( '.$,-="!*( )%'( ,-22/%.,)&.-%X( $#)2(

;.4&"!.%+X( !"'/,.%+( ";;",&$( -;( 4.%8( $#)27( )%'( .2#!-=.%+( -%( &!/$&(

2"&!.,$X( .'"%&.;*.%+( &!"%'$( )%'( "2"!+.%+( &-#.,$( +4-3)44*( )%'(

9.&5.%( ,-22/%.&."$X( )%'( 4-,)&.%+( "@#"!&$( )%'(-#.%.-%( 4")'"!$( .%(

$#",.;.,('-2).%$0(

Y)!.)&.-%$( .%( &5"(2-'"4( '"$,!.3"'( .%( >.+/!"( ?( )!"( #-$$.34"0( >-!(

"@)2#4"7( 4.%8$( 3"&9""%( !"$-/!,"$( ,-/4'( 3"( )3$"%&7( )%'( 4.8"9.$"(

;-!( /$"!$0( Z"="!&5"4"$$7( .%( &5"$"( ,.!,/2$&)%,"$7( 9"( ,)%( $&.44(

-3$"!="( ,-%%",&.-%$( 3"&9""%( /$"!$7( &)+$7( )%'( !"$-/!,"$0( <5"$"(

,-%%",&.-%$( '";.%"( )%( .2#4.,.&( !"4)&.-%$5.#( 3"&9""%( !"$-/!,"$(

&5!-/+5(&5"(/$"!$(&5)&(&)+(&5"2X($.2.4)!4*7(/$"!$()!"(,-%%",&"'(3*(

&5"(!"$-/!,"$(&5"*(&)+0(

[%(-!'"!( &-(3"&&"!( ;!)2"( &5"( $#),"(-;( $-,.)4( &)++.%+( $*$&"2$7(9"(

'"$,!.3"( &9-( -!+)%.W)&.-%)4( &)@-%-2."$( ;-!( $-,.)4( &)++.%+(

$*$&"2$7( '"="4-#"'( 3*( )%)4*W.%+( )%'( ,-2#)!.%+( &5"( '"$.+%( )%'(

;")&/!"$(-;(2)%*("@.$&.%+($-,.)4(&)++.%+($*$&"2$0(<5"(&)@-%-2."$(

'"$,!.3"B(

!!./#0"1) 2"#3-4) ,42) ,00&35%0"#0( Q"( ,4).2( &5)&( &5"( #4),"( -;( )(

&)++.%+( $*$&"2( .%( &5.$( &)@-%-2*(9.44( +!")&4*( );;",&( &5"( %)&/!"(

)%'( '.$&!.3/&.-%( -;( &)+$7( )%'( &5"!";-!"( &5"( )&&!.3/&"$( -;( &5"(

.%;-!2)&.-%(,-44",&"'(3*(&5"($*$&"20(

!!*#"&) 34'"4036"#0( \$"!( 3"5)=.-!$( )!"( 4)!+"4*( '.,&)&"'( 3*( &5"(

;-!2$( -;( ,-%&!.3/&.-%( )44-9"'( )%'( &5"( #"!$-%)4( )%'( $-,.)4(

2-&.=)&.-%$( ;-!( )''.%+( .%#/&( &-( &5"( $*$&"20( <5"( #4),"( -;( )(

&)++.%+( $*$&"2( .%( &5.$( &)@-%-2*( 9.44( );;",&( .&$( -="!)44(

,5)!),&"!.$&.,$()%'(3"%";.&$0(

<-('"2-%$&!)&"(5-9( &5"$"(,4)$$.;.,)&.-%$();;",&( &5"(#!-#"!&."$(-;(

&)+$()%'(/$"!$7(9"(9.44(#!"$"%&()($&/'*(-;(>4.,8!7(-%"(-;(&5"(2-$&(

#-#/4)!( &)++.%+( $*$&"2$( -%( &5"( 9"3( &-')*0( Q"( ,-2#)!"( -/!(

;.%'.%+$( ;!-2( &5"( >4.,8!( $&/'*( &-( &5"( 9-!8( -;( ]-4'"!( )%'(

U/3"!2)%( E^G( -%( O"40.,.-0/$0( >4.,8!( )%'( O"40.,.-0/$( )!"(

,-2#4"2"%&)!*( "@)2#4"$( -;( &)++.%+A$*$&"2$( .%( -/!( &)@-%-2."$X((

9"(#!"$"%&(.%.&.)4("=.'"%,"(&5)&(&5"('*%)2.,$(-;(&5"$"($*$&"2$()!"(

L/.&"('.;;"!"%&0(

<-( &5.$( "%'7( &5"( %"@&( $",&.-%( #!-=.'"$( '"&).4$( -%( !"4)&"'( 9-!87(

2-$&4*(,-%,"%&!)&.%+(-%(),)'"2.,(!"$")!,5(9-!8(.%(!"4)&"'()!")$0(

P",&.-%( _( 3!.";4*( -/&4.%"$( )( %/23"!( -;( ,/!!"%&( &)++.%+( $*$&"2$(

/$"'( )$( .44/$&!)&.-%( .%( '.;;"!"%&( #)!&$( -;( &5.$( #)#"!0( P",&.-%( R(

'"$,!.3"$( -/!( &)@-%-2."$( -;( &)++.%+( $*$&"2( '"$.+%( ,5-.,"$( )%'(

.%,"%&.="$0( ( [%( P",&.-%( J( 9"( #!"$"%&( &5"( !"$/4&$( -;( -/!( $&/'*( -;(

&)++.%+( .%( >4.,8!7( )( #5-&-A$5)!.%+( &)++.%+( $*$&"20(Q"(#!"$"%&( )(

$/22)!*()%'(-/&4.%"(;/&/!"('.!",&.-%$(-;(!"$")!,5(.%(P",&.-%(K0(

5)! 678*97:';<6='O"$#.&"()(,-%$.'"!)34"()2-/%&(-;()&&"%&.-%(.%(),)'"2.,(,.!,4"$7()$(

!"#!"$"%&"'(.%(=)!.-/$(34-+(#-$&$(E?M7ING7(4.&&4"(),)'"2.,(!"$")!,5(

9-!8(5)$(3""%(.%="$&"'(.%(&)++.%+($*$&"2$(&-(')&"0((

D"!5)#$( &5"( 2-$&( $.+%.;.,)%&( ;-!2)4( $&/'*( -;( &)++.%+( $*$&"2$(

)##")!$( .%( &5"( 9-!8( -;( ]-4'"!( )%'( U/3"!2)%( E^G0( <5"( )/&5-!$(

$&/'*( &5"( .%;-!2)&.-%( '*%)2.,$( .%( S,-44)3-!)&.="( &)++.%+(

$*$&"2$T`$#",.;.,)44*7(&5"(O"40.,0.-/$($*$&"20(<5"()/&5-!$('.$,/$$(

&5"(.%;-!2)&.-%('*%)2.,$(.%($/,5()($*$&"27(.%,4/'.%+(5-9(&)+$(3*(

.%'.=.'/)4( /$"!$( )!"( /$"'( -="!( &.2"7( )%'( 5-9( &)+$( ;-!( )%(

.%'.=.'/)4( !"$-/!,"( 6.%( &5"( ,)$"( -;( O"40.,0.-/$7( 9"3( !"$-/!,"$:(

,5)%+"a-!(2-!"( $#",.;.,)44*7( $&)3.4.W"a-="!( &.2"0(Q"( !";"!( &-(

&5".!(;.%'.%+$()+).%(.%(P",&.-%(J0((

]-4'"!( )%'( U/3"!2)%( )4$-( '.$,/$$( &5"( $"2)%&.,( '.;;.,/4&."$( -;(

&)++.%+($*$&"2$0(H$(&5"*(#-.%&(-/&7(7$8/#"1/(695"%()($.%+4"(9-!'(

5)$( 2/4&.#4"( !"4)&"'( 2")%.%+$:( )%'( #/4$4/1/( 695"%( '.;;"!"%&(

9-!'$(5)="(&5"($)2"(2")%.%+:(.%(&5"(&)+(')&)3)$"(3-&5(5.%'"!(&5"(

#!",.$.-%()%'(!",)44(-;(&)++.%+($*$&"2$0([%()''.&.-%7(&5"('.;;"!"%&(

32

(Marlow, Naaman, Boyd, Davis)

A model of tagging systems

A simple taxonomy of tagging systems

The Vocabulary Problem

Probability of two people applying the same term to something

typically below .2

Results in high failure rates in many common situations

(Furnas et al., 1987)

interests. First, we look at users’ activity with respect to their tag use. Next, we examine tags themselves in greater detail.

3.2 User Activity and Tag Quantity As might be expected, users vary greatly in the frequency and nature of their Delicious use. In our “people” dataset, there is only a weak relationship between the age of the user’s account (i.e. the time since they created the account) and the number of days on which they created at least one bookmark (n=229; R2=.52). That is, some users use Delicious very frequently, and others less frequently. Note that these data do not include any users who had previously used Delicious but stopped, as they were all active users at the time the dataset was collected.

More interestingly, there is not a strong relationship between the number of bookmarks a user has created and the number of tags they used in those bookmarks (n=229; R2=.33). The relationship is weak at the low end of the scale, users with fewer than 30 bookmarks (n=39; R2=.33), and even weaker at the upper end, users with more than 500 bookmarks (n=36; R2=.14). Some users have comparatively large sets of tags, and other users have comparatively small sets (Figures 2, 3).

Users’ tag lists grow over time, as they discover new interests and add new tags to categorize and describe them. Tags may exhibit very different growth rates, however, reflecting how users’

interests develop and change over time. Figures 4a and 4b show how use of each tag increases as each user adds more bookmarks over time. For each user, two of those tags’ usages grow steadily, reflecting continual interests tagged in a consistent way. One tag grows rapidly, reflecting a newfound interest or a change in tagging practice. It is possible that the newly growing tag represents a new interest or category to the user. Another possibility is that the user has chosen to draw a new distinction among their bookmarks, which can prove problematic for the user.

Because sensemaking is a retrospective process, information must be observed before one can establish its meaning (Weick et al. forthcoming). Therefore, a distinction may go unnoticed for a long time until it is finally created by the individual, who then continues to find that distinction important in making sense of future information. Since finding previously encountered information is extremely important (Dumais et al. 2003), this is deeply problematic for past information. For example, user # 575 (Figure 4a) did not use “tag 3” until approximately the 2500th bookmark. If ‘tag 3” indeed constitutes a new distinction among the kinds of items this user bookmarks, though Delicious does allow users to alter previous bookmarks, it would be arduous to reconsider each of the earlier 2500 bookmarks to decide whether to add “tag 3” to them. Further, if in the future this user needs to filter his bookmarks by “tag 3”, then no bookmark before the

Figure 2. The number of tags in each user’s tag list, in decreasing order.

Figure 3. Two extreme users’ (#575, #635) tag growth. As they add more bookmarks, the number of tags they

use increases, but at very different rates.

Figure 4a,b. Growth rate of three selected tags for two users (#575, #635).

(Golder and Huberman)

(Marlow, Naaman, Boyd, Davis)

!"#$%&#'()*+*!,!$-#./0+1!"2#$%+$#+#(+34)5#6"&(#%+"#+#"&$#)7#4!"$!38$#

$+9"#.:0+1!"2#$%+$#!"#,+(9&(#$%+3#$%+$#8),,&8$!)3#"!;&<#=)(#&1+5',&>#

$%&#'()*+*!,!$-#$%+$#+#=,!8?(#6"&(#%+"#5)(&#$%+3#@AB#4!"$!38$#$+9"#!"#

()69%,-#B<CD<#E%!"#4!"$(!*6$!)3#!,,6"$(+$&"#$%&#7+8$#$%+$#5)"$#6"&("#

%+F&# F&(-# 7&G# 4!"$!38$# $+9"# G%!,&# +# "5+,,# 9()6'# %+"# &1$(&5&,-#

,+(9&#"&$"#)7#$+9"<#

E%&# (&,+$!)3"%!'#*&$G&&3# $+9#6"+9&#+34#)$%&(# $-'&"#)7# !3'6$#8+3#

*&#+#9))4#!34!8+$)(#)7#%)G#6"&76,#)(#!5')($+3$#6"&("#*&,!&F&#$+9"#

+(&# $)# $%&# &1'&(!&38&# )7# 6"!39# $%&# "-"$&5<# H!$%!3# I&,<!8!)<6">#

J),4&(# +34# K6*&(5+3# 7)634# $%+$# $%&(&# G+"# 3)$# +# "$()39#

+"")8!+$!)3# *&$G&&3# $%&# 365*&(# )7# *))?5+(?"# 5+4&# +34# $%&#

365*&(#)7#$+9"#6"&4#$)#+33)$+$&#$%)"&#*))?5+(?"#LMN<#H&#"$64!&4#

$%(&&# +8$!F!$!&"# G!$%!3# $%&# =,!8?(# &3F!()35&3$O# $%&# 365*&(# )7#

6',)+4&4# '%)$)"># $%&# 8)63$# )7# $%&# 6"&(P"# 4!"$!38$# $+9"># +34# $%&#

365*&(#)7#8)3$+8$"#4&"!93+$&4#*-#$%&#6"&(<#=)(#&1+5',&>#+#8&($+!3#

6"&(#8+3#%+F&#CBB#'%)$)"#G!$%#+# $)$+,#)7#QBB#4!"$!38$# $+9"#+8()""#

$%&"&#'%)$)">#+34#*&#8)33&8$&4#$)#AB#4!77&(&3$#8)3$+8$"<#

#

!"#$%

!"#&%

!"#'%

!"#(%

!"#)%

!"#*%!# !"# !""# !"""# !""""

+,-./0%12%34564786%69:5#

!"#!%

#!"#$%&'()'*"+,%"-$,"./'.0'1"+,"/2,',3#'2.44&2,"./+5'%&6%&+&/,&1'3+',7&'

6%.-3-"4",8',73,'3'%'

E+*,&# Q# "%)G"# $%&# '+!(0G!"&# R&+(")3# 8)((&,+$!)3# LCMN# *&$G&&3#

'%)$)#8),,&8$!)3#"!;&>#4!"$!38$#$+9"#+34#365*&(#)7#8)3$+8$"#+8()""#

$%&#"&$#)7#6"&("<#H&#8)5'6$&4#$%!"#8)((&,+$!)3#7)(#+#"&$#)7#QA>BBB#

6"&("# (+34)5,-# "&,&8$&4# 7()5# )6(# 4+$+"&$<# =)(# &1+5',&># $%&#

8)((&,+$!)3#*&$G&&3#$+9"#+34#'%)$)"#!"#B<ACS>#"699&"$!39#+#"$()39#

,!3&+(# (&,+$!)3"%!'# *&$G&&3# $%&"&# F+(!+*,&"># !<&<# +3# !38(&+"&# !3#

'%)$)#8),,&8$!)3#"!;&#!5',!&"#+3#!38(&+"&#!3#$%&#365*&(#)7#4!"$!38$#

$+9"<#E%&#"$()39&"$#(&,+$!)3"%!'#*&$G&&3#$%&"&#$%(&&#!$&5"#.'%)$)">#

4!"$!38$# $+9"># +34# 8)3$+8$"2# 8)5&"# *&$G&&3# '%)$)"# +34# 4!"$!38$#

$+9"># +# ,!?&,-# (&,+$!)3"%!'#46&# $)# $%&# 7+8$# $%+$# $+99!39#)3&"P#)G3#

'%)$)"# !"# $%&# 4)5!3+3$# 7)(5# )7# $+9"<# E%&# +"")8!+$!)3# *&$G&&3#

8)3$+8$"# +34# '%)$)"# !"# 568%# G&+?&(# $%+3# $%&# )3&# *&$G&&3#

8)3$+8$"# +34# 4!"$!38$# $+9"># G%!8%# 5!9%$# "699&"$# $%+$# $+99!39# !"#

(&,+$&4#$)#")8!+,#+8$!F!$-#$)#")5&#4&9(&&<##

93-4&'()'!4"2:%'$+3#&'2.%%&43,"./'

' 93#+' ;7.,.+' <./,32,+'

93#+' C# <ACS# <TSU#

;7.,.+' <ACS# C# <CMQ#

<./,32,+' <TSU# <CMQ# C#

V#W#X#QA>BBB#

VV#'#Y#B<BBC#7)(#+,,#F+,6&"<#

Z3# +44!$!)3# $)# ")8!+,# !5',!8+$!)3"># +3)$%&(# 7&+$6(&# )7# $+9"#G)($%#

!3F&"$!9+$!39# !"# +3# !34!F!46+,P"# 6"&#)7# $+9"#)F&(# $!5&<#K)G#4)&"#

$%&#7(&[6&38-#)7#$+9"#8%+39&#+"#+#6"&(#*&8)5&"#+88,!5+$&4#$)#$%&#

"-"$&5\#I)#%&(# $+9"#*&8)5&#+#8)%&"!F&# $+1)3)5-#)F&(# $!5&>#)(#

4)#$%&-#8)3$!36&#$)#9()G#+"#%&(#&1'&(!&38&"#8%+39&\#Z3#"$64-!39#

I&,<!8!)<6"># J),4&(# +34# K6*&(5+3# "%)G# &1+5',&"# +"# $)# %)G#

8&($+!3#6"&("P#"&$"#)7#4!"$!38$#$+9"#8)3$!36&#$)#9()G#,!3&+(,-#+"#3&G#

(&")6(8&"#+(&#+44&4<#]$#$%&#"+5&#$!5&>#$%&-#8,+!5#$%+$#$%&#$+9"#7)(#

+# 9!F&3# (&")6(8&# $&34# $)# "$+*!,!;&# +7$&(# )3,-# +# 7&G# 6"&("# %+F&#

$+99&4#!$#LMN<#^!38&#=,!8?(#6"&"#+#"&$05)4&,#7)(#(&'(&"&3$!39#$+9">#

G&#8+33)$#(&&1+5!3&#$%&#,+$$&(#)*"&(F+$!)3>#*6$#G&#8+3#,))?#+$#$%&#

9()G$%#)7#+#6"&(P"#$+9"#)F&(#$!5&<#

=!96(&# T# "%)G"# $%&# 9()G$%# )7# 4!"$!38$# $+9"# 7)(# CB# (+34)5,-#

"&,&8$&4#6"&("#)F&(#$%&#8)6("&#)7#6',)+4&4#'%)$)"<#E%&#6"&("#G&(&#

"&,&8$&4#+"#*)$%#7(&[6&3$#6',)+4&("#.9(&+$&(#$%+3#CBB#'%)$)"2#+34#

7(&[6&3$#$+99&("#.9(&+$&(#$%+3#CBB#$+9"2<#_+8%#')!3$#)3#$%!"#9(+'%#

"%)G"# $%&#365*&(#)7#4!"$!38$# $+9"#./0+1!"2# 7)(#+#9!F&3#6"&(#+7$&(#

$%&#9!F&3#'%)$)#365*&(#.:0+1!"2<#Z$#!"#+''+(&3$#7()5#$%!"#',)$#$%+$#

+# 365*&(#)7# 4!77&(&3$#*&%+F!)("# &5&(9&# 7()5# $%!"# ")8!+,# $+99!39#

"-"$&5<#Z3#")5&#8+"&"#."68%#+"#6"&(#]#!3#=!96(&#T2>#3&G#$+9"#+(&#

+44&4#8)3"!"$&3$,-#+"#'%)$)"#+(&#6',)+4&4>#"699&"$!39#+#"6'',-#)7#

7(&"%#F)8+*6,+(-#+34#8)3"$+3$#!38&3$!F&#7)(#6"!39#$+9"<#^)5&$!5&"#

)3,-#+#7&G#$+9"#+(&#6"&4#!3!$!+,,-#G!$%#+#"644&3#9()G$%#"'6($#,+$&(#

)3># "699&"$!39# $%+$# $%&#6"&(# &!$%&(#4!"8)F&(&4# $+9"#)(# 7)634#3&G#

!38&3$!F&"#7)(#6"!39#$%&5>#+"#G!$%#6"&(#`<#=)(#5+3-#6"&(">#"68%#+"#

$%)"&# G!$%# 7&G# 4!"$!38$# $+9"# !3# $%&# 9(+'%># 4!"$!38$# $+9# 9()G$%#

4&8,!3&"# )F&(# $!5&># !34!8+$!39# &!$%&(# +9(&&5&3$# )3# $%&# $+9#

F)8+*6,+(-># )(# 4!5!3!"%!39# (&$6(3"# )3# $%&!(# 6"+9&<# I&"'!$&# $%&#

%&+F-# 6"+9&# )7# $+9"# 7)(# &+8%# )7# $%&# !34!F!46+,"# G%)"&# $+9"# +(&#

4&'!8$&4# !3# $%&# 7!96(&># +# 365*&(# )7# 8,+""&"# )7# *&%+F!)(# %+F&#

+(!"&3>#!5',-!39#$%+$#$%&#!3$&(+8$!)3#*&$G&&3#6"&(>#$+9>#+34#6$!,!$-#

!"#+#F+(!&4#)3&<#

;4564786%69:5%1</0%64-/

"

'"

!""

!'"

*""

*'"

! *! (! &! =!

>?161%473/@%.A%64-/

B

C

B

#

!"#$%&'=)''>$?-&%'.0'1"+,"/2,',3#+'3,'#"@&/'6."/,+''"/'AB'

%3/1.?'$+&%+C'2.44&2,"./+'

H%&(&+"#J),4&(#%!9%,!9%$&4#)3&# 7)(5#)7# $+9#F)8+*6,+(-#9()G$%>#

3+5&,-# 9()G!39# +$# +# 4!5!3!"%!39# (+$&# )F&(# $!5&># $%&# 9(+'%#

!,,6"$(+$&"# $G)# +44!$!)3+,# 6"&# 8,+""&"# &+8%# G!$%# "&F&(+,# ')""!*,&#

&1',+3+$!)3"<# Z"# $%&# 8+"&# )7# ,!3&+(# 9()G$%# (&,+$&4# $)# $%&# $-'&# )7#

5&4!+# *&!39# $+99&4># 3+5&,-# '%)$)"# $%+$# +(&# $+?&3# )7# 8)3"$+3$,-#

&F),F!39# "6*a&8$#5+$$&(\#b(#4)&"# !$# &F),F&# 7()5#+#5)$!F+$!)3# $)#

8)3$!36+,,-#+$$(+8$#3&G#!34!F!46+,"#$)#$%&#6"&("P#'%)$)"\#c!?&G!"&>#

$%&# 8+"&# )7# 9(+46+,# !38(&+"&# 8)6,4# (&7,&8$# +# 8%+39&# !3# '&(")3+,#

5)$!F+$!)3"# .&<9<># +# 3&&4# $)# "$+($# )(9+3!;!39# '%)$)"# )38&# $%&#

8),,&8$!)3# 9()G"# +*)F&# +# 8&($+!3# "!;&2># )(# +# ")8!+,# )3&# .&<9<># +#

"644&3# (&+,!;+$!)3# $%+$# $+9"# 8+3# *(!39# 3&G# '&)',&# $)# "&&# )3&P"#

'%)$)"2<# E%&"&# [6&"$!)3"# 8)6,4# *&# +3"G&(&4# *-# ,))?!39# +$# $%&#

(&,+$!)3"%!'# *&$G&&3# $%&# 9()G$%# )7# 6"&("P# $+9# 8),,&8$!)3"# +34#

F+(!)6"# 7)(5"# )7# '+($!8!'+$!)3># "68%# +"# $%&# ')'6,+(!$-# )7# $%&!(##

'%)$)"#)(#$%&!(#6"&#)7#$%&#")8!+,#3&$G)(?#"-"$&5<##

37

(Golder and Huberman)

Delicious system for at least six months before reaching their peak of popularity. The URL in our sample that took the longest amount of time to peak in popularity did not do so until it had been in the system for over 33 months.

A burst in popularity may be self-sustaining, as popular URLs are displayed on the “popular” page, which users can visit to learn what others are currently talking about. However, the initial cause of a popularity burst is likely exogenous to Delicious; given that Delicious is a bookmarking service, a mention on a widely read weblog or website is a plausible primary cause. Kumar et al. (2003) demonstrate “burstiness” among links in weblogs, and literature on opinion and fad formation demonstrate how “well-connected” individuals and “fashion leaders” can spread information and influence others (Wu & Huberman 2005; Bikhchandani, Hirshleifer & Welch 1998).

4.2 Stable Patterns in Tag Proportions As a URL receives more and more bookmarks, the set of tags used in those bookmarks, as well as the frequency of each tag’s use within that set, represents the combined description of that URL by many users.

One might expect that individuals’ varying tag collections and personal preferences, compounded by an ever-increasing number of users, would yield a chaotic pattern of tags. However, it turns out that the combined tags of many users’ bookmarks give rise to a stable pattern in which the proportions of each tag are nearly

fixed. Empirically, we found that, usually after the first 100 or so bookmarks, each tag’s frequency is a nearly fixed proportion of the total frequency of all tags used. Figures 7a and 7b show this pattern. Each line represents a tag; as more bookmarks are added (horizontal axis), the proportion of the tags represented by that tag (vertical axis) flattens out. A web tool that visualizes Delicious data, called Cloudalicious also shows this pattern.

This stable pattern can be explained by resorting to the dynamics of a stochastic urn model originally proposed by Eggenberger and Polya to model disease contagion (Eggenberger & Polya 1923). In its simplest form, this probabilistic model consists of an urn initially containing two balls, one red and one black. At each time step, a ball is randomly selected and replaced in the urn along with an additional ball of the same color. Thus, after N steps, the urn contains N+2 balls. The remarkable property of such a model is that, in spite of its random nature, after a number of draws a pattern emerges such that the fraction of balls of a given color becomes stable over time. Furthermore, that fraction converges to a random limit. This implies that if the process is run forever the fraction converges to a limit, but the next time one starts the process over and run it again the stable fraction will converge to a different number.

This behavior is shown in Figures 7a and 7b, which are indistinguishable from what one would obtain from running a computer simulation of the urn model, where the colored balls would correspond to the tags observed.

Figure 7a,b. The stabilization of tags’ relative proportions for two popular URLs (#1310, #1209). The vertical axis denotes fractions and the horizontal axis time in units of bookmarks added.

Figure 6a,b. The addition of bookmarks to two URLs (#1310, #1209) over time. URL #1310 (left) peaks immediately, whereas #1209 (right) is long obscure before peaking.

(Marlow, Naaman, Boyd, Davis)

!"#! $%&'()*'+,-.%+/'01%2-!""# $%# &'(# &)**+,*# -.-&(/-#0(#')1(#/(,&+$,(2# +,# &'+-# 3)3(4# )4(#

)4*5)6".# -$7+)"# +,#,)&54(8# +,# -$/(#7)-(-# &'(# -$7+)"#)-3(7&#7$/(-#

%4$/# "(1(4)*+,*# &'(# 7$//5,+&.9-# 7$""(7&+1(# +,&(""+*(,7(:# ),2# +,#

$&'(4-# &'(4(# +-# (;3"+7+&# -$7+)"# +,&(4)7&+$,# )4$5,2# &'(# 5-(# $%# &)*-<#

=(7)5-(# >"+7?4# )""$0-# 5-(4-# &$# (,5/(4)&(# -$7+)"# ,(&0$4?-# ),2#

2(1("$3# 7$//5,+&+(-# $%# +,&(4(-&:# &'(4(# +-# )# '5*(# 3$&(,&+)"# %$4#

-$7+)"#+,%"5(,7(#+,#&'(#2(1("$3/(,&#$%#&)*#1$7)65")4+(-<#

@,(# %()&54(# $%# &'(# 7$,&)7&# ,(&0$4?# +-# )# 5-(49-# )6+"+&.# &$# ()-+".#

%$""$0#&'(#3'$&$-#6(+,*#53"$)2(2#6.#&'(+4#%4+(,2-<#A'+-#34$1+2(-#)#

7$,&+,5$5-#)0)4(,(--#$%# &'(#3'$&$*4)3'+7#)7&+1+&.#$%# &'(+4#>"+7?4#

7$,&)7&-:# ),2# 6.# &4),-+&+1+&.:# )# 7$,-&),&# (;3$-54(# &$# &)**+,*#

34)7&+7(-<# B$# &'(-(# 4(")&+$,-'+3-# )%%(7&# &'(# %$4/)&+$,# $%# &)*#

1$7)65")4+(-:# $4# )4(# +,2+1+25)"-# *5+2(2# 6.# $&'(4# -&+/5"+C# A$#

(;3),2# $,# &'+-# D5(-&+$,:# 0(# ')1(# 4),2$/".# 7'$-(,# EFGG# 5-(4-#

0+&'#)#7$,-+2(4)6"(#,5/6(4#$%#&)*-#H*4()&(4#&'),#IGGJ#),2#3)+4(2#

&'(/#0+&'# &0$#$&'(4# +,2+1+25)"-K# $,(# 4),2$/".# 7'$-(,# %4$/# &'(#

4(-&#$%#&'(#-(&:#),2#&'(#$&'(4#%4$/#&'(+4#"+-&#$%#7$,&)7&-<#>4$/#&'(-(#

3)+4+,*-# 0(# ')1(# 7)"75")&(2# &'(# $1(4")3# +,# &'(+4# &)*# -(&-8# &'(#

$1(4")3#+-#7$/35&(2#)-#!!"=!L!!#=!:#0'(4(#!#),2#=#)4(#&'(#-(&-#$%#&)*-#%4$/#$54#&0$#5-(4-<#

A'(# 4(-5"&-# $%# &'+-# +,D5+4.# )4(# 2(37+&(2# +,# >+*54(# M<# A'+-# *4)3'#

-'$0-# &0$# %4(D5(,7.# 2+-&4+65&+$,-# %$4# &'(# $1(4")3-# 6(&0((,# -(&-#

$%# 5-(4-K# &'(# $1(4")3# 6(&0((,# &'(# *+1(,# 5-(4# ),2# ),$&'(4#

4),2$/".# 7'$-(,# $,(:# -'$0,# 0+&'# )# 2)-'(2# 6"5(# "+,(:# ),2# &'(#

$1(4")3#6(&0((,# &'(# -)/(#5-(4# ),2#$,(#$%# &'(+4# 7$,&)7&-:# -'$0,#

0+&'#)#-$"+2# 4(2# "+,(<#A'(# 4),2$/#5-(4-#)4(#/57'#/$4(# "+?(".# &$#

')1(#)#-/)""(4#$1(4")3# +,#7$//$,#&)*-:#0'+"(#7$,&)7&-#)4(#/$4(#

2+-&4+65&(2:#),2#')1(#)#'+*'(4#$1(4)""#/(),<##

!

"!

#!!

#"!

$!!

$"!

%!!

! !&!$ !&!' !&!( !&!) !&# !&#$ !&#' !&#(

*+,-./0+123+45-/.643+7-89:

;<64-.<&-=+>?/@

;<64-.<&-A/>B+0B

#

.13)+4-5"-$%&'()*'+,-%64+*'7-8190+1()01%2-:%+-+'28%/-)94+9-

'28-&%20'&09-;2<#!==>-

A'+-#4(-5"&:#0'+"(#-&+""#34("+/+,)4.:#-'$0-#)#4(")&+$,-'+3#6(&0((,#

-$7+)"# )%%+"+)&+$,# ),2# &)*# 1$7)65")4.# %$4/)&+$,# ),2# 5-(# (1(,#

&'$5*'#&'(#3'$&$-#/).#6(#$%#7$/3"(&(".#2+%%(4(,&#-56N(7&#/)&&(4<#

A'+-#7$//$,)"+&.#7$5"2#)4+-(#%4$/#-+/+")4#2(-74+3&+1(#&)*-#H(<*<:#

64+*'&:#7$,&4)-&:#6")7?#),2#0'+&(:#$4#$&'(4#3'$&$#%()&54(-J:#-+/+")4#

7$,&(,&# H3'$&$-# &)?(,# $,# &'(# -)/(#1)7)&+$,J:# $4# -+/+")4# -56N(7&-#

H7$O$77544+,*# %4+(,2-# ),2# %)/+".J:# ()7'# -5**(-&+,*# 2+%%(4(,&#

/$2(-#$%#2+%%5-+$,<#

@&'(4# "+?(".# (;3"),)&+$,-# %$4# &'(# $6-(41(2# 7$44(")&+$,# 6(&0((,#

-$7+)"# 7$,,(7&+$,# ),2# 7$//$,# &)*# 5-)*(# /).# 6(# %$5,2# +,# &'(#

2(-74+3&+1(# 7)&(*$4+(-# $%# -$7+$"+,*5+-&+7-# 0'+7'# -&52+(-# '$0#

2+%%(4(,&#*($*4)3'+7#),2#-$7+)"#%$4/)&+$,-#-&457&54(#&'(#7$'(4(,7(#

),2# 2+%%5-+$,# $%# -(/),&+7# ),2# -.,&)7&+7# -&457&54(-# +,# 1)4+$5-#

P"(7&-P# 0+&'+,# )# ")4*(4# -$7+$"+,*5+-&+7# -.-&(/<# Q$/(# $%# &'(-(#

(;)/3"(# "(7&-# +,7"52(K# 2+)"(7&# H)# "(7&# 5-(2# 6.# )# !"#!$%&'()%**+#

2(%+,(2#7$//5,+&.J8# -$7+$"(7&# H)# "(7&#5-(2#6.#)#,#)(%**+#2(%+,(2#

7$//5,+&.J8# (&',$"(7&# H)# "(7&# -3$?(,# 6.# )# 3)4&+75")4# (&',+7#

*4$53J8#(7$"(7&#H)#"(7&#-3$?(,#0+&'+,#)#'$5-('$"2#$4#%)/+".J8#),2#

+2+$"(7&#H)#"(7&#3)4&+75")4#&$#)#7(4&)+,#3(4-$,J<#R%#0(#7$,7(3&5)"+S(#

-$7+)"# &)**+,*# -.-&(/-# 0+&'+,# &'(# &'($4(&+7)"# %4)/(# $%#

-$7+$"+,*5+-&+7-:# &'(-(# ),2# $&'(4# T"(7&-P# -((/# (-3(7+)"".#

)33"+7)6"(# &$# 5,2(4-&),2+,*# ),2# 7")--+%.+,*# &'(# )33)4(,&#

+-$/$43'+-/#6(&0((,#-$7+)"#),2#"+,*5+-&+7#-&457&54(-#0(#$6-(41(2#

+,# >"+7?4<# A'(# -&457&54(-:# 7'),*(-:# ),2# 2+%%5-+$,# 0+&'+,# ),2#

)/$,*-&#1)4+$5-#T"(7&-P#+,#-$7+)"#&)**+,*#-.-&(/-#0+""#"+?(".#')1(#

-+/+")4#3)&&(4,-# &$# &'$-(#%$5,2#+,#-$7+)"#,(&0$4?#),)".-(-#),2# +,#

-$7+$"+,*5+-&+7# "),*5)*(#/)3-<# #U$,-+2(4+,*#&'(-(#-$7+$"+,*5+-&+7#

7)&(*$4+(-# )-#0(#)&&(/3&# &$# 7$/35&(# -&457&54)"# +-$/$43'+-/#),2#

&'(# +,&(4)7&+$,-# 6(&0((,# -$7+)"# -&457&54(-# ),2# &)**+,*# -&457&54(-#

H%$4# (;)/3"(:# '56-:# 64+2*(-:# ),2# 2+%%5-+$,J# /).# 34$1(#

(;7(3&+$,)"".# 5-(%5"# +,# (;3")+,+,*# &'(# %$4/)&+$,:# (%%+7)7.:# ),2#

2.,)/+7-#$%#-$7+)"#&)**+,*#-.-&(/-<##

A'(-(# D5(-&+$,-# 7)""# %$4# )# /57'# 2((3(4# +,1(-&+*)&+$,# $%# &'+-#

3'(,$/(,$,:#)#-&52.#&')&#7$5"2#),-0(4#/),.#D5(-&+$,-#)6$5&#&'(#

4(")&+$,-'+3#6(&0((,#3($3"(:# 7$//5,+7)&+$,:# ),2# &'(#(/(4*(,7(#

$%#7$//$,#"(7&-#+,#-$7+)"#&)**+,*#-.-&(/-<#

?"! @AB@CDEFABE-Q$7+)"# &)**+,*# -.-&(/-# ')1(# &'(# 3$&(,&+)"# &$# +/34$1(# $,#

&4)2+&+$,)"# -$"5&+$,-# &$# /),.# 0(""O-&52+(2# 0(6# ),2# +,%$4/)&+$,#

-.-&(/-#34$6"(/-<#Q57'#34$6"(/-#+,7"52(#3(4-$,)"+S(2#$4#6+)-(2#

"+,?# ),)".-+-:# $4*),+S+,*# +,%$4/)&+$,:# +2(,&+%.+,*# -.,$,./-# ),2#

'$/$,./-:# 65+"2+,*# ,(&0$4?-# $%# &45-&# &$# 7$/6)&# "+,?# -3)/:#

/$,+&$4+,*#&4(,2-#),2#24+%&#+,#+,%$4/)&+$,#-.-&(/-#),2#/$4(<#A'(#

34$-3(7&-# $%# 4()-$,+,*# )6$5&# &)*-:# 5-(4-:# ),2# 4(-$547(-# +,# 5,+&.#

)4(#(,7$54)*+,*<#

R,# $42(4# &$# -&52.# &'(-(# -.-&(/-:# 4(-()47'(4-# -'$5"2# $6-(41(# &'(#

-.-&(/9-#3")7(#0+&'+,#&'(#&);$,$/.#$%#)47'+&(7&54(-#2(-74+6(2#+,#

Q(7&+$,# V<I<# Q&52+(-# -'$5"2# )"-$# 7$,-+2(4# &'(# +,7(,&+1(-# 24+1+,*#

3)4&+7+3)&+$,:# ),2# &'(# (;&(,&# &$# 0'+7'# &'(# -.-&(/# -533$4&-# $4#

4(-&4)+,-#&'(-(#/$&+1)&+$,-<#R,#-&52.+,*#>"+7?4:#0(#-'$0(2#&')&#&'(#

2.,)/+7-#$%#+,&(4)7&+$,#),2#3)4&+7+3)&+$,#)4(#2+%%(4(,&#&'),#&'$-(#

$%# B("<+7+$<5-<# R,2((2:# >"+7?4# ),2# B("<+7+$<5-# )4(# 4)&'(4# 2+-&+,7&#

0'(,# 3$-+&+$,+,*# &'(/# +,# &'(# 2+/(,-+$,-# $%# $54# &);$,$/.<#

B("<+7+$<5-# +-# )# %4((O%$4O)"":# -5**(-&+1(:# 6)*O/$2("# H&$# /(,&+$,#

N5-&# &'4((#?(.#2+/(,-+$,-J# -.-&(/<#B("<+7+$<5-# +-# &'(4(%$4(# "+?(".#

&$# *(,(4)&(# )# 2+%%(4(,&# 5-(# /$2("# ),2# $5&35&# &'),# >"+7?4:# )#

H/$-&".J#-("%O&)**+,*:#1+(0)6"(:#-(&O/$2("#-.-&(/<#W$4($1(4:#&'(#

+,7(,&+1(#/$2("-# $%# >"+7?4# ),2#B("<+7+$<5-# )4(# )"-$# -56-&),&+)"".#

2+-3)4)&(:# -5**(-&+,*# (1(,# /$4(# (;3(7&(2# 2+%%(4(,7(-# +,# &'(#

-.-&(/-9#$5&35&<#

X(# '$3(# &')&# -.-&(/# 2(-+*,(4-# 0+""# 7$,-+2(4# &'(-(# 2(-+*,#

2(7+-+$,-#+,#)47'+&(7&+,*#&'(+4#&)**+,*#-.-&(/-<#=.#").+,*#$5&#&'(#

+/3"+7)&+$,-# $%# &'(# 7'$+7(-# +,# ()7'# 2+/(,-+$,# $%# $54# '+(4)47'.:#

0(#'$3(#&$#)--+-&#3"),,(4-#)-#0(""#)-#4(-()47'(4-#),2#)7)2(/+7-<##

>+,)"".:#6.#,$#/(),-#2$#0(#7$,&(,2#&')&#&'(#2(-+*,#&);$,$/.#),2#

+,7(,&+1(# &);$,$/.# 0(# 2(-74+6(# )4(# 7$/3"(&(<# Y(0# 5-(-# %$4#

&)**+,*# -.-&(/-# )4(# +,1(,&(2# (1(4.# 2).8# 5-(4-# $%# -57'# -.-&(/-#

)334$34+)&(#&'(/#0+&'#),#(1(4O7'),*+,*#-(&#$%#*$)"-:#/$&+1(-:#),2#

)-3+4)&+$,-<#X(#'$3(#&')&#$54#&);$,$/.#7),#-(41(#)-#)#%$5,2)&+$,#

%$4#4(-()47'(4-#),2#(,)6"(#)#/$4(#7$/3"(&(#5,2(4-&),2+,*#$%#&'(#

7$,-&4)+,&-#),2#)%%$42),7(-#$%#&)*O6)-(2#+,%$4/)&+$,#-.-&(/-<##

38

“Leveraging collective intelligence”

Human Computation(von Ahn)

!

"#$%&'()*!+,-!%#!+&-'(.#$#.!,!/012,-!,34&$()0256!4(7#-!,!

8&$.! ,'! (-91):! ()! &1)91)'! ,! '#)! &;! +&22&-<'#-'#! ;,+)'!

$#3,)#.!)&!)0#!8&$.=!>-')#,.!&;!1'(-4!,!+&291)#$!9$&+#''&$:!

)0&140:! )0('! /,34&$()025! 1'#'! &$.(-,$*! 012,-'! (-)#$,+)(-4!

8()0!+&291)#$'! )0$&140&1)! )0#!?#%=!@1$!'*')#2! )0#$#;&$#!

'(4-(;(+,-)3*! +&-)$(%1)#'! )&! AB>! (-! )8&! 8,*'6! ()! +&33#+)'!

+&22&-<'#-'#!.,),!)0,)!+,-!(29$&7#!AB>!,993(+,)(&-':!,-.!

()!,..$#''#'!,!)*9(+,3!C>!9$&%3#2!8()0!-&7#3!AB>!)&&3'=!

!"#$%&'$()*$+&&,-*.$

B&291)#$'!-&8!0,7#! )0#!,%(3()*! )&!'#,$+0!7,')!,2&1-)'!&;!

.,),!(-!3())3#! )(2#=!D0('!2#,-'!)0,)!9#$0,9'!8#!+&13.!1'#!,!

'#,$+0!#-4(-#!)&!+&33#+)!)0#!(-;&$2,)(&-!8#!-##.=!C3)0&140!

'1+0!,-!,99$&,+0!*(#3.'!'&2#!1'#;13!.,),!EFG:!()!('!-&)!4&&.!

#-&140!;&$!&1$!91$9&'#'!;&$!)0$##!$#,'&-'=!H($'):!2&')!&;!)0#!

I-&83#.4#!)0,)!8#!,$#!)$*(-4!)&!+&33#+)!('!'&!&%7(&1'!)0,)!-&!

&-#!0,'!%&)0#$#.!)&!$#+&$.!()=!J#+&-.:!)0#$#!#K(')'!(-+&$$#+)!

I-&83#.4#!&-!)0#!?#%!L;&$!#K,293#:!,)!)0#!)(2#!&;!8$()(-4!

)0('! 9,9#$:! '#,$+0(-4! ;&$! /+,$'! +,-! '8(25! $#)1$-'! 2&$#!

$#'13)'! &-!M&&43#! )0,-! /+,$'! +,-N)! '8(25O=!D0($.:! )0#! )#K)!

&-! )0#!?#%! ('! 1-')$1+)1$#.! ,-.! )1$-(-4! ()! (-)&! ,! .($#+)3*!

1'#;13! ;&$2,)! ('! ,! -&-<)$(7(,3! ),'I=! @1$! '*')#2! .&#'! -&)!

'1;;#$!;$&2!)0#'#!'0&$)+&2(-4'=!!

/012304$!5/6$

D0#$#!0,7#!%##-! ,! 7,$(#)*! &;! &)0#$!2#)0&.'! ;&$! +&33#+)(-4!

+&22&-<'#-'#!;,+)':!80(+0!.(;;#$!(-!-,)1$#!;$&2!&1$'=!!

7#8$$

B*+! L#=4=:! EPGO!8,'! )0#! ;($')! #;;&$)! ,)! %1(3.(-4! ,! +&22&-<

'#-'#!.,),%,'#=!B*+! '),$)#.!%*!+$#,)(-4!,! '##.!.,),%,'#!&;!

+&22&-<'#-'#!I-&83#.4#!1'(-4!9,(.!#K9#$)'!)&!#-)#$!;,+)'!

(-!B*+Q!R!,!9$&9$(#),$*:!7#$*!9$#+('#!3,-41,4#!.#7#3&9#.!

(-!&$.#$! )&!,7&(.!,-*!,2%(41()*!9$&%3#2'=!S'(-4! )0('!'##.!

.,),%,'#! ,'! ,! '),$)(-4! 9&(-):! 2&$#! .,),! +,-! %#! +&33#+)#.!

L#=4=:!%*!+&2%(-(-4!;,+)'!)&!(-;#$!-#8!&-#'O=!!!

@-#! 9$&%3#2! 8()0! )0('! ,99$&,+0! ('! )0,)! )0#! -12%#$! &;!

+&22&-<'#-'#!;,+)'!$#T1($#.!('!&$.#$'!&;!2,4-()1.#!0(40#$!

)0,-! )0,)! 80(+0! +&13.! %#! #-)#$#.! %*! ,-! ('&3,)#.! '#)! &;!

#K9#$)'!R!&7#$!)0#!+&1$'#!&;!,!.#+,.#:!B*+!0,'!%##-!,%3#!)&!

9&913,)#! ()'!2,(-!.,),%,'#!8()0!,$&1-.!,!2(33(&-!9(#+#'!&;!

(-;&$2,)(&-! EUG=! C'! #7(.#-+#.! %*! )0#! VJW!M,2#! EXG:! &1$!

,99$&,+0! 0,'! )0#! 9&)#-)(,3! )&! +&33#+)!2(33(&-'! &;! 9(#+#'! &;!

I-&83#.4#!8()0(-!)0#!'9,+#!&;!'#7#$,3!8##I'=!

59*:$;<:=$>:=$;<:=9<?*-$

Y&$#! $#+#-)3*:! )0#! @9#-! Y(-.! 9$&Z#+)! 0,'! $#3(#.! &-!

/-#)([#-'5! L>-)#$-#)! 1'#$'O! )&! #-)#$! +&22&-<'#-'#! ;,+)'=!

@9#-!Y(-.!0,'!.&[#-'!&;!,+)(7()(#':!#,+0!.#'(4-#.!)&!4,)0#$!

'9#+(;(+! )*9#'! &;! I-&83#.4#6! '9,)(,3:! 0(#$,$+0(+,3:!

(293(+,)(&-:!#)+=!J&!;,$:!)0#*!0,7#!4,)0#$#.!'#7#$,3!01-.$#.!

)0&1',-.!9(#+#'!&;!I-&83#.4#=!C-&)0#$!9$&Z#+):!Y(-.9(K#3!

E\G:! ('! '(2(3,$! )&! @9#-!Y(-.! (-! )0,)! ()! $#3(#'! &-! &$.(-,$*!

>-)#$-#)! 1'#$'=! D0#*! 0,7#! ,! +&33,%&$,)(7#! '*')#2! (-!80(+0!

2,-*!9,$)(+(9,-)'!+$#,)#!,-.!+3,''(;*!,!'),)#2#-)!,'!)$1#!&$!

;,3'#:!)01'!%1(3.(-4!19!,!3,$4#!.,),%,'#!&;!;,+)'=!",3(.,)(&-!

('! ,! 2,Z&$()*<%,'#.! '*')#2:! ,-.! ()! $#8,$.'! )0&'#! 80&!

+&-'(')#-)3*!7,3(.,)#!,!;,+)!(-3(-#!8()0!)0#!&)0#$!1'#$'=!!

D0#!2,Z&$!(29$&7#2#-)!)0,)!8#!2,I#!&7#$!@9#-!Y(-.!,-.!

Y(-.9(K#3! ('! )0,)! &1$! '*')#2! !"#$%&'"(%! )0#! 9$&+#''! &;!

#-)#$(-4!;,+)'!(-)&!,!)$1#!4,2(-4!#-7($&-2#-)=!C+)(7()(#'!(-!

@9#-! Y(-.! ,-.! Y(-.9(K#3! $#T1($#! 1'#$'! )&! '(293*! #-)#$!

;,+)']!8#!.&-N)!-##.!)&!#K93(+()3*!,'I!1'#$'!)&!.&!'&=!!

!<@<9*=<>$

C-&)0#$! #K,293#! &;! 4,)0#$(-4! I-&83#.4#! ;$&2! 012,-'! ('!

?(I(9#.(,=&$4:! ,! )$#2#-.&1'3*! '1++#'';13! &-3(-#!

#-+*+3&9#.(,!)&!80(+0!,-*%&.*!+,-!+&-)$(%1)#=!!

>-! ,..()(&-! )&! )0#! 4,2(-4! ,'9#+):! &1$! '*')#2! ('! .(;;#$#-)!

;$&2!?(I(9#.(,! (-! )0,)! )0#!I-&83#.4#!8#!,$#!(-)#$#')#.!(-!

('!'(4-(;(+,-)3*!'(293#$=!V-+*+3&9#.(,'!,$#!2&')3*!1'#;13!;&$!

)0#($! 3())3#<I-&8-! ;,+)']! 8#! 8('0! )&! +&33#+)! ;,+)'! )0,)!

#7#$*%&.*!I-&8'!L80(+0!,$#!&;!3())3#!1'#!(-!#-+*+3&9#.(,'O=!!

A2BC7$+2;0$;07D2%C7B$

"#$%&'()*! ('! 2#,-)! )&! %#! 93,*#.! &-3(-#! %*! )8&! 93,*#$'!

'#3#+)#.! ,)! $,-.&2=! @-#! &;! )0#! 93,*#$'! ('! +0&'#-! ,'! )0#!

/^,$$,)&$5!80(3#! )0#! &)0#$! ('! )0#! /M1#''#$=5! D0#!^,$$,)&$!

4#)'! ,! '#+$#)! 8&$.! ,-.!21')! 4#)! )0#! M1#''#$! )&! )*9#! )0,)!

8&$.! %*! '#-.(-4! 0(-)'! )&! )0#! M1#''#$=! D0#! 0(-)'! ),I#! )0#!

;&$2!&;!'#-)#-+#!)#293,)#'!8()0!%3,-I'!)&!%#!;(33#.!(-=!D0#!

^,$$,)&$! +,-! ;(33! (-! )0#! %3,-I'! 8()0! ,-*! 8&$.! )0#*! 8('0!

#K+#9)! )0#! '#+$#)!8&$.! L&$!,-*!')$(-4!+&-),(-(-4! )0#! '#+$#)!

8&$.O=!J##!H(41$#!X=!H&$!#K,293#:!(;!)0#!8&$.!('!QCWD@W:!

)0#!^,$$,)&$!2(40)!',*6!/()!0,'!,!_V`a@Cbc=5!

!

!"#$%&'()'*+%,'-.',/&'0+%%+,-%12'23%&&4)'

D0#!'#-)#-+#!)#293,)#'!)0,)!)0#!-,$$,)&$!+,-!1'#!+&2#!(-!)0#!

;&$2! &;! +,$.'=! D0#! ^,$$,)&$! '),$)'! 8()0! d! +,$.'! (-! )0#($!

0,-.:! ,-.! #,+0! +,$.! +&-),(-'! &-#! '#-)#-+#! )#293,)#=! D&!

4(7#! 0(-)'! ,%&1)! )0#! '#+$#)! 8&$.:! )0#! ^,$$,)&$! +0&&'#'! ,!

+,$.:! ;(33'! (-! )0#! %3,-I'! (-! )0#! +,$.! ,-.! '#-.'! ()! )&! )0#!

M1#''#$=!M(7#-! )0#! 0(-)':! )0#!M1#''#$! +,-! )*9#!80,)! )0#*!

!"#$%#&

'(&)*+(,-+.&,&/012%"34

Google Zeitgeist

Representing data

that changes

over time

SmartMoney treemapMartin Wattenberg

LifeLinesBrett Milash, Catherine Plaisant, Anne Rose, 1996UMD

!"#$%&'()* (+,* %+-* ."$!* /!-)-0"1-'2* 3-* (#4-,* -(5!

6('$"5"6(+$*78-#$"%+#*(9%8$*.!($* $!-:*%9#-'1-,* "+*-(5!

,"#6;(:2

<=()6;-#*%>*#6-5">"5*78-#$"%+#*"+5;8,-•! ?+*@8;:*ABCD*.!($*('-* $!-* $!'--*)%#$*,"#58##-,$!-)-#E

•!3!-'-*"#*(*+-.*$!-)-*"+$'%,85-,E<=()6;-#*%>*)%'-*&-+-'(;*78-#$"%+#*"+5;8,-

•!3!($* ;%%4#* "+$-'-#$"+&* !-'-* F* .!($* ,%* :%8.(+$*$%*-=6;%'-E

•!G%.*.%8;,* :%8* ;"4-* $%* 5!(+&-* %'*)(+"68;($-$!-*1"-.E

3-*5(6$8'-,*1-'9(;*6'%$%5%;*,8'"+&* $!"#* ,"#58##"%+2

H$* $!-* -+,D* .-* (#4-,* 6('$"5"6(+$#* $%* 5%)6;-$-* (* #!%'$

78-#$"%++("'-D*."$!*>--,9(54*(9%8$*$!-*1"#8(;"I($"%+*(+,

6%##"9;-*-+!(+5-)-+$#2

J'%)*$!-*1-'9(;*6'%$%5%;*(+,*>'%)*8#-'*9-!(1"%'D*.-

%9#-'1-,*$!($*$!-*8#-'#*!(,*+%*,">>"58;$:*"+*8+,-'#$(+,K

"+&* $!-* )-$(6!%'2* /!-:* .-'-* (9;-* $%* ",-+$">:* $!-)-#

$!($* .-'-* #$'%+&;:* '-6'-#-+$-,* (+,* (9;-* $%* 8+,-'#$(+,

$!-* '-;($"%+#!"6* 9-$.--+* $!-* .",$!* %>* $!-* 58''-+$* (+,

$!-)-* #$'-+&$!2* /!-* 1"#8(;"I($"%+* (;#%* $'"&&-'-,* 78-#K

$"%+#* (9%8$* $!-* '-(#%+#* 9-!"+,* 5-'$("+* $!-)-* #$'-+&$!#

(+,* 6($$-'+#2* J%'* -=6;%'($%':* 1"#8(;"I($"%+#D* $!"#* "#* (

&%%,*'-#8;$L*.-*9-;"-1-*$!($*(*1"#8(;"I($"%+*#!%8;,*!-;6

$!-*8#-'*",-+$">:*78-#$"%+#*%>*"+$-'-#$*$%*-=6;%'-2

M8-#$"%++("'-* '-#6%+#-#* #!%.-,* $!($* 8#-'#* >%8+,

/!-)-0"1-'* -(#:* $%* 8+,-'#$(+,2* /!-:* (;#%* >%8+,

/!-)-0"1-'* 8#->8;D* 6('$"58;(';:* >%'* ",-+$">:"+&* )(5'%

$'-+,#2* /!-:* $%;,* 8#* $!($* "$* .(#* ;-##* 8#->8;* >%'* ",-+$"K

>:"+&* )"+%'* $'-+,#* 9-5(8#-* $!-* 58'1-#* $-+,* $%* ,-K

-)6!(#"I-*1-':*#)(;;*1(;8-#2*3-*(#4-,*(9%8$*$!-*1(;8-

%>*$!-*'"1-'*)-$(6!%'D*(+,*8#-'#*'($-,*"$*!"&!;:*(#*.-;;2

/!-:* %9#-'1-,* $!($* $!-* 5%++-5$-,+-##* %>* $!-* '"1-'

!-;6-,* $!-)*>%;;%.*(* $'-+,*)%'-*-(#";:*%1-'* $")-* $!(+

"+*$!-*!"#$%&'()L*$!"#*'-#8;$* "#*5%)6($"9;-*."$!*$!-*6-'K

5-6$"%+*6'"+5"6;-#*,-#5'"9-,*9:*3('-*NOPQ2

R#-'#* ;"4-,*#%)-*>-($8'-#*%>* $!-*!"#$%&'()*(+,* '-5K

%))-+,-,*(,,"+&*$!-)*$%*/!-)-0"1-'2*S+-*#85!*>-(K

$8'-* "#* $!-* (9";"$:* $%* #--* +8)-'"5* 1(;8-#* $!($* ,'"1-* $!-

!"#$%&'()*(+,*'"1-'*58''-+$#2*S+-*8#-'*-=6'-##-,*)%'-

$'8#$*"+*$!-*!"#$%&'()D*9-5(8#-*#!-*T4+-.U*$!($*$!-*9('#

.-'-*-=(5$;:*$!-*,($(*1(;8-#D*.!-'-(#*#!-*.(#*+%$*#8'-

-=(5$;:*.!($* $!-*,($(*1(;8-#*.-'-* "+*/!-)-0"1-'2*G-'

6%"+$*"#*(*1(;",*%+-D*-#6-5"(;;:*9-5(8#-*$!-*58'1-,*;"+-#

!"#$%&'()'*"+&',-&.&/"0&%,1'"2'!"#$%&'34'5-"6'-"657#%8.'$6&6'5-&'9865%7':7;;&:5"72'<858'82<

<&=":56':-82#&6'"2'5-&.85":':725&25'70&%'5".&>

Proceedings of the IEEE Symposium on Information Visualization 2000 (InfoVis'00)0-7695-0804-9/00 $10.00 @ 2000 IEEE

ThemeRiverS. Havre et al., 2002

ThemeRiverS. Havre et al., 2002

!"#$%&'&()*&+# ,!# +&-.)+&# /%0/#1&# )2/&+3!40/&# 5&/1&&2

,0/0# 3!)2/6# /!# 3+!,.7&# /%&# 7.+*&68#9&# %0*&# 0,,&,# /%&

70305)4)/:# "!+# .6&+6# /!# 6&&# /%&# &;07/# ,0/0# 3!)2/6# !2

,&'02,8<4/%!.=%# .6&+6# 4)>&,# /%&# 056/+07/)!2# /!# /%&# 1%!4&

7!44&7/)!2# 02,# /%.6# 010:# "+!'# )2,)*),.04# ,!7.'&2/6?5!/%# .6&+6# 6.==&6/&,# 0,,)2=# "&0/.+&6# /!# 077&66# ,!7.@'&2/6#)"#,&6)+&,8#$%&:#102/&,#/%&#05)4)/:#/!#6&&#/%&#/!/042.'5&+#!"#,!7.'&2/6#,.+)2=#02:#/)'&#3&+)!,#02,#/!#=&//%&# /&;/#!"#&07%#,!7.'&2/#!2#,&'02,8#$%&:#102/&,# /!6&4&7/#0#7.++&2/#02,#6&&#/%&#,!7.'&2/6# /%0/#7!2/+)5./&,/!#)/8

A6&+6# 046!# 102/&,# /%&# 05)4)/:# /!# +&!+,&+# /%&# /%&'&

7.++&2/68#B3/)!26# /%&:# ,)67.66&,# )274.,&,# .6&+@,&")2&,

!+,&+)2=# 02,# !+,&+)2=# 5:# 7!++&40/)!2?# 6!# /%0/# /%&'&6

033&0+)2=#/!=&/%&+#)2#/%&#,!7.'&2/6#1!.4,#5&#2&0+5:#)2

/%&#+)*&+8

!"!#$%&'()%*+$,-($.-/(012&-3,(4&

C06&,# !2# .605)4)/:# &*04.0/)!2# +&6.4/6?# 1&# 0,,&,# 0

2.'5&+#!"#"&0/.+&6#/!#7!'5)2&#/%&#5&6/#!"#5!/%#/%&#+)*&+

'&/03%!+# 02,# %)6/!=+0'# 70305)4)/)&68# $%)6# 6&7/)!2# 3+&@

6&2/6# 0# 60'34&# .60=&# 67&20+)!?# )44.6/+0/)2=# /%&

70305)4)/)&6#!"#/%&#7.++&2/#*&+6)!28

9&# .6&,# $%&'&()*&+# /!# &;34!+&# /%&# DEEF# <66!7)@

0/&,#G+&66# H<GI#2&161)+&#,0/0# "+!'#/%&#$(JKL#,)6/+)@

5./)!2# ,)6>6?# 0# 6&/# !"# !*&+# DFF?FFF# ,!7.'&2/6# H6&&

M)=.+&#NI8#$!#&;34!+&#/%&#6&4&7/&,#/%&'&6#)2#/%)6#7!44&7@

/)!2?#0#.6&+#')=%/#5&=)2#1)/%#0#%)=%@4&*&4#6.+*&:#!"#/%&

*)6.04)O0/)!2#5:#3022)2=# 04!2=# /%&# 7!.+6&# !"# /%&# +)*&+8

$%&#.6&+#')=%/#4!!>#"!+#1),&+#7.++&2/6#/%0/#6)=204#%&0*:

.6&#!"#0#/!3)7?#6.7%#06#/%&#!2&#"!+#P50=%,0,Q#)2#M)=.+&

N8#K%02=&6# )2# /%&# 7!4!+# ,)6/+)5./)!2# !"# /%&# +)*&+# 6)=204

7%02=&6# )2# /%&'&68# 9&# 6&&# 6.7%# 0# 7%02=&# )2# <.=.6/

DEEF?#1%&2#/%&#P>.10)/Q#7.++&2/?#1%)7%#%0,#*02)6%&,#)2

40/&# R.4:?# 6.,,&24:# 033&0+6# 02,# +03),4:# 1),&268# $%&

.6&+#7!.4,#046!#4!!>#"!+#20++!1#7.++&2/6#)2#/%&#+)*&+#/%0/

6)=204#+&40/)*&4:#4)=%/#.6&#!"#30+/)7.40+#/%&'&68

S2#02#&0+4)&+#303&+?#T&/O4&+#&/#048#ULV#&;34!+&,#/%&#<G

,0/0# 6&/#1)/%# 0# *0+)&/:#!"# !.+# *)6.04# 0204:6)6# /!!46?# "!@

7.6)2=# !2# 40+=&# /%&'&# 7%02=&6# 6.++!.2,)2=# /%&# S+0-)

)2*06)!2# !"#W.10)/# !2#<.=.6/# X8# $%&'&()*&+# 046!# +&@

"4&7/6#/%&6&#40+=&#/%&'&#7%02=&68#Y&0+#/%&#+)=%/#6),&#!"

M)=.+&#N?#1&#6&&#6&*&+04#7.++&2/6#/%0/#&;302,#,+0'0/)7@

"#$%&'!()!*+!,-.-!/&01!2%34!5!*%$%6.!7889:!*!;#,'!<%&&'=.!#=!.>'!&#?'&!#=,#<-.'6!>'-?4!%6'!0/!-!.0@#<A

;>#3'!<>-=$'6!#=!<030&!,#6.&#B%.#0=!<0&&'3-.'!.0!<>-=$'6!#=!.>'1'6:

Proceedings of the IEEE Symposium on Information Visualization 2000 (InfoVis'00)0-7695-0804-9/00 $10.00 @ 2000 IEEE

history flowFernanda Viégas, Martin Wattenberg, 2003IBM, MIT

http://research.yahoo.com/taglines

Interestingness

# occurrences in interval

# occurrences in entire collection + C

compare TF-IDF

# of all documents# of documents containing termIDF := log( )

# of occurences of term

# of wordsTF :=

TF-IDF := TF * IDF

(term frequency * inverse document frequency)

The Dataset

86.8 M tag-photo instances1.26 M unique tags=> each tag used 70 times on average

data spans 472 days => 1.2 M incoming tags / week

Bulk Uploads

Notice that the aggregation function is given by the ac-tual decomposition: if I = (!J!J+

J) \ (!J!J!

J), thenInt(u, I) =

P

J!J+Int(u, J) "

P

J!J!

Int(u, J). Therefore,

the aggregation function f is no longer monotone and so wecannot use Algorithm TA.

However, notice that f is of a very special form: f =f+ " f", where f+, f" are monotone. Now we have two op-tions to find the k objects with the highest f scores. Thereare two algorithms based on the kind of access model thatis available. The first algorithm works when the only accessavailable is to objects sorted by their f+ and f" scores. Forinstance, this is the case if we were to use TA to find theobjects with highest/lowest f+ and f" scores. The second,more e!cient algorithm works if, in addition to sorted ac-cess, we can random access the f+ or f" score of an object;this is inspired by TA.

Threshold Algorithm (Non-monotone, sorted access).Access in parallel the objects ordered by f+ and those

ordered by -f". Continue until the same k objects appearin both lists. Output these k objects with top aggregatedscores.

Threshold Algorithm (Non-monotone, random access).Access in parallel the objects ordered by decreasing values

of f+ and those ordered by increasing values of f". As anobject is seen under sorted access in one of the lists, dorandom access to the other list to find its score. For f+, f",let x+, x" be the score of the last object seen under sortedaccess. Define the threshold value ! = x+ " x". As soonas at least k objects have been seen whose aggregate scoreis at least ! , stop. Finally, output the k objects with topaggregated scores.

It is easy to show:

Proposition 3. For the aggregation function f = f+ "f", the above algorithms correctly find the top k answers.

4.5 Computation of Sliding IntervalsIt is also possible to obtain incremental versions of both

the additive and subtractive algorithms. These algorithmstake advantage of the fact that most of the time, the optimalrepresentation for an interval [t, t+w] is related to the opti-mal representation for the next timestamp [t + 1, t + 1 + w].We omit the details of this algorithm in this version, andsimply state the following proposition:

Proposition 4. For a sequence of intervals Ii = [a +i, a + w + i] for i = 0, . . . , m, with m # w, let Ji repre-sent the optimal collection of intervals to cover Ii. ThenEi[|Ji"Ji+1|] $ c for some constant c.

That is, as we slide an interval, the additive and sub-tractive algorithms produce a cover that changes on averageby only a small (constant) number of members. Thus, ifwe count the number of pre-computed intervals that mustbe accessed to compute the most interesting objects at aparticular query interval o#set, the amortized cost of thoseaccesses is small for each additional slide of the query inter-val.

5. EXPERIMENTS

5.1 DatasetThe data was obtained from Flickr (flickr.com), a Ya-

hoo! property. Flickr is an extremely popular online photo-sharing site. Users can post photos on this site and annotatethem with tags. Flickr also permits users to tag photos cre-ated by others.

The snapshot of Flickr data we analyzed had the followingcharacteristics. The data consists of a table with the follow-ing columns: date, photo id, tagger id, and tag. There were86.8M rows in this table, where each row represents an in-stance of a photo being tagged. Of these tags, around 1.26Mwere unique; thus each tag is repeated, on average, 70 times.The data spanned 472 days, starting June 3, 2004; thus, onaverage, more than 1.2M tags arrive per week.

First, we performed some simple data cleaning operationson this table. We extracted the raw tags and removed whitespace, punctuation, and converted them to lower case. Thisperforms some simple normalization. We did not attemptto stem or spell-correct the tags since many of the tags wereproper nouns (e.g., Katrina) or abbreviations (e.g., DILO,which means ‘Day In the Life Of’).

Figure 6: Histogram of unique tag-user-date combi-nations, suggesting bulk uploads.

We used the ‘user id’ field to ensure that on each day,a tag is counted only once per user, i.e., for a given useron a given day, we do not allow the user to place the sametag on di#erent photos. This is to discount bulk uploadsof many photos with same tag that might make these tagsappear interesting for the wrong reason; in fact, as Figure 6shows, this is a very common occurrence in Flickr. In otherwords, we implicitly want to capture the “interestingness”from a social perspective, and not an individual perspective.A global analysis of tag occurrences reveals the most com-monly occurring tags are: 2005, 2004, and flowers. Thesetags, while very common, are unlikely to be particularly in-teresting for any reasonable interval in time. The ‘idf’ termin our definition of interestingness guards against this: tagsdon’t become interesting by virtue of occurring too oftenglobally. As we mentioned earlier, the regularization pa-rameter C in Equation 1 ensures that tags occur su!cientlymany times globally before they can be considered interest-ing; we set C = 50. We set k, the number of tags to displayper interval, to be 8. We also compute their interestingnessscore for use in the visualization.

Interesting Tags

EventsValentine’s Day, Thanksgiving, Xmas, Hanukkah, Republican National ConventionWeb_2.0, Lunar eclipse, NYC Marathon, GoogleFest ...

PersonalitiesJeanne Claude, Pope ...

Social media taggingWhat’s in your fridge?, Faces_in_holes, dilodec04, What’s in your bag?, Badge, Do your worst ...

Discussion

Is the “interestingness” measureappropriate?

Do the results make sense to you?

What are the strengths and weaknessesof the visualization?