What's in a place? Adventures with Location-Aware Media

1yahooresearchberkeley.comRahul Nair

What’s in a place?Adventures with Location-Aware Media

Rahul Nair

Yahoo! Research Berkeley


Flickr “geotagged”

20+ million images

Can we do better?


Talk Outline

• Extracting information from geo-tagged photos

• World Explorer: Visualization

• ZoneTag: Creation

• Zurfer: Consumption


Attraction Map of Paris

Stanley Milgram, 1976. Psychological Maps of Paris


Attraction Map of Paris

Y!RB, 2007.


Location-driven Modeling

• Derive meaningful data about map regions

• E.g., representative tags, photos


Data Description


Issues

• Sparse data set

• Photographer bias

– In location

– In tags

• Incorrect data


Heuristics

• Number of photographs denotes the “importance” of a location

• Users will use a common subset of tags to describe objects/locations

• Concentrated tag usage indicates descriptiveness


Algorithm

• Clustering: k-Means, get set of k clusters

• “Document” C is bag of all tags in cluster

• For each tag in C calculate:

– TF = |P(C,t)|

– IDF = |P(R)| / |P(R, t)|

– UF = |U(C,t)|/|U(C)|


Scoring

• Score (t) = TF * IDF * UF

• Threshold values

– 30+ photographs

– Minimum 3 users

– Score > 1

• Final dataset: (tag, score, latitude, longitude)


Talk Outline

• Mining information from geo-tagged photos




13

DEMO


Precomputation

• Divide the world into equal sized non-overlapping tiles

• Compute and store the tags for each tile

• Repeat for different zoom

levels


Retrieval

• Find the tile level closest in size to the request area

• Select the tiles that fully cover the request area

• Return the tags that fall within the request area


User Study

10 subjects

• 6 female, 4 male

• Ages 20-60

• Varying technical knowledge

• No geotagged photos of their own


Experiment tasks

• Vacation recap

• San Francisco tour

• Explore a new city


Recall

Reminded the subject about locations

• “It brings out memories”

• “Oh my God! This place has the best restaurants”

• “We wanted to see the Polynesian Cultural Center"


Discovery

Participants discovered previously unknown locations and events

– “I’ve never heard of this festival”

– “There is car racing which I'd probably go see”


Needle & Haystack

• Excellent visualization of the Haystack

• Hard to find specific information

– “Where was Culver City again?”

• No way to search

– “I guess what I’m looking for are bull fighting pictures”


Other Responses

• Gets the “vibe” of a place

• Share with other people

• Tags did not always match the mental model of a location

• Wanted more tags

• Want more info about tags


Conclusions

• Extract meaningful aggregate information from georeferenced data

• Allows users to explore locations in a new way

• Users like using the overview but also want the ability to search


Future work

• Adding search capability

• Show photos in places with no tags

• Differentiate locations and events

• Apply to other types of georeferenced data


Talk Outline






Current Mobile Experience

• Difficult to share (or even save!)

• Hard to find

– No context

– No semantic information

Current mobile experience?


Current Mobile Experience

• Difficult to share (or even save!)

• Hard to find

– No context

– No semantic information


ZoneTag Experience

• 2-click upload (same key!)

• Photo uploaded with location and time metadata


Where does location come from?

• Bluetooth GPS (when available)

• User-contributed cell tower mapping


ZoneTag Experience

• Tagging made easy

– Tag/annotate your photos from the phone


Where do tags come from?

• Tags I used in this context (`home’)

• Tags my friends used in this context

• Tags other people used in this context (‘Ricoh’, ‘California Research Center’)

– E.g., TagMaps data


Where do tags come from?

• Stuff around you:

– Yahoo! Local

– Upcoming.org

• Stuff from you (any RSS 2.0 feed):

– Calendar

– Favorite hangouts (Wayfaring, Plazes, Socialight)


Example of Tagging


Talk Outline






Flickr “geotagged”

20+ million images

Can we do mobile?


Strengths of Mobile

• Personal

• Easy to access

• Networked

• Context aware

The ultimate photo wallet


Design Goals

• Engagement and Discovery

– Spatial

– Social

• Allow customization

• Complete access to your photos

• Search and Filtering capability


Channel Metaphor

• Each row is a single channel

• Navigate using 4 way

– Left & right to browse channel (infinite scroll)

– Up & down to change channels


Detail view

• Enlarge photo

– Scroll through channel

• Photo details

• Add comments

• Add to favourites


Nearby photos channel

• Photos from the users current location

• Local Highlights

• My Nearby


Social Channels

• Contacts photos

– Expanded view

• Recent Activity


My Stuff Channel

• My Photos

– By Location

– By Tag

• My Photo Wallet

• My Favourites


Custom Channels

• Users can create channels to match their interests

– Tags

– Groups

– Location aware (dogs near me)


Search and Filter


Try it out

http://zurfer.research.yahoo.com


Conclusions

• It is possible to extract information from georeferenced media

• Users like browsing the extracted data

• It can help users tag new media

• We hope it helps them browse on a mobile device

47

Questions?

Rahul [email protected]

http://tagmaps.research.yahoo.comhttp://zonetag.research.yahoo.com http://zurfer.research.yahoo.com

http://tagmaps.research.yahoo.com/

http://zonetag.research.yahoo.com/

http://zurfer.research.yahoo.com/

Technology

What's in a place? Adventures with Location-Aware Media