(CC BY 2.0) Rachel Elaine, flic.kr/p/fw8HSn

Describing & Understanding Neighborhood Characteristics through Online Social MediaMohamed Kafsi Henriette CramerBart Thomee, David A. Shamma

1

CC Steve, flic.kr/p/5tm2Ci

2

What’s unique about Florence? What’s similar to other cities?

CC Pedro, flic.kr/p/85QqvB

CC Mitch Altman, flic.kr/p/ffHDeY

Lisboa

Paris

in Florence?

Which neighborhoods have characteristic street art?

masses of geotagged, user community-generated content

User-generated content *finding, describing, defining locales & boundaries (e.g. ZoneTag, TagMaps ‘07, Livehoods'12, Thomee & Rae’13 etc.)

CC Momo, flic.kr/p/6bKMtX

But #1: locally frequent is not always specifically descriptive blackandwhite, gotham, tribeca

Brett Weinstein, flic.kr/p/RHWan

6

Bert Kaufmann, flic.kr/p/dEfd12

#2 regions within regions Is ‘desert’ a descriptor of Las Vegas?or rather the surrounding area?

7

Problem: let’s describe neighborhoods with flickr content go beyond local, compare neighborhoods

focus on user-defined noisy tags -ignore the pixels-

build model to distinguish ‘specifically descriptive’ local content work with the geo-hierarchies that people are familiar with compare with human reasoning (interviews & survey)

Goal: -Find specific descriptions of pre-defined regions -Quantify their uniqueness -Map similar regions

Idea: -Define any geographical hierarchy of regions -Quantify the descriptiveness of tags with respect to a given geographical level

Geographical hierarchy of tags

9

Country

Cou

ntry

City A City B City C

Citi

es

Hood B.1

Hood B.2

Hoo

d

10

USA

Cou

ntry

Chicago

SF New York

Citi

es

Mission SOMA

Hoo

d

11

Country

Cou

ntry

City A City B City C

Citi

es

Hood B.1

NeighborhoodB.2H

ood

Randomly sample tags from the nodes along the path from leaf to root

12

+

Distribution of tags in neighborhood n

θcountry(n)

πcountry(n)

θn

πn

θcity(n)

πcity(n)

13

14

(cc by 2.0) S J Pinkney, flic.kr/p/8cNAgd (CC BY 2.0) !STORAX, flic.kr/p/39Wstq

SF & Manhattan

- Sample of 8M geo-tagged photos - 20M tags, - vocabulary of 8000 unique tags

Probability of tag t in neighborhood n

p(t|n) =X

v2Rn

✓v(t) p(z = d(v) | n)

depth of node v

path from the leaf n to the root of the geo-tree multinomial distribution associated with node v

15

Training

- Expectation-Maximization to learn the model’s parameters

- Fast convergence

- Scales. Worst case running time O(N V D)

16

Results: assigning tags to a level

17

Mission, SF

California Mission SF USA Graffiti Art Mural Valencia Food Car

David McSpadden, flic.kr/p/oVLorr

Most frequent tags

18

Country: 0.06 City: 0.33Neighborhood: 0.61

California Mission SF USA Graffiti Art Mural Valencia Food Car

Mission, SFMost frequent tags

19

Country: 0.06 City: 0.33Neighborhood: 0.61

David McSpadden, flic.kr/p/oVLorr

New York Manhattan USA Midtown Skyscraper Time square Light Moma Broadway Rockfeller

Midtown South, Manhattan

CC Jeffrey Zeldman, flic.kr/p/s1eE5W

Most frequent tags

20

Country: 0.21City: 0.32Neighborhood: 0.47

New York Manhattan USA Midtown Skyscraper Time square Light Moma Broadway Rockfeller

Midtown South, ManhattanMost frequent tags

21

Country: 0.21City: 0.32Neighborhood: 0.47

CC Jeffrey Zeldman, flic.kr/p/s1eE5W

1. Where do I find tag t? (i.e. where is tag t most locally descriptive)

22

Food

23

Hipster

Mapping Tags in San Francisco

2. where are the unique ‘hoods?quantifying uniqueness

24

25

Uniqueness

3. What’s the east-village of SF? mapping & comparing neighborhoods

26

San Francisco Manhattan Top common tags

Mission East Village (0.23) graffiti, food, restaurant, mural, bar

Golden gate park Washington heights (0.26)Upper west side (0.22)

park, museum, nature, flower, bird

Financial district Battery park (0.29) Midtown Manhattan (0.27)

downtown, building, skyscraper, city, street

Chinatown Chinatown (0.85)Chinatown, Chinese, downtown, dragons, lantern

Castro West village (0.06) park, gay, halloween, pride, bar

sim(n, n0) =

PVt=1 ✓n(t) ✓n0(t)qPV

t=1 ✓n(t)2

qPVt=1 ✓n0(t)2

27

Does it match human reasoning?

presentation -or-

model adaptation

locals’ reasoning in classifying tags’ local specificity

10 interviews

Survey: 22 human classifiers

3 neighborhoods: classify 32 tags 1291 tag classifications

########

########

###### s

an-franc

isco, Ca

stro/Upp

er Marke

t ######

########

########

sfmoma

sf

embarc

adero

baseb

all

san

mission

market

tram

rainbow

muni

pride

city

sanfranc

isco

gay

califo

rnia

alcat

raz

streetca

r

francis

co

male

dolor

espark

castro

soma

church

sign

street

theater

flag

usa

dolores

night

movie

hallo

ween

########

########

########

#### san

-francis

co, Mari

na #####

########

########

########

boat

sanfran

cisco

flower

s

bird

water

califor

nia

usa

night

francisc

o

embarca

dero

street

bridg

e

alcatraz

presidi

o

pond

explo

ratorium

sf

columns

mason

fort

gate

swan

golden

bay

city

sign

san

finan

cial

sailing

palace

marina

sfmom

a

roomfars

outh-lm:

scripts

kafsi$

Principle 1. While users’ tagging is personal, varied, local;

(e.g. Naaman et al., 07)

- the community generates similar meta-content

(e.g. Rost et al., 2013)

2. Personal experiences shape classifications

this slides features 3 different churches.

CC Justin Pickard, flic.kr/p/6hWpoa

CC torbakhopper, flic.kr/p/9PYSjx

(CC BY 2.0) Dustin Gaffke, flic.kr/p/aCvuxP

No human ground truth.

SF: North Beach?

P1: ’party land!’

P2: ‘I don’t believe there’s much of a nightlife there’

3. Teaching opportunity? ‘mistakes’ or perspectives?

32

Proposed geo-model extensions (i.e. humans aren’t wrong)

1. even when using existing regions, detecting sub-regions is important

this is (not) North Beach

(CC BY 2.0) David Ohmer, flic.kr/p/2xT9rU

Most ‘rejected’ by our human classifiers:

Fisherman’s Wharf in North Beach in our official neighborhood dataset… but it’s its own thing.

San Francisco neighborhood map (1960) From The Urban Aesthetic: Evolution of a Survey System.

(CC BY 2.0) Eric Fischer, flic.kr/p/dTxhBg

Night, The Mission:

“before…you wouldn’t be caught there at night-time… 20 years ago it was a different neighborhood.”

2. boundaries, character, data change over time

(CC BY 2.0) Σταύρος, flic.kr/p/6ZZX6A

3. Topography & closeness matters: make neighborhood boundaries permeable

Geographic Hierarchical model with Adjacency, extension.

(CC BY 2.0) Rachel Elaine, flic.kr/p/fw8HSn

Mohamed Kafsi - mohamed.kafsi@epfl.ch

Henriette Cramer@hsmcramer henriette@yahoo-inc.com

Bart Thomeebthomee@yahoo-inc.com

David A. Shammashamma@yahoo-inc.com

SUMMARIZING: Geographical Hierarchy Model - resilient & scalable probabilistic model (with adjacency extension) - applied on sample Flickr photos SF & NY - we can describe & compare neighborhoods, quantify uniqueness & similarity - mind the individual human interpretation gap: we present an aggregate view based on community-generated content.

37