Embed Size (px)
Citation preview
evolution
lexica
lear
ningword
NLP
model
node
network
syntax
POS
@
complex
semanti
edge
bangla
PA
DD
zulu
Social Computing for Linguistics & Linguistics for Social Computing
Monojit ChoudhuryMicrosoft Research [email protected]
evolution
lexica
learni
ngword
NLP
model
node
networ
k
syntax
POS
@
complex
semanti
edge
bangla
PA
DD
zulu
Language is shaped by social interaction patterns;emails, blogs, sms, queries – CMC’sLinguistics for SoC
SoC for LinguisticsSoC platforms for linguistic experiments; SoC Models
for linguistic theories
evolution
lexica
learni
ngword
NLP
model
node
networ
k
syntax
POS
@
complex
semanti
edge
bangla
PA
DD
zulu
Language is shaped by social interaction patterns;emails, blogs, sms, queries – CMC’sLinguistics for SoC
SoC for LinguisticsSoC platforms for linguistic experiments; SoC Models
for linguistic theories
Web Search Queries as an Emerging
Language
evolution
lexica
learni
ngword
NLP
model
node
networ
k
syntax
POS
@
complex
semanti
edge
bangla
PA
DD
zulu
Language is shaped by social interaction patterns;emails, blogs, sms, queries – CMC’sLinguistics for SoC
SoC for LinguisticsSoC platforms for linguistic experiments; SoC Models
for linguistic theories
2000 2006 20100
1
2
3
4
5
2.21
3.54
Mean length of queries
Web Search Queries as an Emerging
Language
evolution
lexica
learni
ngword
NLP
model
node
networ
k
syntax
POS
@
complex
semanti
edge
bangla
PA
DD
zulu
Language is shaped by social interaction patterns;emails, blogs, sms, queries – CMC’sLinguistics for SoC
SoC for LinguisticsSoC platforms for linguistic experiments; SoC Models
for linguistic theories
2000 2006 20100
1
2
3
4
5
2.21
3.54
Mean length of queries
Web Search Queries as an Emerging
Language
6
evolution
lexica
learni
ngword
NLP
model
node
networ
k
syntax
POS
@
complex
semanti
edge
bangla
PA
DD
zulu
Language is shaped by social interaction patterns;emails, blogs, sms, queries – CMC’sLinguistics for SoC
SoC for LinguisticsSoC platforms for linguistic experiments; SoC Models
for linguistic theories
Web Search Queries as an Emerging
Language
Unsupervised segmentation
Distributional PatternsTaxonomy of Function units
Word co-occurrence NetworkQueries are difficult to interpret
User Behavior Studies
Basic units
Search expertise language acquisition
7
evolution
lexica
learni
ngword
NLP
model
node
networ
k
syntax
POS
@
complex
semanti
edge
bangla
PA
DD
zulu
SoC for LinguisticsSoC platforms for linguistic experiments; SoC Models
for linguistic theories
Web Search Queries as an Emerging
Language
Web Search Queries are an evolving Protolanguage
Collaborators: Rishiraj Saha Roy & Niloy Ganguly (IIT Kharagpur), Srivatsan Laxman & Kalika Bali (MSR India)
8
evolution
lexica
learni
ngword
NLP
model
node
networ
k
syntax
POS
@
complex
semanti
edge
bangla
PA
DD
zulu
Language is shaped by social interaction patterns;emails, blogs, sms, queries – CMC’sLinguistics for SoC
SoC for LinguisticsSoC platforms for linguistic experiments; SoC Models
for linguistic theories
Use of Indian languages on online
social media
Code mixing
Transliteration
Spelling Change
Indian English
Collaborators: Kalika Bali & Nimmi Rangaswamy (MSR India)
9
evolution
lexica
learni
ngword
NLP
model
node
networ
k
syntax
POS
@
complex
semanti
edge
bangla
PA
DD
zulu
Language is shaped by social interaction patterns;emails, blogs, sms, queries – CMC’sLinguistics for SoC
SoC for LinguisticsSoC platforms for linguistic experiments; SoC Models
for linguistic theories
Sociolinguistic ExperimentsOSN Games with a
Purpose
CodeGambler is an OSN Game to study the population-level emergence of categorization of a continuous space (colors) into discrete category terms (color names)
Collaborators: Animesh Mukherjee (IIT Kgp), Vittorio Loreto, (University of Rome)
10
evolution
lexica
learni
ngword
NLP
model
node
networ
k
syntax
POS
@
complex
semanti
edge
bangla
PA
DD
zulu
Language is shaped by social interaction patterns;emails, blogs, sms, queries – CMC’sLinguistics for SoC
SoC for LinguisticsSoC platforms for linguistic experiments; SoC Models
for linguistic theories
Sociolinguistic ExperimentsCrowdsourcing for
data
Flat segmentation (queries and sentences)
Nested segmentation
11
evolution
lexica
learni
ngword
NLP
model
node
networ
k
syntax
POS
@
complex
semanti
edge
bangla
PA
DD
zulu
Language is shaped by social interaction patterns;emails, blogs, sms, queries – CMC’sLinguistics for SoC
SoC for LinguisticsSoC platforms for linguistic experiments; SoC Models
for linguistic theories
SoC inspired Linguistic TheoryWord Co-occurrence
Network
Small world
word
language
in
human
treat
as
is
can
evolving
neighbori
ng
distinct
interacting
web
sentences
such
structur
e
acomplex
network
Two-regime power law degree distribution
Kernel-Periphery structure
Low rank
Collaborators: Animesh, Niloy (IIT Kgp), Chris Biemann (University of Darmstadt), Ravi Kannan (MSR India)
12
evolution
lexica
learni
ngword
NLP
model
node
networ
k
syntax
POS
@
complex
semanti
edge
bangla
PA
DD
zulu
Language is shaped by social interaction patterns;emails, blogs, sms, queries – CMC’sLinguistics for SoC
SoC inspired Linguistic TheoryWord Co-occurrence
NetworkCollaborators: Animesh, Niloy (IIT Kgp), Chris Biemann (University of Darmstadt), Ravi Kannan (MSR India)
WCN of Bollywood Lyrics is a very small world with a tiny kernel.
Learn 1000 Hindi words and you will understand every Bollywood song!
13
evolution
lexica
learni
ngword
NLP
model
node
networ
k
syntax
POS
@
complex
semanti
edge
bangla
PA
DD
zulu
Language is shaped by social interaction patterns;emails, blogs, sms, queries – CMC’sLinguistics for SoC
SoC inspired Linguistic TheoryWord Co-occurrence
NetworkCollaborators: Animesh, Niloy (IIT Kgp), Chris Biemann (University of Darmstadt), Ravi Kannan (MSR India)
WCN of Web search queries is not (yet) small world!
It has a tiny kernel but very large periphery
Thank You!http://research.microsoft.com/people/monojitc/
evolution
lexica
lear
ningword
NLP
model
node
network
syntax
POS
@
complex
semanti
edge
bangla
PA
DD
zulu
