Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
A text-based model of foreign-affairs sentiment
Sean Gerrish and David BleiPrinceton University Computer Science
17 December 2011
These news articles tell a story.
A spatial model of foreign relations sentiment
This work develops a model of the sentimentbetween countries over time.
• It models dynamic relationships in aninterpretable way
• It infers sentiment from printed media
• Sentiment is defined by Mechanical Turkers
A spatial model of foreign relations sentiment
To do this, our plan is to:
• Collect a bunch of newspaper articles
• Define a latent variable model to captureinteresting structure in these articles
• Perform posterior inference to estimate thevalue of these random variables
Countries take latent positions x̄ct over time
Time
x
0
-2-1
1 2
x̄c ,t|x̄c ,t−1 ∼ N(x̄c ,t−1, σ2K)
The relationship between countries is observed in the news.
Time
x
0
-2-1
1 2
xc1,d ∼ N(x̄c1,t, σ2D)
xc2,d ∼ N(x̄c2,t, σ2D)
Sentiment sd := xc1,dTxc2,d
The relationship between countries is observed in the news.
Time
x
0
-2-1
1 2
xc1,d ∼ N(x̄c1,t, σ2D)
xc2,d ∼ N(x̄c2,t, σ2D)
Sentiment sd := xc1,dTxc2,d
The relationship between countries over time
xc,t-1 xc,t xc,t+1
xc,t-1 xc,t xc,t+1
C
Dt-1 t t+1
w
sss
ww
zerozero zero
2 2 2
DD
β
Latent position
Position duringinteraction
Sentiment
Observed text
Regularization
Labeling sentiment
1. We found all pairs of paragraphs from the New York Timeswhich discussed exactly two countries
2. A random sample of 3607 paragraphs from New York Timesarticles from 1988 to 2008 were labeled by AmazonMechanical Turk workers
3. Raters rated news articles on the scale −5,−3,−1, 1, 3, 5
Labeling sentiment: typical task
Sentiment and news articles: text regression
sd = wTd β + ε
• wd ∈ RV is the text of a news paragraph
• sd ∈ R is the sentiment between two countries
• β ∈ RV is the “weight” of each word
Sentiment and news articles: text parameter β
towns
looking
eligible
electricity
increasing
specialist
hero
reporterreported
elaborate
reports
military
criticism
divide
explained
replace
cocaine
unit
passport
successful
hurt
midst
hold
locked
pursue
plunged
revenues
owesorganized
vehemently
currency
absolute
complaining
travel
machine
effective
headquarters
rescued
easier
represents
burial
financial
series
laboratory
foundation
threatened
estimate
enormous
ministries
speedy
channels
service
engagement
euro
bitter
ranging
collapse
legally
peril
nations
projectwillingness
dozen
affairs
person
nonproliferation
mouth
letter
organization
drought
professor
fugitives
bomb
reactor
lessons
altogether
societies
release
respond
blew
fair
result
preserve
extend
extentdebt
accident
country
conclusions
demanded
planned
argue
union
life
bans
eastern
commerce
appeals
flooding
played
trusted
rebellion democracies
split
fairly
echoed
academic
ease
easy
prison
possibly
steps
people
captive
brigades
losing
shaken
dollars
citizens
facility
son
raises
reducing
occupyingauthorized
launchingoffer
delegation
inside
devices
adopt
proved
proven
accounting
dealer
negotiationsflood
republic
ambitious
intend
time
chain
veterans
cheap
choice
minute
tear
unify
leave
settle
team
sign
falling
drafted
funeralunderstanding
address
passengers
reclaim
positive
angry
scope
afford
apparent
refrain
believed
gunfire
allowed
monitoring
winter
divided
spot
natural
pulled
insurgency
course
decades
civilian
maintaining
shouted
didn
establishingquarter
square
peacekeepers
breakthrough
entering investigation
formula
superiors
siege
million
possibility
complicated
training
spoken
vote
city
meddling
favors
future
argument
noteroadside
handing
printed
drivelaws
walking
aggressive
dispute
guarded
freezing
prime
worried
bombs
surgery
ostensibly
representatives
sponsorship
jobs
jets
article
considers
stretch
airlines
thousand
tribal
policies
newspaper
situation
engagedtechnology
missiles
summer
overthrow
sums
traffic
vacuum
worldexit
restoring
contingent
power
leadership
stone
package
industryneighbor
mean
burning
image
technically
punished
completesurvived
politically
citizenship
wartime
graduate
tight
external
causedcondition
cable
joined
penalties
past
section
scientists
contrast
hours
civilians
concluding
experience
social
initiated
company
tested
foundations
resourceslearn
male
accept
states
sense
dress
imminent
dismissed
plant
plans
plane
director
fundamental
trade
signs
roots
rapidly
followers
entire
found
investigators
week
reduce
operation
research
plotting
risen
retained
anonymity
heads
jet
threatening
checkpoint
tough
self
internal
play
plan
singled
seize
sometimes
coverresearchers
golf
defended
occupation
writer
failed
banned
interviews
preparing
enemy
surrendering
kill
approaching
set administrator
migration
muted
available
targeted
incident
nonexistent legislature
prospects
allegiancemonthly
acceptance
convicted
champion
fire
representative
consulate
expanded
technical
pressed
owed
binding
renounced
read
virtually
levels
recent
arrests
eager
victimsdemands
couple
projects
imposedconsensus
communications
continue
tribes
names
shipments
spring
sight
transactions
religion
agreement
majorspending
crackdown
considerably
elsewhere
link
line
considerablecamp
nationalist
influence
nationalism
elements
energetic
prosecution
ago
land
fighter
age
summit
confederation
resultsexisting
longstanding
send
citing
reopened
continued
eve
fewer
try
severed
imply
assembly
expressed
led
gathered
poverty
receive
involved
defeat
opinion
makes
involves
standing
confidence
dealings
carrying
baby
retreated
weaker
process
purposes
weaken
delay
halted
ties
allow
produces
move
chosen
hijackersoutlined
accounts
doing
society
agriculture
fundamentally
evacuation
shut
mainlandassumed
enforcement
elite
colleagues
listeners
false
tonight
viewed
studying
visa
atmosphere
bringeconomistdecade
principal
bombings
hope
means
familiar exchanges
following
taxes
temporarily subcommittee
nuclear
onetime
restrictions
invited
drugconclude
picking
willing
waste
rioting
delaying site
terrorism
conflicting
ball
command
credibility
drawing
arrest
increased
government
five
replacement
indications
separate
demonstrators
accompanied
loans
stage
holdings
concession
revise
allianceletters
roads
airline
frustration
building
delivery
official
smooth
convince
distribution
recognize
slowing
details
chance
rule
write
rural
doubts
votes
invaders
worth
announced
established
told
protection
obtained
stressed
uniforms
secure
servicemen
total
word
worn
deprived
indicated
cited
campaigning
totaled
carriersdisappear
interviewordinary
lay
law
green
south
office
bombing
production
scene
break
rocky
reasonable
lifted
crimes
networkvessels
barrier
veto
disputes
created
renew
oppose
system
electronic
rejects
target
powers
forced
forces
involving
sun
respondedanniversary
yen
exchanged
secretary
observer
discussions
cooperation
accordterms
received
ill
receives
requests
tons
speak
mobilization
condemned
dozens
families
separatist
confirmed
incorrectly
independent
raidrail
hand
documents
blamed
kept
stringent
adding
belongs
night
security
born
boredenies
denied
confusion
participation
post
months
prisoners
deception
fight
lowest
peninsula
maximum
promises
evidence
promised
incursion
apartment
stake
reality
holding
test
brothers
welcome
reception
uprisings
diplomacy
global
battle
devoted
killingszone
compensate
trouble blast
fragile
gun
regret
cost
appeal
inauguration
appears
change
regardlessmarket
prove
territories
towers
organizations
carincidents
heart
rebuffed
recalled
occur
economy
southern
produce
commit
lifting
embassies
serving
indeedaiding
ratified
cold
halt
wealthy
discuss
drop
entirely
fired
happen
calling
space
increase
receiving
rebuilding
cars
marine
card
tries support
sprawling
flying
impossible
message
size
silent
atrocities
caught
national
friend
televisiontangible
contentious
remained
overnight
equipmenttourist
begin
deterioration
price
rationale
slavesprofessional
typically
discovered
excerpts
ground
title
developed
scientist
electedconcern
import
justified
blame
comes
nearby
rebuild
suspected
commander
media
inquiry
figures closestcomplained
charges
foot
speed
struck momentum
real
laundering
ruledheightened
pounds
passions
deficit
business
sixth
equivalent
strained
unease
negotiated
comparison
central
supervised greatly
testimonyrolling
involvement
intervention
prepare
start
low
lot
delayed
trying
moved
sales
moves
intercontinental
forcing
pool
surrounded
month
unrest
deadline
clerics
study
fence
streets
witnessed
learned
strong
ahead
inspired
soldierbase
family
aimed
trained
gunmen
excusebroke
coalition
nine historyboycottpushed
reject
tried
terrorist
invasion
banks
dream
help
soon
actually systems
foodsweeping
traveled
imprisoned
stoppeddie
referred
heavy
inspections
event
safety
favored
heroin
launch
liaison
scheduled
reflected
station
hundred
sentenced
alongside
approved
unreasonable
northern
assertion
flights
circle
ouster
engineers
close
probably
conditions
missing
abruptly
collaborators
sensitive
forgotten
headed
contras
vessel
described
stamp
dams
exert
maintenance
airstrikescollected
threaten
territory
empty
lives
pact
look
pace
endanger
voters
rulers
ready
conflict
anxious
obviously
ambassador
settlements
reviewedwounding
cemetery
march
outright
provoking
popular
parade
antiaircraft
fathers
urgent
economic
deliveredrun
stem
abroad
troopsrivalries
collapsed
properly
doesn politicianselectrical
recognition
reportedly
visitsregulations
approving
requiredorbit
depth
airliners
excluded
friendly
positions
enforce
commercial
radicals
kidnapped
collaborating
addressed
buildup
remains formally
started
missile
appointed
crosses
announcing
sanctions
crossed
meet
links
arrangement
farm
peace
gunman
mentioned
helicopters
guerrilla
hands
front
universitymagnitude
crossing
globe
constitute
variety
special
activist
cause
alleged
completely
resumed
hostile
route
times
counterpart
management
finalexactly
ultimatum
providing
border
pursued
instance
tactic
intentions
expiredgesture
businessman
emerging
sent
soldiers
based
earned
bases
employed
joint
procedures
tobacco
accused
written
tens
importance
strains
diplomats
addition
immense
controlled
releasing
guerrillas
smoking
surface
finance
capture
party
effect
frequently
indigenous
destruction
trials
restore
sources
fate
propelled
historic
immediately
loss
lost
payments
lose
refugees
home
broad
reaching
monetaryexpansion
celebrations
previously additional
inner
north
evacuate
blaming
demilitarized
signedlimit
urging
reservationsoffensive
deter
friends
persistent
portion
aided
protest
seriously
misunderstanding
toppling
vast
enhance
clothes
force
prisons
spokesmandeemed
met
governing
detonated
economics
credit
permit
campaign
guests
call calm
survive
tell
disruption
advancing
answer
coup
president
attempt
maintain
operate
operations
personal
weeks
overcome
meat
wage
lawyers
crucial
struggle
arrived
girls matter
emphasizing
seen
regular
don
medical
dog
principle
consumer
jurisdiction
treaties
attempts
judges
jailed
exports
conductstop
comply
fields
bad
troop
bantesting
decided
restricting
suggested
appeared
initiative
basis
patrolling
basic
deeperprosperous
aviation
tank
economies
nearmoratorium
seven
grown
potentially
advocate
declaring
opportunity
unconditionally
failing
punishment
materials
claims
investments
left
candidate
assassination
save
boasted
interior
destroy
www
deal
dead
devastationlies
distributing
initial
form
failure
stages
occupiers
grenades
sale
ship
marking
handed
fell
authorities
weekend
billion
assume
precise
mileassailants
bomber
hijacked
suicide
answered
finally
marks
advised
governments
round
fearing
targets
suspect
international
refused
earthquake
wait
box
shift
bow
convoys
sympathetic
wealth
visit
sharing
rigid
effort
fly
soul
arrive
agent
council
chemical hostage
sworn
map
designed
hailed
stability
argued
policy
main
boatskilled
possess
living
interference
emerged
spies
advance
language
frequent
suspended
carry
visas
speaking
venture
voices
occupied
fortified
consortium
channel
investigations
arguing
paid
assault
hardships
especially
massacres
shop
shot
sites
impunity
black
enthusiasm
contractsnearly
morning
declared
seal
indicted
checks
infrastructure
killing
nationality
according
tour
unusual
capable
borders
gripped
wake
sound
coca
coffee
pressing
traffickers pay
speech
running
roughly
substantive
money
resolution
grid
identifying
maintainedimages
colonial
critical
expressing
island
mixed
road
decisive
deepened
deadlock
powerless
personnel
narcotics
starting dollar
expressions
children
fall
neighborhood
returned
issues
movement
promising
speeches
significantly
published
buses
establish
controlling
particular
town
hour
intercepted
guards
remain
abandon
purchases
share
purchased
acts
advice
response
bleak
responsibilitysect
playing
inspectors
industrial
existence
late
smuggle
curfews
seeking
compound
easilyspy
deteriorated
house
energy
hard
oil reparations
beginning
computersconductedembraced
militant
assumption
statement
rates
doctors
contrary
treated
mountain
build
deepening
province
significant
joining
particularly
giant
resolve
common
scenes
surrendered
forthcoming
feared
expert
fans
aircraft
solely
annual
simply
throughout
expensive
create
secret
collaborator
meeting
gas
moment
understand
resisted
politician
bill
elections
cluster
replaced
consular century
suggests
corridor
development
yesterday
helped
spent
flagswithdraw
entryfundamentalists
shape
mob
alternative
cutdanger
source
deliberately
bid
matters
foreigners
bit
absolutely
scale
contacts
decision
remembered
hostages
forward
invite
immigration
hopes
subsidiarydirected
requesting
planes
autonomous
units
expedition
stripped
single
desert
vehicles
notion
brigade
warheads
rivers
givingaccessbody
exchange
jointlyhonor
richest
privately
resolutions
breakup
themselves
account
tunnel
praise
closing
effectively
reserves native
holds
regions
responsibilities
profile
collection
chief
bridges
labor
criminal
extradition
descendants
mandate
day
warned
identified
approachedallies
mortar
allied
reached
gear
acquiredoctorate
unlikely
apparently
mid
mix
eight
prisoner
enthusiastic
stranded request
normally
staff
controls
regularly
teams
shellingincluding
intensity
instant equal
passing
commentdisarmament
proposal
curtail
assert
finished
assessments
appearance
value
captors
arrangements
claimed
administration
injured
material
cubic
insure
executive
interim
delegations
resolved
tests
journalist
authority
proper students
panelactual
baseless
billions
buy
bus
repeated
dangerous
minutes
supreme
deaths
shooting
misstated
weaponry
stirredaccidents
limited
facilities
violate
rise
encounter
school
reciprocal
barracks
enjoy
leaders
consistent
direct
street
estimated
supplied
endorse
supplies
generals
changedhospital
assessment
casualties
shoot
joinestate
illicit
accusations
widespread
badly
stronghold
amidacknowledged
enter
executed
expectsexpectations
destroyed
prevented
tourism
notified
leading
victory
satellite
laid
newly
closings
days
relations
militancy
murder
servenegotiating
abandoning
tourists
bridge
ran
relatively
broadly
flow
reputation
pope
radio
earth
delays
announce
criticized
despite
reportcountries
twice
corrupt
capacityimprove
protect
games
expense
quickly
confident
expected
drugs
teachers
stopping
beside
complex
bands
action
insisted
shortly
forceful
detained
humanity
cultural
judge
advanced
apart
gift
officer
meters
election
escape
armed
espionage
core
attacks
efforts
civil
laborers
rely
fought
head
medium
differences
hear
removed
suspicions
setting
bullet
vigorously
backward
leads
environment
charge
tanksdenounced
level
posts
brother
water
baseballguilty
shootings
crisishandled
memory
ferry
figure
accepts
unarmed
critic
dropped
requirementsvital
information
tipped
succeededmainly
coordinated
happened
intended
tensions
factory
state
view
blood
suspend
requirement
season
copy
population
wide
require
officials
cards
outcomepre
armored
blunt
urge
stormed
consideredsenior
convoy
allegations
gulf
crime
cautioned
embassy
imposing
college
insuring
farmers
federal
review
commitments
weapons
outside
missions
region
provocation
period
insistpilgrims
peacefuldirection
minister
spirit
cash
tentatively
couples
postwarevents
status
driver
vowed
relief
inability
model
justify
guided
violent
cease
captured
announcement
disrupt
inflation
acceptedpicture
competition
table
provided
legal
discussed
stand
tariffs
communication
rising
determine
strict
regard
promote
strongly
spends
deserve
briefly
detaineessymbol
bullets included
callsfollow
settlement
decisions
acting
opportunities
program
activities
woman
worse
worst
list
prolonged
rate
design
supposedly
options
proceed
markets
minor
flag
glad
division
breaking
short
retaliate
mission
outraged
pretext
clashes
resort
soccer
framework
signing
instructions
friendship
expect
threat
feel
linkingstory
option
exploded
king
double
arsenal
outstanding
strengthen
chairman
donor
added
electric
measures
enduring
measured
stands
institutions
lying
industrialized
hit
militants
investigate
retaliated
declined
youth
unions
determined
various
numerous
corruption
recently
sold
attentionrapid
opposition
oversee
reasons
village
declinepolitical
due
strategy
flight
demand
plants
frozen
warrant
obstacle
widely
protestersflown
stating
questions
workers
amended
refuses
pipeline
firing
shed
question
fast
repeatedly
liberty
called
humanitarian
commissioner
warning
allowing
fertilizer
issued
resistance
disposal
dissidents languages
stable
include
fences
seized
deals
underscored
settlers
reunion
growers
universities
waiting
capital
chose
degree
pushing
larger
leaving
deputy
apply
timetable
journalistscooperate
women
tax
sitsix
struggled
attend
ethnic
lightsuperior
ships
permanent
covered
crash
flew
satellites
flee
fled
successor
articles
transitional
artists
related
doubled
disclose
averted
promptly
conflicts
proposing
reversed
institution
isolation
linked
afraid
agency
appealing
combat
alarmedneighboring
gain
electoral
agreed
fearoperational
local
ones
married
barrels
hardship
closer
favor
identification
closed
ability
agencies
job
grain
tactical
unclear
wall
withdrawn
improved
cross
unity
largest
diplomat
student
correspondent
fighting
rocket
heavily
obtain
republics
press
taxpayers
agents
employment
ravaged
lead
leader
murdered
extension
getting
spilled
sectarian
shipping
surge
carries
promisetransfer
intention
funds
challenges
record
demonstrate
firm
percent
book
conclusion
illegally
−0.5 0.0 0.5
Experiments
• Randomly select 3607 paragraphs discussing pairs of 245countries and territories.
• Label each of these paragraphs’ sentiment with two ratingsfrom Amazon Mechanical Turk.
• Hold out 42 random pairs (244 paragraphs) for testing.
• Fit sentiment model parameters β on training paragraphs.
• Infer the spatial sentiment model with these parameters on all257,472 paragraphs from 1988 to 2008.
Analysis with this model
To perform analysis with this model:
1. Fit the posterior (we used MAP)
2. Inspect countries’ means x̄c,·
3. Inspect the relationship between countries’ means x̄c1,t x̄c2,t
Results: selected countries’ latent positions
Ip1
Ip2
−0.4
−0.2
0.0
0.2
0.4
0.6
0.8
afghanistan
canada
china
france
indiairaq
israel
japannorth_koreapakistanrussia/ussr
south_korea
switzerland
united_states
●
−0.2 0.0 0.2 0.4 0.6 0.8 1.0Ip1
Ip2
−0.2
0.0
0.2
0.4
0.6
0.8
1.0
afghanistancanada
china
franceindia
iraq
israeljapan
north_korea
pakistan
russia/ussr
south_koreaswitzerland
united_states
●
0.0 0.5 1.0
1987 2007
Results: selected countries’ mutual sentiment with the U.S.
Date
Mut
ual s
entim
ent w
ith th
e U
nite
d S
tate
s
−0.6
−0.4
−0.2
0.0
0.2
0.4
−0.6
−0.4
−0.2
0.0
0.2
0.4
−0.6
−0.4
−0.2
0.0
0.2
0.4
afghanistan
iraq
russia/ussr
1990 1995 2000 2005
canada
israel
switzerland
1990 1995 2000 2005
china
japan
yugoslavia
1990 1995 2000 2005
Results: selected countries’ mutual sentiment with Spain
Date
Mut
ual s
entim
ent w
ith S
pain
−0.3
−0.2
−0.1
0.0
0.1
0.2
0.3
−0.3
−0.2
−0.1
0.0
0.1
0.2
0.3
−0.3
−0.2
−0.1
0.0
0.1
0.2
0.3
afghanistan
great_britain
portugal
1990199520002005
china
greece
united_states
1990199520002005
france
iraq
1990199520002005
germany
italy
1990199520002005
Evaluation
The model does better than text regression and individualMechanical Turk workers compared against one another.
Model Mean Squared Error Mean Absolute Error
Inter-rater agreement 1.77 (7.11) 1.037 (2.07)
Text regression 5.53 1.94
Reversion variance 0.1 2.36 1.09Reversion variance 1 2.32 1.07
Reversion variance 10 2.32 1.08Reversion variance 100 2.34 1.09
Reversion variance 1000 2.33 1.08
Current work and future directions
• Sentiment intercepts for each country
• Infer asymmetric relationships
• Application to other dyads
• Infer unsupervised relations• Sentiment is only one dimension• Similar to relational topic models [1]
Bibliography
Chang, J., D. M. Blei.
“Relational topic models for document networks.”
Proceedings of the 12th International Conference on ArtificialIntelligence and Statistics (AIStats) 2009, 5, 2009.