Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
125 250 500 1000 2000 4000 8000 16000number of training sequences
9
8
7
6
5
log-l
ikelih
ood p
er
event
Experiments on real-world social media datasets • Retweet (top): long sequences with • MemeTrack (bottom): short sequences with
1000 2000 4000 8000 16000 32000number of training sequences
2000
1500
1000
500
0
500
log-l
ikelih
ood p
er
event
Type-1 Type-2
Intensity-1 Intensity-2
BaseRate-1 BaseRate-2
LSTM-Unit
Neural Hawkes process: • Continuous-time LSTM • Hidden state • Cell memory • • Extra gates to compute and
Hawkes
An event stream from a neural Hawkes process Time and type depend on details of past history and on each other
Events happen at random times At time , there occurs an event of type Given past events, what might happen next, and when? • Generative model • Medical: patient’s visits, tests and diagnoses • Online shopping: purchasing and feedback • Social media: posts, shares, comments • Other: quantified self, news, dialogue, music, etc Traditional model is a Hawkes process [1] • Each event type has an intensity • Each event token occurs with probability • Past events temporarily excite future events •
Neural Hawkes process vs. similar work [2] • Prediction error for type (upper) and time (lower)
Train the model by maximizing log-likelihood
• Total intensity • Integral estimation by Monte Carlo simulation Minimum Bayes Risk prediction • Density for is • Time prediction • Type prediction Thinning algorithm for sampling sequences
Experiments on artificial datasets • Models try to fit data generated by each other • Oracle model performance —
The Neural Hawkes ProcessA Neurally Self-Modulating Multivariate Point Process Hongyuan Mei and Jason Eisner Center for Language and Speech Processing, Department of Computer Science, Johns Hopkins University
Overview Model Algorithms
Experiments (many more in paper)
125 250 500 1000 2000 4000 8000 16000number of training sequences
40
30
20
10
0
log-l
ikelih
ood p
er
event
Missing data experiments ★ Censor all events of some types ★ neural Hawkes > Hawkes process ★ Consistent over all combos
h(t) = oi � (2�(2c(t))� 1)
c̄i+1
�k(t) = fk(w>k h(t))
Intensity
Time
t̂i =R1t=ti�1
tpi(t)dt
ˆki = argmaxk
R1t=ti�1
pi(t)�k(t)/�(t)dt
pi(t) = �(t) exp(�R ts=ti�1
�(s)ds)ti
` =P
i:tiT log �ki(ti)�R Tt=0 �(t)dt
�(t) =PK
k=1 �k(t)
[1] Hawkes, Alan G. Spectra of some self-exciting and mutually exciting point processes. 1971. [2] Du, Nan, et. al. Recurrent marked temporal point processes: Embedding event history to vector. 2016.
P ((ki, ti) | (k1, t1), . . . , (ki�1, ti�1))
�k(t) = µk +
Ph:th<t ↵kh,k exp(��kh,k(t� th))
ti ki
Intensity
Time
�
t
Intensity
Time
�
t
�
t
Erro
r Rat
e %
RMSE
Financial MIMIC-II Stack Overflow
neural Hawkes
Log-
likel
ihoo
d
# of training sequences
K = 3
K = 5000
zoom in
Hawkes+inhibitionLog-
likel
ihoo
d
Data: Hawkes Hawkes+inhibition neural Hawkes
Haw
kes
neural Hawkes
�1(t) �2(t)
t
c(t) = c̄i+1 + ⌫i+1(t)
⌫i+1(t) = (ci+1 � ¯ci+1) exp(��i+1(t� ti))
�k(t)
�k(t)dtc1(t)
c2(t)
�1(t)
�2(t)
0 < t1 < t2 . . .
ti ki 2 {1, 2, . . . ,K}
�i+1
t
cell c(t) ! hidden h(t) ! intensity �(t) ! event? ! updated c(t+ dt)
c3(t)
Neural Hawkes is winner (4/5, 5/5, and 5/5) on type prediction No clear winner on time prediction