Upload
jay
View
218
Download
0
Embed Size (px)
Citation preview
8/16/2019 Nips Tutorial05.Ps
1/127
8/16/2019 Nips Tutorial05.Ps
2/127
8/16/2019 Nips Tutorial05.Ps
3/127
8/16/2019 Nips Tutorial05.Ps
4/127
8/16/2019 Nips Tutorial05.Ps
5/127
8/16/2019 Nips Tutorial05.Ps
6/127
8/16/2019 Nips Tutorial05.Ps
7/127
8/16/2019 Nips Tutorial05.Ps
8/127
8/16/2019 Nips Tutorial05.Ps
9/127
8/16/2019 Nips Tutorial05.Ps
10/127
00
00
11
11
01 0011
00
00
11
11
0011
00
00
11
11
0011
01
00
00
11
11
01
0
0
1
1
8/16/2019 Nips Tutorial05.Ps
11/127
φ2φ1 φ3 φ01
01 0011
0011
0011
0011
00
00
11
11
0
0
1
1
0
0
1
1
0
0
1
1
01
4
8/16/2019 Nips Tutorial05.Ps
12/127
8/16/2019 Nips Tutorial05.Ps
13/127
Æ
8/16/2019 Nips Tutorial05.Ps
14/127
8/16/2019 Nips Tutorial05.Ps
15/127
8/16/2019 Nips Tutorial05.Ps
16/127
8/16/2019 Nips Tutorial05.Ps
17/127
1 X
2 X
3 X
X 4
X 5
X 6
8/16/2019 Nips Tutorial05.Ps
18/127
xiθ
N
θ
x N
x1
x2
8/16/2019 Nips Tutorial05.Ps
19/127
8/16/2019 Nips Tutorial05.Ps
20/127
8/16/2019 Nips Tutorial05.Ps
21/127
0 0.2 0.4 0.6 0.8 1
0
1
2
3
4
5
pdf of Beta(1,α)
8/16/2019 Nips Tutorial05.Ps
22/127
8/16/2019 Nips Tutorial05.Ps
23/127
8/16/2019 Nips Tutorial05.Ps
24/127
8/16/2019 Nips Tutorial05.Ps
25/127
8/16/2019 Nips Tutorial05.Ps
26/127
8/16/2019 Nips Tutorial05.Ps
27/127
8/16/2019 Nips Tutorial05.Ps
28/127
Æ
Æ
8/16/2019 Nips Tutorial05.Ps
29/127
G
θi
xi
Æ
8/16/2019 Nips Tutorial05.Ps
30/127
8/16/2019 Nips Tutorial05.Ps
31/127
Æ
G
θi
xi
α 0
G0
8/16/2019 Nips Tutorial05.Ps
32/127
8/16/2019 Nips Tutorial05.Ps
33/127
8/16/2019 Nips Tutorial05.Ps
34/127
Æ
8/16/2019 Nips Tutorial05.Ps
35/127
1 2...
1β β (1−β )
8/16/2019 Nips Tutorial05.Ps
36/127
Æ
8/16/2019 Nips Tutorial05.Ps
37/127
G
φk
α 0
G0
π k
θi
xi
8/16/2019 Nips Tutorial05.Ps
38/127
8/16/2019 Nips Tutorial05.Ps
39/127
8/16/2019 Nips Tutorial05.Ps
40/127
8/16/2019 Nips Tutorial05.Ps
41/127
Æ
8/16/2019 Nips Tutorial05.Ps
42/127
Æ
8/16/2019 Nips Tutorial05.Ps
43/127
A
A
A
A
A
1
2
3
4
5
Ω
8/16/2019 Nips Tutorial05.Ps
44/127
Æ
Æ
8/16/2019 Nips Tutorial05.Ps
45/127
℄
Æ
Æ
Æ
8/16/2019 Nips Tutorial05.Ps
46/127
8/16/2019 Nips Tutorial05.Ps
47/127
Gα 0
G0
θi
xi
8/16/2019 Nips Tutorial05.Ps
48/127
Gα 0
G0
θi
xi
α 0
G0
θi
xi
8/16/2019 Nips Tutorial05.Ps
49/127
℄
Æ
8/16/2019 Nips Tutorial05.Ps
50/127
8/16/2019 Nips Tutorial05.Ps
51/127
8/16/2019 Nips Tutorial05.Ps
52/127
8/16/2019 Nips Tutorial05.Ps
53/127
8/16/2019 Nips Tutorial05.Ps
54/127
8/16/2019 Nips Tutorial05.Ps
55/127
Æ
8/16/2019 Nips Tutorial05.Ps
56/127
8/16/2019 Nips Tutorial05.Ps
57/127
8/16/2019 Nips Tutorial05.Ps
58/127
℄
℄
℄
℄
8/16/2019 Nips Tutorial05.Ps
59/127
8/16/2019 Nips Tutorial05.Ps
60/127
VDP
CDP
TDP
5 10 20 30 40 50
2 0 0
3 0 0
5 0 0
7 0 0
1 0 0
0
1 5 0 0
2 2 0 0
3 2 0 0
4 7 0 0
Dimension
T i m e i n s e c o n d s
5 10 20 30 40 50
− 1 2 0
0
− 8 0 0
− 4
0 0
Dimension
H e l d − o u t l o g l i k e l i h o
o d
8/16/2019 Nips Tutorial05.Ps
61/127
8/16/2019 Nips Tutorial05.Ps
62/127
8/16/2019 Nips Tutorial05.Ps
63/127
8/16/2019 Nips Tutorial05.Ps
64/127
8/16/2019 Nips Tutorial05.Ps
65/127
A B c
b Ca
{A, a}
{B, b}
{C, c}
8/16/2019 Nips Tutorial05.Ps
66/127
A B c
b Ca
{A, a}
{B, b}
{C, c}
8/16/2019 Nips Tutorial05.Ps
67/127
8/16/2019 Nips Tutorial05.Ps
68/127
8/16/2019 Nips Tutorial05.Ps
69/127
8/16/2019 Nips Tutorial05.Ps
70/127
2 θ m1
x2 xm j j x1 j
θ θ
8/16/2019 Nips Tutorial05.Ps
71/127
θ θ 2 θ m1
x2 xmj j x1 j
θ
8/16/2019 Nips Tutorial05.Ps
72/127
θ
θ i
xij
θ θ2 θ m1
x2 xmj j x1 j
θ
8/16/2019 Nips Tutorial05.Ps
73/127
1 j
mj
mj
1 j
2 j
2 j
π π2 πm1
z z
x
z
x x
8/16/2019 Nips Tutorial05.Ps
74/127
x
G
ij
ij
i
θ
0α
G 0
8/16/2019 Nips Tutorial05.Ps
75/127
8/16/2019 Nips Tutorial05.Ps
76/127
H
8/16/2019 Nips Tutorial05.Ps
77/127
Gα0
G0
θ
x
i
ij
ij
γ
H
8/16/2019 Nips Tutorial05.Ps
78/127
Æ
Æ
8/16/2019 Nips Tutorial05.Ps
79/127
Gα 0
G0
θ
x
i
ij
ij
γ
H
α 0
θ
x
ij
ij
γ
H
18 θθ
θ
8/16/2019 Nips Tutorial05.Ps
80/127
φ
φ
φ
22
21 23
24
26
25 27
28
33
34
31
32
3536
= =ψ 12 =ψ
=ψ =ψ =ψ =ψ
=ψ =ψ
11 131 2 1
3 121 22 23 24
31
3 1
1 232
ψ
θ
θ
θ θ
θ
θ
θ
θθ
θ θ
θ
17
15
16
1211
13
1418
θθθ
θθ
θθ
φ φ
φφ φ φ
φφ
8/16/2019 Nips Tutorial05.Ps
81/127
Æ
Æ
8/16/2019 Nips Tutorial05.Ps
82/127
Æ
8/16/2019 Nips Tutorial05.Ps
83/127
8/16/2019 Nips Tutorial05.Ps
84/127
8/16/2019 Nips Tutorial05.Ps
85/127
8/16/2019 Nips Tutorial05.Ps
86/127
8/16/2019 Nips Tutorial05.Ps
87/127
α z wθ
β
M N
8/16/2019 Nips Tutorial05.Ps
88/127
x
x x
x
x
x
x x
x
x
x
x
x
x
x
x
xx
x
x
x
x
xx
x
x
x
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 11 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
0 0
0 0
1 1
1 1
0011 0 00 01 11 1
topic 2
topic 1
topic 3
topic simplex
word simplex
8/16/2019 Nips Tutorial05.Ps
89/127
8/16/2019 Nips Tutorial05.Ps
90/127
8/16/2019 Nips Tutorial05.Ps
91/127
8/16/2019 Nips Tutorial05.Ps
92/127
G j G j
G0
VS Trainingdocuments
x jin j
jiθ
VS Testdocuments
x jin j
jiθ
γ
0α
H
G j G j
G0
VS Trainingdocuments
Additional trainingdocuments
x jin j
jiθ
G j
x jin j
jiθ
VS Testdocuments
x jin j
jiθ
γ H
0α
G j
G1 G2
G0
G j G j
VS Trainingdocuments
Additional trainingdocuments
x jin j
jiθ
x jin j
jiθ
VS Testdocuments
x jin j
jiθ
γ
0α
α1
H
6000Average perplexity over NIPS sections of 3 models
none, no other sectionfl ddi i l i
5000Relationship to LT, AA, and AP
LTAA
8/16/2019 Nips Tutorial05.Ps
93/127
0 20 40 60 802500
3000
3500
4000
4500
5000
5500
P e
r p l e x i t y
Number of VS training documents
,flat, additional sectionhierarchical, additional section
0 20 40 60 802500
3000
3500
4000
4500
P e
r p l e x i t y
Number of VS training documents
AAAP
8/16/2019 Nips Tutorial05.Ps
94/127
8/16/2019 Nips Tutorial05.Ps
95/127
xT x 2 x1
z zT 2
z1