Mathematical Statistics Team - Riken...IX. ICCS 2018. Springer Proceedings in Complexity. Springer),...

•

•

•

•

•

数理統計学チーム 下平英寿Mathematical Statistics Team

Hidetoshi Shimodaira

@

• selective inference••••••

Thong PhamKIM GEEWOOK

View-1 View-2 View-Dℝ𝑝1 ℝ𝑝2 ℝ𝑝𝐷

Neural networks

ℝ𝐾Shared space

IPS

Stu

den

t/C

ours

eU

niv

ersi

ties

Hyperbolic SIPS IPDS WIPS

●

●●

●●

●●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●●●

●

●

●●

●●

●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●● ●

●

●

●

●

●

●●

●

●

●

●

●

●● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

● ●

●

●●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●

●

● ●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●●

● ●

●

●

●

●

●

●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

● ●

●

●

●

●●

●●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●●

●

●

●

●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

● ●

●

●●

-4 -2 0 2 4

-4-2

02

4

Y

A (c

ase

1)

: z=1, : z=0

Y

density

-3 -2 -1 0 1 2 3

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

Y, ZYY, A (case 1)

-4 -2 0 2 4

-4-2

02

4

Y

A (c

ase

2)

: z=1, : z=0

Y

density

-3 -2 -1 0 1 2 3

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

Y, ZYY, A (case 2)

●●●●●●●●

●

●

●

●

●

0 2 4 6 8

−0.5

0.0

0.5

1.0

1.5

2.0

2.5

extrapolation k=2poly.3

σ2

ψ(σ

2 )

k.2

A Fitting to Tree T1 B Fitting to Edge E2

●●●●●●

●●

●

●

●

●

0 2 4 6 8

−1.6

−1.4

−1.2

−1.0

−0.8

−0.6

extrapolation k=2poly.2+poly.3+sing.3

σ2

ψ(σ

2 )

k.2

1 human

2 seal3 cow

4 rabbit

5 mouse6 opssum

1 human

2 seal

3 cow

4 rabbit 5 mouse

6 opssum0.1

E1 E3

E2 E6

T7T1

E8E1

❗

linear algebra learn study -ing !number

learnalgebralinear algebra

endeavor

learn

studymathmatics

algebra

linear algebra

studying

Corpus

(a)

(b)

(c)

xi�nleft , · · · , xi�2, xi�1, xi, xi+1, · · · , xj�1, xj , xj+1, xj+2, · · · , xj+nrightTarget n-gramContext n-gram Context n-gram

xi, xi+1, xi+2, xi+3, · · · , xj�3, xj�2, xj�1, xjSub-n-grams 2 S(x (i:j)) ⇢ V

AAACk3ichVFBS+NAFP6MuqvVXesuguAlWJV6KS9eVtyLuB68CGptFayUJDvtzpomIZkWNfQP7HnBgycFEfEfeNXD/oE9+BPEo4IXD76mAVFRX8i8b75535tvZizfkaEiuuzQOru6P3zs6U319X/6PJAe/FIMvXpgi4LtOV6wbpmhcKQrCkoqR6z7gTBrliPWrK0frfW1hghC6bmrascXmzWz6sqKtE3FVDmdGytJV89nS0psK6sSxVmqaLvZLEdZOfN7sjmpl8K6FQqlF8fK6QzlKA79JTASkEESS176GCX8hAcbddQg4EIxdmAi5G8DBgg+c5uImAsYyXhdoIkUa+tcJbjCZHaLxyrPNhLW5XmrZxirbd7F4T9gpY5x+k8ndEP/6JSu6P7VXlHco+Vlh7PV1gq/PPBnOH/3rqrGWeHXo+pNzwoVTMdeJXv3Y6Z1Crutb+zu3eRnVsajCTqka/Z/QJd0zidwG7f20bJY2UeKH8B4ft0vQXEqZ1DOWJ7KzM4lT9GDEYwiy/f9DbNYwBIKvO9fnOEcF9qQ9l2b0+bbpVpHovmKJ6EtPgAbLps9AAACk3ichVFBS+NAFP6MuqvVXesuguAlWJV6KS9eVtyLuB68CGptFayUJDvtzpomIZkWNfQP7HnBgycFEfEfeNXD/oE9+BPEo4IXD76mAVFRX8i8b75535tvZizfkaEiuuzQOru6P3zs6U319X/6PJAe/FIMvXpgi4LtOV6wbpmhcKQrCkoqR6z7gTBrliPWrK0frfW1hghC6bmrascXmzWz6sqKtE3FVDmdGytJV89nS0psK6sSxVmqaLvZLEdZOfN7sjmpl8K6FQqlF8fK6QzlKA79JTASkEESS176GCX8hAcbddQg4EIxdmAi5G8DBgg+c5uImAsYyXhdoIkUa+tcJbjCZHaLxyrPNhLW5XmrZxirbd7F4T9gpY5x+k8ndEP/6JSu6P7VXlHco+Vlh7PV1gq/PPBnOH/3rqrGWeHXo+pNzwoVTMdeJXv3Y6Z1Crutb+zu3eRnVsajCTqka/Z/QJd0zidwG7f20bJY2UeKH8B4ft0vQXEqZ1DOWJ7KzM4lT9GDEYwiy/f9DbNYwBIKvO9fnOEcF9qQ9l2b0+bbpVpHovmKJ6EtPgAbLps9AAACk3ichVFBS+NAFP6MuqvVXesuguAlWJV6KS9eVtyLuB68CGptFayUJDvtzpomIZkWNfQP7HnBgycFEfEfeNXD/oE9+BPEo4IXD76mAVFRX8i8b75535tvZizfkaEiuuzQOru6P3zs6U319X/6PJAe/FIMvXpgi4LtOV6wbpmhcKQrCkoqR6z7gTBrliPWrK0frfW1hghC6bmrascXmzWz6sqKtE3FVDmdGytJV89nS0psK6sSxVmqaLvZLEdZOfN7sjmpl8K6FQqlF8fK6QzlKA79JTASkEESS176GCX8hAcbddQg4EIxdmAi5G8DBgg+c5uImAsYyXhdoIkUa+tcJbjCZHaLxyrPNhLW5XmrZxirbd7F4T9gpY5x+k8ndEP/6JSu6P7VXlHco+Vlh7PV1gq/PPBnOH/3rqrGWeHXo+pNzwoVTMdeJXv3Y6Z1Crutb+zu3eRnVsajCTqka/Z/QJd0zidwG7f20bJY2UeKH8B4ft0vQXEqZ1DOWJ7KzM4lT9GDEYwiy/f9DbNYwBIKvO9fnOEcF9qQ9l2b0+bbpVpHovmKJ6EtPgAbLps9AAACk3ichVFBS+NAFP6MuqvVXesuguAlWJV6KS9eVtyLuB68CGptFayUJDvtzpomIZkWNfQP7HnBgycFEfEfeNXD/oE9+BPEo4IXD76mAVFRX8i8b75535tvZizfkaEiuuzQOru6P3zs6U319X/6PJAe/FIMvXpgi4LtOV6wbpmhcKQrCkoqR6z7gTBrliPWrK0frfW1hghC6bmrascXmzWz6sqKtE3FVDmdGytJV89nS0psK6sSxVmqaLvZLEdZOfN7sjmpl8K6FQqlF8fK6QzlKA79JTASkEESS176GCX8hAcbddQg4EIxdmAi5G8DBgg+c5uImAsYyXhdoIkUa+tcJbjCZHaLxyrPNhLW5XmrZxirbd7F4T9gpY5x+k8ndEP/6JSu6P7VXlHco+Vlh7PV1gq/PPBnOH/3rqrGWeHXo+pNzwoVTMdeJXv3Y6Z1Crutb+zu3eRnVsajCTqka/Z/QJd0zidwG7f20bJY2UeKH8B4ft0vQXEqZ1DOWJ7KzM4lT9GDEYwiy/f9DbNYwBIKvO9fnOEcF9qQ9l2b0+bbpVpHovmKJ6EtPgAbLps9

2 VAAACanichVG7SgNBFD1Z3/EVY6OkCSYRq3DXRrESbSx95QGJyO466uBmd9ndBDT4A3ZWgqkURMTPsPEHLPIJol0EGwtvNguiQb3DzJw5c8+dMzO6Y0rPJ2pGlJ7evv6BwaHo8Mjo2HhsIp737KpriJxhm7Zb1DVPmNISOV/6pig6rtAquikK+tFqe79QE64nbWvbP3bETkU7sOS+NDSfqUK6LK1kPr0bS1GWgkh2AzUEKYSxbsduUcYebBioogIBCz5jExo8biWoIDjM7aDOnMtIBvsCp4iytspZgjM0Zo94POBVKWQtXrdreoHa4FNM7i4rk8jQE91Rix7pnp7p49da9aBG28sxz3pHK5zd8bOprfd/VRWefRx+qf707GMfi4FXyd6dgGnfwujoaycXra2lzUx9lq7phf1fUZMe+AZW7c242RCbDUT5A9Sfz90N8vNZlbLqxnxqeSX8ikEkMIM5fu8FLGMN68gF7s5xiUbkVYkr00qik6pEQs0kvoWS/gTObourAAACanichVG7SgNBFD1Z3/EVY6OkCSYRq3DXRrESbSx95QGJyO466uBmd9ndBDT4A3ZWgqkURMTPsPEHLPIJol0EGwtvNguiQb3DzJw5c8+dMzO6Y0rPJ2pGlJ7evv6BwaHo8Mjo2HhsIp737KpriJxhm7Zb1DVPmNISOV/6pig6rtAquikK+tFqe79QE64nbWvbP3bETkU7sOS+NDSfqUK6LK1kPr0bS1GWgkh2AzUEKYSxbsduUcYebBioogIBCz5jExo8biWoIDjM7aDOnMtIBvsCp4iytspZgjM0Zo94POBVKWQtXrdreoHa4FNM7i4rk8jQE91Rix7pnp7p49da9aBG28sxz3pHK5zd8bOprfd/VRWefRx+qf707GMfi4FXyd6dgGnfwujoaycXra2lzUx9lq7phf1fUZMe+AZW7c242RCbDUT5A9Sfz90N8vNZlbLqxnxqeSX8ikEkMIM5fu8FLGMN68gF7s5xiUbkVYkr00qik6pEQs0kvoWS/gTObourAAACanichVG7SgNBFD1Z3/EVY6OkCSYRq3DXRrESbSx95QGJyO466uBmd9ndBDT4A3ZWgqkURMTPsPEHLPIJol0EGwtvNguiQb3DzJw5c8+dMzO6Y0rPJ2pGlJ7evv6BwaHo8Mjo2HhsIp737KpriJxhm7Zb1DVPmNISOV/6pig6rtAquikK+tFqe79QE64nbWvbP3bETkU7sOS+NDSfqUK6LK1kPr0bS1GWgkh2AzUEKYSxbsduUcYebBioogIBCz5jExo8biWoIDjM7aDOnMtIBvsCp4iytspZgjM0Zo94POBVKWQtXrdreoHa4FNM7i4rk8jQE91Rix7pnp7p49da9aBG28sxz3pHK5zd8bOprfd/VRWefRx+qf707GMfi4FXyd6dgGnfwujoaycXra2lzUx9lq7phf1fUZMe+AZW7c242RCbDUT5A9Sfz90N8vNZlbLqxnxqeSX8ikEkMIM5fu8FLGMN68gF7s5xiUbkVYkr00qik6pEQs0kvoWS/gTObourAAACanichVG7SgNBFD1Z3/EVY6OkCSYRq3DXRrESbSx95QGJyO466uBmd9ndBDT4A3ZWgqkURMTPsPEHLPIJol0EGwtvNguiQb3DzJw5c8+dMzO6Y0rPJ2pGlJ7evv6BwaHo8Mjo2HhsIp737KpriJxhm7Zb1DVPmNISOV/6pig6rtAquikK+tFqe79QE64nbWvbP3bETkU7sOS+NDSfqUK6LK1kPr0bS1GWgkh2AzUEKYSxbsduUcYebBioogIBCz5jExo8biWoIDjM7aDOnMtIBvsCp4iytspZgjM0Zo94POBVKWQtXrdreoHa4FNM7i4rk8jQE91Rix7pnp7p49da9aBG28sxz3pHK5zd8bOprfd/VRWefRx+qf707GMfi4FXyd6dgGnfwujoaycXra2lzUx9lq7phf1fUZMe+AZW7c242RCbDUT5A9Sfz90N8vNZlbLqxnxqeSX8ikEkMIM5fu8FLGMN68gF7s5xiUbkVYkr00qik6pEQs0kvoWS/gTObour

• Okuno, Hada, Shimodaira (ICML 2018) A probabilistic framework for multi-view feature learning with many-to-many associations via neural networks

• Okuno, Shimodaira (ICML workshop Theoretical Foundations and Applications of Deep Generative Models 2018) On representation power of neural network-based graph embedding and beyond

• Okuno, Kim, Shimodaira (to appear AISTATS 2019) Graph Embedding with Shifted Inner Product Similarity and Its Improved Approximation Capability

• Kim, Okuno, Fukui, Shimodaira (arXiv:1902.10409) Representation Learning with Weighted Inner Product for Universal Approximation of General Similarities

• Okuno, Shimodaira (to appear AISTATS 2019) Robust Graph Embedding with Noisy Link Weights

•

Universal Approximation Mercer

• mini-batch SGD)

•

•

• (IPS) positive definite, PD) ( )

• Shifted Inner Product Similarity (SIPS)

• SIPS (conditionally PD, CPD)

• CPD ( ) PoincareWasserstein

• Inner Product Difference Similarity (IPDS)

• IPDS (indefinite)

IPS ( SIPS (

• Weighted Inner Product Similarity (WIPS)

•(indefinite)

•

• (β- )( β-

• density power divergence

Cosine sim., Gauss kernel, …

Negative Poincare distance, …

Negative Jeffrey’s divergence, …CPD similarities

General similarities

PD similarities IPS

SIPS

IPDS and WIPS (Proposed)

β=0 β=0.5 β=1.0

β

• Kim, Fukui, Shimodaira (EMNLP Workshop on Noisy User-generated Text W-NUT 2018), Word-like character n-gram embedding

• Kim, Fukui, Shimodaira (to appear NAACL-HLT 2019), Segmentation-free compositional n-gram embedding

• Shimodaira, Terada (arXiv:1902.04964), Selective Inference for Testing Trees and Edges in Phylogenetics

•

p

•(selective inference)

• (Terada, Shimodaira arXiv:1711.00949)(Shimodaira 2002, 2004,

2008)p-

•p-

• Pham, Sheridan, Shimodaira (to appear Journal of Statistical Software 2019), PAFit: an R Package for Estimating Preferential Attachment and Node Fitness in Temporal Complex Networks

• Inoue, Pham, Shimodaira (Unifying Themes in Complex Systems IX. ICCS 2018. Springer Proceedings in Complexity. Springer), Transitivity vs Preferential Attachment: Determining the Driving Force Behind the Evolution of Scientific Co-Authorship Networks

•(PAFit)

•)

•

• Imori, Shimodaira (arXiv:1902.07954), An information criterion for auxiliary variable selection in incomplete data analysis

•

•

•

•

•

• AIC

• CMP:• HEP:• SMJ: