Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
TreeConnect: A Sparse Alternative to Fully Connected Layers
Oliver Richter, Roger Wattenhofer
Distributed Computing Group (DisCo)ETH Zurich
Switzerland 1
“cat”
“dog”
Convolutional Neural Network (CNN)
Fully Connected Layers (FC)
Convolutional Neural Network (CNN)
Fully Connected Layers (FC)
Channel
n 𝜣(n2)m = r n.
n
mhc
1
2
3
= n0.5𝜣(n1.5)
= (n m)0.5. = r n.
c . n hc c
. + c m. = n . r 0.5 nn0.5
. + n0.5 r n. .
L
1 2 3
𝜣(n(2 L - 1) / L). 𝜣(n2)
IMDB sentiment classification
IMDB sentiment classification
Reuters News Categorization
Reuters News Categorization
47,150
25,806
Reuters News Categorization
CIFAR-10 Image Classification
CIFAR-10 Image Classification
CIFAR-10 Image Classification
CIFAR-10 Image Classification
Related Work
Questions & Answers
https://github.com/OliverRichter/TreeConnect
Code?
Further Questions?