View
167
Download
2
Category
Preview:
Citation preview
End-to-end convolutional network for saliency prediction
Junting Pan Xavier Giró-i-Nieto
Slides online@DocXavi
Large-scale Scene Understanding (LSUN)
Challenge 2015
http://bit.ly/juntingnet
2
Financial supportTechnical support
Albert Gil Josep Pujal
ACKNOWLEDGMENTS
3
LSUN SALIENCY CHALLENGE: A Déjà vu ?
John Markoff, “Scientists see promise in deep learning Programs”, The New York Times (Nov2012).
Photo: Keith Penner
4
LSUN SALIENCY CHALLENGE: A Déjà vu ?
[Mohedano’14]
5
LSUN SALIENCY CHALLENGE: A Déjà vu ?
6
RELATED WORK: Deep Saliency
Kümmerer, Matthias, Lucas Theis, and Matthias Bethge. "Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet." arXiv preprint arXiv:1411.1045 (2014).
7
RELATED WORK: Deep Saliency
Vig, Eleonora, Michael Dorr, and David Cox. "Large-scale optimization of hierarchical features for saliency prediction in natural images." Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on. IEEE, 2014.
8
RELATED WORK: Fully convolutional
Long, Jonathan, Evan Shelhamer, and Trevor Darrell. "Fully convolutional networks for semantic segmentation." Computer Vision and Pattern Recognition (CVPR), 2015 IEEE Conference on. IEEE, 2015.
9
RELATED WORK: Image Classification
CaffeNet
ARCHITECTURE[Khrizevsky’12]
DATA[Deng’09]
FRAMEWORK[Jia’14]
10
SALIENCY PREDICTION: JuntingNet
JuntingNet
11
SALIENCY PREDICTION: JuntingNet
JuntingNet
DATAiSun [Xu’15]
SALICON [Jiang’15]
12
SALIENCY PREDICTION: Data
TRAIN VALIDATION TEST
SALICON [Jiang’15] 10,000 5,000 5,000
iSun [Xu’15] 6,000 926 2,000
CAT2000 [Borji’15] 2,000 - 2,000
MIT300 [Judd’12] 300 . -
LargeScale
13
SALIENCY PREDICTION: JuntingNet
JuntingNet
ARCHITECTURE[Pan’15]
DATAiSun [Xu’15]
SALICON [Jiang’15]
14
SALIENCY PREDICTION: Architecture
15
SALIENCY PREDICTION: Architecture
End to end + regression = JuntingNet
16
SALIENCY PREDICTION: Architecture
Resize
96x96
Upsample + filter
4608 = 48x48
2D map
17
SALIENCY PREDICTION: JuntingNet
JuntingNet
ARCHITECTURE[Pan’15] (soon)
DATAiSun [Xu’15]
SALICON [Jiang’15]
FRAMEWORK[Bergstra’10][Bastien’12]
18
SALIENCY PREDICTION: Framework
Tutorial by Daniel Nouri (*) on regression for facial points for Kaggle.
(*) Daniel Nouri, “Using convolution networks to detect facil points” (Dec 2014).
on Lasagne
19
SALIENCY PREDICTION: Training
Data augmentation with horizontal mirroring.
20
SALIENCY PREDICTION: Training
Loss function Mean Square Error (MSE)
Weight initialization Gaussian distribution
Learning rate 0.03 to 0.0001
Mini batch size 128
Training time 7h (SALICON) / 3h (iSUN)
Acceleration Sigmoid + nesterov momentum 0.9
Regularisation Maxout norm
GPU NVidia GTX 980
21
RESULTS: Qualitative (iSUN)
JuntingNetGround TruthPixels
22
RESULTS: Qualitative (iSUN)
JuntingNetGround TruthPixels
23
RESULTS: Quantitative (iSUN)
24
RESULTS: Qualitative (SALICON)
JuntingNetGround TruthPixels
25
RESULTS: Qualitative (SALICON)
JuntingNetGround TruthPixels
26
RESULTS: Quantitative (SALICON)
27
RESULTS: Publications by end of June
http://bit.ly/juntingnet
28
Thank you LSUN ! Thank you Boston !
http://bit.ly/juntingnetSlides online @DocXavi
Recommended