Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Model (UPC Reading Group 2017)

Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Model

Junting Pan

M Cornia, L Baraldi, G Serra, R Cucchiara. Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Model. arXiv preprint arXiv:1611.09571

https://scholar.google.it/citations?view_op=view_citation&hl=it&user=DzgmSJEAAAAJ&citation_for_view=DzgmSJEAAAAJ:9yKSN-GCB0IC



1. Introduction

Outline1. Introduction2. Model Architecture3. Experimental Evaluation4. Implementation Details5. Results 6. Conclusion

Outline1. Introduction2. Model Architecture

a. Dilated Residual Convolutional Networksb. Attentive Convolutional LSTMc. Learned Priors

3. Experimental Evaluation4. Implementation Details5. Results 6. Conclusion

2. Model Architecture

2. Model Architecture - DRCN


ResNet-50



conv4 use dilated rate = 1conv5 use dilated rate = 3

2. Model Architecture - Attentive ConvLSTM







Filter size of 5x5 with dilated rate =3, which means a receptive field of 17x17


- SALICON 5.000 testing images- MIT1003 1003 images- MIT300 300 testing images- CAT2000 2000 testing images


4. Implementation Details- Batch size : 10- RMSprop optimizer- Residual Network are initialized using ResNet-50 pre trained on ImageNet- Loss Function : KL-Divergence - Learning rate : 10e-4, decay by a factor of 10 every two epochs


5. Results - Model Ablation Analysis

5. Results - Model Ablation AnalysisDRCN +Conv LSTM +Priors Ground Truth

5. Results - SALICON MIT300 CAT200

5. Results


6. Conclusions

- New Saliency Attentive Model for fixation prediction.

- Attentive Convolutional LSTM that is specifically designed to sequentially focus on the most salient regions of input images.

- Residual Architecture with dilated filters that maintains the spatial resolution.

Thanks for your attention!

Data & Analytics

Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Model (UPC Reading Group 2017)