Visualizing and Understanding Convolutional Networks and... · Visualizing and Understanding...

Introduction Visualization Training Details CNN Visualization Experiments Discussion

Visualizing and Understanding ConvolutionalNetworks

Mattew D. Zeiler and Rob Fergus12 Nov 2013

Baek Gyuseung

Seoul National University

December 2, 2016

1 Introduction

2 Visualization

3 Training Details

4 CNN Visualization

5 Experiments

6 Discussion

Outline for section 1

1 Introduction

2 Visualization

3 Training Details

4 CNN Visualization

5 Experiments

6 Discussion

Introduction

• Since their introduction by LeCun in the early 1990’s,Convolution Neural Networks - CNNs have demonstratedexcellent performance at image classification

• CNNs have developed continuously - Krizhevsky et al. win theImageNet 2012 classification benchmark with their own CNN(AlexNet)

• There are several reasons that CNNs perform well - complexstructure, parsimonious coefficient

Introduction

• However, from a scientific standpoint, this is deeplyunsatisfactory

• Little insight into the internal operation and behavior• Have no idea of how they achieve such good performance

• Without clear understanding of how and why they work, thedevelopment of better models is reduced to trial-and-error

• Visualization - reveals the input stimuli that excite individualfeature maps at any layer in the model

• Result of visualization is no just crops of input images, rathertop-down projection

1 Introduction

2 Visualization

3 Training Details

4 CNN Visualization

5 Experiments

6 Discussion

Deconvolution Network

• Proposed by Zeiler et al.

• Inverse mapping of CNN

• It is not a learning - just use already trained CNN

• Unpoolingmax pooling is non-invertible - recording the locations of themaxima (switch variable)

• Rectification

• Filteringuse tranposed versions of the same filters (like Auto-Encoder)

• Contrast normalization does not need

Figure: Example of unpooling by using switch variable

1 Introduction

2 Visualization

3 Training Details

4 CNN Visualization

5 Experiments

6 Discussion

Training details

• Compare with the AlexNet

• Data set - ImageNet 2012

• model fitting by stochastic gradient descent

• mini-batch size : 128• learning rate : 10−2, momentum term : 0.9• Dropout with a rate of 0.5• initial weight : 10−2, bias : 0• produce multiple crops

• Renormalize each filter in conv. layers whose RMS valueexceeds a fixed radius of 10−1 to this fixed radius

Architecture

A little difference to AlexNet

• Sparse connection in AlexNet are replaced with denseconnections

• Filter size of input image and stride are adjusted

1 Introduction

2 Visualization

3 Training Details

4 CNN Visualization

5 Experiments

6 Discussion

Feature Visualization

Figure: Top 9 activations in a random subset of feature maps across thevalidation data

Feature Visualization

Feature Evolution during Training

Figure: Evolution of features through training. The visualization showsthe strongest activation for a given feature map at epochs[1,2,5,10,20,30,40,64]

Feature invariance

Figure: translation, scale, rotation invariance of CNN

Architecture selection

• (b) and (d) is obtained by visualizing AlexNet

• (b) : too many features are dead // (d) : Aliasing

• Fix it by adjusting filter size and stride of 1st layer

Occlusion Sensitivity

Figure: (a) Original image (b) Strongest feature map, layer5 (c)Visualization of b. (d) Prob. of correct class (e) Most probable class

CNN identify the location of the object in the image.

Correspondence Analysis

• Occlude same configuration (eyes and nose) and calculateHamming distance

• ∆ =∑

i ̸=j H(sign(ϵi ), sign(ϵj)), where ϵi = xi − x̃i

• Lower value indicates greater consistency in the changeresulting from the masking operation

1 Introduction

2 Visualization

3 Training Details

4 CNN Visualization

5 Experiments

6 Discussion

ImageNet 2012

Varying ImageNet Model Sizes

Feature Generalization - Caltech-101

• Foregoing results show that the features of CNN representunique topological properties.

• Use this image classifier trained by ImageNet 2012 to otherdifferent image datasets!

Caltech-256

PASCAL 2012

Feature Analysis

• Compare models having different number of layers

• As the feature hierarchies become deeper, they learnincreasingly powerful features!

1 Introduction

2 Visualization

3 Training Details

4 CNN Visualization

5 Experiments

6 Discussion

Discussion

• Introduce a novel way to cisualize the activity within themodel

• By visualizing the CNN, we improve the AlexNet

• CNN is highly sensitive to local structure

• Deep model shows good performance

• ImageNet trained model can generalize well to other datasets

• Shortcoming of our visualization : It only visualize a singleactivation, not the joint activity

Bibliography

• Krizhevsky, A., Sutskever, I., Hinton, G.: Imagenetclassification with deep convolutional neural networks.In:NIPS(2012)

• LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard,R.E., Hubbard, W., Jackel, L.D.: Backpropagation applied tohandwritten zip code recognition. Neural Comput. 1(4),541551 (1989)

• Zeiler, M., Taylor, G., Fergus, R.: Adaptive deconvolutionalnetworks for mid and high level feature learning. In: ICCV(2011)

Visualizing and Understanding Convolutional Networks and... · Visualizing and Understanding...

Documents

CONVOLUTIONAL CODES

LNCS 8689 - Visualizing and Understanding Convolutional Networksfergus/papers/zeilerECCV2014.pdf · 2014-09-10 · Visualizing and Understanding Convolutional Networks ... alization

DeepTracker: Visualizing the Training Process of ...huamin/tist_2018_dongyu_deeptracker.pdf · 1 DeepTracker: Visualizing the Training Process of Convolutional Neural Networks DONGYU

Convolutional Code

deepViz: Visualizing Convolutional Neural Networks for Image … · Deep learning, or the use of deep (i.e., many-layered) convolutional neural networks for machine recognition and

Convolutional Networks: Motivationsrihari/CSE676/9.2 CNN-Motivation.pdf · Deep Learning Srihari Overview of Convolutional Networks • Convolutional networks, also known as Convolutional

Visualizing and Understanding Convolutional Networks de faculteit/Afdelinge… · et al., 2006) and others (Bengio et al., 2007; Vincent et al., 2008). The generalization ability

Visualizing and Understanding Convolutional Networksgraphics.cs.cmu.edu/courses/16-824/2016_spring/slides/... · 2016-02-10 · •The components of convolutional neural networks

CDC: Convolutional-De-Convolutional Networks for Precise ...zs2262/files/research/cvpr17_CDC... · CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization

TensorViz: Visualizing the Training of Convolutional

Mattew West ( Yahoo ) at ro:newmedia 5.0

Robust Graph Convolutional Networks Against Adversarial ...pengcui.thumedialab.com/papers/RGCN.pdf · convolutional networks and graph adversarial attacks. 2.1 Graph Convolutional

Mattew SlideShow

Accelerating Convolutional Neural Network Systemshgrg1/publications/honours.pdf · 2015-07-29 · Accelerating Convolutional Neural Network Systems ... Convolutional Neural Networks

Convolutional Deep Belief Networks for Scalable ... · •Convolutional Restricted Boltzmann Machine –Probabilistic max-pooling •Convolutional Deep Belief Networks –Scalable

Sequentially Aggregated Convolutional Networksopenaccess.thecvf.com/content_ICCVW_2019/papers/... · art networks [10, 19, 14, 3], with its effectiveness further conﬁrmed by visualizing

Visualizing Convolutional Neural Networks to Improve ... · Visualizing Convolutional Neural Networks 5 (a) 6, 44 (b) 7, 33 Fig.3: Feature maps with high activations on skin types

Visualizing and Understanding Convolutional Networksqye/MA721/presentations/... · Visualizing and Understanding Convolutional Networks MatthewD. Zeiler, Rob Fergus Presented by Huan

Visualizing and Understanding Convolutional Neural ... - arXiv and Understanding Convolutional Neural ... - arXiv

CS4670/5670: Computer Vision - Cornell University[Zeiler and Fergus, “Visualizing and Understanding Convolutional Networks”, 2013] Convolutional Neural Networks CS 4670 Sean Bell