• Home

  • Documents

  • Fully Convolutional Networks for Semantic Segmentation [1] › ~yjlee › teaching ›...
14
Fully Convolutional Networks for Semantic Segmentation [1] Jonathan Long, Evan Shelhamer and Trevor Darrell ECE 289G: Paper Presentation Philipp Gysel

Fully Convolutional Networks for Semantic Segmentation [1] › ~yjlee › teaching › ecs289g... · Fully Convolutional Networks for Semantic Segmentation [1] Jonathan Long, Evan

  • Upload
    others

  • View
    0

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Fully Convolutional Networks for Semantic Segmentation [1] › ~yjlee › teaching › ecs289g... · Fully Convolutional Networks for Semantic Segmentation [1] Jonathan Long, Evan

Fully Convolutional Networks for Semantic Segmentation [1] Jonathan Long, Evan Shelhamer and Trevor Darrell

ECE 289G: Paper Presentation Philipp Gysel

Page 2: Fully Convolutional Networks for Semantic Segmentation [1] › ~yjlee › teaching › ecs289g... · Fully Convolutional Networks for Semantic Segmentation [1] Jonathan Long, Evan

ECE 289G Paper Presentation, Philipp Gysel Slide 2

Analyse Genome of C Elegans

1) Nucleus

2) Nucleus membrane

3) Cytoplasm

4) Cell wall

5) External medium

[2]

an

d h

ttp

://w

ww

.go

da

nd

scie

nce.o

rg

Page 3: Fully Convolutional Networks for Semantic Segmentation [1] › ~yjlee › teaching › ecs289g... · Fully Convolutional Networks for Semantic Segmentation [1] Jonathan Long, Evan

PASCAL VOC 2011

ECE 289G Paper Presentation, Philipp Gysel Slide 3

[3]

Person

Motorbike

Chair

Page 4: Fully Convolutional Networks for Semantic Segmentation [1] › ~yjlee › teaching › ecs289g... · Fully Convolutional Networks for Semantic Segmentation [1] Jonathan Long, Evan

Accuracy Metric

▪ Mean intersection over union (IU):

▪ Pixel Accuracy:

ECE 289G Paper Presentation, Philipp Gysel Slide 4

[1]

Prediction Ground truth

Page 5: Fully Convolutional Networks for Semantic Segmentation [1] › ~yjlee › teaching › ecs289g... · Fully Convolutional Networks for Semantic Segmentation [1] Jonathan Long, Evan

FCN: Fully Connected CNN

ECE 289G Paper Presentation, Philipp Gysel Slide 5

[1]

“Cat”

“Cat”

“Dog”

CNN:

FCN:

Page 6: Fully Convolutional Networks for Semantic Segmentation [1] › ~yjlee › teaching › ecs289g... · Fully Convolutional Networks for Semantic Segmentation [1] Jonathan Long, Evan

FCN Speedup

ECE 289G Paper Presentation, Philipp Gysel Slide 6

1.2

22

110

0

20

40

60

80

100

120

1 227x227 100 500x500 100 500x500

Runtime of FCN vs naïve CNNs

Inference [ms]

• Keep kernel sizes

and strides

• Replace dense

layer with

convolution

[1]

Page 7: Fully Convolutional Networks for Semantic Segmentation [1] › ~yjlee › teaching › ecs289g... · Fully Convolutional Networks for Semantic Segmentation [1] Jonathan Long, Evan

Upsampling: Backwards strided convolution

ECE 289G Paper Presentation, Philipp Gysel Slide 7

[1],

htt

ps:/

/de

ve

lop

er.

apple

.com

In-network upsampling

pixelwise loss

Page 8: Fully Convolutional Networks for Semantic Segmentation [1] › ~yjlee › teaching › ecs289g... · Fully Convolutional Networks for Semantic Segmentation [1] Jonathan Long, Evan

Deep jet

ECE 289G Paper Presentation, Philipp Gysel Slide 8

[1]

Page 9: Fully Convolutional Networks for Semantic Segmentation [1] › ~yjlee › teaching › ecs289g... · Fully Convolutional Networks for Semantic Segmentation [1] Jonathan Long, Evan

Experiment setup

▪ Pre-trained networks: AlexNet, VGG and GoogLeNet

▪ Convert dense to convolutional layer

▪ Discard final classifier

▪ Add deconvolution layer for up-sampling

▪ Fine-tuning end-to-end on

▪ PASCAL VOC

▪ NYUDv2

▪ SIFT

ECE 289G Paper Presentation, Philipp Gysel Slide 9

[4],

[5

]

Page 10: Fully Convolutional Networks for Semantic Segmentation [1] › ~yjlee › teaching › ecs289g... · Fully Convolutional Networks for Semantic Segmentation [1] Jonathan Long, Evan

Experiment #1: PASCAL VOC 2011

▪ 20 classes (e.g. airplane, boat, bicycle, person, cat)

▪ Relative margin of 20% to previous state-of-art

▪ Inference time is reduced 114x and 286x respectively

ECE 289G Paper Presentation, Philipp Gysel Slide 10

[1],

[3

]

Page 11: Fully Convolutional Networks for Semantic Segmentation [1] › ~yjlee › teaching › ecs289g... · Fully Convolutional Networks for Semantic Segmentation [1] Jonathan Long, Evan

Experiment #2: NYUDv2

▪ 40 classes from indoor scenes

▪ Densely labeled RGB and depth images

ECE 289G Paper Presentation, Philipp Gysel Slide 11

[1],

htt

p:/

/cs.n

yu

.ed

u/~

silb

erm

an/d

ata

sets

/nyu

_depth

_v2

.htm

l

Page 12: Fully Convolutional Networks for Semantic Segmentation [1] › ~yjlee › teaching › ecs289g... · Fully Convolutional Networks for Semantic Segmentation [1] Jonathan Long, Evan

Experiment #3: SIFT Flow

▪ 33 semantic categories (e.g. bridge, mountain)

▪ 3 geometric categories (horizontal, vertical)

▪ Two-headed FCN

ECE 289G Paper Presentation, Philipp Gysel Slide 12

[1]

Page 13: Fully Convolutional Networks for Semantic Segmentation [1] › ~yjlee › teaching › ecs289g... · Fully Convolutional Networks for Semantic Segmentation [1] Jonathan Long, Evan

Questions?

ECE 289G Paper Presentation, Philipp Gysel Slide 13

Page 14: Fully Convolutional Networks for Semantic Segmentation [1] › ~yjlee › teaching › ecs289g... · Fully Convolutional Networks for Semantic Segmentation [1] Jonathan Long, Evan

References

[1] Long, Jonathan, Evan Shelhamer, and Trevor Darrell. "Fully convolutional networks for semantic segmentation." arXiv preprint arXiv:1411.4038 (2014).

[2] Ning, Feng, et al. "Toward automatic phenotyping of developing embryos from videos." Image Processing, IEEE Transactions on 14.9 (2005): 1360-1371.

[3] Everingham, Mark, et al. "The pascal visual object classes (voc) challenge." International journal of computer vision 88.2 (2010): 303-338.

[4] Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. "Imagenet classification with deep convolutional neural networks." Advances in neural information processing systems. 2012.

[5] Szegedy, Christian, et al. "Going deeper with convolutions." arXiv preprint arXiv:1409.4842 (2014).

ECE 289G Paper Presentation, Philipp Gysel Slide 14


Cell nuclei segmentation using deep convolutional ... - SINTEF

Cell nuclei segmentation using deep convolutional ... - SINTEF

Documents
Few-Shot Semantic Segmentation with Prototype Learning · 2018. 8. 6. · in the few-shot semantic segmentation.2 Related Work. Semantic Segmentation. Semantic segmentation is the

Few-Shot Semantic Segmentation with Prototype Learning · 2018. 8. 6. · in the few-shot semantic segmentation.2 Related Work. Semantic Segmentation. Semantic segmentation is the

Documents
Introduction to Semantic Segmentation using …machinelearning.math.rs/Radovic-SemanticSegmentation.pdfSemantic segmentation - problem definition The goal of semantic segmentation

Introduction to Semantic Segmentation using …machinelearning.math.rs/Radovic-SemanticSegmentation.pdfSemantic segmentation - problem definition The goal of semantic segmentation

Documents
Medical Image Segmentation using 3D Convolutional Neural

Medical Image Segmentation using 3D Convolutional Neural

Documents
Future Semantic Segmentation with Convolutional LSTM {Sutton … · 2018-07-23 · focuses on using temporal correlation to improve current frame semantic segmentation sta-bility

Future Semantic Segmentation with Convolutional LSTM {Sutton … · 2018-07-23 · focuses on using temporal correlation to improve current frame semantic segmentation sta-bility

Documents
1 Fully Convolutional Networks for Semantic Segmentation ·  · 2016-05-231 Fully Convolutional Networks for Semantic Segmentation Evan Shelhamer , Jonathan Long , and Trevor Darrell,

1 Fully Convolutional Networks for Semantic Segmentation ·  · 2016-05-231 Fully Convolutional Networks for Semantic Segmentation Evan Shelhamer , Jonathan Long , and Trevor Darrell,

Documents
Fully Convolutional Networks for Semantic Segmentationjhoffman/yahooJapan... · 2016. 3. 25. · Fully Convolutional Networks for Semantic Segmentation Jonathan Long* Evan Shelhamer*

Fully Convolutional Networks for Semantic Segmentationjhoffman/yahooJapan... · 2016. 3. 25. · Fully Convolutional Networks for Semantic Segmentation Jonathan Long* Evan Shelhamer*

Documents
Hand segmentation with structured convolutional learning · Hand segmentation with structured convolutional learning 1;2Natalia Neverova 1;2Christian Wolf 3Graham W. Taylor 4Florian

Hand segmentation with structured convolutional learning · Hand segmentation with structured convolutional learning 1;2Natalia Neverova 1;2Christian Wolf 3Graham W. Taylor 4Florian

Documents
3D Shape Segmentation With Projective Convolutional …openaccess.thecvf.com/content_cvpr_2017/papers/...3D Shape Segmentation with Projective Convolutional Networks Evangelos Kalogerakis1

3D Shape Segmentation With Projective Convolutional …openaccess.thecvf.com/content_cvpr_2017/papers/...3D Shape Segmentation with Projective Convolutional Networks Evangelos Kalogerakis1

Documents
Semantic segmentation of mFISH images using convolutional … · 2018. 5. 4. · arXiv:1805.01220v1 [cs.CV] 3 May 2018 Semantic segmentation of mFISH images using convolutional networks

Semantic segmentation of mFISH images using convolutional … · 2018. 5. 4. · arXiv:1805.01220v1 [cs.CV] 3 May 2018 Semantic segmentation of mFISH images using convolutional networks

Documents
Lecture 7: Semantic Segmentation - POSTECHLecture 7: Semantic Segmentation BohyungHan Computer Vision Lab. ... J. Long, E. Shelhamer, and T. Darrell, Fully Convolutional Network for

Lecture 7: Semantic Segmentation - POSTECHLecture 7: Semantic Segmentation BohyungHan Computer Vision Lab. ... J. Long, E. Shelhamer, and T. Darrell, Fully Convolutional Network for

Documents
Problem: Semantic Segmentation

Problem: Semantic Segmentation

Documents
Semi-supervised semantic segmentation needs strong, varied … · 2020. 9. 3. · FRENCH ET AL.: SEMI-SUPERVISED SEMANTIC SEGMENTATION 1 Semi-supervised semantic segmentation needs

Semi-supervised semantic segmentation needs strong, varied … · 2020. 9. 3. · FRENCH ET AL.: SEMI-SUPERVISED SEMANTIC SEGMENTATION 1 Semi-supervised semantic segmentation needs

Documents
Image segmentation and convolutional neural networks as ...publications.lib.chalmers.se/records/fulltext/242058/242058.pdf · Image segmentation and convolutional neural networks

Image segmentation and convolutional neural networks as ...publications.lib.chalmers.se/records/fulltext/242058/242058.pdf · Image segmentation and convolutional neural networks

Documents
Generating 3D City Models Based on the Semantic Segmentation … · 2019-09-22 · Generating 3D City Models Based on the Semantic Segmentation of LiDAR Data using Convolutional Neural

Generating 3D City Models Based on the Semantic Segmentation … · 2019-09-22 · Generating 3D City Models Based on the Semantic Segmentation of LiDAR Data using Convolutional Neural

Documents
Johnson2008 Semantic Segmentation

Johnson2008 Semantic Segmentation

Documents
UC Berkeley Fully Convolutional Networks for Semantic Segmentation Jonathan Long* Evan Shelhamer* Trevor Darrell 1 Presented by: Gordon Christie Slide

UC Berkeley Fully Convolutional Networks for Semantic Segmentation Jonathan Long* Evan Shelhamer* Trevor Darrell 1 Presented by: Gordon Christie Slide

Documents
Semantic Segmentation of Small Objects and Modeling … Segmentation of Small Objects and Modeling of Uncertainty in Urban Remote Sensing Images Using Deep Convolutional Neural Networks

Semantic Segmentation of Small Objects and Modeling … Segmentation of Small Objects and Modeling of Uncertainty in Urban Remote Sensing Images Using Deep Convolutional Neural Networks

Documents
Fully Convolutional Networks for Semantic Segmentationai.ms.mff.cuni.cz/~sui/leibl.pdf · 2018. 1. 17. · Presentation Outline Introduction... convolutional neural networks Semantic

Fully Convolutional Networks for Semantic Segmentationai.ms.mff.cuni.cz/~sui/leibl.pdf · 2018. 1. 17. · Presentation Outline Introduction... convolutional neural networks Semantic

Documents
Taxonomy-Regularized Semantic Deep Convolutional Neural Networksvision.snu.ac.kr/assets/papers/eccv16_taxonomy.pdf · 2018-01-16 · Taxonomy-Regularized Semantic Deep Convolutional

Taxonomy-Regularized Semantic Deep Convolutional Neural Networksvision.snu.ac.kr/assets/papers/eccv16_taxonomy.pdf · 2018-01-16 · Taxonomy-Regularized Semantic Deep Convolutional

Documents
Tumor Semantic Segmentation in Hyperspectral Images ......Tumor Semantic Segmentation in HSI using Deep Learning et al.,2017) applied convolutional network with leaving-one-patient-out

Tumor Semantic Segmentation in Hyperspectral Images ......Tumor Semantic Segmentation in HSI using Deep Learning et al.,2017) applied convolutional network with leaving-one-patient-out

Documents
Brain Tumor Segmentation Using Convolutional Neural Network

Brain Tumor Segmentation Using Convolutional Neural Network

Documents
Fully Convolutional Networks for Semantic Segmentation [1]web.cs.ucdavis.edu/~yjlee/teaching/ecs289g-fall2015/...References [1] Long, Jonathan, Evan Shelhamer, and Trevor Darrell

Fully Convolutional Networks for Semantic Segmentation [1]web.cs.ucdavis.edu/~yjlee/teaching/ecs289g-fall2015/...References [1] Long, Jonathan, Evan Shelhamer, and Trevor Darrell

Documents
1 Fully Convolutional Networks for Semantic Segmentation · Fully Convolutional Networks for Semantic Segmentation Evan Shelhamer , Jonathan Long , and Trevor Darrell, Member, IEEE

1 Fully Convolutional Networks for Semantic Segmentation · Fully Convolutional Networks for Semantic Segmentation Evan Shelhamer , Jonathan Long , and Trevor Darrell, Member, IEEE

Documents
Fully Convolutional Networks for Semantic Segmentationweb.cs.ucdavis.edu/~yjlee/teaching/ecs289g-fall2016/zilong.pdf · Fully Convolutional Networks for Semantic Segmentation By Jonathan

Fully Convolutional Networks for Semantic Segmentationweb.cs.ucdavis.edu/~yjlee/teaching/ecs289g-fall2016/zilong.pdf · Fully Convolutional Networks for Semantic Segmentation By Jonathan

Documents
SEMANTIC SEGMENTATION OF RAILWAY TRACK ......SEMANTIC SEGMENTATION OF RAILWAY TRACK IMAGES WITH DEEP CONVOLUTIONAL NEURAL NETWORKS Xavier Gibert, Vishal M. Patel, and Rama Chellappa

SEMANTIC SEGMENTATION OF RAILWAY TRACK ......SEMANTIC SEGMENTATION OF RAILWAY TRACK IMAGES WITH DEEP CONVOLUTIONAL NEURAL NETWORKS Xavier Gibert, Vishal M. Patel, and Rama Chellappa

Documents
Dual Graph Convolutional Network for Semantic Segmentation ... · ZHANG ET AL.: DUAL GRAPH CONVOLUTIONAL NETWORK 3. 2 Related Work. Following the success of deep neural networks for

Dual Graph Convolutional Network for Semantic Segmentation ... · ZHANG ET AL.: DUAL GRAPH CONVOLUTIONAL NETWORK 3. 2 Related Work. Following the success of deep neural networks for

Documents
SCENE SEMANTIC SEGMENTATION FROM INDOOR · PDF fileSCENE SEMANTIC SEGMENTATION FROM ... we propose an Encode-Decoder Fully Convolutional Networks for RGB-D image classification.

SCENE SEMANTIC SEGMENTATION FROM INDOOR · PDF fileSCENE SEMANTIC SEGMENTATION FROM ... we propose an Encode-Decoder Fully Convolutional Networks for RGB-D image classification.

Documents
A Deep Convolutional Neural Network for Semantic Pixel ......for semantic segmentation of road and pavement surface cracks using a Convolutional Neural Network, namely U-Net. The algorithm

A Deep Convolutional Neural Network for Semantic Pixel ......for semantic segmentation of road and pavement surface cracks using a Convolutional Neural Network, namely U-Net. The algorithm

Documents
High Dimensional Convolutional Neural Networks · 2020-03-23 · Very deep convolutional neural networks possible in 3D 42-layer deep neural networks for semantic segmentation 101

High Dimensional Convolutional Neural Networks · 2020-03-23 · Very deep convolutional neural networks possible in 3D 42-layer deep neural networks for semantic segmentation 101

Documents