Upload
others
View
9
Download
0
Embed Size (px)
Citation preview
Convolutional Neural Networks in MATLAB
28 March 2019
Welcome
• Danielle Winter
Application Engineer – AI and Data Science in Engineering
• Adri van Nieuwkerk
Business Development Consultant - Education
Copyright 2018 Opti-Num Solutions (Pty) Ltd 2
Why are we here today?
Copyright 2018 Opti-Num Solutions (Pty) Ltd 3
Shell
Musashi Seimitsu Industry Co.,Ltd.
Why are we here today?
• Feature extraction is painful
Copyright 2018 Opti-Num Solutions (Pty) Ltd 4
Why are we here today?
Copyright 2018 Opti-Num Solutions (Pty) Ltd 5
• Deep learning is getting more accurate
Why are we here today?
Copyright 2018 Opti-Num Solutions (Pty) Ltd 6
• Deep Learning is Cool!
Agenda
• Introduction to concepts
– Deep Learning checklist
– Convolutional Neural Networks and layer architecture
• CNNs for non-image applications
• Some challenges and how to mitigate them
• Take Aways
Copyright 2018 Opti-Num Solutions (Pty) Ltd 7
Agenda
• Introduction to concepts
– Deep Learning checklist
– Convolutional Neural Networks and layer architecture
• CNNs for non-image applications
• Some challenges and how to mitigate them
• Take Aways
Copyright 2018 Opti-Num Solutions (Pty) Ltd 8
Deep Learning Workflow
Copyright 2018 Opti-Num Solutions (Pty) Ltd 9
validation data
testing data
DataNetwork Architecture• Pretrained• Transfer Learning• From Scratch
Training Options• Algorithm• Learning Rate• Mini-Batch• Validation• Environment
Data
Training options
Network Layers
Deep Learning Checklist
Car Tree Dog Cat Bus Duck
Deep Learning Workflow
Copyright 2018 Opti-Num Solutions (Pty) Ltd 10
validation data testing data
Convolutional Neural Networks
Copyright 2018 Opti-Num Solutions (Pty) Ltd 11
CNN Architecture – Input Layer
Copyright 2018 Opti-Num Solutions (Pty) Ltd 12
imageInputLayer([5,5])Image size: m*n*a
Colour image: a = 3Multispectral image: a = many
CNN Architecture – Middle Layers
• Convolution of spatial filters with input image
Copyright 2018 Opti-Num Solutions (Pty) Ltd 13
convolution2dLayer([2,2],50)
CNN Architecture – Middle Layers
• Convolution of spatial filters with input image
Copyright 2018 Opti-Num Solutions (Pty) Ltd 14
convolution2dLayer([2,2],50) activations(net, X, layer)
CNN Architecture – Middle Layers
Copyright 2018 Opti-Num Solutions (Pty) Ltd 15
reluLayer maxPooling2dLayer([3,3])
𝑓 𝑥 = ቊ𝑥, 𝑥 ≥ 00, 𝑥 < 0
CNN Architecture – Final Layers
Copyright 2018 Opti-Num Solutions (Pty) Ltd 16
fullyConnectedLayer(2) softmaxLayer()
classificationLayer()
Agenda
• Introduction to concepts
– Deep Learning checklist
– Convolutional Neural Networks and layer architecture
• CNNs for non-image applications
• Some challenges and how to mitigate them
• Take Aways
Copyright 2018 Opti-Num Solutions (Pty) Ltd 17
CNNs for Non-Image Applications
Copyright 2018 Opti-Num Solutions (Pty) Ltd 18
CNNs for Non-Image Applications
Copyright 2018 Opti-Num Solutions (Pty) Ltd 19
Agenda
• Introduction to concepts
– Deep Learning checklist
– Convolutional Neural Networks and layer architecture
• CNNs for non-image applications
• Some challenges and how to mitigate them
• Take Aways
Copyright 2018 Opti-Num Solutions (Pty) Ltd 20
Some Challenges
• Building layer architecture
Copyright 2018 Opti-Num Solutions (Pty) Ltd 21
Some Challenges
• Building layer architecture
• Training Time
– Use GPU
Copyright 2018 Opti-Num Solutions (Pty) Ltd 22
Some Challenges
• Building layer architecture
• Training Time
– Use GPU
• Getting better performance
– Learning rate
– Epochs
– Amount of data
Copyright 2018 Opti-Num Solutions (Pty) Ltd 23
NaNs/ spike in
loss?
↑
learn rate
↓ learn rate
Loss ↓ at end of
training?
↑ Epochs↓ learn rate at plateau
Training loss >
validation loss?Augment data,
↓ epochs
YN
Y
Y
N
Agenda
• Introduction to concepts
– Deep Learning checklist
– Convolutional Neural Networks and layer architecture
• CNNs for non-image applications
• Some challenges and how to mitigate them
• Take Aways
Copyright 2018 Opti-Num Solutions (Pty) Ltd 24
Take-Aways
• Growing the Deep Learning Community in South Africa
– Application-based research
• Deep Learning in MATLAB
– Deep Learning Onramp Tutorial
– MATLAB Onramp Tutorial
Copyright 2018 Opti-Num Solutions (Pty) Ltd 25
Q&A
Copyright 2018 Opti-Num Solutions (Pty) Ltd 26
Questions & Answers