Deep learning for mere mortals - Devoxx Belgium 2015

@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning

Shallow Dive into Deep Learning

Sam Bessalah - @samklr


“ This talk is filled with buzzwords and terms, that might make you sound too nerdy or pedantic at dinner. ”

Use with caution.




In the news, lately ...







Skype Translator Demo https://www.youtube.com/watch?v=eu9kMIeS0wQ

https://www.youtube.com/watch?v=eu9kMIeS0wQ


Google DeepMind's Deep Q-learning playing Atari

https://www.youtube.com/watch?v=V1eYniJ0Rnk

https://www.youtube.com/watch?v=V1eYniJ0Rnk


Deep Learning?


Multiple Definitions :

- A set of algorithms that try to model high-level abstractions in data by using architectures composed of multiple non linear transformations.

- A set of machine learning algorithms that automatically learn features hierarchies.

- Usually built using Neural Networks.

- Representation Learning: Automatically learning good representations of the data for your classifier, i.e learn good features

- Deep Learning : Learning multiple levels of representation with

complex layers architectures.


Typical Machine Learning Workflow


Typical Machine Learning Workflow


A Neural Networks Primer


Loose imitation of the brain neurons


Loose imitation of the brain neurons

@YourTwitterHandle#Devoxx #YourTag @samklr#Devoxx #deepLearning

@nivdul

Perceptron


@nivdul

Perceptron

Output Error computed with E, and Network error

minimized with a stochastic gradient descent.


Stochastic gradient descent and Backpropagation

- The optimal value for each weight where at error achieves a global minimum.

- Backpropagation : Compute error in the output, then propagate it back through the network, to update the weights during the training phase.



@nivdulMulti-layer Perceptron


Problem with Large Networks

- Vanishing Gradient : More layers kills the back propagation. As information is passed back; gradient value startts to vanish and become smaller compared to weights.

- Overfitting : Algorithm fits too closely the training data, but will fail miserably on real examples



Problem with Large Networks

- Vanishing Gradient : More layers kills the back propagation. As information is passed back; gradient value startts to vanish and become smaller compared to weights.

- Overfitting : Algorithm fits too closely the training data, but will fail miserably on real examples

This led to another AI

winter



Why does work suddenly ?

- A lot more labeled data- More compute power CPU and GPU- Clever new idea on how to train deep

architectures


Convolution Neural Networks









Recurrent Neural Networks (RNN)

- Learn from arbitrary sequential inputs, by using their internal states

- Cannot look far back (back propagation limited)- Long Short Term Memory Networks help solving this- Good for NLP, hand writing, etc


Recurrent Neural Networks (RNN)


Deep Visual-Semantic Alignments for Generating Image Descriptions CNN + RNN

http://cs.stanford.edu/people/karpathy/cvpr2015.pdf


Other Techniques

- Auto Encoders

- Restricted Boltzmann Machine (RBM)

- Deep Belief

- Hierarchical temporal memory (HTM)

- LSTM

- More and more ...



Frameworks and Libraries





H20.ai Deep Learning


DeepLearning4j



Getting Started

- Coursera course on Neural Networks from G. Hinton

- Deeplearning4j.org

- Deeplearning.net


Deep Dream


CONCLUSION

Data & Analytics

Deep learning for mere mortals - Devoxx Belgium 2015