Embed Size (px)
Citation preview
13/08/15
1
A Short(er) Introduc0on To Deep Learning Dr. Brian Mac Namee University College Dublin
13/08/15
2
Deep Learning
Google Trends: http://www.google.com/trends/
2005 2007 2009 2011 2013 2015
Kaggle Digit Recogniser Contest https://www.kaggle.com/c/digit-recognizer
MNIST Dataset from Yan LeCun http://yann.lecun.com/exdb/mnist/index.html
13/08/15
3
The standard approach to using machine learning to build a system to recognise these different digits is to first engineer a high-level respresentation
Percent filled: 0.37 Number of loops: 2 Direction 0: 0.2 Direction 1: 0.6 Direction 2: 0.1 Direction 3: 0.4
Percent filled: 0.11 Number of loops: 0 Direction 0: 0.1 Direction 1: 0.4 Direction 2: 0.0 Direction 3: 0.5
0 1
2
3
Percent filled: 0.29 Number of loops: 1 Direction 0: 0.2 Direction 1: 0.5 Direction 2: 0.4 Direction 3: 0.2
The standard approach to using machine learning to build a system to recognise these different digits is to first engineer a high-level respresentation
Percent filled: 0.37 Number of loops: 2 Direction 0: 0.2 Direction 1: 0.6 Direction 2: 0.1 Direction 3: 0.4
Percent filled: 0.11 Number of loops: 0 Direction 0: 0.1 Direction 1: 0.4 Direction 2: 0.0 Direction 3: 0.5
0 1
2
3
Percent filled: 0.29 Number of loops: 1 Direction 0: 0.2 Direction 1: 0.5 Direction 2: 0.4 Direction 3: 0.2
Using this reperesentation (6 features) we could train a decision tree that would manage to correctly
recognise about 8 out of every 10 digits
13/08/15
4
Engineering representa0ons, is one of the most important and >me consuming jobs in most
predic>ve analy>cs projects, and needs a blend of technical exper>se and domain exper>se
Representa0on learning is a set of methods that allows a machine to be fed with raw data and to
automa>cally discover the representa>ons needed for detec>on or classifica>on
[LeCun etal, 2014] Deep Learning Yann LeCun, Yoshua Bengio & Geoffrey Hinton http://www.nature.com/nature/journal/v521/n7553/full/nature14539.html
Rosenbla='s perceptron from 1957 was the earliest example of representa0on learning, and the first neural network
13/08/15
5
Each image is composed of 28 x 28 = 784 pixels each
containing a grayscale value between 0 and 255
13/08/15
6
This low-level representation uses a vector of 784 features, each with values
between 0 and 255
13/08/15
7
0
1 2
3
4 5
6 7
8
9
A simple representation learning
approach to the digit recognition problem could use a multi-
layer perceptron to make predictions using
the low-level representation
0
1 2
3
4 5
6 7
8
9
13/08/15
8
0
1 2
3
4 5
6 7
8
9
This neural network could manage to
correctly recognise about 9 out of every 10 digits
Deep-‐learning methods are representa0on-‐learning methods with mul>ple levels of
representa>on, obtained by composing simple but non-‐linear modules that each transform the representa>on at one level (star>ng with the raw input) into a representa>on at a higher, slightly
more abstract level. [LeCun etal, 2014]
Deep Learning Yann LeCun, Yoshua Bengio & Geoffrey Hinton http://www.nature.com/nature/journal/v521/n7553/full/nature14539.html
13/08/15
9
0 1 2 3 4 5 6 7 8
9
0 1 2 3 4 5 6 7 8
9
A deep learning appraoch could
manage to correctly recognise about 10
out of every 10 digits
13/08/15
10
Deep neural networks seem to brilliantly address the selec0vity-‐invariance dilemma that is fundamental to all efforts to learn to classify objects: they produce representa>ons that are selec>ve to the aspects of the image that are important for discrimina>on, but that are
invariant to irrelevant aspects
Deep networks hold records for problems in image recogni0on, speech recogni0on, and text
classifica0on amongst other areas
Hardware Data Algorithms
Applica>ons
13/08/15
11
Thank You
Questions? Fundamentals of Machine Learning for Predictive Data Analytics: Algorithms, Worked Examples, and Case Studies John D. Kelleher, Brian Mac Namee and Aoife D'Arcy www.machinelearningbook.com
