View
15
Download
1
Category
Preview:
Citation preview
An NVIDIA Tech Talk @
Boston Imaging and Vision Meetup
Thursday, October 22nd, 2015
Accelerated Deep Learning With GPUs
2
PC DATA CENTER MOBILE
DESKTOP VIRTUALIZATION
AUTONOMOUS MACHINES
HPC & CLOUD SERVICE PROVIDERS GAMING DESIGN
The World Leader in Visual Computing
3
Academic Collaboration
https://developer.nvidia.com/academia
4
CUDA Zone
5
Accelerated Deep Learning with GPUs
6
• Data Science & Deep Learning Overview
• The Promise of Machine Learning
• What Makes Deep Learning Deep?
• Why Is Deep Learning Hot Now? AGENDA
7
Data science landscape (simplified)
• Regression • SVM • Recommendation systems
Data Analytics
Machine
Learning Graph Analytics SQL Query
Traditional
Methods
Deep Neural
Networks
8
How GPU Acceleration Works (highly simplified)
Application Code
+
GPU CPU 5% of Code
Compute-Intensive Functions
Rest of Sequential CPU Code
~ 80% of run-time
9
3 Drivers for Deep Learning
More Data Better Models Powerful GPU Accelerators
10
“Machine Learning” (ML) is in some sense a rebranding of AI.
The focus is now on more specific, often perceptual tasks, and there are many successes.
Today, some of the world’s largest internet companies, as well as the foremost research institutions, are using GPUs for machine learning.
GPU Accelerated Deep Learning CUDA for
Deep Learning
11
Neural Networks
GPUs
Inherently Parallel
Matrix Operations
FLOPS
Why are GPUs Useful for Deep Learning?
0 0 4
60
110 28%
26%
16%
12%
7%
2010 2011 2012 2013 2014
bird
frog
person
dog
chair
GPUs deliver —
Same or better prediction accuracy
Faster results
Smaller footprint
Lower power
12
Deep learning with COTS HPC systems
A. Coates, B. Huval, T. Wang, D. Wu, A. Ng, B. Catanzaro
ICML 2013
GOOGLE DATACENTER
1,000 CPU Servers 2,000 CPUs • 16,000
cores
600 kWatts
$5,000,00
0
STANFORD AI LAB
3 GPU-Accelerated Servers 12 GPUs • 18,432 cores
4 kWatts
$33,000
Now You Can Build Google’s
$1M Artificial Brain on the Cheap
“ “ GPUs Make Deep Learning Accessible
13
NVIDIA & IBM Cloud Support
“NVIDIA is excited to announce we have teamed up with IBM Cloud to provide qualifying competitors with access to the SoftLayer cloud server infrastructure. The servers are outfitted with dual Intel Xeon E5-2690 CPUs, 128GB RAM, two 1TB SATA HDD/RAID 0, and two NVIDIA Tesla K80s, NVIDIA’s fastest dual-GPU accelerators.” NVIDIA Parallel For All Blog, August 13th, 2015
14
The Promise of Machine Learning ML Systems Extract Value From Big Data
350 millions images uploaded per day
2.5 Petabytes of customer data hourly
100 hours of video uploaded every minute
15
“Delving Deep into Rectifiers: Surpassing Human-Level
Performance on ImageNet Classification”
— Microsoft: 4.94%, Feb. 6, 2015
“Deep Image: Scaling up Image Recognition”
— Baidu: 5.98%, Jan. 13, 2015
“Batch Normalization: Accelerating Deep Network
Training by Reducing Internal Covariant Shift”
— Google: 4.82%, Feb. 11, 2015
IMAGENET
CHALLENGE
Accuracy %
2010 2014 2012 2011 2013
74%
84%
DNN
CV
72%
16
The Early Big Data Narrative
2.5 Exabytes of Web Data Created Daily 2.5 Petabytes of Customer Data Hourly
350 Million Images Uploaded a Day 100 Hours Video Uploaded Every Minute
How can we organize, analyze, understand, and
benefit from such a trove of data?
?
TRAINING DATA
The promise of machine learning
LIVE DATA PREDICTION ANALYSIS
MACHINE LEARNING
Walmart • 2.5 PB of customer transaction
data every hr
Facebook • 340 million photos uploaded
every day
Orange France • >1 billion call data records every
month
18
Image Classification, Object Detection, Localization
Facial Recognition Speech & Natural Language
Processing
Medical Imaging & Interpretation
Seismic Imaging & Interpretation Recommendation
Machine Learning Use Cases …machine learning is pervasive!
19
Traditional ML – Hand Tuned Features
Image Vision features Detection
Images/video
Audio sample Audio features Speaker ID
Audio
Text
Text Text features
Text classification, Machine
translation, Information
retrieval, ....
Slide courtesy of Andrew Ng, Stanford University
20
What is Deep Learning? Systems that learn to recognize objects which are important, without us telling the system explicitly what the object is ahead of time
Key components
Task
Features
Model
Learning Algorithm
21
Deep Learning Advantages
Don’t have to figure out the features ahead of time
Use same neural net approach for many different problems
Fault tolerant
Scales well
Simplicity & Scalability
Linear classifier
Regression
Support Vector Machine Bayesian
Decision Trees Association Rules
Clustering
22
Convolutional Neural Networks (CNNs)
Biologically inspired
Neuron only connected to a small region of neurons in layer below it called the receptive field A given layer can have many convolutional filters/kernels Each filter has the same weights across the whole layer Bottom layers are convolutional, top layers are fully connected
Generally trained via supervised learning Supervised Unsupervised Reinforcement …ideal system automatically switches modes…
23
Convolutional Networks Breakthrough
Y. LeCun et al. 1989-1998 : Handwritten digit reading
A. Krizhevsky, G. Hinton et al. 2012 : Imagenet classification winner
24
Training
Image Classification with DNNs
Cars Buses Trucks Motor cycles
Truck
Inference
25
Image Classification with DNNs Training
Typical training run
Pick a DNN design
Input 100 million training images spanning 1,000 categories
One week of computation
Test accuracy
If bad: modify DNN, fix training set or update training parameters
Cars Buses Trucks Motor cycles
26
What makes Deep Learning deep?
Input Result
Hinton et al., 2006; Bengio et al., 2007; Bengio & LeCun, 2007; Lee et al., 2008; 2009
Visual Object Recognition Using Deep Convolutional Neural Networks
Rob Fergus (New York University / Facebook) http://on-demand-gtc.gputechconf.com/gtcnew/on-demand-gtc.php#2985
Networks can have ~10 layers 1B parameters 10M images ~30 Exaflops ~30 GPU days Human brain has trillions of parameters.
27
Tree
Cat
Dog
Deep Learning Framework
“turtle”
Forward Propagation
Compute weight update to nudge
from “turtle” towards “dog”
Backward Propagation
Trained Model
“dog”
Repeat
Training
Classification
28
CNNs dominate in perceptual tasks
Slide credit: Yann Lecun, Facebook & NYU
29
Why is Deep Learning Hot Now ?
30
ConvNet Benchmark comments from github
“…9 months ago, we were ~3x slower on Alex net and ~4x slower on overfeat. Training that took 3 weeks, takes 1 week (on the same 1-GPU metric). That is a huge fundamental speedup results in several man-hours saved in waiting for experiments to finish.”
“Pushing these boundaries so far, in such a short time-frame is quite something. There’s two sets of teams who have made this happen:
• NVIDIA, with their Maxwell cards that are as fast as #*%!
• Nervana Systems (Scott Gray and team) who have pushed the CUDA kernels to the limits of the GPUs with efficiencies > 95%”
“Rejigging the marks… #56” – Soumith Chintala, August 2015 Artificial Intelligence Research Engineer at Facebook
31
Why is Deep Learning Hot Now? Three Driving Factors…
Big Data Availability New ML Techniques Compute Density
350 millions images uploaded per day
2.5 Petabytes of customer data hourly
100 hours of video uploaded every minute
Deep Neural Networks GPUs
ML systems extract value from Big Data
32
Deep Learning Revolutionizing Medical Research
Detecting Mitosis in
Breast Cancer Cells
— IDSIA
Predicting the Toxicity
of New Drugs
— Johannes Kepler University
Understanding Gene Mutation
to Prevent Disease
— University of Toronto
33
ILSVRC12 winning model: “Supervision”
7 layers
5 convolutional layers + 2 fully-connected
ReLU, pooling, drop-out, response normalization
Implemented with Caffe
GPU Acceleration Training a Deep, Convolutional Neural Network
Batch Size Training Time
CPU Training Time
GPU GPU
Speed Up
64 images 64 s 7.5 s 8.5X
128 images 124 s 14.5 s 8.5X
256 images 257 s 28.5 s 9.0X
Dual 10-core Ivy Bridge CPUs
1 Tesla K40 GPU
CPU times utilized Intel MKL BLAS library
GPU acceleration from CUDA matrix libraries (cuBLAS)
34
GPU-Accelerated Deep Learning Frameworks
CAFFE TORCH THEANO CUDA-
CONVNET2 KALDI
Domain Deep Learning
Framework
Scientific Computing
Framework
Math Expression
Compiler
Deep Learning
Application
Speech Recognition
Toolkit
cuDNN R2 R2 R2 -- --
Multi-GPU In Progress Partial Partial (nnet2)
Multi-CPU (nnet2)
License BSD-2 GPL BSD Apache 2.0 Apache 2.0
Interface(s)
Text-based
definition files,
Python, MATLAB
Python, Lua,
MATLAB Python C++ C++, Shell scripts
Embedded (TK1)
http://developer.nvidia.com/deeplearning
35
36
Deep Learning Use Cases for Vision & Graphics
37
Street Number Detection
[Goodfellow 2014]
38
Object Classification
[Krizhevsky 2012]
39
Image Retrieval
[Krizhevsky 2012]
40
Pose Estimation
[Toshev, Szegedy 2014]
41
Object Detection
[Toshev, Szegedy 2014]
42
Object Detection
[Huval et al. 2015]
43
Face Recognition
[Taigman et al. 2014]
44
Action Recognition
[Simonyan et al. 2014]
Thank You!
Recommended