14
Journée ROMEO, 9 juin 2016 Frédéric Parienté, Tesla Accelerated Computing, NVIDIA Corporation LA RÉVOLUTION DU DEEP LEARNING

The Revolution of Deep Learning

Embed Size (px)

Citation preview

Page 1: The Revolution of Deep Learning

Journée ROMEO, 9 juin 2016

Frédéric Parienté, Tesla Accelerated Computing, NVIDIA Corporation

LA RÉVOLUTION DU DEEP LEARNING

Page 2: The Revolution of Deep Learning

Deep Learning Achieves “Superhuman” Results

A NEW COMPUTING MODEL

Traditional Computer VisionExperts + Time

Deep Learning Object DetectionDNN + Data + HPC

ImageNet

Page 3: The Revolution of Deep Learning

KEY DRIVERS FOR DEEP LEARNING

Big Data GPU AccelerationBetter Algorithms

350 millions images uploaded per day

2.5 Petabytes of customer data hourly

300 hours of video uploaded every minute

“The Three Breakthroughs that have Finally Unleashed A.I. on the World”

Page 4: The Revolution of Deep Learning

NVIDIA DEEP LEARNING SDKHigh Performance GPU-Acceleration for Deep Learning

COMPUTER VISION SPEECH AND AUDIO BEHAVIOR

Object Detection Voice Recognition TranslationRecommendation

EnginesSentiment Analysis

DEEP LEARNING

cuDNN

MATH LIBRARIES

cuBLAS cuSPARSE

MULTI-GPU

NCCL

cuFFT

Mocha.jl

Image Classification

DEEP LEARNING

SDK

FRAMEWORKS

APPLICATIONS

Page 5: The Revolution of Deep Learning

EVERY INDUSTRY WANTS DEEP LEARNINGCloud Service Provider Medicine Media & Entertainment Security & Defense Autonomous Machines

Cancer cell detection

Diabetic grading

Drug discovery

Pedestrian detection

Lane tracking

Recognize traffic sign

Face recognition

Video surveillance

Cyber security

Video captioning

Content based search

Real time translation

Image/Video classification

Speech recognition

Natural language processing

Page 6: The Revolution of Deep Learning

AUTONOMOUS DRIVING

https://www.youtube.com/watch?v=qhUvQiKec2U

Page 7: The Revolution of Deep Learning

ROBOTICS

https://www.youtube.com/watch?v=JeVppkoloXs

Page 8: The Revolution of Deep Learning

SPEECH RECOGNITION

https://www.youtube.com/watch?v=l5eparXPc7Q

Page 9: The Revolution of Deep Learning

ART

https://www.youtube.com/watch?v=Khuj4ASldmU

Page 10: The Revolution of Deep Learning

MEDICAL

https://www.youtube.com/watch?v=BmH5_BQUYT0

Page 11: The Revolution of Deep Learning

BAIDU: DL DEVELOPERS NEED HPC

“Investments in computer systems — and I think the bleeding-edge of AI, and deep learning specifically, is shifting to HPC (high performance computing) — can cut down the time to run an experiment, and therefore go around the circle, from a week to a day and sometimes even faster.”

“Those of us that grew up doing machine learning often didn’t grow up with an HPC or computer systems background … partnerships between machine learning researchers and computer systems researchers tend to help both teams drive a lot of machine learning progress.”

— Andrew Ng

Page 12: The Revolution of Deep Learning

MICROSOFT: “SUPER DEEP NETWORKS”

Microsoft Deep ResNet

http://arxiv.org/pdf/1512.03385v1.pdf

18 LAYERS1.8 GF

152 LAYERS11.3 GF

>6X MORE FLOPSRevolution of Depth

http://arxiv.org/pdf/1512.03385v1.pdf
http://arxiv.org/pdf/1512.03385v1.pdf
Page 13: The Revolution of Deep Learning

NVIDIA DGX-1WORLD’S FIRST DEEP LEARNING SUPERCOMPUTER

170 TFLOPS FP16

8x Tesla P100 16GB

NVLink Hybrid Cube Mesh

Accelerates Major AI Frameworks

Dual Xeon

7 TB SSD Deep Learning Cache

Dual 10GbE, Quad IB 100Gb

3RU – 3200W

Page 14: The Revolution of Deep Learning

DEEP LEARNING &

ARTIFICIAL INTELLIGENCE

Sep 28-29, 2016 | Amsterdam

www.gputechconf.eu #GTC16

SELF-DRIVING CARS VIRTUAL REALITY &

AUGMENTED REALITY

SUPERCOMPUTING & HPC

GTC Europe is a two-day conference designed to expose the innovative ways developers, businesses and academics are

using parallel computing to transform our world.

EUROPE’S BRIGHTEST MINDS & BEST IDEAS

2 Days | 800 Attendees | 50+ Exhibitors | 50+ Speakers | 15+ Tracks | 15+ Workshops | 1-to-1 Meetings

http://www.gputechconf.eu/

An Introduction to Deep Learning · Deep Learning An Introduction to ... •Deep Learning •Feed Forward Network •Problems of Deep Learning •AutoEncoders •Restricted Boltzmann

An Introduction to Deep Learning · Deep Learning An Introduction to ... •Deep Learning •Feed Forward Network •Problems of Deep Learning •AutoEncoders •Restricted Boltzmann

Documents
Intro to Deep Learning - bpTT Bright Minds … · 17/08/2020  · Intro to Deep Learning Deep Sequence Modeling Deep Computer Vision Deep Generative Modeling Deep Learning Applications

Intro to Deep Learning - bpTT Bright Minds … · 17/08/2020  · Intro to Deep Learning Deep Sequence Modeling Deep Computer Vision Deep Generative Modeling Deep Learning Applications

Documents
Deep Learning for AI - Data Science Summer School · Deep Learning for AI Yoshua Bengio August 28th, 2017 @ DS3 Data Science Summer School. A new revolution ... unlocked a breakthrough

Deep Learning for AI - Data Science Summer School · Deep Learning for AI Yoshua Bengio August 28th, 2017 @ DS3 Data Science Summer School. A new revolution ... unlocked a breakthrough

Documents
Introduction to Deep Learning - Joyce Hojoyceho.github.io/cs534_s17/slide/25-deep-learning.pdf · Deep Learning: Overview ... Review: Backpropogation https: ... • Deep Learning

Introduction to Deep Learning - Joyce Hojoyceho.github.io/cs534_s17/slide/25-deep-learning.pdf · Deep Learning: Overview ... Review: Backpropogation https: ... • Deep Learning

Documents
20151216 Computer Vision Meetup Deep Learning Deep Learning - Rene...Deep Learning . René Donner Deep Learning Overview 2 ... coursera ... 20151216 Computer Vision Meetup Deep Learning

20151216 Computer Vision Meetup Deep Learning Deep Learning - Rene...Deep Learning . René Donner Deep Learning Overview 2 ... coursera ... 20151216 Computer Vision Meetup Deep Learning

Documents
Deep Learning: An Introduction from the NLP Perspective Deep Learning - An...Deep Learning Approach 1 Deep Learning Approach 2 Disclaimer I am not (yet) an expert in Deep Learning

Deep Learning: An Introduction from the NLP Perspective Deep Learning - An...Deep Learning Approach 1 Deep Learning Approach 2 Disclaimer I am not (yet) an expert in Deep Learning

Documents
Deep Learning for Reinforcement Learning in · PDF fileDeep Learning for Reinforcement Learning in ... Deep Learning for Reinforcement Learning in Pacman Deep Learning für ... Während

Deep Learning for Reinforcement Learning in · PDF fileDeep Learning for Reinforcement Learning in ... Deep Learning for Reinforcement Learning in Pacman Deep Learning für ... Während

Documents
Deep Learning for Laser Based Odometry Estimation - …juxi.net/workshop/deep-learning-rss-2016/papers/Nicolai - Deep... · Deep Learning for Laser Based Odometry Estimation ... deep

Deep Learning for Laser Based Odometry Estimation - …juxi.net/workshop/deep-learning-rss-2016/papers/Nicolai - Deep... · Deep Learning for Laser Based Odometry Estimation ... deep

Documents
Executing Deep Learning Strategies Masterclass Preview - Enterprise Deep Learning

Executing Deep Learning Strategies Masterclass Preview - Enterprise Deep Learning

Software
Deep Learning for Generic Object Detection: A Survey · deep learning, probabilistic models, autoencoders, manifold learning, and deep networks 17 Deep learning LeCun et al. (2015)

Deep Learning for Generic Object Detection: A Survey · deep learning, probabilistic models, autoencoders, manifold learning, and deep networks 17 Deep learning LeCun et al. (2015)

Documents
Tutorial: Deep Reinforcement Learning - Machine Learning ...hunch.net/~beygel/deep_rl_tutorial.pdfTutorial: Deep Reinforcement Learning - Machine Learning

Tutorial: Deep Reinforcement Learning - Machine Learning ...hunch.net/~beygel/deep_rl_tutorial.pdfTutorial: Deep Reinforcement Learning - Machine Learning

Documents
Demystifying Deep Learning - MathWorks · Demystifying Deep Learning. 2 What’s Deep Learning and why should I care? A practical approach to Deep Learning (for images) –Transfer

Demystifying Deep Learning - MathWorks · Demystifying Deep Learning. 2 What’s Deep Learning and why should I care? A practical approach to Deep Learning (for images) –Transfer

Documents
Deep Learning Lab Course 2017 (Deep Learning …ais.informatik.uni-freiburg.de/teaching/ws17/deep...Deep Learning Lab Course 2017 (Deep Learning Practical) Labs: (Computer Vision)

Deep Learning Lab Course 2017 (Deep Learning …ais.informatik.uni-freiburg.de/teaching/ws17/deep...Deep Learning Lab Course 2017 (Deep Learning Practical) Labs: (Computer Vision)

Documents
The Deep Learning Revolution: Progress, Promise, and Profligate … · 2019-03-11 · Very Brief History of Machine Learning • Single-layer neural networks (1957- 1969) • Symbolic

The Deep Learning Revolution: Progress, Promise, and Profligate … · 2019-03-11 · Very Brief History of Machine Learning • Single-layer neural networks (1957- 1969) • Symbolic

Documents
Lecture 3: Deep Learning Optimizations - GitHub Pagesuvadlc.github.io/lectures/nov2019/lecture3.pdf · Lecture 3: Deep Learning Optimizations Deep Learning @ UvA. UVA DEEP LEARNING

Lecture 3: Deep Learning Optimizations - GitHub Pagesuvadlc.github.io/lectures/nov2019/lecture3.pdf · Lecture 3: Deep Learning Optimizations Deep Learning @ UvA. UVA DEEP LEARNING

Documents
Introduction to Deep Learning - Computer Graphics€¦ · Introduction to Deep Learning CS468 Spring 2017 Charles Qi. What is Deep Learning? Deep learning allows computational models

Introduction to Deep Learning - Computer Graphics€¦ · Introduction to Deep Learning CS468 Spring 2017 Charles Qi. What is Deep Learning? Deep learning allows computational models

Documents
Deep Learning for Reinforcement Learning in Pacman · Deep Learning for Reinforcement Learning in Pacman Deep Learning für Reinforcement Learning in Pacman Vorgelegte Bachelor-Thesis

Deep Learning for Reinforcement Learning in Pacman · Deep Learning for Reinforcement Learning in Pacman Deep Learning für Reinforcement Learning in Pacman Vorgelegte Bachelor-Thesis

Documents
Deep Graph Laplacian Regularization for Robust Denoising ...openaccess.thecvf.com/content_CVPRW_2019/papers/...Recent developments in deep learning have revolution-ized the aforementioned

Deep Graph Laplacian Regularization for Robust Denoising ...openaccess.thecvf.com/content_CVPRW_2019/papers/...Recent developments in deep learning have revolution-ized the aforementioned

Documents
Lecture 1: Introduction to Deep Learning · uva deep learning course –efstratios gavves introduction to deep learning - 1

Lecture 1: Introduction to Deep Learning · uva deep learning course –efstratios gavves introduction to deep learning - 1

Documents
The Military Technical Revolution- Can Corps Deep

The Military Technical Revolution- Can Corps Deep

Documents
The Learning Revolution

The Learning Revolution

Education
Lecture 1: Introduction to Deep Learning - uvadlc.github.io · Lecture 1: Introduction to Deep Learning Efstratios Gavves. UVA DEEP LEARNING COURSE UVA DEEP LEARNING COURSE –EFSTRATIOS

Lecture 1: Introduction to Deep Learning - uvadlc.github.io · Lecture 1: Introduction to Deep Learning Efstratios Gavves. UVA DEEP LEARNING COURSE UVA DEEP LEARNING COURSE –EFSTRATIOS

Documents
Deploying Enterprise Deep Learning Masterclass Preview - Enterprise Deep Learning

Deploying Enterprise Deep Learning Masterclass Preview - Enterprise Deep Learning

Software
LEADING THE LEARNING REVOLUTION - Michael …...LEADING THE LEARNING REVOLUTION MICHAEL FULLAN QUEST 2015 DEEP LEARNING IN A DIGITAL WORLD NOVEMBER 18-20, 2015 TORONTO 1 IRRESISTIBLE

LEADING THE LEARNING REVOLUTION - Michael …...LEADING THE LEARNING REVOLUTION MICHAEL FULLAN QUEST 2015 DEEP LEARNING IN A DIGITAL WORLD NOVEMBER 18-20, 2015 TORONTO 1 IRRESISTIBLE

Documents
Learning Revolution

Learning Revolution

Business
UNSUPERVISED DEEP LEARNING - TAUweb.eng.tau.ac.il/.../uploads/2016/12/Unsupervised-Deep-Learning.pdf · UNSUPERVISED DEEP LEARNING Erez Aharonov Noam Eilon Deep Learning Seminar School

UNSUPERVISED DEEP LEARNING - TAUweb.eng.tau.ac.il/.../uploads/2016/12/Unsupervised-Deep-Learning.pdf · UNSUPERVISED DEEP LEARNING Erez Aharonov Noam Eilon Deep Learning Seminar School

Documents
Regression in Deep Learning: Siamese and Triplet Networks · Revolution of deep learning in classification 3 28.19 25.7 15.3 11.19 6.7 4.86 2.99 2.25 2010 2011 2012 2013 2014 2015

Regression in Deep Learning: Siamese and Triplet Networks · Revolution of deep learning in classification 3 28.19 25.7 15.3 11.19 6.7 4.86 2.99 2.25 2010 2011 2012 2013 2014 2015

Documents
Deep Architectures for Natural Language Processingsergey/slides/N17_DataFest...plan • The deep learning revolution has not left natural language processing alone. • DL in NLP has

Deep Architectures for Natural Language Processingsergey/slides/N17_DataFest...plan • The deep learning revolution has not left natural language processing alone. • DL in NLP has

Documents
THE DEEP LEARNING REVOLUTION - Nvidia...THE DEEP LEARNING REVOLUTION Superhuman Breakthroughs in Modern Artificial Intelligence Powered by GPUs JOIN THE DEEP LEARNING ERA DEEP LEARNING

THE DEEP LEARNING REVOLUTION - Nvidia...THE DEEP LEARNING REVOLUTION Superhuman Breakthroughs in Modern Artificial Intelligence Powered by GPUs JOIN THE DEEP LEARNING ERA DEEP LEARNING

Documents
Methods and Resources Distributed Deep Learningsergey/slides/N17_AIUkraineDistributed.pdf · 10 years ago machine learning underwent a deep learning revolution Neural networks are

Methods and Resources Distributed Deep Learningsergey/slides/N17_AIUkraineDistributed.pdf · 10 years ago machine learning underwent a deep learning revolution Neural networks are

Documents