Deep Learning: a brief overview on the possibility for ... · Deep learning for studies of galaxy...

Deep Learning: a brief overview on the possibilityfor astrophysics

Francois-Xavier Dupe

(LIS/Aix-Marseille Universite, France)

Journee SKA @ LAM

Introduction

What is Machine Learning?

The aim of Machine Learning is to build a mathematical functionwhich solve a human task.

Today tasks include

classification/regression;

representation (or feature) learning;

transfert learning;

reinforcement learning;

F.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 2 / 25

Introduction

What is Machine Learning?

The aim of Machine Learning is to build a mathematical functionwhich solve a human task.

Today tasks include

classification/regression;

representation (or feature) learning;

transfert learning;

reinforcement learning;

Introduction

Some history about Machine Learning

1956: the Dartmouth workshop

Proposal

We propose that a 2 month, 10 man study of artificial intelligence becarried out during the summer of 1956 at Dartmouth College inHanover, New Hampshire. The study is to proceed on the basis of theconjecture that every aspect of learning or any other feature ofintelligence can in principle be so precisely described that a machinecan be made to simulate it. An attempt will be made to find how tomake machines use language, form abstractions and concepts, solvekinds of problems now reserved for humans, and improve themselves.We think that a significant advance can be made in one or more ofthese problems if a carefully selected group of scientists work on ittogether for a summer.

Introduction

Some history about Machine Learning (2)

Introduction

A brief timeline

from Andrew L. Beam

Introduction

Today’s talk

1 Deep learning

2 Interactions with astrophysics

3 What next?

Deep learning

Today’s talk

1 Deep learning

3 What next?

Deep learning

What is it?

Deep learning ⇒ hierarchical learning with high order features.

Deep learning

What is it?

Deep learning ⇒ hierarchical learning with high order features.

Deep learning

The location of deep learning

From https://www.machinecurve.com/index.php/2017/09/30/

the-differences-between-artificial-intelligence-machine-learning-more/

Deep learning

The zoo

From http://www.asimovinstitute.org/neural-network-zoo/

Deep learning

AlexNet

ImageNet Classification with Deep Convolutional Neural Networks byA. Krizhevsky et al (NIPS 2012)

Deep learning

AlexNet (results)

Deep learning

Long Short-Time Memory (LSTM)

Example of a recurrent neural network.

From http://colah.github.io/posts/2015-08-Understanding-LSTMs/

Deep learning

Long Short-Time Memory (LSTM): details

Input gate: encode theinput data.

Output gate: create thenext output (from theinput).

Forget gate: removeinformation from inputdata.

Block input: output datafrom another block.

Deep learning

Generative Adversarial Networks (GAN)

Idea: building a generator that can fool a discriminator

Generator: a NN thatproduce new data fromnoise.

Discriminator: aclassifier which distinguishfake data from true.

A set of real samples.

Generative adversarial nets by I. Goodfellow et al (NIPS 2014)

Deep learning

Generative Adversarial Networks (GAN): some results

From https://adeshpande3.github.io/Deep-Learning-Research-Review-Week-1-Generative-Adversarial-Nets

Interactions with astrophysics

Today’s talk

1 Deep learning

3 What next?

Some examples

Star-galaxy Classification Using Deep Convolutional Neural Networksby Edward J. Kim Robert J. Brunner (MNRAS 2016)

Idea: Use ConvNet on the reduced calibrated pixel values.

Some examples

Star-galaxy Classification Using Deep Convolutional Neural Networksby Edward J. Kim Robert J. Brunner (MNRAS 2016)

Idea: Use ConvNet on the reduced calibrated pixel values.

Some examples (2)

Fast Cosmic Web Simulations with Generative Adversarial Networksby A.C. Rodrıguez et al (arXiv:1801.09070)

Idea: use deep learning to avoid the computational cost of N-bodysimulation.

Some examples (2)

Fast Cosmic Web Simulations with Generative Adversarial Networksby A.C. Rodrıguez et al (arXiv:1801.09070)

Idea: use deep learning to avoid the computational cost of N-bodysimulation.

Some other examples from arXiv

Deep learning for studies of galaxy morphology (arXiv:1701.05917)

Detecting Solar-like Oscillations in Red Giants with Deep Learning(arXiv:1804.07495)

Lunar Crater Identification via Deep Learning (arXiv:1803.02192)

Deep learning from 21-cm images of the Cosmic Dawn (arXiv:1805.02699)

Fast Point Spread Function Modeling with Deep Learning (arXiv:1801.07615)

What next?

Today’s talk

1 Deep learning

3 What next?

What next?

Explainable Machine Learning

Deep learning a black-box?

Source: Department of Defense, Advanced Research Projects Agency.

What next?

Evaluation

One of the open question in Machine Learning: how to fullyevaluate a method?

No free lunch theorem (Wolpert and Macready 1997) which ”statesthat any two optimization algorithms are equivalent when theirperformance is averaged across all possible problems”.Ugly Duckling theorem (Watanabe 1969) which states that perfectclassification is impossible without some sort of bias.

By OswaldLR - From: A Corny Concerto (2).png, Public Domain

What next?

Evaluation

No free lunch theorem (Wolpert and Macready 1997) which ”statesthat any two optimization algorithms are equivalent when theirperformance is averaged across all possible problems”.

Ugly Duckling theorem (Watanabe 1969) which states that perfectclassification is impossible without some sort of bias.

By OswaldLR - From: A Corny Concerto (2).png, Public Domain

What next?

Evaluation

No free lunch theorem (Wolpert and Macready 1997) which ”statesthat any two optimization algorithms are equivalent when theirperformance is averaged across all possible problems”.Ugly Duckling theorem (Watanabe 1969) which states that perfectclassification is impossible without some sort of bias.

By OswaldLR - From: A Corny Concerto (2).png, Public DomainF.-X. Dupe (AMU) Deep learning: an overview 16 May 2018 23 / 25

Conclusion

Take away message

Evaluation is primordial! Must be coherent with the task.

Learning representation is a way to avoid some bias.

Deep learning asks for big datasets, but scale very well.

There is no free lunch!

Conclusion

Take away message

Conclusion

Take away message

Conclusion

Take away message

Conclusion

Thank you for your attention.

Any questions?

Deep Learning: a brief overview on the possibility for ... · Deep learning for studies of galaxy...

Documents

Tutorial: Deep Reinforcement Learning · Outline Introduction to Deep Learning Introduction to Reinforcement Learning Value-Based Deep RL Policy-Based Deep RL Model-Based Deep RL

THE DEEP LEARNING REVOLUTION - Nvidia...THE DEEP LEARNING REVOLUTION Superhuman Breakthroughs in Modern Artiﬁcial Intelligence Powered by GPUs JOIN THE DEEP LEARNING ERA DEEP LEARNING

fast.ai - Learning Deep Learning

Deep Learning

Lecture 1: Introduction to Deep Learning · uva deep learning course –efstratios gavves introduction to deep learning - 1

Deep Learning with H2O j Installation collaborating closely with academic researchers and industrial data scientists. Stanford university giants Stephen Boyd, Trevor Hastie, Rob Tibshirani

Introducing Deep Learning with MATLAB - MathWorks · Introducing Deep Learning with MATLAB7 How A Deep Neural Network ... relevant features of an image. With deep learning, ... Deep

Deep Learning - Oxford Statisticspalamara/teaching/SML19/HT19_lecture15.pdf · SB2b/SM4 - Deep Learning Deep Learning Infrastructure! Computational Infrastructure critical to deep

VISITOR LEARNING GUIDE - Ecotypeecotype.com.au/.../2015/11/Styx-Visitor-Learning-Guide-spreads.pdf · STY VALLE F THE GIANTS - VISITOR LEARNING GUIDE The Styx Valley of the Giants

Lecture 1: Introduction to Deep Learning - uvadlc.github.io · Lecture 1: Introduction to Deep Learning Efstratios Gavves. UVA DEEP LEARNING COURSE UVA DEEP LEARNING COURSE –EFSTRATIOS

Hierarchical Deep Reinforcement Learning: Integrating ...papers.nips.cc/paper/6233-hierarchical-deep-reinforcement-learning... · work that integrates deep reinforcement learning

Lecture 3: Deep Learning Optimizations · UVA DEEP LEARNING COURSE EFSTRATIOS GAVVES DEEP LEARNING OPTIMIZATIONS -3 Empirical Risk Minimization

Deep Learning: An Introduction from the NLP Perspective Deep Learning - An...Deep Learning Approach 1 Deep Learning Approach 2 Disclaimer I am not (yet) an expert in Deep Learning

Deep Learning - South Dakota State UniversityDeep Learning Deep learning (also known as deep structured learning, hierarchical learning or deep machine learning) is a branch of machine

Deep Learning for Reinforcement Learning in Pacman · Deep Learning for Reinforcement Learning in Pacman Deep Learning für Reinforcement Learning in Pacman Vorgelegte Bachelor-Thesis

An Introduction to Deep Learning · Deep Learning An Introduction to ... •Deep Learning •Feed Forward Network •Problems of Deep Learning •AutoEncoders •Restricted Boltzmann

Deep Learning Early Work Why Deep Learning Stacked Auto Encoders Deep Belief Networks CS 678 – Deep Learning1

Deploying Enterprise Deep Learning Masterclass Preview - Enterprise Deep Learning

Intro to Deep Learning - bpTT Bright Minds … · 17/08/2020 · Intro to Deep Learning Deep Sequence Modeling Deep Computer Vision Deep Generative Modeling Deep Learning Applications

Deep Learning Lab Course 2017 (Deep Learning …ais.informatik.uni-freiburg.de/teaching/ws17/deep...Deep Learning Lab Course 2017 (Deep Learning Practical) Labs: (Computer Vision)