Deep Learning in the Natural Sciences: Applications to High-Energy Physics

Peter SadowskiAssistant Professor of Computer Science

Galileo Galiliei, 1600 Isaac Newton, 1670 Gregor Mendel, 1860

Models

Data

Models

Data

Models𝜃

DataD

P(𝜃|D) P(D|𝜃) Bayes Rule:

𝜃

DataD

P(𝜃|D)

P(𝜑|𝜃)

Bayes Rule:𝜑

P(D|𝜑)

1) Science is an optimization problem:

2) In general, this optimization is computationally intractable.3) Machine learning is the study of tractable solutions.

Neural Network Models

Deep Learning

Deep Learning

Understanding Neural Networks through Deep Visualization, Yosinski, et. al. 2016

Deep LearningSome of the advantages of deep neural networks:

1. Depth seems to help generalization.

2. Architecture motifs for constraining the learning problem.

3. Better scaling to large data sets.

4. 10× − 100× speed up with Graphics Processing Units (GPUs).

5. Differentiable.

Computer vision

Voice recognition

Natural language processing

Optimization

Music synthesis

Finance

Predictive maintenance

Drug/material discovery

HealthcareChat bots

RoboticsControl systems Brain-computer

interfacesAugmented reality

Deep LearningGames (AI players, graphics)

Deep Learning in High-Energy Physics

Deep Learning for Exotic Particle Searches

Large Hadron Collider at CERN, outside Geneva, Switzerland

Deep Learning for Exotic Particle Searches

Decay Type𝜃

DataD

P(𝜃|D)

𝜑1

P(D|𝜑k)

𝜑k

...

P(𝜑1|𝜃)

One approach:

1) Simulate data generation process.2) Estimate P(𝜃|D) using

feature-engineering and machine learning.

Other possible approaches:

● Matrix-Element Methods● Approximate Bayesian Computation

Decay Type𝜃

DataD

P(𝜃|D)

𝜑1

P(D|𝜑k)

𝜑k

...

P(𝜑1|𝜃)

Example: Higgs Boson Detection

Data: 11M labeled Monte-Carlo simulation events.

Features: Trajectories of observed particles.

Engineered Features: Mass estimates, missing mass, sphericity, etc.

Higgs Event Background Event

Example: Higgs Boson Detection

Example: Higgs Boson Detection

Example: Higgs Boson Detection

Example: Higgs Boson Detection

Deep Learning in High-Energy Physics1. Searching for Exotic Particles in HEP with DL, Nature Comm. 20142. Deep Learning, Dark Knowledge, and Dark Matter, JMLR 20143. Enhanced Higgs Boson to τ+τ− Search with Deep Learning, PRL 20154. Parameterized Neural Networks for High-Energy Physics, Eur. Phys. J. C, 20165. Jet Substructure Classification in HEP with Deep NN, Physical Rev. D, 20166. Jet flavor classification in high-energy physics with deep neural networks, Physical Rev. D, 20167. Decorrelated Jet Tagging using Adversarial Neural Networks, Physical Rev. D, 2017

Common theme:

1. Improves performance by extracting more information.2. Reduces workload of physicists.

Neural Network Architecture Design

Structured Input

“The first and only 3D Smart Water Sensor that allows you to date and convert your phone on the go!”

Antihydrogen Detection with Deep Learning

ASACUSA experiment

Antihydrogen Detection with Deep Learning

Efficient Antihydrogen Detection in Antimatter Physics by Deep LearningSadowski, et. al. 2017

Antihydrogen Detection with Deep Learning

Dark Matter Detection with Deep Learning

XENON1T Dark Matter Detector Detector data.

Dark Matter Detection with Deep Learning

Disentangling Deep Representations

Deep Generative Models𝜑1

𝜑k

...

Generative Adversarial Networks

Brock, et. al. 2018, Large Scale GAN Training for High-Fidelity Natural Image Synthesis

Generative Adversarial Networks

(Real or generated?)

(Gradient reversal layer)

Result: Convincing fake samples.

Conditional Generative Adversarial Networks

(Real or generated?)

Class C

Result: Convincing class-conditional fake samples.

Transfer Learning

X Y

GRL

Classification:Is input X Real or Simulated?

Result: Classifier that can be trained on simulated data and perform well on real data.

Protected Attributes

X, A Y

GRL

Classification: Predicted attribute A

Result: Classifier that is “blind” to the protected attribute.

Factored/Disentangled Representations

X Y GRL GRL

Result: Intermediate representations zero mutual information, I(H1, H2)=0.

H1

H2

Jets

Single Jet Double JetClustering

(Feature Engineering)Jet Substructure Classification in High-Energy Physics with Deep Neural Networks, 2016

Adversarial Networks for Jet-Tagging

Adversarial Networks for Jet-Tagging

Adversarial Networks for Jet-Tagging

Decorrelated Jet Substructure Tagging using Adversarial Neural Networks, Shimmin, et. al. 2017

Adversarial Networks for Jet-Tagging

Adversarial Networks for Jet-Tagging

Other Applications

Tasks for ML:1) Likelihood free inference.2) Disentangled representations.

Also:

3) Learn fast approximations of slow simulations.4) Optimization.5) Learn generative models of distributions. 6) Anomaly detection.

Deep Learning for Weather Analysis

Giuseppe TorriUniv. of Hawaii

Deep Learning for Chemical Reaction Prediction

Synergies Between Quantum Mechanics and Machine Learning in Reaction Prediction,Sadowski, et. al. 2016

Other Projects

Hyperparameter Optimization

1. Automate experiments on a cluster.2. Visualize results.3. Explore high-dimensional hyperparameter space.

www.github.com/LarsHH/sherpa

Neural Density Estimators

Neural Density Estimators

Neural Density Estimators

Potential application: clustering metagenomic data.