Machine Learning for Operations

J. Wenninger

With input from E. Piselli, A. Akroh, B. Salvachua, V. Kain, S. Hirlander

Presentation prepared by V.Kain

Operating accelerators = controlling

parameters

• Many successful examples

• Orbit and trajectory steering with YASP,

• Tune control in the higher energy

machines, but also in PS with combined

functions magnets

• Spill control in the SPS through Br

parameter

• …many others

• What counts is reproducibility of approach, efficiency of approach and maintainability.• E.g. different operators can get to same performance within the same time

• Well established CERN approach from LHC, SPS,.. - work with higher level physics parameters and deterministic algorithms, models

If the model is known, the problem is solved within 2-3 clicks.

Deterministic algorithms and model based correction is always the fastest

and easiest to implement (given a number of prerequisites).

Jörg Wenninger BE Machine Learning Workshop – May’19

However…

Some of the accelerators and the involved processes difficult to model and not available online

Examples:

• Space charge dominated beam dynamics in LINACs

• Controlling e-cooling

• Transmission optimization during transition crossing

• Alignment of collimators, electrostatic septa with many degrees of freedom

• Multi-turn injection with almost diagnostics except accumulated intensity measurement

Usually solved by trial-and-error manual scans in the control room

Time consuming

Depends heavily on experience of operator

Jörg Wenninger BE Machine Learning Workshop – May’19 4

Alternative?

• Reproducibility, flexibility and efficiency are more and more important• The number or projects, facilities and beam types to be

operated in parallel is constantly increasing

Advanced algorithms for tuning and diagnostics

Numerical optimizers and reinforcement learning combined with machine vision (or more conventional ML techniques) for complex diagnostics

OP has started deploying some of these techniques to problems mentioned before.

These techniques are game changers!

Jörg Wenninger BE Machine Learning Workshop – May’19 5

Generic beam optimization for scalars based on

Simplex Algorithm

Jörg Wenninger 6

-1.00E-11

0.00E+00

1.00E-11

2.00E-11

3.00E-11

4.00E-11

5.00E-11

6.00E-11

7.00E-11

8.00E-11

9.00E-11

0 100 200 300 400

12 parameters optimized

at the same time!

#iterations

inte

nsity

Started on PSB and PS and was

recently extended to ISOLDE

and HIE-ISOLDE

E. Piselli & A. Akroh

About 1200 scans performed

BE Machine Learning Workshop – May’19

Generic python tool for functions and scalars

optimization with variety of algorithms

• Including state-of-the-art algorithms BOBYQA, COBYLA,…• BOBYQA = Bounded Optimization by Quadratic Approximation,

• COBYLA = Constrained Optimization by Linear Approximation,

• And more in the family : UOBYQA, NEWUOA,

Jörg Wenninger 7

S. HirlanderS. Hirlander & D. Kuchler

BE Machine Learning Workshop – May’19

Comparison of numerical optimizers after tuning

of hyperparameters

2019 LINAC4 beam tests for chopping efficiency optimisation

Jörg Wenninger BE Machine Learning Workshop – May’19 8

S. Hirlander, V. Kain,

S. Appel (GSI),

G. Valentino (U. Malta)

BOBYQA

and COBYLA

fast and robust

And more results…

SPS electrostatic septum anode alignment:

9 free parameters in 40 minutes, before it took 8 h

Jörg Wenninger 9

TE-ABT with V. Kain & S. Hirlander

BE Machine Learning Workshop – May’19

Next step: reinforcement learning

Testing Reinforcement Learning at LEIR for

injection and electron cooler set up:

• First steps done with deep double Q learning

(proof of principle)

• Potential use for future e-cooler optimization

Jörg Wenninger 10

S. Hirlander & M. Schenk

Real machine learning. Optimization in a handful of iterations!

BE Machine Learning Workshop – May’19

Reinforcement learning – the challenge

• Tuning accelerators reinforcement learning with

continuous action space (not like playing computer

games)

• Challenge is the required sample efficiency until

“agent” has learned

• Simulation of accelerator problem (target steering,

injection matching) in OpenAI gym framework to

choose best adopted algorithm

Jörg Wenninger BE Machine Learning Workshop – May’19 11

episodes

ite

ratio

ns

Norm

. in

t.

Example of training for target steering

with NAF algorithm

S. Hirlander, V. Kain, M. Schenk

Preparing for FCC with Machine Learning

Jörg Wenninger 12

• FCC-ee (T. Tydecks): activity stopped (fellow not extended)

• Training an algorithm to learn correct a raw orbit towards a ‘golden orbit’

(good rms, good dispersion, good coupling).

• Tricky - using a model is still the best option !

• DA optimization (sextupole settings) with particle swarm optimizers provides

a gain of ~20% wrt simplex optimization.

• Optimization of ~ 900 sextupole settings.

Initial = simplex

J. Wenninger & T. Tydecks

BE Machine Learning Workshop – May’19

Machine Learning at the LHC

Jörg Wenninger 13

• Loss spike detection by ML for automated collimator alignment

(G. Azzopardi – shared with ABP/collimation).

• Working point with optimal lifetime at injection with ML

(L. Coyle, B. Salvachua together with T. Pieloni / EPFL).

• Proof of principle promising. Algorithm converged towards the region of expected

optimum (not usable directly due to other constraints).

Features: Spike Height,

Exponential Decay,

Collimator position in σ

exponential

decay

spike

height

G. Azzopardi, B. Salvachua, G. Valentino

BE Machine Learning Workshop – May’19

Collimator alignment

Jörg Wenninger 14

2018 Injection Commissioning

-> 79 collimators: 1.5 hours

2018 Parallel MD

-> 79 collimators: 50 minutes

2018 Angular Alignment MD

-> 1 collimator at 13 angles x 3

methods: 28 minutes

2018 ION Commissioning

-> 38 IR7 collimators: 1 hour

G. Azzopardi, et al.

From fully manual to fully automatic LHC Collimator alignment

BE Machine Learning Workshop – May’19

Where is the data coming from?

• Optimization and reinforcement learning: “online data”

• Eventually all ‘op tools’ aim to run online (predictive part).

• From simulation for testing or building surrogate models or mocking environments

• More standard ML, training: logging, ObsBox data, …

• Fast data access needed, plus storage of cleaned data samples.

Additional infrastructure need:

• How to use ML online in the control room with the JAVA world? UCAP? JAVA ML libraries to exploit and python to train? • Currently reinforcement learning is all in python.

Jörg Wenninger BE Machine Learning Workshop – May’19 15

It really pays off, but…

This work needs a "new” profile in the accelerator sector:

Hybrid of: accelerator physicist (with practical

experience in the control room) and data scientist

Currently working with PJAS (U. Malta) and collaboration

with EPFL (PhD for LHC, PostDoc for Injectors & LHC)

This is not enough: we need people at CERN doing this!

Otherwise we will not even exploit half of what it could offer…

Jörg Wenninger BE Machine Learning Workshop – May’19 16

Conclusion

• The application of ML in OP and the control room is (can be) wide ranging

• From supervised and unsupervised learning for diagnostics to optimization and reinforcement learning for accelerator control, fault identification and early fault prediction.

• In the injectors the use cases are numerous, with large payoff in terms of optimization time.

• Example: The setting up of the ISOLDE high intensity beam in the PSB with the optimisers was reduced from 3 weeks to 1 week!

• Why are we not fully embracing ML in OP?

If it is possible to have a model with a deterministic algorithm, it is still better and will always be!

• Unfortunately we do not have models everywhere, and might never have.

• Still a lot of improvements achievable with deterministic algorithms.

People are sceptical towards the “black box” – need to explain what is inside (no black magic) and train people who are expected to use such algorithms.

Jörg Wenninger BE Machine Learning Workshop – May’19 17